2024-05-09T01:03:18.148 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T01:03:18.149 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932 --name pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/test_o_trunc}} --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/orig.config.yaml 2024-05-09T01:03:18.204 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T01:03:18.295 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932 branch: wip-pdonnell-testing-20240508.183908-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698932' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240508.183908-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.183908-debug sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1722 sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.183908-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPLS0+K77/DESGgHxYUz2V/9GaiCe9KiuzH/xMTS88RhKJCp48JnUmO3mlp+fFELCfup45TxPGb0AgpnJbfBTa8= smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGyNA7CI/ChV04v+HmNSV2gZyGNpYNl5IDDJys0TQucOVG8sgDbgVwhM7RfxGClk0NhehyvH/Ta8JqWTNSKe4a4= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGMBfBAWvnJVWq8snTIAjeQEGoWJuEAhCQemdKDMeu+SqiunK3OhyGe3aWYgv3ud6GA4kq4ZQD/tLRkBpfKrOQ0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_22:06:20 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T01:03:18.296 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa; will attempt to use it 2024-05-09T01:03:18.296 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks 2024-05-09T01:03:18.296 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T01:03:18.298 INFO:teuthology.task.internal:Checking packages... 2024-05-09T01:03:18.321 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c9da4049f638c0f84bdd9b4e33c42e86046ac554' 2024-05-09T01:03:18.321 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T01:03:18.321 INFO:teuthology.packaging:ref: None 2024-05-09T01:03:18.321 INFO:teuthology.packaging:tag: None 2024-05-09T01:03:18.322 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.183908-debug 2024-05-09T01:03:18.322 INFO:teuthology.packaging:sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:03:18.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240508.183908-debug 2024-05-09T01:03:18.470 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3594.gc9da4049 2024-05-09T01:03:18.471 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T01:03:18.477 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T01:03:18.477 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T01:03:18.482 INFO:teuthology.task.internal:Saving configuration 2024-05-09T01:03:18.502 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T01:03:18.508 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T01:03:18.533 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 00:58:19.205689', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPLS0+K77/DESGgHxYUz2V/9GaiCe9KiuzH/xMTS88RhKJCp48JnUmO3mlp+fFELCfup45TxPGb0AgpnJbfBTa8='} 2024-05-09T01:03:18.553 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 00:58:19.204065', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGyNA7CI/ChV04v+HmNSV2gZyGNpYNl5IDDJys0TQucOVG8sgDbgVwhM7RfxGClk0NhehyvH/Ta8JqWTNSKe4a4='} 2024-05-09T01:03:18.576 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 00:58:19.202159', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGMBfBAWvnJVWq8snTIAjeQEGoWJuEAhCQemdKDMeu+SqiunK3OhyGe3aWYgv3ud6GA4kq4ZQD/tLRkBpfKrOQ0='} 2024-05-09T01:03:18.576 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T01:03:18.583 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-09T01:03:18.583 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-09T01:03:18.583 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-09T01:03:18.583 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T01:03:18.695 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff8161070d0>, signals=[15]) 2024-05-09T01:03:18.695 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T01:03:18.701 INFO:teuthology.task.internal:Opening connections... 2024-05-09T01:03:18.701 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T01:03:18.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:03:18.779 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-05-09T01:03:18.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:03:18.852 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-05-09T01:03:18.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:03:18.924 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T01:03:18.931 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-05-09T01:03:18.949 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-05-09T01:03:18.950 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-05-09T01:03:19.007 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T01:03:19.008 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T01:03:19.009 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-05-09T01:03:19.030 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-05-09T01:03:19.048 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-05-09T01:03:19.049 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-05-09T01:03:19.106 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-05-09T01:03:19.106 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-05-09T01:03:19.106 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-05-09T01:03:19.106 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-05-09T01:03:19.106 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-05-09T01:03:19.107 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-05-09T01:03:19.107 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T01:03:19.107 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-05-09T01:03:19.107 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-05-09T01:03:19.107 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T01:03:19.107 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-05-09T01:03:19.107 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T01:03:19.107 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T01:03:19.107 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T01:03:19.107 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-05-09T01:03:19.129 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-05-09T01:03:19.147 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-05-09T01:03:19.147 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-05-09T01:03:19.204 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T01:03:19.205 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T01:03:19.206 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-05-09T01:03:19.230 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T01:03:19.240 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T01:03:19.245 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T01:03:19.245 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-05-09T01:03:19.248 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-05-09T01:03:19.250 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-05-09T01:03:19.266 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T01:03:19.272 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T01:03:19.272 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-05-09T01:03:19.305 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-05-09T01:03:19.307 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-05-09T01:03:19.339 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T01:03:19.417 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T01:03:19.431 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T01:03:19.431 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-09T01:03:19.431 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-09T01:03:19.431 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T01:03:19.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:03:19.449 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-05-09T01:03:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-05-09T01:03:19.507 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-437.el9.x86_64 2024-05-09T01:03:19.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-05-09T01:03:22.152 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.1 MB 00:01 2024-05-09T01:03:34.320 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 19 MB 00:11 2024-05-09T01:03:39.378 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-05-09T01:03:40.978 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-05-09T01:03:43.705 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 21 MB 00:02 2024-05-09T01:03:49.079 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-09T01:03:51.203 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T01:03:51.203 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T01:03:51.310 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-05-09T01:03:51.319 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-05-09T01:03:51.319 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-05-09T01:03:51.319 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-05-09T01:03:51.319 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-05-09T01:03:51.319 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T01:03:51.319 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-05-09T01:03:51.319 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T01:03:51.319 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T01:03:51.319 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T01:03:51.320 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:03:51.320 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-05-09T01:03:51.320 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-05-09T01:03:51.320 INFO:teuthology.orchestra.run.smithi019.stdout:Install 4 Packages 2024-05-09T01:03:51.320 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:03:51.320 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 78 M 2024-05-09T01:03:51.321 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 123 M 2024-05-09T01:03:51.321 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-05-09T01:03:51.531 INFO:teuthology.orchestra.run.smithi019.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.8 MB/s | 484 kB 00:00 2024-05-09T01:03:52.123 INFO:teuthology.orchestra.run.smithi019.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 20 MB/s | 15 MB 00:00 2024-05-09T01:03:52.743 INFO:teuthology.orchestra.run.smithi019.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 23 MB/s | 28 MB 00:01 2024-05-09T01:03:53.548 INFO:teuthology.orchestra.run.smithi019.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 16 MB/s | 34 MB 00:02 2024-05-09T01:03:53.549 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:03:53.549 INFO:teuthology.orchestra.run.smithi019.stdout:Total 35 MB/s | 78 MB 00:02 2024-05-09T01:03:54.261 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-05-09T01:03:54.665 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-05-09T01:03:54.665 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-05-09T01:03:55.622 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-05-09T01:03:55.622 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-05-09T01:03:58.192 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-05-09T01:03:58.856 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T01:03:58.940 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T01:04:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T01:04:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T01:04:06.962 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T01:04:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:04:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:04:43.479 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:04:43.520 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:04:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:04:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T01:04:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T01:04:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T01:04:44.682 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:04:44.682 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:04:44.682 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-05-09T01:04:44.682 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T01:04:44.682 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T01:04:44.682 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T01:04:44.682 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T01:04:44.682 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:04:44.682 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-05-09T01:04:45.020 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T01:04:45.095 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T01:04:45.095 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T01:04:45.095 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T01:04:45.095 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T01:04:45.095 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T01:04:45.095 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-09T01:04:45.096 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T01:04:45.096 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T01:04:45.147 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T01:04:45.147 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T01:04:45.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-05-09T01:04:45.177 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 68.el9 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8264254 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-05-09T01:04:45.178 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:04:45.179 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T01:04:45.180 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T01:04:45.180 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 5.14.0-444.el9.x86_64 2024-05-09T01:04:45.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T01:04:45.770 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-05-09T01:04:47.108 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T01:04:47.139 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-05-09T01:04:47.141 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-05-09T01:04:47.156 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.6LBY9YlEi4 2024-05-09T01:04:47.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6LBY9YlEi4 2024-05-09T01:04:47.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.6LBY9YlEi4 2024-05-09T01:04:47.367 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.6LBY9YlEi4 is 6KB 2024-05-09T01:04:47.416 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.6LBY9YlEi4 2024-05-09T01:04:47.431 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T01:04:47.500 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T01:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T01:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T01:04:47.501 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T01:04:47.502 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T01:04:47.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-05-09T01:05:17.666 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T01:05:17.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:05:17.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:05:36.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-05-09T01:05:45.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:05:45.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:06:45.035 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T01:06:57.036 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:06:57.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:06:57.344 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-05-09T01:06:57.687 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:06:57.687 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-09T01:06:57.687 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-05-09T01:06:57.704 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-444.el9.x86_64 2024-05-09T01:06:57.704 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T01:06:57.704 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T01:06:57.704 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T01:06:58.705 DEBUG:teuthology.parallel:result is None 2024-05-09T01:06:58.705 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T01:06:58.713 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T01:06:58.713 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T01:06:58.718 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T01:06:58.720 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T01:06:58.739 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T01:06:58.745 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T01:06:58.752 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T01:06:58.752 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T01:06:58.775 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T01:06:58.779 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T01:06:58.991 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T01:06:58.997 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T01:06:58.997 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T01:06:59.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:06:59.015 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T01:06:59.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:06:59.032 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T01:06:59.034 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T01:06:59.057 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T01:06:59.089 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:06:59.093 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:06:59.108 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:06:59.113 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:06:59.121 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:06:59.143 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:06:59.144 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T01:06:59.151 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T01:06:59.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T01:06:59.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T01:06:59.157 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T01:06:59.211 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T01:06:59.219 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T01:06:59.220 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T01:06:59.223 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T01:06:59.225 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T01:06:59.272 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:06:59.342 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:06:59.410 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:06:59.511 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:06:59.554 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:06:59.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T01:06:59.628 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:06:59.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:06:59.727 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:06:59.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:06:59.874 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:06:59.874 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T01:06:59.943 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:06:59.987 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:07:00.042 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:07:00.122 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:07:00.169 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:07:00.169 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T01:07:00.236 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-05-09T01:07:00.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-05-09T01:07:00.241 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-05-09T01:07:00.277 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:07:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:07:00.318 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:07:00.736 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T01:07:00.744 INFO:teuthology.task.internal:Starting timer... 2024-05-09T01:07:00.744 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T01:07:00.754 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T01:07:00.762 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T01:07:00.762 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-05-09T01:07:00.828 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-05-09T01:07:00.830 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-05-09T01:07:00.887 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-05-09T01:07:00.889 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-05-09T01:07:00.965 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-05-09T01:07:00.967 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T01:07:00.967 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-05-09T01:07:00.992 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-05-09T01:07:00.993 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-05-09T01:07:01.022 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-05-09T01:07:01.022 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-05-09T01:07:01.049 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-05-09T01:07:01.049 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-05-09T01:07:01.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:07:01.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:07:01.078 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:07:01.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:07:01.106 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:07:01.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:07:01.135 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T01:07:01.135 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-05-09T01:07:01.162 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-05-09T01:07:01.189 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-05-09T01:07:01.218 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T01:07:01.227 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T01:07:01.247 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T01:07:01.249 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi059.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T01:11:31.510 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-05-09T01:11:31.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:11:31.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:11:31.590 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-05-09T01:11:31.670 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:11:31.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-09T01:11:31.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:11:31.752 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-05-09T01:11:31.838 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-09T01:11:31.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-09T01:11:31.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:11:31.929 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-05-09T01:11:32.015 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-09T01:11:32.016 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T01:11:32.028 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T01:11:32.028 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T01:11:32.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:11:32.031 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T01:11:32.031 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:11:32.033 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T01:11:32.033 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:11:32.060 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T01:11:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T01:11:32.075 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T01:11:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T01:11:32.087 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T01:11:32.099 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-05-09T01:11:32.102 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T01:11:32.102 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-05-09T01:11:32.109 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-05-09T01:11:32.114 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-05-09T01:11:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T01:11:32.126 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-05-09T01:11:32.129 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T01:11:32.137 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-05-09T01:11:32.137 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T01:11:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T01:11:32.150 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T01:11:32.163 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T01:11:32.192 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:11:32.195 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:11:32.195 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-05-09T01:11:32.195 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.195 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.195 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.195 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:11:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:11:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-05-09T01:11:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.216 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:11:32.218 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:11:32.219 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-05-09T01:11:32.219 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.219 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.219 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.219 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:11:32.220 INFO:teuthology.run_tasks:Running task install... 2024-05-09T01:11:32.228 DEBUG:teuthology.task.install:project ceph 2024-05-09T01:11:32.228 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'}} 2024-05-09T01:11:32.228 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'} 2024-05-09T01:11:32.229 INFO:teuthology.task.install:Using flavor: default 2024-05-09T01:11:32.238 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T01:11:32.238 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T01:11:32.238 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T01:11:32.239 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:11:32.241 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T01:11:32.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:11:32.242 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T01:11:32.242 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:11:32.398 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T01:11:32.398 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T01:11:32.406 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T01:11:32.406 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T01:11:32.475 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T01:11:32.476 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T01:11:32.524 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T01:11:32.524 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:11:32.524 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T01:11:32.551 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T01:11:32.552 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T01:11:32.618 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T01:11:32.656 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T01:11:32.656 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:11:32.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T01:11:32.683 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T01:11:32.683 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T01:11:32.695 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T01:11:32.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T01:11:32.760 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-05-09T01:11:32.762 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-05-09T01:11:32.824 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T01:11:32.837 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T01:11:32.837 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:11:32.837 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T01:11:32.866 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T01:11:32.866 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T01:11:32.888 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-05-09T01:11:32.889 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-05-09T01:11:32.936 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T01:11:33.013 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T01:11:33.014 INFO:teuthology.orchestra.run.smithi019.stdout:54 files removed 2024-05-09T01:11:33.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T01:11:33.078 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-05-09T01:11:33.080 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-05-09T01:11:33.125 INFO:teuthology.orchestra.run.smithi059.stdout:54 files removed 2024-05-09T01:11:33.153 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T01:11:33.312 INFO:teuthology.orchestra.run.smithi196.stdout:54 files removed 2024-05-09T01:11:33.340 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T01:11:33.793 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 180 kB/s | 84 kB 00:00 2024-05-09T01:11:33.901 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 177 kB/s | 84 kB 00:00 2024-05-09T01:11:34.496 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 94 kB/s | 84 kB 00:00 2024-05-09T01:11:34.668 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-05-09T01:11:34.735 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 15 kB/s | 12 kB 00:00 2024-05-09T01:11:35.279 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-05-09T01:11:35.360 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-05-09T01:11:35.428 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-05-09T01:11:35.846 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-05-09T01:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.1 MB 00:01 2024-05-09T01:11:37.129 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.1 MB 00:01 2024-05-09T01:11:37.863 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.1 MB 00:01 2024-05-09T01:11:39.647 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-09T01:11:40.420 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-05-09T01:11:46.489 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.2 MB 00:03 2024-05-09T01:11:47.687 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.2 MB 00:03 2024-05-09T01:11:47.865 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 16 kB 00:00 2024-05-09T01:11:49.066 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-09T01:11:49.188 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-05-09T01:11:50.589 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-09T01:11:54.141 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 50 kB/s | 1.7 kB 00:00 2024-05-09T01:11:55.989 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-05-09T01:11:56.253 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-05-09T01:11:56.400 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:11:56.401 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:11:56.411 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T01:11:56.412 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T01:11:56.413 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T01:11:56.414 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T01:11:56.415 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T01:11:56.417 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T01:11:56.418 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T01:11:56.478 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-09T01:11:56.496 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-09T01:11:56.496 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-09T01:11:56.496 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T01:11:56.497 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T01:11:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T01:11:56.499 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T01:11:56.500 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T01:11:56.500 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-05-09T01:11:56.500 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T01:11:56.500 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T01:11:56.500 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T01:11:56.500 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T01:11:56.500 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T01:11:56.500 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T01:11:56.502 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T01:11:56.503 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T01:11:56.504 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T01:11:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T01:11:56.506 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T01:11:56.507 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T01:11:56.508 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T01:11:56.509 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-09T01:11:56.510 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-05-09T01:11:56.511 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 8 Packages 2024-05-09T01:11:56.511 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:11:56.511 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 219 M 2024-05-09T01:11:56.511 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-09T01:11:58.400 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:11:58.401 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:11:58.412 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T01:11:58.413 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T01:11:58.413 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T01:11:58.414 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T01:11:58.416 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T01:11:58.418 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T01:11:58.419 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T01:11:58.484 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-05-09T01:11:58.500 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-09T01:11:58.500 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-05-09T01:11:58.500 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-09T01:11:58.500 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T01:11:58.501 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T01:11:58.502 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T01:11:58.503 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T01:11:58.504 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T01:11:58.505 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T01:11:58.506 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T01:11:58.506 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T01:11:58.506 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T01:11:58.506 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T01:11:58.506 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T01:11:58.506 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T01:11:58.506 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T01:11:58.507 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T01:11:58.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T01:11:58.509 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T01:11:58.510 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T01:11:58.511 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T01:11:58.512 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T01:11:58.513 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:11:58.514 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-05-09T01:11:58.514 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-09T01:11:58.514 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-05-09T01:11:58.514 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 8 Packages 2024-05-09T01:11:58.514 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:11:58.518 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 219 M 2024-05-09T01:11:58.518 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-05-09T01:11:59.026 INFO:teuthology.orchestra.run.smithi196.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-05-09T01:11:59.400 INFO:teuthology.orchestra.run.smithi019.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-05-09T01:11:59.400 INFO:teuthology.orchestra.run.smithi196.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 2.2 MB/s | 863 kB 00:00 2024-05-09T01:11:59.492 INFO:teuthology.orchestra.run.smithi196.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 1.6 MB/s | 147 kB 00:00 2024-05-09T01:11:59.735 INFO:teuthology.orchestra.run.smithi196.stdout:(4/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 5.7 MB/s | 5.4 MB 00:00 2024-05-09T01:11:59.850 INFO:teuthology.orchestra.run.smithi019.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 1.9 MB/s | 863 kB 00:00 2024-05-09T01:11:59.983 INFO:teuthology.orchestra.run.smithi019.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 1.1 MB/s | 147 kB 00:00 2024-05-09T01:11:59.993 INFO:teuthology.orchestra.run.smithi196.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 5.8 MB/s | 1.5 MB 00:00 2024-05-09T01:12:00.209 INFO:teuthology.orchestra.run.smithi019.stdout:(4/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 5.3 MB/s | 5.4 MB 00:01 2024-05-09T01:12:00.310 INFO:teuthology.orchestra.run.smithi196.stdout:(6/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 2.8 MB/s | 2.3 MB 00:00 2024-05-09T01:12:00.567 INFO:teuthology.orchestra.run.smithi019.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 4.2 MB/s | 1.5 MB 00:00 2024-05-09T01:12:00.677 INFO:teuthology.orchestra.run.smithi196.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 7.1 MB/s | 4.9 MB 00:00 2024-05-09T01:12:00.951 INFO:teuthology.orchestra.run.smithi019.stdout:(6/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 2.3 MB/s | 2.3 MB 00:00 2024-05-09T01:12:01.776 INFO:teuthology.orchestra.run.smithi019.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 4.0 MB/s | 4.9 MB 00:01 2024-05-09T01:12:01.870 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.2 MB 00:01 2024-05-09T01:12:02.401 INFO:teuthology.orchestra.run.smithi019.stdout:(8/156): ceph-common-19.0.0-3594.gc9da4049.el9. 6.1 MB/s | 20 MB 00:03 2024-05-09T01:12:02.485 INFO:teuthology.orchestra.run.smithi019.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 303 kB/s | 25 kB 00:00 2024-05-09T01:12:02.602 INFO:teuthology.orchestra.run.smithi196.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 5.3 MB/s | 10 MB 00:01 2024-05-09T01:12:02.711 INFO:teuthology.orchestra.run.smithi196.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 232 kB/s | 25 kB 00:00 2024-05-09T01:12:03.111 INFO:teuthology.orchestra.run.smithi196.stdout:(10/156): ceph-common-19.0.0-3594.gc9da4049.el9 4.5 MB/s | 20 MB 00:04 2024-05-09T01:12:03.228 INFO:teuthology.orchestra.run.smithi196.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 1.9 MB/s | 224 kB 00:00 2024-05-09T01:12:03.353 INFO:teuthology.orchestra.run.smithi196.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 216 kB/s | 27 kB 00:00 2024-05-09T01:12:03.703 INFO:teuthology.orchestra.run.smithi196.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 91 kB/s | 32 kB 00:00 2024-05-09T01:12:03.735 INFO:teuthology.orchestra.run.smithi019.stdout:(10/156): ceph-radosgw-19.0.0-3594.gc9da4049.el 5.2 MB/s | 10 MB 00:01 2024-05-09T01:12:03.810 INFO:teuthology.orchestra.run.smithi019.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 2.9 MB/s | 224 kB 00:00 2024-05-09T01:12:03.869 INFO:teuthology.orchestra.run.smithi019.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 465 kB/s | 27 kB 00:00 2024-05-09T01:12:03.928 INFO:teuthology.orchestra.run.smithi019.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 543 kB/s | 32 kB 00:00 2024-05-09T01:12:04.045 INFO:teuthology.orchestra.run.smithi196.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 2.2 MB/s | 756 kB 00:00 2024-05-09T01:12:04.078 INFO:teuthology.orchestra.run.smithi019.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 4.9 MB/s | 756 kB 00:00 2024-05-09T01:12:04.137 INFO:teuthology.orchestra.run.smithi196.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.7 MB/s | 158 kB 00:00 2024-05-09T01:12:04.145 INFO:teuthology.orchestra.run.smithi019.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 2.3 MB/s | 158 kB 00:00 2024-05-09T01:12:04.211 INFO:teuthology.orchestra.run.smithi019.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.9 MB/s | 127 kB 00:00 2024-05-09T01:12:04.212 INFO:teuthology.orchestra.run.smithi196.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.7 MB/s | 127 kB 00:00 2024-05-09T01:12:04.346 INFO:teuthology.orchestra.run.smithi196.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 3.6 MB/s | 497 kB 00:00 2024-05-09T01:12:04.404 INFO:teuthology.orchestra.run.smithi019.stdout:(17/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 5.1 MB/s | 18 MB 00:03 2024-05-09T01:12:04.437 INFO:teuthology.orchestra.run.smithi019.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 2.2 MB/s | 497 kB 00:00 2024-05-09T01:12:04.488 INFO:teuthology.orchestra.run.smithi019.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 906 kB/s | 45 kB 00:00 2024-05-09T01:12:04.520 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-05-09T01:12:04.546 INFO:teuthology.orchestra.run.smithi019.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 2.3 MB/s | 135 kB 00:00 2024-05-09T01:12:04.605 INFO:teuthology.orchestra.run.smithi019.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 2.7 MB/s | 162 kB 00:00 2024-05-09T01:12:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 2.9 MB/s | 321 kB 00:00 2024-05-09T01:12:04.789 INFO:teuthology.orchestra.run.smithi019.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 3.9 MB/s | 301 kB 00:00 2024-05-09T01:12:04.855 INFO:teuthology.orchestra.run.smithi019.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.5 MB/s | 99 kB 00:00 2024-05-09T01:12:04.906 INFO:teuthology.orchestra.run.smithi019.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.7 MB/s | 86 kB 00:00 2024-05-09T01:12:05.065 INFO:teuthology.orchestra.run.smithi019.stdout:(26/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 7.8 MB/s | 5.2 MB 00:00 2024-05-09T01:12:05.115 INFO:teuthology.orchestra.run.smithi019.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 3.3 MB/s | 169 kB 00:00 2024-05-09T01:12:05.157 INFO:teuthology.orchestra.run.smithi019.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 818 kB/s | 34 kB 00:00 2024-05-09T01:12:05.207 INFO:teuthology.orchestra.run.smithi019.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 2.8 MB/s | 142 kB 00:00 2024-05-09T01:12:05.546 INFO:teuthology.orchestra.run.smithi196.stdout:(18/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 4.3 MB/s | 5.2 MB 00:01 2024-05-09T01:12:05.574 INFO:teuthology.orchestra.run.smithi019.stdout:(30/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 4.6 MB/s | 3.0 MB 00:00 2024-05-09T01:12:05.605 INFO:teuthology.orchestra.run.smithi196.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 776 kB/s | 45 kB 00:00 2024-05-09T01:12:05.688 INFO:teuthology.orchestra.run.smithi196.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.6 MB/s | 135 kB 00:00 2024-05-09T01:12:05.780 INFO:teuthology.orchestra.run.smithi196.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.7 MB/s | 162 kB 00:00 2024-05-09T01:12:05.930 INFO:teuthology.orchestra.run.smithi196.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 2.1 MB/s | 321 kB 00:00 2024-05-09T01:12:05.991 INFO:teuthology.orchestra.run.smithi019.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 4.5 MB/s | 3.5 MB 00:00 2024-05-09T01:12:06.050 INFO:teuthology.orchestra.run.smithi019.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 4.6 MB/s | 274 kB 00:00 2024-05-09T01:12:06.100 INFO:teuthology.orchestra.run.smithi019.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 989 kB/s | 49 kB 00:00 2024-05-09T01:12:06.150 INFO:teuthology.orchestra.run.smithi019.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 330 kB/s | 16 kB 00:00 2024-05-09T01:12:06.217 INFO:teuthology.orchestra.run.smithi019.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 4.0 MB/s | 273 kB 00:00 2024-05-09T01:12:06.360 INFO:teuthology.orchestra.run.smithi019.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 5.3 MB/s | 763 kB 00:00 2024-05-09T01:12:06.397 INFO:teuthology.orchestra.run.smithi196.stdout:(23/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 2.9 MB/s | 18 MB 00:06 2024-05-09T01:12:06.410 INFO:teuthology.orchestra.run.smithi019.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 366 kB/s | 18 kB 00:00 2024-05-09T01:12:06.422 INFO:teuthology.orchestra.run.smithi196.stdout:(24/156): python3-rbd-19.0.0-3594.gc9da4049.el9 611 kB/s | 301 kB 00:00 2024-05-09T01:12:06.481 INFO:teuthology.orchestra.run.smithi196.stdout:(25/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.2 MB/s | 99 kB 00:00 2024-05-09T01:12:06.560 INFO:teuthology.orchestra.run.smithi019.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 7.5 MB/s | 7.4 MB 00:00 2024-05-09T01:12:06.597 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-09T01:12:06.618 INFO:teuthology.orchestra.run.smithi019.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 385 kB/s | 80 kB 00:00 2024-05-09T01:12:06.740 INFO:teuthology.orchestra.run.smithi196.stdout:(26/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 270 kB/s | 86 kB 00:00 2024-05-09T01:12:06.807 INFO:teuthology.orchestra.run.smithi196.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 2.5 MB/s | 169 kB 00:00 2024-05-09T01:12:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 902 kB/s | 195 kB 00:00 2024-05-09T01:12:06.882 INFO:teuthology.orchestra.run.smithi196.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 457 kB/s | 34 kB 00:00 2024-05-09T01:12:06.927 INFO:teuthology.orchestra.run.smithi019.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-09T01:12:06.957 INFO:teuthology.orchestra.run.smithi196.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 1.9 MB/s | 142 kB 00:00 2024-05-09T01:12:06.961 INFO:teuthology.orchestra.run.smithi019.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-05-09T01:12:07.019 INFO:teuthology.orchestra.run.smithi019.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-09T01:12:07.078 INFO:teuthology.orchestra.run.smithi019.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-09T01:12:07.138 INFO:teuthology.orchestra.run.smithi019.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-09T01:12:07.229 INFO:teuthology.orchestra.run.smithi019.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-05-09T01:12:07.282 INFO:teuthology.orchestra.run.smithi196.stdout:(30/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 3.8 MB/s | 3.0 MB 00:00 2024-05-09T01:12:07.671 INFO:teuthology.orchestra.run.smithi019.stdout:(47/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 8.3 MB/s | 43 MB 00:05 2024-05-09T01:12:07.683 INFO:teuthology.orchestra.run.smithi196.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 4.9 MB/s | 3.5 MB 00:00 2024-05-09T01:12:07.704 INFO:teuthology.orchestra.run.smithi019.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 1.6 MB/s | 1.2 MB 00:00 2024-05-09T01:12:07.796 INFO:teuthology.orchestra.run.smithi019.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 689 kB 00:00 2024-05-09T01:12:07.808 INFO:teuthology.orchestra.run.smithi196.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 2.2 MB/s | 274 kB 00:00 2024-05-09T01:12:07.830 INFO:teuthology.orchestra.run.smithi019.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 853 kB/s | 107 kB 00:00 2024-05-09T01:12:07.855 INFO:teuthology.orchestra.run.smithi019.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-05-09T01:12:07.875 INFO:teuthology.orchestra.run.smithi196.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 737 kB/s | 49 kB 00:00 2024-05-09T01:12:07.888 INFO:teuthology.orchestra.run.smithi019.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-05-09T01:12:07.905 INFO:teuthology.orchestra.run.smithi019.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-05-09T01:12:07.931 INFO:teuthology.orchestra.run.smithi019.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-05-09T01:12:07.942 INFO:teuthology.orchestra.run.smithi196.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 247 kB/s | 16 kB 00:00 2024-05-09T01:12:07.948 INFO:teuthology.orchestra.run.smithi019.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 431 kB/s | 18 kB 00:00 2024-05-09T01:12:07.965 INFO:teuthology.orchestra.run.smithi019.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 332 kB/s | 11 kB 00:00 2024-05-09T01:12:07.999 INFO:teuthology.orchestra.run.smithi019.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-09T01:12:08.024 INFO:teuthology.orchestra.run.smithi019.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-05-09T01:12:08.042 INFO:teuthology.orchestra.run.smithi196.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 2.7 MB/s | 273 kB 00:00 2024-05-09T01:12:08.083 INFO:teuthology.orchestra.run.smithi019.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-05-09T01:12:08.116 INFO:teuthology.orchestra.run.smithi019.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-05-09T01:12:08.141 INFO:teuthology.orchestra.run.smithi019.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-05-09T01:12:08.167 INFO:teuthology.orchestra.run.smithi019.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-05-09T01:12:08.192 INFO:teuthology.orchestra.run.smithi019.stdout:(63/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-05-09T01:12:08.201 INFO:teuthology.orchestra.run.smithi196.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 4.7 MB/s | 763 kB 00:00 2024-05-09T01:12:08.217 INFO:teuthology.orchestra.run.smithi019.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-09T01:12:08.242 INFO:teuthology.orchestra.run.smithi019.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-05-09T01:12:08.276 INFO:teuthology.orchestra.run.smithi019.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-05-09T01:12:08.276 INFO:teuthology.orchestra.run.smithi196.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 243 kB/s | 18 kB 00:00 2024-05-09T01:12:08.368 INFO:teuthology.orchestra.run.smithi196.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 879 kB/s | 80 kB 00:00 2024-05-09T01:12:08.369 INFO:teuthology.orchestra.run.smithi019.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-05-09T01:12:08.410 INFO:teuthology.orchestra.run.smithi019.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-09T01:12:08.477 INFO:teuthology.orchestra.run.smithi019.stdout:(69/156): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-05-09T01:12:08.502 INFO:teuthology.orchestra.run.smithi019.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-09T01:12:08.527 INFO:teuthology.orchestra.run.smithi196.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2024-05-09T01:12:08.535 INFO:teuthology.orchestra.run.smithi019.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-05-09T01:12:08.578 INFO:teuthology.orchestra.run.smithi019.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-05-09T01:12:08.603 INFO:teuthology.orchestra.run.smithi019.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-09T01:12:08.653 INFO:teuthology.orchestra.run.smithi019.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-09T01:12:08.720 INFO:teuthology.orchestra.run.smithi019.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-05-09T01:12:08.744 INFO:teuthology.orchestra.run.smithi196.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 902 kB/s | 195 kB 00:00 2024-05-09T01:12:08.745 INFO:teuthology.orchestra.run.smithi019.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-05-09T01:12:08.770 INFO:teuthology.orchestra.run.smithi019.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-09T01:12:08.796 INFO:teuthology.orchestra.run.smithi019.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-09T01:12:08.821 INFO:teuthology.orchestra.run.smithi019.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-09T01:12:08.836 INFO:teuthology.orchestra.run.smithi196.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-09T01:12:08.846 INFO:teuthology.orchestra.run.smithi019.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-05-09T01:12:08.930 INFO:teuthology.orchestra.run.smithi019.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-09T01:12:08.972 INFO:teuthology.orchestra.run.smithi019.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-05-09T01:12:08.989 INFO:teuthology.orchestra.run.smithi019.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-09T01:12:09.014 INFO:teuthology.orchestra.run.smithi019.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-09T01:12:09.019 INFO:teuthology.orchestra.run.smithi196.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-05-09T01:12:09.040 INFO:teuthology.orchestra.run.smithi019.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 837 kB/s | 42 kB 00:00 2024-05-09T01:12:09.065 INFO:teuthology.orchestra.run.smithi019.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-09T01:12:09.090 INFO:teuthology.orchestra.run.smithi019.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-09T01:12:09.103 INFO:teuthology.orchestra.run.smithi196.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-05-09T01:12:09.116 INFO:teuthology.orchestra.run.smithi019.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-09T01:12:09.133 INFO:teuthology.orchestra.run.smithi019.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-05-09T01:12:09.149 INFO:teuthology.orchestra.run.smithi019.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-05-09T01:12:09.187 INFO:teuthology.orchestra.run.smithi196.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-09T01:12:09.217 INFO:teuthology.orchestra.run.smithi019.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-09T01:12:09.233 INFO:teuthology.orchestra.run.smithi019.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-05-09T01:12:09.250 INFO:teuthology.orchestra.run.smithi019.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-05-09T01:12:09.270 INFO:teuthology.orchestra.run.smithi196.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-09T01:12:09.284 INFO:teuthology.orchestra.run.smithi019.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-09T01:12:09.309 INFO:teuthology.orchestra.run.smithi019.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-05-09T01:12:09.334 INFO:teuthology.orchestra.run.smithi019.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-09T01:12:09.360 INFO:teuthology.orchestra.run.smithi019.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-09T01:12:09.362 INFO:teuthology.orchestra.run.smithi196.stdout:(46/156): ceph-mgr-diskprediction-local-19.0.0- 3.6 MB/s | 7.4 MB 00:02 2024-05-09T01:12:09.385 INFO:teuthology.orchestra.run.smithi019.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-09T01:12:09.427 INFO:teuthology.orchestra.run.smithi019.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-09T01:12:09.437 INFO:teuthology.orchestra.run.smithi196.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-05-09T01:12:09.461 INFO:teuthology.orchestra.run.smithi019.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 934 kB/s | 94 kB 00:00 2024-05-09T01:12:09.486 INFO:teuthology.orchestra.run.smithi019.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-05-09T01:12:09.695 INFO:teuthology.orchestra.run.smithi019.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-05-09T01:12:09.746 INFO:teuthology.orchestra.run.smithi196.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-05-09T01:12:09.805 INFO:teuthology.orchestra.run.smithi196.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-05-09T01:12:09.830 INFO:teuthology.orchestra.run.smithi196.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-05-09T01:12:09.872 INFO:teuthology.orchestra.run.smithi196.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 323 kB/s | 21 kB 00:00 2024-05-09T01:12:09.886 INFO:teuthology.orchestra.run.smithi019.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 52 kB/s | 22 kB 00:00 2024-05-09T01:12:09.906 INFO:teuthology.orchestra.run.smithi196.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-09T01:12:09.931 INFO:teuthology.orchestra.run.smithi196.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 556 kB/s | 33 kB 00:00 2024-05-09T01:12:09.995 INFO:teuthology.orchestra.run.smithi019.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-05-09T01:12:10.006 INFO:teuthology.orchestra.run.smithi196.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 243 kB/s | 18 kB 00:00 2024-05-09T01:12:10.054 INFO:teuthology.orchestra.run.smithi019.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-05-09T01:12:10.073 INFO:teuthology.orchestra.run.smithi196.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-05-09T01:12:10.123 INFO:teuthology.orchestra.run.smithi196.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-05-09T01:12:10.149 INFO:teuthology.orchestra.run.smithi196.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-09T01:12:10.154 INFO:teuthology.orchestra.run.smithi019.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-05-09T01:12:10.191 INFO:teuthology.orchestra.run.smithi196.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-05-09T01:12:10.221 INFO:teuthology.orchestra.run.smithi019.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-05-09T01:12:10.232 INFO:teuthology.orchestra.run.smithi196.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-05-09T01:12:10.274 INFO:teuthology.orchestra.run.smithi196.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-05-09T01:12:10.288 INFO:teuthology.orchestra.run.smithi019.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-05-09T01:12:10.300 INFO:teuthology.orchestra.run.smithi196.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 379 kB/s | 25 kB 00:00 2024-05-09T01:12:10.313 INFO:teuthology.orchestra.run.smithi019.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-05-09T01:12:10.325 INFO:teuthology.orchestra.run.smithi196.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-05-09T01:12:10.346 INFO:teuthology.orchestra.run.smithi019.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-09T01:12:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-09T01:12:10.389 INFO:teuthology.orchestra.run.smithi019.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-09T01:12:10.401 INFO:teuthology.orchestra.run.smithi196.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-09T01:12:10.443 INFO:teuthology.orchestra.run.smithi196.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 469 kB/s | 35 kB 00:00 2024-05-09T01:12:10.473 INFO:teuthology.orchestra.run.smithi019.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-05-09T01:12:10.523 INFO:teuthology.orchestra.run.smithi019.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 833 kB/s | 863 kB 00:01 2024-05-09T01:12:10.557 INFO:teuthology.orchestra.run.smithi019.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-05-09T01:12:10.573 INFO:teuthology.orchestra.run.smithi019.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-05-09T01:12:10.593 INFO:teuthology.orchestra.run.smithi196.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-05-09T01:12:10.624 INFO:teuthology.orchestra.run.smithi019.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 302 kB/s | 15 kB 00:00 2024-05-09T01:12:10.649 INFO:teuthology.orchestra.run.smithi019.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-09T01:12:10.682 INFO:teuthology.orchestra.run.smithi019.stdout:(118/156): python3-cachetools-4.2.4-1.el9.noarc 202 kB/s | 32 kB 00:00 2024-05-09T01:12:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-09T01:12:10.708 INFO:teuthology.orchestra.run.smithi019.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-09T01:12:10.733 INFO:teuthology.orchestra.run.smithi019.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-05-09T01:12:10.766 INFO:teuthology.orchestra.run.smithi019.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-09T01:12:10.783 INFO:teuthology.orchestra.run.smithi019.stdout:(122/156): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-05-09T01:12:10.794 INFO:teuthology.orchestra.run.smithi196.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-09T01:12:10.826 INFO:teuthology.orchestra.run.smithi019.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-09T01:12:10.835 INFO:teuthology.orchestra.run.smithi196.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 5.9 MB/s | 911 kB 00:00 2024-05-09T01:12:10.842 INFO:teuthology.orchestra.run.smithi019.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-05-09T01:12:10.861 INFO:teuthology.orchestra.run.smithi196.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-05-09T01:12:10.893 INFO:teuthology.orchestra.run.smithi019.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-05-09T01:12:10.918 INFO:teuthology.orchestra.run.smithi019.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-09T01:12:10.943 INFO:teuthology.orchestra.run.smithi019.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-09T01:12:10.969 INFO:teuthology.orchestra.run.smithi196.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-09T01:12:10.977 INFO:teuthology.orchestra.run.smithi019.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-05-09T01:12:11.002 INFO:teuthology.orchestra.run.smithi019.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-05-09T01:12:11.036 INFO:teuthology.orchestra.run.smithi019.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 791 kB/s | 46 kB 00:00 2024-05-09T01:12:11.045 INFO:teuthology.orchestra.run.smithi196.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-05-09T01:12:11.061 INFO:teuthology.orchestra.run.smithi019.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-09T01:12:11.070 INFO:teuthology.orchestra.run.smithi196.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-05-09T01:12:11.095 INFO:teuthology.orchestra.run.smithi019.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-05-09T01:12:11.159 INFO:teuthology.orchestra.run.smithi019.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-09T01:12:11.170 INFO:teuthology.orchestra.run.smithi196.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-09T01:12:11.187 INFO:teuthology.orchestra.run.smithi019.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-05-09T01:12:11.229 INFO:teuthology.orchestra.run.smithi019.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-09T01:12:11.254 INFO:teuthology.orchestra.run.smithi019.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-05-09T01:12:11.262 INFO:teuthology.orchestra.run.smithi196.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-05-09T01:12:11.287 INFO:teuthology.orchestra.run.smithi196.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-05-09T01:12:11.288 INFO:teuthology.orchestra.run.smithi019.stdout:(137/156): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-05-09T01:12:11.313 INFO:teuthology.orchestra.run.smithi019.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-05-09T01:12:11.329 INFO:teuthology.orchestra.run.smithi196.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-05-09T01:12:11.338 INFO:teuthology.orchestra.run.smithi019.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-09T01:12:11.354 INFO:teuthology.orchestra.run.smithi196.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-05-09T01:12:11.363 INFO:teuthology.orchestra.run.smithi019.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-05-09T01:12:11.389 INFO:teuthology.orchestra.run.smithi019.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-05-09T01:12:11.396 INFO:teuthology.orchestra.run.smithi196.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-05-09T01:12:11.414 INFO:teuthology.orchestra.run.smithi019.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-09T01:12:11.422 INFO:teuthology.orchestra.run.smithi196.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 517 kB/s | 35 kB 00:00 2024-05-09T01:12:11.439 INFO:teuthology.orchestra.run.smithi019.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-09T01:12:11.465 INFO:teuthology.orchestra.run.smithi019.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-09T01:12:11.498 INFO:teuthology.orchestra.run.smithi019.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-09T01:12:11.514 INFO:teuthology.orchestra.run.smithi196.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-09T01:12:11.524 INFO:teuthology.orchestra.run.smithi019.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-09T01:12:11.549 INFO:teuthology.orchestra.run.smithi019.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-09T01:12:11.589 INFO:teuthology.orchestra.run.smithi196.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-05-09T01:12:11.608 INFO:teuthology.orchestra.run.smithi019.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-09T01:12:11.614 INFO:teuthology.orchestra.run.smithi196.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-09T01:12:11.656 INFO:teuthology.orchestra.run.smithi196.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-05-09T01:12:11.682 INFO:teuthology.orchestra.run.smithi196.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 629 kB/s | 42 kB 00:00 2024-05-09T01:12:11.723 INFO:teuthology.orchestra.run.smithi196.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-05-09T01:12:11.748 INFO:teuthology.orchestra.run.smithi196.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-09T01:12:11.791 INFO:teuthology.orchestra.run.smithi196.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-09T01:12:11.816 INFO:teuthology.orchestra.run.smithi196.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-05-09T01:12:11.858 INFO:teuthology.orchestra.run.smithi196.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 799 kB/s | 54 kB 00:00 2024-05-09T01:12:11.891 INFO:teuthology.orchestra.run.smithi019.stdout:(149/156): librados2-19.0.0-3594.gc9da4049.el9. 9.1 MB/s | 3.3 MB 00:00 2024-05-09T01:12:11.906 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-09T01:12:11.925 INFO:teuthology.orchestra.run.smithi196.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-05-09T01:12:11.992 INFO:teuthology.orchestra.run.smithi196.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-05-09T01:12:12.067 INFO:teuthology.orchestra.run.smithi196.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-05-09T01:12:12.083 INFO:teuthology.orchestra.run.smithi019.stdout:(150/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 5.8 MB/s | 3.1 MB 00:00 2024-05-09T01:12:12.142 INFO:teuthology.orchestra.run.smithi196.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 479 kB/s | 36 kB 00:00 2024-05-09T01:12:12.201 INFO:teuthology.orchestra.run.smithi196.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-05-09T01:12:12.225 INFO:teuthology.orchestra.run.smithi019.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 295 kB/s | 42 kB 00:00 2024-05-09T01:12:12.277 INFO:teuthology.orchestra.run.smithi196.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-05-09T01:12:12.308 INFO:teuthology.orchestra.run.smithi019.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 3.3 MB/s | 2.3 MB 00:00 2024-05-09T01:12:12.343 INFO:teuthology.orchestra.run.smithi196.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 869 kB/s | 58 kB 00:00 2024-05-09T01:12:12.359 INFO:teuthology.orchestra.run.smithi019.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.8 MB/s | 248 kB 00:00 2024-05-09T01:12:12.410 INFO:teuthology.orchestra.run.smithi196.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-09T01:12:12.469 INFO:teuthology.orchestra.run.smithi196.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-09T01:12:12.509 INFO:teuthology.orchestra.run.smithi019.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 3.0 MB/s | 1.8 MB 00:00 2024-05-09T01:12:12.527 INFO:teuthology.orchestra.run.smithi196.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-05-09T01:12:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 2.4 MB 00:00 2024-05-09T01:12:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 1.3 MB 00:00 2024-05-09T01:12:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:12:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:Total 15 MB/s | 219 MB 00:14 2024-05-09T01:12:12.936 INFO:teuthology.orchestra.run.smithi196.stdout:(101/156): ceph-test-19.0.0-3594.gc9da4049.el9. 4.2 MB/s | 43 MB 00:10 2024-05-09T01:12:12.953 INFO:teuthology.orchestra.run.smithi196.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 52 kB/s | 22 kB 00:00 2024-05-09T01:12:13.112 INFO:teuthology.orchestra.run.smithi196.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-05-09T01:12:13.287 INFO:teuthology.orchestra.run.smithi196.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-05-09T01:12:13.529 INFO:teuthology.orchestra.run.smithi196.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2024-05-09T01:12:13.563 INFO:teuthology.orchestra.run.smithi196.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 900 kB/s | 548 kB 00:00 2024-05-09T01:12:13.588 INFO:teuthology.orchestra.run.smithi196.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 688 kB/s | 309 kB 00:00 2024-05-09T01:12:13.663 INFO:teuthology.orchestra.run.smithi196.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-05-09T01:12:13.688 INFO:teuthology.orchestra.run.smithi196.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-05-09T01:12:13.780 INFO:teuthology.orchestra.run.smithi196.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-09T01:12:13.806 INFO:teuthology.orchestra.run.smithi196.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 474 kB/s | 67 kB 00:00 2024-05-09T01:12:13.873 INFO:teuthology.orchestra.run.smithi196.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-09T01:12:13.973 INFO:teuthology.orchestra.run.smithi196.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-05-09T01:12:13.998 INFO:teuthology.orchestra.run.smithi196.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-05-09T01:12:14.023 INFO:teuthology.orchestra.run.smithi196.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-05-09T01:12:14.024 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-05-09T01:12:14.048 INFO:teuthology.orchestra.run.smithi196.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 427 kB/s | 32 kB 00:00 2024-05-09T01:12:14.074 INFO:teuthology.orchestra.run.smithi196.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 198 kB/s | 15 kB 00:00 2024-05-09T01:12:14.116 INFO:teuthology.orchestra.run.smithi196.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-09T01:12:14.141 INFO:teuthology.orchestra.run.smithi196.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-05-09T01:12:14.166 INFO:teuthology.orchestra.run.smithi196.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-09T01:12:14.190 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-05-09T01:12:14.190 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-05-09T01:12:14.192 INFO:teuthology.orchestra.run.smithi196.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 466 kB/s | 35 kB 00:00 2024-05-09T01:12:14.217 INFO:teuthology.orchestra.run.smithi196.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T01:12:14.267 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:12:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:12:14.275 INFO:teuthology.orchestra.run.smithi196.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-09T01:12:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T01:12:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T01:12:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T01:12:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T01:12:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T01:12:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T01:12:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T01:12:14.301 INFO:teuthology.orchestra.run.smithi196.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-05-09T01:12:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-09T01:12:14.351 INFO:teuthology.orchestra.run.smithi196.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-09T01:12:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-09T01:12:14.371 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-09T01:12:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-09T01:12:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-09T01:12:14.371 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T01:12:14.371 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T01:12:14.371 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T01:12:14.371 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T01:12:14.372 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T01:12:14.373 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T01:12:14.374 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T01:12:14.375 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T01:12:14.376 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T01:12:14.377 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T01:12:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T01:12:14.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T01:12:14.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T01:12:14.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T01:12:14.382 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:12:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-09T01:12:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-09T01:12:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:Install 148 Packages 2024-05-09T01:12:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 8 Packages 2024-05-09T01:12:14.384 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:12:14.384 INFO:teuthology.orchestra.run.smithi196.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-09T01:12:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 219 M 2024-05-09T01:12:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-09T01:12:14.401 INFO:teuthology.orchestra.run.smithi196.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-09T01:12:14.427 INFO:teuthology.orchestra.run.smithi196.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-09T01:12:14.469 INFO:teuthology.orchestra.run.smithi196.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 295 kB/s | 20 kB 00:00 2024-05-09T01:12:14.503 INFO:teuthology.orchestra.run.smithi196.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-05-09T01:12:14.545 INFO:teuthology.orchestra.run.smithi196.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-05-09T01:12:14.595 INFO:teuthology.orchestra.run.smithi196.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 628 kB/s | 79 kB 00:00 2024-05-09T01:12:14.620 INFO:teuthology.orchestra.run.smithi196.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 493 kB/s | 58 kB 00:00 2024-05-09T01:12:14.645 INFO:teuthology.orchestra.run.smithi196.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-09T01:12:14.671 INFO:teuthology.orchestra.run.smithi196.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-05-09T01:12:14.696 INFO:teuthology.orchestra.run.smithi196.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-09T01:12:14.738 INFO:teuthology.orchestra.run.smithi196.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-05-09T01:12:14.763 INFO:teuthology.orchestra.run.smithi196.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-05-09T01:12:14.788 INFO:teuthology.orchestra.run.smithi196.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-09T01:12:14.814 INFO:teuthology.orchestra.run.smithi196.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-05-09T01:12:14.839 INFO:teuthology.orchestra.run.smithi196.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-09T01:12:14.873 INFO:teuthology.orchestra.run.smithi196.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-05-09T01:12:14.897 INFO:teuthology.orchestra.run.smithi196.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-09T01:12:14.923 INFO:teuthology.orchestra.run.smithi196.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-09T01:12:14.948 INFO:teuthology.orchestra.run.smithi196.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-05-09T01:12:14.965 INFO:teuthology.orchestra.run.smithi196.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-05-09T01:12:15.015 INFO:teuthology.orchestra.run.smithi196.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-09T01:12:15.057 INFO:teuthology.orchestra.run.smithi059.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-05-09T01:12:15.074 INFO:teuthology.orchestra.run.smithi196.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-09T01:12:15.258 INFO:teuthology.orchestra.run.smithi059.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 4.2 MB/s | 863 kB 00:00 2024-05-09T01:12:15.308 INFO:teuthology.orchestra.run.smithi059.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 2.9 MB/s | 147 kB 00:00 2024-05-09T01:12:15.466 INFO:teuthology.orchestra.run.smithi196.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 5.9 MB/s | 2.3 MB 00:00 2024-05-09T01:12:15.583 INFO:teuthology.orchestra.run.smithi059.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 8.2 MB/s | 2.3 MB 00:00 2024-05-09T01:12:15.650 INFO:teuthology.orchestra.run.smithi196.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 10 MB/s | 1.8 MB 00:00 2024-05-09T01:12:15.716 INFO:teuthology.orchestra.run.smithi196.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 630 kB/s | 42 kB 00:00 2024-05-09T01:12:15.767 INFO:teuthology.orchestra.run.smithi059.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-05-09T01:12:15.908 INFO:teuthology.orchestra.run.smithi196.stdout:(152/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 3.5 MB/s | 3.1 MB 00:00 2024-05-09T01:12:15.967 INFO:teuthology.orchestra.run.smithi196.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 993 kB/s | 248 kB 00:00 2024-05-09T01:12:16.009 INFO:teuthology.orchestra.run.smithi059.stdout:(6/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 4.8 MB/s | 5.4 MB 00:01 2024-05-09T01:12:16.242 INFO:teuthology.orchestra.run.smithi059.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 10 MB/s | 4.9 MB 00:00 2024-05-09T01:12:16.351 INFO:teuthology.orchestra.run.smithi196.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-05-09T01:12:16.434 INFO:teuthology.orchestra.run.smithi196.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.1 MB/s | 2.4 MB 00:00 2024-05-09T01:12:16.450 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-05-09T01:12:16.450 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-05-09T01:12:16.918 INFO:teuthology.orchestra.run.smithi196.stdout:(156/156): librados2-19.0.0-3594.gc9da4049.el9. 1.7 MB/s | 3.3 MB 00:01 2024-05-09T01:12:16.923 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:12:16.924 INFO:teuthology.orchestra.run.smithi196.stdout:Total 11 MB/s | 219 MB 00:20 2024-05-09T01:12:17.417 INFO:teuthology.orchestra.run.smithi059.stdout:(8/156): ceph-common-19.0.0-3594.gc9da4049.el9. 7.7 MB/s | 20 MB 00:02 2024-05-09T01:12:17.518 INFO:teuthology.orchestra.run.smithi059.stdout:(9/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 7.9 MB/s | 10 MB 00:01 2024-05-09T01:12:17.534 INFO:teuthology.orchestra.run.smithi059.stdout:(10/156): ceph-selinux-19.0.0-3594.gc9da4049.el 215 kB/s | 25 kB 00:00 2024-05-09T01:12:17.610 INFO:teuthology.orchestra.run.smithi059.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 2.9 MB/s | 224 kB 00:00 2024-05-09T01:12:17.677 INFO:teuthology.orchestra.run.smithi059.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 406 kB/s | 27 kB 00:00 2024-05-09T01:12:17.752 INFO:teuthology.orchestra.run.smithi059.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 423 kB/s | 32 kB 00:00 2024-05-09T01:12:17.861 INFO:teuthology.orchestra.run.smithi059.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 6.8 MB/s | 756 kB 00:00 2024-05-09T01:12:17.953 INFO:teuthology.orchestra.run.smithi059.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.7 MB/s | 158 kB 00:00 2024-05-09T01:12:18.036 INFO:teuthology.orchestra.run.smithi059.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.5 MB/s | 127 kB 00:00 2024-05-09T01:12:18.051 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-09T01:12:18.202 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-09T01:12:18.203 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-09T01:12:18.237 INFO:teuthology.orchestra.run.smithi059.stdout:(17/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 8.0 MB/s | 18 MB 00:02 2024-05-09T01:12:18.270 INFO:teuthology.orchestra.run.smithi059.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 2.1 MB/s | 497 kB 00:00 2024-05-09T01:12:18.362 INFO:teuthology.orchestra.run.smithi059.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 494 kB/s | 45 kB 00:00 2024-05-09T01:12:18.454 INFO:teuthology.orchestra.run.smithi059.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.4 MB/s | 135 kB 00:00 2024-05-09T01:12:18.538 INFO:teuthology.orchestra.run.smithi059.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.9 MB/s | 162 kB 00:00 2024-05-09T01:12:18.638 INFO:teuthology.orchestra.run.smithi059.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 3.1 MB/s | 321 kB 00:00 2024-05-09T01:12:18.738 INFO:teuthology.orchestra.run.smithi059.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 2.9 MB/s | 301 kB 00:00 2024-05-09T01:12:18.814 INFO:teuthology.orchestra.run.smithi059.stdout:(24/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 8.9 MB/s | 5.2 MB 00:00 2024-05-09T01:12:18.839 INFO:teuthology.orchestra.run.smithi059.stdout:(25/156): python3-rgw-19.0.0-3594.gc9da4049.el9 987 kB/s | 99 kB 00:00 2024-05-09T01:12:18.889 INFO:teuthology.orchestra.run.smithi059.stdout:(26/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-09T01:12:18.990 INFO:teuthology.orchestra.run.smithi059.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 1.6 MB/s | 169 kB 00:00 2024-05-09T01:12:19.098 INFO:teuthology.orchestra.run.smithi059.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 316 kB/s | 34 kB 00:00 2024-05-09T01:12:19.157 INFO:teuthology.orchestra.run.smithi059.stdout:(29/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 9.6 MB/s | 3.0 MB 00:00 2024-05-09T01:12:19.182 INFO:teuthology.orchestra.run.smithi059.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 1.7 MB/s | 142 kB 00:00 2024-05-09T01:12:19.950 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-09T01:12:19.950 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-09T01:12:19.965 INFO:teuthology.orchestra.run.smithi059.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-05-09T01:12:20.033 INFO:teuthology.orchestra.run.smithi059.stdout:(32/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 4.0 MB/s | 3.5 MB 00:00 2024-05-09T01:12:20.058 INFO:teuthology.orchestra.run.smithi059.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3594.gc9 2.9 MB/s | 274 kB 00:00 2024-05-09T01:12:20.100 INFO:teuthology.orchestra.run.smithi059.stdout:(34/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 738 kB/s | 49 kB 00:00 2024-05-09T01:12:20.219 INFO:teuthology.orchestra.run.smithi059.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3594.gc 102 kB/s | 16 kB 00:00 2024-05-09T01:12:20.250 INFO:teuthology.orchestra.run.smithi059.stdout:(36/156): ceph-volume-19.0.0-3594.gc9da4049.el9 1.8 MB/s | 273 kB 00:00 2024-05-09T01:12:20.367 INFO:teuthology.orchestra.run.smithi059.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 156 kB/s | 18 kB 00:00 2024-05-09T01:12:20.401 INFO:teuthology.orchestra.run.smithi059.stdout:(38/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 4.1 MB/s | 763 kB 00:00 2024-05-09T01:12:20.818 INFO:teuthology.orchestra.run.smithi059.stdout:(39/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 13 MB/s | 43 MB 00:03 2024-05-09T01:12:20.985 INFO:teuthology.orchestra.run.smithi059.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 130 kB/s | 80 kB 00:00 2024-05-09T01:12:21.152 INFO:teuthology.orchestra.run.smithi059.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 586 kB/s | 195 kB 00:00 2024-05-09T01:12:21.177 INFO:teuthology.orchestra.run.smithi059.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-05-09T01:12:21.269 INFO:teuthology.orchestra.run.smithi059.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-05-09T01:12:21.328 INFO:teuthology.orchestra.run.smithi059.stdout:(44/156): libgfortran-11.4.1-3.el9.x86_64.rpm 867 kB/s | 803 kB 00:00 2024-05-09T01:12:21.353 INFO:teuthology.orchestra.run.smithi059.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-09T01:12:21.395 INFO:teuthology.orchestra.run.smithi059.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-09T01:12:21.462 INFO:teuthology.orchestra.run.smithi059.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-09T01:12:21.513 INFO:teuthology.orchestra.run.smithi059.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-05-09T01:12:21.596 INFO:teuthology.orchestra.run.smithi059.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-05-09T01:12:21.638 INFO:teuthology.orchestra.run.smithi059.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 519 kB/s | 21 kB 00:00 2024-05-09T01:12:21.663 INFO:teuthology.orchestra.run.smithi059.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 711 kB/s | 107 kB 00:00 2024-05-09T01:12:21.689 INFO:teuthology.orchestra.run.smithi059.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-09T01:12:21.714 INFO:teuthology.orchestra.run.smithi059.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 647 kB/s | 33 kB 00:00 2024-05-09T01:12:21.747 INFO:teuthology.orchestra.run.smithi059.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-05-09T01:12:21.773 INFO:teuthology.orchestra.run.smithi059.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-05-09T01:12:21.798 INFO:teuthology.orchestra.run.smithi059.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-09T01:12:21.823 INFO:teuthology.orchestra.run.smithi059.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-09T01:12:21.849 INFO:teuthology.orchestra.run.smithi059.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-09T01:12:21.891 INFO:teuthology.orchestra.run.smithi059.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-09T01:12:21.941 INFO:teuthology.orchestra.run.smithi059.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-09T01:12:21.966 INFO:teuthology.orchestra.run.smithi059.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-05-09T01:12:21.991 INFO:teuthology.orchestra.run.smithi059.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-09T01:12:22.025 INFO:teuthology.orchestra.run.smithi059.stdout:(63/156): libtool-2.4.6-45.el9.x86_64.rpm 3.2 MB/s | 582 kB 00:00 2024-05-09T01:12:22.059 INFO:teuthology.orchestra.run.smithi059.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-09T01:12:22.084 INFO:teuthology.orchestra.run.smithi059.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-09T01:12:22.184 INFO:teuthology.orchestra.run.smithi059.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-05-09T01:12:22.209 INFO:teuthology.orchestra.run.smithi059.stdout:(67/156): openblas-0.3.21-2.el9.x86_64.rpm 191 kB/s | 35 kB 00:00 2024-05-09T01:12:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-05-09T01:12:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-09T01:12:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-09T01:12:22.394 INFO:teuthology.orchestra.run.smithi059.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 4.8 MB/s | 911 kB 00:00 2024-05-09T01:12:22.436 INFO:teuthology.orchestra.run.smithi059.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-05-09T01:12:22.461 INFO:teuthology.orchestra.run.smithi059.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-09T01:12:22.512 INFO:teuthology.orchestra.run.smithi059.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-09T01:12:22.587 INFO:teuthology.orchestra.run.smithi059.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-05-09T01:12:22.604 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-05-09T01:12:22.662 INFO:teuthology.orchestra.run.smithi059.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-05-09T01:12:22.679 INFO:teuthology.orchestra.run.smithi059.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 279 kB/s | 26 kB 00:00 2024-05-09T01:12:22.681 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T01:12:22.704 INFO:teuthology.orchestra.run.smithi059.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-05-09T01:12:22.720 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T01:12:22.729 INFO:teuthology.orchestra.run.smithi059.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-09T01:12:22.755 INFO:teuthology.orchestra.run.smithi059.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-05-09T01:12:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T01:12:22.805 INFO:teuthology.orchestra.run.smithi059.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-09T01:12:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T01:12:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T01:12:22.889 INFO:teuthology.orchestra.run.smithi059.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-05-09T01:12:22.914 INFO:teuthology.orchestra.run.smithi059.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-09T01:12:22.931 INFO:teuthology.orchestra.run.smithi059.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-09T01:12:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T01:12:22.956 INFO:teuthology.orchestra.run.smithi059.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-09T01:12:22.981 INFO:teuthology.orchestra.run.smithi059.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-09T01:12:23.007 INFO:teuthology.orchestra.run.smithi059.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-09T01:12:23.032 INFO:teuthology.orchestra.run.smithi059.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-09T01:12:23.057 INFO:teuthology.orchestra.run.smithi059.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-09T01:12:23.083 INFO:teuthology.orchestra.run.smithi059.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-09T01:12:23.124 INFO:teuthology.orchestra.run.smithi059.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-09T01:12:23.166 INFO:teuthology.orchestra.run.smithi059.stdout:(92/156): python3-jinja2-2.11.3-5.el9.noarch.rp 379 kB/s | 248 kB 00:00 2024-05-09T01:12:23.184 INFO:teuthology.orchestra.run.smithi059.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 753 kB/s | 44 kB 00:00 2024-05-09T01:12:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-05-09T01:12:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-09T01:12:23.302 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T01:12:23.418 INFO:teuthology.orchestra.run.smithi059.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-05-09T01:12:23.443 INFO:teuthology.orchestra.run.smithi059.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 799 kB/s | 221 kB 00:00 2024-05-09T01:12:23.460 INFO:teuthology.orchestra.run.smithi059.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-05-09T01:12:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:12:23.485 INFO:teuthology.orchestra.run.smithi059.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-05-09T01:12:23.510 INFO:teuthology.orchestra.run.smithi059.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T01:12:23.536 INFO:teuthology.orchestra.run.smithi059.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-05-09T01:12:23.561 INFO:teuthology.orchestra.run.smithi059.stdout:(102/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 797 kB/s | 94 kB 00:00 2024-05-09T01:12:23.603 INFO:teuthology.orchestra.run.smithi059.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 241 kB/s | 22 kB 00:00 2024-05-09T01:12:23.611 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:12:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:12:23.662 INFO:teuthology.orchestra.run.smithi059.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-05-09T01:12:23.735 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:12:23.764 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:12:23.904 INFO:teuthology.orchestra.run.smithi059.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-05-09T01:12:23.944 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T01:12:24.002 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T01:12:24.029 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:12:24.137 INFO:teuthology.orchestra.run.smithi059.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 578 kB/s | 309 kB 00:00 2024-05-09T01:12:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:12:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:12:24.237 INFO:teuthology.orchestra.run.smithi059.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-05-09T01:12:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:12:24.262 INFO:teuthology.orchestra.run.smithi059.stdout:(108/156): abseil-cpp-20211102.0-3.el9.x86_64.r 781 kB/s | 548 kB 00:00 2024-05-09T01:12:24.273 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:12:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:12:24.329 INFO:teuthology.orchestra.run.smithi059.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-05-09T01:12:24.355 INFO:teuthology.orchestra.run.smithi059.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-05-09T01:12:24.489 INFO:teuthology.orchestra.run.smithi059.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 951 kB/s | 151 kB 00:00 2024-05-09T01:12:24.492 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:12:24.589 INFO:teuthology.orchestra.run.smithi059.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-05-09T01:12:24.649 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:12:24.689 INFO:teuthology.orchestra.run.smithi059.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-05-09T01:12:24.790 INFO:teuthology.orchestra.run.smithi059.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-05-09T01:12:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:12:24.815 INFO:teuthology.orchestra.run.smithi059.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-05-09T01:12:24.856 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:12:24.898 INFO:teuthology.orchestra.run.smithi059.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-05-09T01:12:24.924 INFO:teuthology.orchestra.run.smithi059.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.4 MB/s | 839 kB 00:00 2024-05-09T01:12:24.974 INFO:teuthology.orchestra.run.smithi059.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-05-09T01:12:24.984 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:12:24.999 INFO:teuthology.orchestra.run.smithi059.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-09T01:12:25.065 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:12:25.066 INFO:teuthology.orchestra.run.smithi059.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-05-09T01:12:25.100 INFO:teuthology.orchestra.run.smithi059.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-09T01:12:25.125 INFO:teuthology.orchestra.run.smithi059.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-05-09T01:12:25.200 INFO:teuthology.orchestra.run.smithi059.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-09T01:12:25.226 INFO:teuthology.orchestra.run.smithi059.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-05-09T01:12:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:12:25.301 INFO:teuthology.orchestra.run.smithi059.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-09T01:12:25.326 INFO:teuthology.orchestra.run.smithi059.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-09T01:12:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T01:12:25.401 INFO:teuthology.orchestra.run.smithi059.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-05-09T01:12:25.427 INFO:teuthology.orchestra.run.smithi059.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-09T01:12:25.433 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T01:12:25.474 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T01:12:25.527 INFO:teuthology.orchestra.run.smithi059.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 464 kB/s | 46 kB 00:00 2024-05-09T01:12:25.548 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T01:12:25.588 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T01:12:25.631 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T01:12:25.635 INFO:teuthology.orchestra.run.smithi059.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 728 kB/s | 79 kB 00:00 2024-05-09T01:12:25.664 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T01:12:25.691 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T01:12:25.753 INFO:teuthology.orchestra.run.smithi059.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 494 kB/s | 58 kB 00:00 2024-05-09T01:12:25.762 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T01:12:25.823 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T01:12:25.852 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T01:12:25.878 INFO:teuthology.orchestra.run.smithi059.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-05-09T01:12:25.896 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T01:12:25.903 INFO:teuthology.orchestra.run.smithi059.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-05-09T01:12:25.945 INFO:teuthology.orchestra.run.smithi059.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 1.9 MB/s | 1.0 MB 00:00 2024-05-09T01:12:25.975 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T01:12:25.987 INFO:teuthology.orchestra.run.smithi059.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-05-09T01:12:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T01:12:26.013 INFO:teuthology.orchestra.run.smithi059.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 829 kB/s | 90 kB 00:00 2024-05-09T01:12:26.037 INFO:teuthology.orchestra.run.smithi059.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-05-09T01:12:26.051 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T01:12:26.079 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T01:12:26.079 INFO:teuthology.orchestra.run.smithi059.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-05-09T01:12:26.121 INFO:teuthology.orchestra.run.smithi059.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-09T01:12:26.146 INFO:teuthology.orchestra.run.smithi059.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-05-09T01:12:26.153 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T01:12:26.158 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-09T01:12:26.172 INFO:teuthology.orchestra.run.smithi059.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 387 kB/s | 36 kB 00:00 2024-05-09T01:12:26.211 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T01:12:26.214 INFO:teuthology.orchestra.run.smithi059.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 552 kB/s | 51 kB 00:00 2024-05-09T01:12:26.238 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T01:12:26.264 INFO:teuthology.orchestra.run.smithi059.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 977 kB/s | 90 kB 00:00 2024-05-09T01:12:26.276 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T01:12:26.288 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T01:12:26.315 INFO:teuthology.orchestra.run.smithi059.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-09T01:12:26.329 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T01:12:26.340 INFO:teuthology.orchestra.run.smithi059.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-05-09T01:12:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T01:12:26.365 INFO:teuthology.orchestra.run.smithi059.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-05-09T01:12:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T01:12:26.424 INFO:teuthology.orchestra.run.smithi059.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-09T01:12:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T01:12:26.433 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T01:12:26.489 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T01:12:26.649 INFO:teuthology.orchestra.run.smithi059.stdout:(148/156): librados2-19.0.0-3594.gc9da4049.el9. 12 MB/s | 3.3 MB 00:00 2024-05-09T01:12:26.759 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T01:12:26.865 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:12:26.874 INFO:teuthology.orchestra.run.smithi059.stdout:(149/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 6.9 MB/s | 3.1 MB 00:00 2024-05-09T01:12:26.990 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:12:27.025 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:12:27.045 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T01:12:27.058 INFO:teuthology.orchestra.run.smithi059.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 5.7 MB/s | 2.3 MB 00:00 2024-05-09T01:12:27.116 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:12:27.117 INFO:teuthology.orchestra.run.smithi059.stdout:(151/156): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-05-09T01:12:27.136 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T01:12:27.137 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:12:27.142 INFO:teuthology.orchestra.run.smithi059.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 500 kB/s | 42 kB 00:00 2024-05-09T01:12:27.170 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T01:12:27.224 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T01:12:27.233 INFO:teuthology.orchestra.run.smithi059.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 5.1 MB/s | 1.8 MB 00:00 2024-05-09T01:12:27.259 INFO:teuthology.orchestra.run.smithi059.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.7 MB/s | 248 kB 00:00 2024-05-09T01:12:27.260 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T01:12:27.283 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T01:12:27.303 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T01:12:27.342 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T01:12:27.359 INFO:teuthology.orchestra.run.smithi059.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 1.3 MB 00:00 2024-05-09T01:12:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:12:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T01:12:27.475 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:12:27.505 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:12:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T01:12:27.567 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:12:27.589 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:12:27.610 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:12:27.617 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T01:12:27.626 INFO:teuthology.orchestra.run.smithi059.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.1 MB/s | 2.4 MB 00:00 2024-05-09T01:12:27.628 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:12:27.629 INFO:teuthology.orchestra.run.smithi059.stdout:Total 17 MB/s | 219 MB 00:13 2024-05-09T01:12:27.678 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T01:12:27.752 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T01:12:27.789 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T01:12:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:12:27.825 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T01:12:27.933 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:12:28.050 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T01:12:28.095 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:12:28.138 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:12:28.151 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T01:12:28.196 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T01:12:28.246 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T01:12:28.254 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:12:28.323 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:12:28.392 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T01:12:28.432 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T01:12:28.462 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T01:12:28.504 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T01:12:28.543 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:12:28.603 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T01:12:28.645 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T01:12:28.669 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T01:12:28.688 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T01:12:28.703 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T01:12:28.727 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T01:12:28.737 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T01:12:28.752 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T01:12:28.797 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-09T01:12:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T01:12:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T01:12:28.873 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T01:12:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T01:12:28.929 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T01:12:28.957 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T01:12:28.964 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-09T01:12:28.964 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-09T01:12:29.027 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T01:12:29.080 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T01:12:29.118 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T01:12:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T01:12:29.240 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T01:12:29.240 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T01:12:29.273 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T01:12:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T01:12:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T01:12:29.316 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T01:12:29.345 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T01:12:29.356 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T01:12:29.395 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T01:12:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T01:12:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T01:12:29.459 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T01:12:29.464 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T01:12:29.499 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T01:12:29.532 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T01:12:29.533 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T01:12:29.574 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T01:12:29.608 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T01:12:29.665 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T01:12:29.690 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T01:12:29.719 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T01:12:29.739 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T01:12:29.769 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T01:12:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T01:12:29.928 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T01:12:29.980 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T01:12:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T01:12:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T01:12:30.345 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T01:12:30.387 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T01:12:30.425 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T01:12:30.467 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T01:12:30.616 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T01:12:30.670 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T01:12:30.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T01:12:30.760 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T01:12:30.801 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T01:12:30.825 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-09T01:12:30.825 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-09T01:12:30.826 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T01:12:30.835 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T01:12:30.900 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T01:12:30.929 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T01:12:30.939 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T01:12:30.967 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T01:12:31.013 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T01:12:31.018 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T01:12:31.049 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T01:12:31.092 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T01:12:31.110 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T01:12:31.143 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T01:12:31.154 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T01:12:31.278 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T01:12:31.322 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T01:12:31.338 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T01:12:31.353 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T01:12:31.367 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T01:12:31.394 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T01:12:31.448 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T01:12:31.489 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T01:12:31.586 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T01:12:31.626 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T01:12:31.660 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T01:12:31.685 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T01:12:31.760 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T01:12:31.886 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T01:12:31.925 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T01:12:31.953 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T01:12:31.987 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T01:12:32.026 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T01:12:32.058 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T01:12:32.088 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T01:12:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T01:12:32.149 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T01:12:32.156 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T01:12:32.197 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T01:12:32.232 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T01:12:32.278 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T01:12:32.319 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T01:12:32.370 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T01:12:32.402 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T01:12:32.448 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T01:12:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T01:12:32.594 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T01:12:33.321 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T01:12:33.367 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T01:12:33.400 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T01:12:33.499 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T01:12:33.566 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T01:12:33.620 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T01:12:33.654 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T01:12:33.685 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T01:12:33.689 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T01:12:33.718 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T01:12:33.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T01:12:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T01:12:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T01:12:33.969 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T01:12:34.086 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T01:12:34.130 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:12:34.222 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:12:34.252 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T01:12:34.288 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T01:12:34.614 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T01:12:34.725 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T01:12:34.805 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T01:12:34.842 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:12:34.913 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:12:34.935 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T01:12:35.957 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T01:12:36.039 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T01:12:36.067 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T01:12:36.100 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T01:12:36.416 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T01:12:36.454 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:12:36.544 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:12:36.575 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T01:12:36.611 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T01:12:37.075 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-09T01:12:37.098 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T01:12:37.132 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:12:37.164 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T01:12:37.202 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:12:37.203 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T01:12:37.225 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T01:12:37.257 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T01:12:37.290 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T01:12:37.316 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T01:12:37.325 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:12:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T01:12:37.364 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:12:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T01:12:37.833 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:12:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:12:37.962 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:12:37.984 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:12:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:12:38.077 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:12:38.097 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:12:38.098 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T01:12:38.098 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T01:12:38.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:38.105 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:12:38.136 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:12:38.262 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T01:12:38.311 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T01:12:38.338 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:12:38.428 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:12:38.456 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:12:38.502 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:12:38.524 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:12:38.547 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:12:38.758 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:12:38.886 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:12:39.029 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:12:39.090 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:12:39.185 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:12:39.274 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:12:39.473 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:12:39.517 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:12:39.548 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:12:39.584 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T01:12:39.612 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T01:12:39.647 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T01:12:39.722 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T01:12:39.763 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T01:12:39.807 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T01:12:39.847 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T01:12:39.875 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T01:12:39.947 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T01:12:39.998 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T01:12:40.028 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T01:12:40.064 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T01:12:40.106 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:12:40.141 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T01:12:40.143 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:12:40.174 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T01:12:40.218 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T01:12:40.237 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:12:40.237 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T01:12:40.237 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T01:12:40.237 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:12:40.246 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T01:12:40.269 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:12:40.320 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T01:12:40.369 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T01:12:40.447 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T01:12:40.582 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T01:12:40.631 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T01:12:41.187 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T01:12:41.270 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T01:12:41.304 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T01:12:41.348 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T01:12:41.377 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T01:12:41.410 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T01:12:41.566 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T01:12:41.638 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T01:12:41.733 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T01:12:41.795 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T01:12:41.869 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T01:12:41.914 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T01:12:41.951 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T01:12:41.999 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T01:12:42.035 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T01:12:42.078 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T01:12:42.119 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T01:12:42.263 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T01:12:42.307 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T01:12:42.338 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T01:12:42.379 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T01:12:42.477 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T01:12:42.513 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T01:12:42.553 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T01:12:42.578 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T01:12:42.603 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T01:12:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T01:12:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T01:12:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T01:12:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T01:12:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T01:12:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T01:12:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T01:12:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T01:12:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T01:12:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T01:12:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T01:12:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T01:12:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T01:12:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T01:12:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T01:12:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T01:12:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T01:12:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T01:12:44.507 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T01:12:44.560 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T01:12:44.594 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T01:12:44.696 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T01:12:44.753 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T01:12:44.842 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T01:12:44.880 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T01:12:45.064 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T01:12:45.085 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T01:12:45.165 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T01:12:45.839 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T01:12:45.953 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T01:12:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T01:12:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T01:12:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T01:12:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T01:12:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T01:12:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:12:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:12:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T01:12:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T01:12:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T01:12:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:12:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:12:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T01:12:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:12:50.406 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-05-09T01:12:50.406 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-05-09T01:12:50.406 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-05-09T01:12:50.406 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-05-09T01:12:50.406 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-05-09T01:12:50.406 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-05-09T01:12:50.406 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-05-09T01:12:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:50.584 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:12:50.601 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:12:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:50.601 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T01:12:50.601 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:50.830 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:12:50.863 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:12:51.451 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:12:51.736 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:12:51.827 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:12:51.827 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T01:12:51.827 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T01:12:51.828 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:12:51.862 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:12:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:12:51.944 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-05-09T01:12:51.944 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-05-09T01:12:51.944 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-05-09T01:12:51.944 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-05-09T01:12:51.944 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-05-09T01:12:51.944 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-05-09T01:12:51.944 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-05-09T01:12:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:12:52.123 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:12:52.138 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:12:52.138 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:52.138 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T01:12:52.138 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:12:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:12:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:12:52.291 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:52.291 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T01:12:52.291 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:12:52.291 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:12:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:52.530 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:12:52.556 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:12:52.556 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:52.556 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T01:12:52.557 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:12:52.557 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:12:52.557 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:53.039 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:12:53.064 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:12:53.064 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:53.064 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T01:12:53.064 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:12:53.064 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:12:53.064 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:53.171 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T01:12:53.221 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T01:12:53.285 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T01:12:53.483 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T01:12:53.533 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T01:12:53.615 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T01:12:53.713 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T01:12:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:12:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:12:53.753 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:53.753 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T01:12:53.753 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:12:53.753 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:12:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:12:53.857 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T01:12:53.918 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T01:12:53.957 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T01:12:53.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:12:53.983 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T01:12:54.000 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:12:54.000 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:54.000 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T01:12:54.000 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:12:54.000 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:12:54.000 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:12:54.407 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:12:54.432 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:12:54.462 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:12:54.465 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T01:12:54.483 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:12:54.484 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:54.484 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T01:12:54.484 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:12:54.484 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:12:54.484 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:12:54.529 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T01:12:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:12:54.583 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T01:12:54.632 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T01:12:54.687 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T01:12:54.873 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T01:12:54.929 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T01:12:55.007 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T01:12:55.104 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T01:12:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T01:12:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T01:12:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T01:12:55.347 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T01:12:55.655 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:12:55.690 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:12:55.791 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:12:55.820 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:12:55.853 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T01:12:55.915 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T01:12:55.943 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:12:56.707 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:12:56.732 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:12:56.851 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:12:56.984 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:12:57.020 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:12:57.024 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T01:12:57.060 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:12:57.085 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:12:57.086 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:57.086 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T01:12:57.086 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:12:57.086 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:12:57.086 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:57.136 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:12:57.148 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:12:57.249 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T01:12:57.311 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T01:12:57.988 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:12:58.012 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:12:58.123 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:12:58.296 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T01:12:58.300 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T01:12:58.330 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:12:58.331 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:12:58.353 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:12:58.353 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:58.353 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T01:12:58.353 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:12:58.353 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:12:58.353 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:12:58.355 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:12:58.355 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:58.355 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T01:12:58.355 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:12:58.355 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:12:58.355 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:58.397 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:12:58.398 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:12:58.409 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:12:58.421 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:12:58.421 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:58.421 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T01:12:58.421 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:58.473 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:12:58.498 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:12:58.498 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:58.498 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T01:12:58.498 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:12:58.498 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:12:58.498 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:58.508 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T01:12:58.563 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T01:12:58.823 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:12:58.848 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:12:58.848 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:58.848 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T01:12:58.848 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:12:58.848 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:12:58.848 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:12:59.502 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T01:12:59.534 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:12:59.556 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:12:59.556 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:59.556 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T01:12:59.556 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:12:59.556 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:12:59.556 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:12:59.601 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:12:59.623 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:12:59.623 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:59.623 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T01:12:59.623 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:12:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:12:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:12:59.691 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:12:59.691 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T01:12:59.691 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:12:59.691 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:12:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:13:00.011 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:13:00.033 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:13:00.033 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:00.033 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T01:13:00.033 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:13:00.033 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:13:00.033 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:13:03.378 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T01:13:03.428 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T01:13:03.469 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T01:13:03.500 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T01:13:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T01:13:03.737 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T01:13:03.790 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T01:13:03.821 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T01:13:03.850 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T01:13:03.888 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T01:13:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:13:03.901 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-05-09T01:13:03.901 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-05-09T01:13:03.901 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-05-09T01:13:03.901 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-05-09T01:13:03.901 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-05-09T01:13:03.901 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /root 2024-05-09T01:13:03.901 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-05-09T01:13:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:13:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:13:04.111 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:04.111 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T01:13:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:04.306 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T01:13:04.348 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T01:13:04.388 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T01:13:04.420 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T01:13:04.448 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T01:13:04.517 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:13:04.561 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:13:04.583 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:13:04.631 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T01:13:04.685 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T01:13:04.715 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T01:13:04.745 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T01:13:04.783 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T01:13:05.350 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:13:05.389 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:13:05.408 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:13:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T01:13:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T01:13:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T01:13:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T01:13:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:13:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T01:13:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:13:05.789 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:05.789 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T01:13:05.789 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:13:05.789 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:13:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T01:13:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:13:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:13:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T01:13:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:13:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:13:06.029 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:13:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T01:13:06.052 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:13:06.053 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:06.053 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T01:13:06.053 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:13:06.053 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:13:06.053 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:13:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:13:06.230 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T01:13:06.310 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T01:13:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:13:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:13:06.351 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T01:13:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:13:06.432 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T01:13:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T01:13:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:13:06.497 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T01:13:06.497 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:13:06.538 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:13:06.561 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:13:06.562 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:06.562 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T01:13:06.562 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:13:06.562 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:13:06.562 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:06.590 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:13:06.623 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T01:13:06.623 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:13:06.677 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T01:13:06.707 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:13:06.732 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T01:13:06.735 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T01:13:06.737 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:13:06.737 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:13:06.799 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T01:13:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:13:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:13:06.995 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T01:13:07.010 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:13:07.048 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T01:13:07.082 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:13:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T01:13:07.218 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T01:13:07.359 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T01:13:07.414 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T01:13:07.446 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T01:13:07.474 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T01:13:07.870 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:13:07.897 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:13:07.930 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T01:13:07.993 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T01:13:08.020 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:13:09.113 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:13:09.147 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:13:10.159 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:13:10.180 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:13:10.298 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:13:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T01:13:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:13:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:13:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T01:13:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:13:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:13:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:10.578 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:13:10.590 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:13:10.697 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T01:13:10.750 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T01:13:11.735 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T01:13:11.761 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:13:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:13:11.785 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:11.785 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T01:13:11.785 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:13:11.785 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:13:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:13:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:13:11.852 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:11.852 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T01:13:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:11.904 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:13:11.927 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:13:11.927 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:11.928 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T01:13:11.928 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:13:11.928 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:13:11.928 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:13:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T01:13:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T01:13:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T01:13:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T01:13:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T01:13:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T01:13:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T01:13:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T01:13:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T01:13:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T01:13:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T01:13:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T01:13:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T01:13:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T01:13:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T01:13:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T01:13:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T01:13:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T01:13:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T01:13:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T01:13:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:13:12.286 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:13:12.287 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:13:12.287 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T01:13:12.287 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:13:12.287 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:13:12.287 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:13:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T01:13:12.652 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T01:13:12.653 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T01:13:12.654 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T01:13:12.655 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T01:13:12.656 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T01:13:12.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T01:13:12.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T01:13:12.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T01:13:12.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T01:13:12.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T01:13:12.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T01:13:12.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T01:13:12.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T01:13:12.657 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T01:13:12.658 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T01:13:12.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T01:13:12.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T01:13:12.661 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T01:13:12.662 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T01:13:12.663 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T01:13:12.664 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T01:13:12.664 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T01:13:12.664 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T01:13:12.664 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T01:13:12.664 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T01:13:12.664 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T01:13:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:13:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:13:14.948 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-05-09T01:13:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T01:13:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T01:13:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T01:13:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T01:13:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T01:13:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T01:13:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T01:13:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T01:13:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T01:13:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:13:14.960 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-05-09T01:13:15.344 DEBUG:teuthology.parallel:result is None 2024-05-09T01:13:15.454 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T01:13:15.455 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.456 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T01:13:15.457 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T01:13:15.458 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T01:13:15.459 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T01:13:15.460 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T01:13:15.461 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T01:13:15.462 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T01:13:15.463 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T01:13:15.464 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:15.465 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T01:13:15.466 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T01:13:15.466 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T01:13:15.466 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T01:13:15.466 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T01:13:15.466 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T01:13:15.466 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T01:13:15.466 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:13:15.466 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-09T01:13:15.863 DEBUG:teuthology.parallel:result is None 2024-05-09T01:13:16.792 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T01:13:16.834 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T01:13:16.874 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T01:13:16.906 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T01:13:16.933 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T01:13:17.130 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T01:13:17.187 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T01:13:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T01:13:17.247 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T01:13:17.285 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T01:13:17.918 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:13:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:13:17.978 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:13:18.870 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T01:13:18.970 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T01:13:19.013 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T01:13:19.096 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T01:13:19.132 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T01:13:19.158 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T01:13:19.158 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:13:19.242 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:13:19.275 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T01:13:19.275 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:13:19.385 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:13:19.410 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T01:13:19.414 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:13:19.415 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:13:19.595 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:13:19.595 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:13:19.679 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:13:19.745 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:13:25.311 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T01:13:25.312 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T01:13:25.313 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T01:13:25.313 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T01:13:25.313 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T01:13:25.313 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T01:13:25.313 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T01:13:25.326 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T01:13:25.326 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T01:13:25.326 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T01:13:25.326 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T01:13:25.326 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T01:13:25.326 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T01:13:25.326 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T01:13:25.327 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T01:13:25.328 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T01:13:25.329 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T01:13:25.330 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T01:13:25.331 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T01:13:25.332 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T01:13:25.333 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T01:13:25.334 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T01:13:25.335 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T01:13:25.336 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T01:13:25.336 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T01:13:27.906 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:13:27.906 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T01:13:27.907 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.908 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T01:13:27.909 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T01:13:27.910 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T01:13:27.911 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T01:13:27.912 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T01:13:27.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T01:13:27.914 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T01:13:27.914 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T01:13:27.914 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T01:13:27.914 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T01:13:27.915 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T01:13:27.915 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T01:13:27.915 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T01:13:27.915 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T01:13:27.915 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T01:13:27.915 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T01:13:27.916 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T01:13:27.917 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T01:13:27.918 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.919 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.920 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:13:27.920 INFO:teuthology.orchestra.run.smithi059.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T01:13:27.920 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T01:13:27.920 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T01:13:27.920 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T01:13:27.920 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T01:13:27.920 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T01:13:27.920 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T01:13:27.920 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:13:27.921 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-09T01:13:28.425 DEBUG:teuthology.parallel:result is None 2024-05-09T01:13:28.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:13:28.584 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T01:13:28.611 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T01:13:28.612 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T01:13:28.612 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T01:13:28.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:13:28.764 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T01:13:28.788 INFO:teuthology.orchestra.run.smithi059.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T01:13:28.788 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T01:13:28.788 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T01:13:28.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:13:29.109 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T01:13:29.135 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T01:13:29.135 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T01:13:29.135 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T01:13:29.137 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T01:13:29.137 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:13:29.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T01:13:29.170 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:13:29.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T01:13:29.202 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:13:29.202 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T01:13:29.231 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T01:13:29.232 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:13:29.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T01:13:29.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T01:13:29.332 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:13:29.332 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T01:13:29.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T01:13:29.428 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:13:29.429 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T01:13:29.456 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T01:13:29.523 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T01:13:29.523 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:13:29.524 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T01:13:29.555 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T01:13:29.624 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:13:29.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T01:13:29.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T01:13:29.722 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:13:29.722 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T01:13:29.746 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T01:13:29.810 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T01:13:29.810 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:13:29.810 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T01:13:29.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T01:13:29.904 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:13:29.904 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T01:13:29.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T01:13:30.002 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:13:30.002 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T01:13:30.025 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T01:13:30.090 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T01:13:30.194 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'subvols': {'create': 2}} 2024-05-09T01:13:30.195 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:13:30.195 INFO:tasks.cephadm:Cluster fsid is 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:13:30.195 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T01:13:30.195 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.59', 'mon.c': '172.21.15.196'} 2024-05-09T01:13:30.196 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-05-09T01:13:30.196 INFO:tasks.cephadm:First mgr is x 2024-05-09T01:13:30.196 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T01:13:30.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-05-09T01:13:30.223 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname $(hostname -s) 2024-05-09T01:13:30.252 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-05-09T01:13:30.276 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-09T01:13:30.277 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:13:30.595 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79226/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 00:57:42.125322', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79224/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:52:58.989794', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-09T01:13:30.893 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240508.183908-debug, sha1 c9da4049f638c0f84bdd9b4e33c42e86046ac554 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:13:30.894 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm 2024-05-09T01:13:30.894 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T01:13:32.055 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 01:13 /home/ubuntu/cephtest/cephadm 2024-05-09T01:13:32.056 DEBUG:teuthology.orchestra.run.smithi059:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T01:13:32.901 INFO:teuthology.orchestra.run.smithi059.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 01:13 /home/ubuntu/cephtest/cephadm 2024-05-09T01:13:32.901 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T01:13:33.824 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 01:13 /home/ubuntu/cephtest/cephadm 2024-05-09T01:13:33.825 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T01:13:33.846 DEBUG:teuthology.orchestra.run.smithi059:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T01:13:33.865 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T01:13:33.902 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 on all hosts... 2024-05-09T01:13:33.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T01:13:33.904 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T01:13:33.908 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T01:13:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:13:35.368 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:13:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:14:05.047 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T01:14:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T01:14:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T01:14:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-05-09T01:14:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T01:14:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-05-09T01:14:05.047 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T01:14:07.840 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T01:14:07.840 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T01:14:07.840 INFO:teuthology.orchestra.run.smithi059.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T01:14:07.840 INFO:teuthology.orchestra.run.smithi059.stdout: "repo_digests": [ 2024-05-09T01:14:07.840 INFO:teuthology.orchestra.run.smithi059.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T01:14:07.840 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-09T01:14:07.840 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T01:14:09.377 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:14:09.377 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T01:14:09.377 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T01:14:09.377 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-05-09T01:14:09.377 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T01:14:09.378 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:14:09.378 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:14:09.402 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-05-09T01:14:09.455 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph 2024-05-09T01:14:09.488 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-05-09T01:14:09.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-05-09T01:14:09.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /etc/ceph 2024-05-09T01:14:09.575 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-05-09T01:14:09.603 INFO:tasks.cephadm:Writing seed config... 2024-05-09T01:14:09.605 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T01:14:09.605 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T01:14:09.605 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T01:14:09.605 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T01:14:09.605 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T01:14:09.605 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-09T01:14:09.605 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T01:14:09.605 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-09T01:14:09.606 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-09T01:14:09.607 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-09T01:14:09.608 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-09T01:14:09.609 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:14:09.609 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T01:14:09.625 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 58c4b1c8-0da1-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T01:14:09.626 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.a.service 2024-05-09T01:14:09.668 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.x.service 2024-05-09T01:14:09.710 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T01:14:09.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 -v bootstrap --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:14:09.909 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:14:09.909 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554', '-v', 'bootstrap', '--fsid', '58c4b1c8-0da1-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-05-09T01:14:09.910 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T01:14:09.910 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-05-09T01:14:09.935 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T01:14:09.935 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-05-09T01:14:09.935 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-05-09T01:14:09.943 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T01:14:09.944 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T01:14:09.950 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T01:14:09.950 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:14:09.957 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-05-09T01:14:09.964 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-05-09T01:14:09.964 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-05-09T01:14:09.964 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-05-09T01:14:09.988 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T01:14:09.989 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T01:14:09.989 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-05-09T01:14:09.989 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-05-09T01:14:09.995 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T01:14:09.995 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T01:14:10.002 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T01:14:10.002 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:14:10.008 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-05-09T01:14:10.015 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-05-09T01:14:10.015 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-05-09T01:14:10.015 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-05-09T01:14:10.016 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:10.016 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 140081682685664 on /run/cephadm/58c4b1c8-0da1-11ef-bc97-c7b262605968.lock 2024-05-09T01:14:10.016 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 140081682685664 acquired on /run/cephadm/58c4b1c8-0da1-11ef-bc97-c7b262605968.lock 2024-05-09T01:14:10.016 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-05-09T01:14:10.016 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-05-09T01:14:10.016 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-05-09T01:14:10.020 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-05-09T01:14:10.020 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-05-09T01:14:10.022 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T01:14:10.023 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-09T01:14:10.023 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-09T01:14:10.025 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T01:14:10.025 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T01:14:10.025 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T01:14:10.025 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-09T01:14:10.025 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-09T01:14:10.026 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T01:14:10.026 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-05-09T01:14:10.026 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-05-09T01:14:10.026 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T01:14:10.027 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T01:14:10.028 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:14:10.539 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T01:14:10.539 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:14:10.539 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T01:14:10.539 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:0411e0fd7a5d45f8ab31b23cec89f97c70494de39461f644c7cdc77de3b9cbca 2024-05-09T01:14:10.539 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-09T01:14:10.539 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T01:14:10.539 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T01:14:11.519 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T01:14:11.519 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T01:14:11.519 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T01:14:12.378 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-05-09T01:14:12.378 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-05-09T01:14:13.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQBkIzxmb+NkKhAAgIg7TO1kt1qNT/JQ3X4phw== 2024-05-09T01:14:14.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQBlIzxmn8rUJBAA1jgDpn81kGMdqm0LXULo6w== 2024-05-09T01:14:15.104 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQBmIzxm8A7JHxAAE3jV8uazh4iRDMG2nU5yfA== 2024-05-09T01:14:15.105 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-05-09T01:14:15.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T01:14:15.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T01:14:15.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:15.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T01:14:15.944 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T01:14:15.944 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-05-09T01:14:15.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:15.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T01:14:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:14:15.944 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-05-09T01:14:16.925 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-05-09T01:14:17.146 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T01:14:17.311 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T01:14:17.500 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968.target. 2024-05-09T01:14:17.500 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968.target. 2024-05-09T01:14:17.734 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.a 2024-05-09T01:14:17.734 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-09T01:14:17.913 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968.target.wants/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@.service. 2024-05-09T01:14:17.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:17 smithi019 systemd[1]: Starting Ceph mon.a for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:14:18.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:18 smithi019 podman[29761]: 2024-05-09 01:14:18.066938014 +0000 UTC m=+0.018669297 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:14:18.291 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:18 smithi019 podman[29761]: 2024-05-09 01:14:18.165059054 +0000 UTC m=+0.116790423 container create 7710815d74e26ea40f1a4546bdf6d8c36bb67e1a9508c151c3e27ce3dec03302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux ) 2024-05-09T01:14:18.532 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T01:14:18.533 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-09T01:14:18.541 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T01:14:18.541 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:14:18.541 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-09T01:14:18.541 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T01:14:18.542 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-05-09T01:14:18.542 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-05-09T01:14:18.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:18 smithi019 podman[29761]: 2024-05-09 01:14:18.434452014 +0000 UTC m=+0.386183290 container init 7710815d74e26ea40f1a4546bdf6d8c36bb67e1a9508c151c3e27ce3dec03302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:14:18.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:18 smithi019 podman[29761]: 2024-05-09 01:14:18.437922629 +0000 UTC m=+0.389653902 container start 7710815d74e26ea40f1a4546bdf6d8c36bb67e1a9508c151c3e27ce3dec03302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:14:18.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:18 smithi019 ceph-mon[29780]: mkfs 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:18.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:18 smithi019 ceph-mon[29780]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T01:14:18.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:18 smithi019 bash[29761]: 7710815d74e26ea40f1a4546bdf6d8c36bb67e1a9508c151c3e27ce3dec03302 2024-05-09T01:14:18.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:18 smithi019 systemd[1]: Started Ceph mon.a for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:14:19.607 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T01:14:19.607 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: monmap epoch 1 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: last_changed 2024-05-09T01:14:15.452380+0000 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: created 2024-05-09T01:14:15.452380+0000 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: min_mon_release 19 (squid) 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: election_strategy: 1 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: fsmap 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: mgrmap e1: no daemons active 2024-05-09T01:14:19.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:19 smithi019 ceph-mon[29780]: from='client.? 172.21.15.19:0/4261326092' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T01:14:19.609 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T01:14:19.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:19.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T01:14:19.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:19.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-05-09T01:14:19.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.556358s) 2024-05-09T01:14:19.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T01:14:19.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T01:14:19.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:19.616 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.968+0000 7f7fa890a700 1 Processor -- start 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.968+0000 7f7fa890a700 1 -- start start 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.968+0000 7f7fa890a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa410cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.968+0000 7f7fa890a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa4077130 con 0x7f7fa410a7a0 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.969+0000 7f7fa259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa410cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.969+0000 7f7fa259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa410cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35210/0 (socket says 172.21.15.19:35210) 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.969+0000 7f7fa259c700 1 -- 172.21.15.19:0/1208546009 learned_addr learned my addr 172.21.15.19:0/1208546009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:19.617 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.969+0000 7f7fa259c700 1 -- 172.21.15.19:0/1208546009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa410d100 con 0x7f7fa410a7a0 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.969+0000 7f7fa259c700 1 --2- 172.21.15.19:0/1208546009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa410cbc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7f940098a0 tx=0x7f7f94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a73d9034d91836 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.969+0000 7f7fa159a700 1 -- 172.21.15.19:0/1208546009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f94004160 con 0x7f7fa410a7a0 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.969+0000 7f7fa159a700 1 -- 172.21.15.19:0/1208546009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f94004300 con 0x7f7fa410a7a0 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.969+0000 7f7fa159a700 1 -- 172.21.15.19:0/1208546009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f940036a0 con 0x7f7fa410a7a0 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.970+0000 7f7fa890a700 1 -- 172.21.15.19:0/1208546009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 msgr2=0x7f7fa410cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.970+0000 7f7fa890a700 1 --2- 172.21.15.19:0/1208546009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa410cbc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7f940098a0 tx=0x7f7f94009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.970+0000 7f7fa890a700 1 -- 172.21.15.19:0/1208546009 shutdown_connections 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.970+0000 7f7fa890a700 1 --2- 172.21.15.19:0/1208546009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa410cbc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.970+0000 7f7fa890a700 1 -- 172.21.15.19:0/1208546009 >> 172.21.15.19:0/1208546009 conn(0x7f7fa4104240 msgr2=0x7f7fa4106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.970+0000 7f7fa890a700 1 -- 172.21.15.19:0/1208546009 shutdown_connections 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.970+0000 7f7fa890a700 1 -- 172.21.15.19:0/1208546009 wait complete. 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.971+0000 7f7fa890a700 1 Processor -- start 2024-05-09T01:14:19.618 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.971+0000 7f7fa890a700 1 -- start start 2024-05-09T01:14:19.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.971+0000 7f7fa890a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:19.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.971+0000 7f7fa890a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa410d510 con 0x7f7fa410a7a0 2024-05-09T01:14:19.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.971+0000 7f7fa259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:19.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7fa259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35212/0 (socket says 172.21.15.19:35212) 2024-05-09T01:14:19.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7fa259c700 1 -- 172.21.15.19:0/4261326092 learned_addr learned my addr 172.21.15.19:0/4261326092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:19.619 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7fa259c700 1 -- 172.21.15.19:0/4261326092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa419dc80 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7fa259c700 1 --2- 172.21.15.19:0/4261326092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa419d740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f94000c00 tx=0x7f7f94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7f937fe700 1 -- 172.21.15.19:0/4261326092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f94003db0 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7f937fe700 1 -- 172.21.15.19:0/4261326092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f94011630 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7fa890a700 1 -- 172.21.15.19:0/4261326092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa419df10 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7f937fe700 1 -- 172.21.15.19:0/4261326092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f94011950 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7fa890a700 1 -- 172.21.15.19:0/4261326092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa41a0b20 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.972+0000 7f7f937fe700 1 -- 172.21.15.19:0/4261326092 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7f94011af0 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.973+0000 7f7f937fe700 1 -- 172.21.15.19:0/4261326092 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7f9401c790 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.973+0000 7f7fa890a700 1 -- 172.21.15.19:0/4261326092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa4064e20 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:18.976+0000 7f7f937fe700 1 -- 172.21.15.19:0/4261326092 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7f94032020 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:19.049+0000 7f7fa890a700 1 -- 172.21.15.19:0/4261326092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7fa402e8f0 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:19.050+0000 7f7f937fe700 1 -- 172.21.15.19:0/4261326092 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7f94004260 con 0x7f7fa410a7a0 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:19.050+0000 7f7fa890a700 1 -- 172.21.15.19:0/4261326092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 msgr2=0x7f7fa419d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:19.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:19.050+0000 7f7fa890a700 1 --2- 172.21.15.19:0/4261326092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa419d740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f94000c00 tx=0x7f7f94004750 comp rx=0 tx=0).stop 2024-05-09T01:14:19.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:19.050+0000 7f7fa890a700 1 -- 172.21.15.19:0/4261326092 shutdown_connections 2024-05-09T01:14:19.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:19.050+0000 7f7fa890a700 1 --2- 172.21.15.19:0/4261326092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa410a7a0 0x7f7fa419d740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:19.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:19.050+0000 7f7fa890a700 1 -- 172.21.15.19:0/4261326092 >> 172.21.15.19:0/4261326092 conn(0x7f7fa4104240 msgr2=0x7f7fa4105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:19.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:19.051+0000 7f7fa890a700 1 -- 172.21.15.19:0/4261326092 shutdown_connections 2024-05-09T01:14:19.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:19.051+0000 7f7fa890a700 1 -- 172.21.15.19:0/4261326092 wait complete. 2024-05-09T01:14:19.621 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-05-09T01:14:19.621 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T01:14:20.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:20 smithi019 ceph-mon[29780]: from='client.? 172.21.15.19:0/2938591309' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T01:14:20.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:20 smithi019 ceph-mon[29780]: from='client.? 172.21.15.19:0/2938591309' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T01:14:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.050+0000 7fd7e3b40700 1 Processor -- start 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.050+0000 7fd7e3b40700 1 -- start start 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.050+0000 7fd7e3b40700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc108f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.050+0000 7fd7e3b40700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc109570 con 0x7fd7dc108b70 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.050+0000 7fd7e18dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.050+0000 7fd7e18dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc108f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36790/0 (socket says 172.21.15.19:36790) 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.050+0000 7fd7e18dc700 1 -- 172.21.15.19:0/2113664820 learned_addr learned my addr 172.21.15.19:0/2113664820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.051+0000 7fd7e18dc700 1 -- 172.21.15.19:0/2113664820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7dc109d40 con 0x7fd7dc108b70 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.051+0000 7fd7e18dc700 1 --2- 172.21.15.19:0/2113664820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc108f80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80098a0 tx=0x7fd7d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62b4c67b0a0caa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.051+0000 7fd7e08da700 1 -- 172.21.15.19:0/2113664820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7d8004160 con 0x7fd7dc108b70 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.051+0000 7fd7e08da700 1 -- 172.21.15.19:0/2113664820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d8004300 con 0x7fd7dc108b70 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.051+0000 7fd7e08da700 1 -- 172.21.15.19:0/2113664820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7d80036a0 con 0x7fd7dc108b70 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.052+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2113664820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 msgr2=0x7fd7dc108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.052+0000 7fd7e3b40700 1 --2- 172.21.15.19:0/2113664820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc108f80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80098a0 tx=0x7fd7d8009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.052+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2113664820 shutdown_connections 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.052+0000 7fd7e3b40700 1 --2- 172.21.15.19:0/2113664820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc108f80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.052+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2113664820 >> 172.21.15.19:0/2113664820 conn(0x7fd7dc104250 msgr2=0x7fd7dc1066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:20.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.052+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2113664820 shutdown_connections 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.052+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2113664820 wait complete. 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.053+0000 7fd7e3b40700 1 Processor -- start 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.053+0000 7fd7e3b40700 1 -- start start 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.053+0000 7fd7e3b40700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.053+0000 7fd7e3b40700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc10a270 con 0x7fd7dc108b70 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.053+0000 7fd7e18dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.053+0000 7fd7e18dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc19d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36792/0 (socket says 172.21.15.19:36792) 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.053+0000 7fd7e18dc700 1 -- 172.21.15.19:0/2938591309 learned_addr learned my addr 172.21.15.19:0/2938591309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.053+0000 7fd7e18dc700 1 -- 172.21.15.19:0/2938591309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7dc19dcc0 con 0x7fd7dc108b70 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.053+0000 7fd7e18dc700 1 --2- 172.21.15.19:0/2938591309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc19d780 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8000c00 tx=0x7fd7d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.054+0000 7fd7d2ffd700 1 -- 172.21.15.19:0/2938591309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7d8003db0 con 0x7fd7dc108b70 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.054+0000 7fd7d2ffd700 1 -- 172.21.15.19:0/2938591309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d8011630 con 0x7fd7dc108b70 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.054+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2938591309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7dc19df50 con 0x7fd7dc108b70 2024-05-09T01:14:20.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.054+0000 7fd7d2ffd700 1 -- 172.21.15.19:0/2938591309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7d8011950 con 0x7fd7dc108b70 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.054+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2938591309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7dc1a0b60 con 0x7fd7dc108b70 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.054+0000 7fd7d2ffd700 1 -- 172.21.15.19:0/2938591309 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd7d8011af0 con 0x7fd7dc108b70 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.054+0000 7fd7d2ffd700 1 -- 172.21.15.19:0/2938591309 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd7d801c450 con 0x7fd7dc108b70 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.055+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2938591309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7dc064e20 con 0x7fd7dc108b70 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.057+0000 7fd7d2ffd700 1 -- 172.21.15.19:0/2938591309 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd7d8032020 con 0x7fd7dc108b70 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.129+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2938591309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd7dc02e8f0 con 0x7fd7dc108b70 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.133+0000 7fd7d2ffd700 1 -- 172.21.15.19:0/2938591309 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7fd7d8004260 con 0x7fd7dc108b70 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.134+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2938591309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 msgr2=0x7fd7dc19d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.134+0000 7fd7d2ffd700 1 -- 172.21.15.19:0/2938591309 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd7d801c650 con 0x7fd7dc108b70 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.134+0000 7fd7e3b40700 1 --2- 172.21.15.19:0/2938591309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc19d780 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8000c00 tx=0x7fd7d8004750 comp rx=0 tx=0).stop 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.134+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2938591309 shutdown_connections 2024-05-09T01:14:20.686 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.134+0000 7fd7e3b40700 1 --2- 172.21.15.19:0/2938591309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc108b70 0x7fd7dc19d780 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:20.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.134+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2938591309 >> 172.21.15.19:0/2938591309 conn(0x7fd7dc104250 msgr2=0x7fd7dc193e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:20.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.134+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2938591309 shutdown_connections 2024-05-09T01:14:20.687 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:20.134+0000 7fd7e3b40700 1 -- 172.21.15.19:0/2938591309 wait complete. 2024-05-09T01:14:20.687 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-05-09T01:14:21.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.091+0000 7f43065a3700 1 Processor -- start 2024-05-09T01:14:21.719 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.091+0000 7f43065a3700 1 -- start start 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.091+0000 7f43065a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f4300108f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.091+0000 7f43065a3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4300109500 con 0x7f4300108b00 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.091+0000 7f42fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f4300108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.091+0000 7f42fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f4300108f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36800/0 (socket says 172.21.15.19:36800) 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.091+0000 7f42fffff700 1 -- 172.21.15.19:0/2909478268 learned_addr learned my addr 172.21.15.19:0/2909478268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.091+0000 7f42fffff700 1 -- 172.21.15.19:0/2909478268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4300109cd0 con 0x7f4300108b00 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.092+0000 7f42fffff700 1 --2- 172.21.15.19:0/2909478268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f4300108f10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42f000bb40 tx=0x7f42f000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997f1632e71aecbc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.092+0000 7f42feffd700 1 -- 172.21.15.19:0/2909478268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42f000dbe0 con 0x7f4300108b00 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.092+0000 7f42feffd700 1 -- 172.21.15.19:0/2909478268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f42f00044c0 con 0x7f4300108b00 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.092+0000 7f42feffd700 1 -- 172.21.15.19:0/2909478268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42f0010470 con 0x7f4300108b00 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.093+0000 7f43065a3700 1 -- 172.21.15.19:0/2909478268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 msgr2=0x7f4300108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.093+0000 7f43065a3700 1 --2- 172.21.15.19:0/2909478268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f4300108f10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42f000bb40 tx=0x7f42f000be50 comp rx=0 tx=0).stop 2024-05-09T01:14:21.720 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.093+0000 7f43065a3700 1 -- 172.21.15.19:0/2909478268 shutdown_connections 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.093+0000 7f43065a3700 1 --2- 172.21.15.19:0/2909478268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f4300108f10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.093+0000 7f43065a3700 1 -- 172.21.15.19:0/2909478268 >> 172.21.15.19:0/2909478268 conn(0x7f43001041e0 msgr2=0x7f4300106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.093+0000 7f43065a3700 1 -- 172.21.15.19:0/2909478268 shutdown_connections 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.093+0000 7f43065a3700 1 -- 172.21.15.19:0/2909478268 wait complete. 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.093+0000 7f43065a3700 1 Processor -- start 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.093+0000 7f43065a3700 1 -- start start 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.094+0000 7f43065a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f430019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.094+0000 7f43065a3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f430007c140 con 0x7f4300108b00 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.094+0000 7f42fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f430019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.094+0000 7f42fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f430019be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36816/0 (socket says 172.21.15.19:36816) 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.094+0000 7f42fffff700 1 -- 172.21.15.19:0/1193000213 learned_addr learned my addr 172.21.15.19:0/1193000213 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.094+0000 7f42fffff700 1 -- 172.21.15.19:0/1193000213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430019c380 con 0x7f4300108b00 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.094+0000 7f42fffff700 1 --2- 172.21.15.19:0/1193000213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f430019be40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42f000b340 tx=0x7f42f0003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:21.721 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.094+0000 7f42fd7fa700 1 -- 172.21.15.19:0/1193000213 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42f0027070 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.095+0000 7f42fd7fa700 1 -- 172.21.15.19:0/1193000213 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f42f0010bb0 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.095+0000 7f42fd7fa700 1 -- 172.21.15.19:0/1193000213 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42f0022ac0 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.095+0000 7f43065a3700 1 -- 172.21.15.19:0/1193000213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430019c610 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.095+0000 7f43065a3700 1 -- 172.21.15.19:0/1193000213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430019f220 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.095+0000 7f42fd7fa700 1 -- 172.21.15.19:0/1193000213 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f42f002a070 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.095+0000 7f42fd7fa700 1 -- 172.21.15.19:0/1193000213 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42f001f070 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.096+0000 7f43065a3700 1 -- 172.21.15.19:0/1193000213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4300064e20 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.098+0000 7f42fd7fa700 1 -- 172.21.15.19:0/1193000213 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f42f0031030 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.170+0000 7f43065a3700 1 -- 172.21.15.19:0/1193000213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"} v 0) -- 0x7f4300195050 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.170+0000 7f42fd7fa700 1 -- 172.21.15.19:0/1193000213 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]=0 v2) ==== 126+0+175 (secure 0 0 0) 0x7f4300195050 con 0x7f4300108b00 2024-05-09T01:14:21.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.171+0000 7f43065a3700 1 -- 172.21.15.19:0/1193000213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 msgr2=0x7f430019be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:21.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.171+0000 7f43065a3700 1 --2- 172.21.15.19:0/1193000213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f430019be40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42f000b340 tx=0x7f42f0003dc0 comp rx=0 tx=0).stop 2024-05-09T01:14:21.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.171+0000 7f43065a3700 1 -- 172.21.15.19:0/1193000213 shutdown_connections 2024-05-09T01:14:21.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.171+0000 7f43065a3700 1 --2- 172.21.15.19:0/1193000213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4300108b00 0x7f430019be40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:21.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.171+0000 7f43065a3700 1 -- 172.21.15.19:0/1193000213 >> 172.21.15.19:0/1193000213 conn(0x7f43001041e0 msgr2=0x7f4300192110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:21.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.171+0000 7f43065a3700 1 -- 172.21.15.19:0/1193000213 shutdown_connections 2024-05-09T01:14:21.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:21.171+0000 7f43065a3700 1 -- 172.21.15.19:0/1193000213 wait complete. 2024-05-09T01:14:21.723 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-05-09T01:14:21.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:21 smithi019 ceph-mon[29780]: from='client.? 172.21.15.19:0/1193000213' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]: dispatch 2024-05-09T01:14:21.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:21 smithi019 systemd[1]: Stopping Ceph mon.a for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:14:22.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:21 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a[29776]: 2024-05-09T01:14:21.989+0000 7f50ff55d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:14:22.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:21 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a[29776]: 2024-05-09T01:14:21.989+0000 7f50ff55d700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T01:14:22.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:22 smithi019 podman[30028]: 2024-05-09 01:14:22.185122603 +0000 UTC m=+0.293498507 container died 7710815d74e26ea40f1a4546bdf6d8c36bb67e1a9508c151c3e27ce3dec03302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:14:22.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:22 smithi019 podman[30028]: 2024-05-09 01:14:22.514452381 +0000 UTC m=+0.622828284 container cleanup 7710815d74e26ea40f1a4546bdf6d8c36bb67e1a9508c151c3e27ce3dec03302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T01:14:22.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:22 smithi019 bash[30028]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a 2024-05-09T01:14:23.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:22 smithi019 podman[30043]: 2024-05-09 01:14:22.748789783 +0000 UTC m=+0.558948598 container remove 7710815d74e26ea40f1a4546bdf6d8c36bb67e1a9508c151c3e27ce3dec03302 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:14:23.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 systemd[1]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T01:14:23.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 systemd[1]: Stopped Ceph mon.a for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:14:23.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 systemd[1]: Starting Ceph mon.a for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:14:23.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 podman[30138]: 2024-05-09 01:14:23.135698322 +0000 UTC m=+0.018391412 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:14:23.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 podman[30138]: 2024-05-09 01:14:23.232843003 +0000 UTC m=+0.115536075 container create 308e13fd3f8c631b12dfa25c4a23bb7ed4cabb10f49a74beb513e4f2670ad903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True) 2024-05-09T01:14:23.539 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T01:14:23.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 podman[30138]: 2024-05-09 01:14:23.451596646 +0000 UTC m=+0.334289719 container init 308e13fd3f8c631b12dfa25c4a23bb7ed4cabb10f49a74beb513e4f2670ad903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T01:14:23.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 podman[30138]: 2024-05-09 01:14:23.454808462 +0000 UTC m=+0.337501525 container start 308e13fd3f8c631b12dfa25c4a23bb7ed4cabb10f49a74beb513e4f2670ad903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T01:14:23.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T01:14:23.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T01:14:23.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: pidfile_write: ignore empty --pid-file 2024-05-09T01:14:23.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: load: jerasure load: lrc 2024-05-09T01:14:23.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: RocksDB version: 7.9.2 2024-05-09T01:14:23.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Git sha 0 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: DB SUMMARY 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: DB Session ID: 2K2NQVXYMPLNMYTSZS6D 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: CURRENT file: CURRENT 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105206 ; 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.error_if_exists: 0 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.create_if_missing: 0 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.paranoid_checks: 1 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T01:14:23.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.env: 0x5597d9f2c500 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.info_log: 0x5597da9fc020 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.statistics: (nil) 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.use_fsync: 0 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_log_file_size: 0 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.allow_fallocate: 1 2024-05-09T01:14:23.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.use_direct_reads: 0 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.db_log_dir: 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.wal_dir: 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T01:14:23.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.write_buffer_manager: 0x5597da8c68c0 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.unordered_write: 0 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T01:14:23.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.row_cache: None 2024-05-09T01:14:23.654 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.wal_filter: None 2024-05-09T01:14:23.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T01:14:23.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T01:14:23.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.two_write_queues: 0 2024-05-09T01:14:23.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T01:14:23.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.wal_compression: 0 2024-05-09T01:14:23.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.atomic_flush: 0 2024-05-09T01:14:23.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.log_readahead_size: 0 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_background_jobs: 2 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_background_compactions: -1 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_subcompactions: 1 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T01:14:23.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_open_files: -1 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_background_flushes: -1 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Compression algorithms supported: 2024-05-09T01:14:23.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: kZSTD supported: 0 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: kXpressCompression supported: 0 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: kZlibCompression supported: 1 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: kSnappyCompression supported: 1 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: kLZ4Compression supported: 1 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: kBZip2Compression supported: 0 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.merge_operator: 2024-05-09T01:14:23.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_filter: None 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5597da9fc1e0) 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T01:14:23.659 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x5597da8f91f0 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-05-09T01:14:23.660 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T01:14:23.661 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression: NoCompression 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.num_levels: 7 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T01:14:23.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T01:14:23.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T01:14:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T01:14:23.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T01:14:23.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T01:14:23.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T01:14:23.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T01:14:23.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T01:14:23.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T01:14:23.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T01:14:23.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T01:14:23.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T01:14:23.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.inplace_update_support: 0 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.bloom_locality: 0 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.max_successive_merges: 0 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.ttl: 2592000 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T01:14:23.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T01:14:23.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T01:14:23.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.enable_blob_files: false 2024-05-09T01:14:23.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.min_blob_size: 0 2024-05-09T01:14:23.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T01:14:23.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T01:14:23.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T01:14:23.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T01:14:23.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T01:14:23.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4c009989-d8aa-4a34-8d44-66c66f9cfc3c 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715217263497902, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715217263499055, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100383, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98267, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14161, "raw_average_key_size": 49, "raw_value_size": 90550, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715217263, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4c009989-d8aa-4a34-8d44-66c66f9cfc3c", "db_session_id": "2K2NQVXYMPLNMYTSZS6D", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715217263499141, "job": 1, "event": "recovery_finished"} 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T01:14:23.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5597da8ddc00 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: DB pointer 0x5597df6d0000 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:14:23.670 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 99.89 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 99.89 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:14:23.671 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 28.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 28.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x5597da8f91f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:23.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mon.a@-1(???) e1 preinit fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mon.a@-1(???).mds e1 new map 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mon.a@-1(???).mds e1 print_map 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:14:23.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 bash[30138]: 308e13fd3f8c631b12dfa25c4a23bb7ed4cabb10f49a74beb513e4f2670ad903 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 systemd[1]: Started Ceph mon.a for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: monmap epoch 1 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: last_changed 2024-05-09T01:14:15.452380+0000 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: created 2024-05-09T01:14:15.452380+0000 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: min_mon_release 19 (squid) 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: election_strategy: 1 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: fsmap 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T01:14:23.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:23 smithi019 ceph-mon[30158]: mgrmap e1: no daemons active 2024-05-09T01:14:24.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.997+0000 7f8731b6a700 1 Processor -- start 2024-05-09T01:14:24.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.997+0000 7f8731b6a700 1 -- start start 2024-05-09T01:14:24.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.997+0000 7f8731b6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c10a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:24.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.997+0000 7f8731b6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f872c10ab10 con 0x7f872c10a0d0 2024-05-09T01:14:24.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.998+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c10a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:24.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.998+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c10a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36822/0 (socket says 172.21.15.19:36822) 2024-05-09T01:14:24.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.998+0000 7f872b7fe700 1 -- 172.21.15.19:0/2996316960 learned_addr learned my addr 172.21.15.19:0/2996316960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:24.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.998+0000 7f872b7fe700 1 -- 172.21.15.19:0/2996316960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c108de0 con 0x7f872c10a0d0 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.998+0000 7f872b7fe700 1 --2- 172.21.15.19:0/2996316960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c10a500 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f871c0098a0 tx=0x7f871c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd0e4324f28e0d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.998+0000 7f872affd700 1 -- 172.21.15.19:0/2996316960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f871c004160 con 0x7f872c10a0d0 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.998+0000 7f872affd700 1 -- 172.21.15.19:0/2996316960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f871c004300 con 0x7f872c10a0d0 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.998+0000 7f872affd700 1 -- 172.21.15.19:0/2996316960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f871c011af0 con 0x7f872c10a0d0 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.999+0000 7f8731b6a700 1 -- 172.21.15.19:0/2996316960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 msgr2=0x7f872c10a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.999+0000 7f8731b6a700 1 --2- 172.21.15.19:0/2996316960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c10a500 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f871c0098a0 tx=0x7f871c009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.999+0000 7f8731b6a700 1 -- 172.21.15.19:0/2996316960 shutdown_connections 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.999+0000 7f8731b6a700 1 --2- 172.21.15.19:0/2996316960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c10a500 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.999+0000 7f8731b6a700 1 -- 172.21.15.19:0/2996316960 >> 172.21.15.19:0/2996316960 conn(0x7f872c103900 msgr2=0x7f872c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.999+0000 7f8731b6a700 1 -- 172.21.15.19:0/2996316960 shutdown_connections 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:23.999+0000 7f8731b6a700 1 -- 172.21.15.19:0/2996316960 wait complete. 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.000+0000 7f8731b6a700 1 Processor -- start 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.000+0000 7f8731b6a700 1 -- start start 2024-05-09T01:14:24.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.000+0000 7f8731b6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c108650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.000+0000 7f8731b6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f872c10d1a0 con 0x7f872c10a0d0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.000+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c108650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.000+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c108650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36826/0 (socket says 172.21.15.19:36826) 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.000+0000 7f872b7fe700 1 -- 172.21.15.19:0/1390675940 learned_addr learned my addr 172.21.15.19:0/1390675940 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.001+0000 7f872b7fe700 1 -- 172.21.15.19:0/1390675940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c108b90 con 0x7f872c10a0d0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.001+0000 7f872b7fe700 1 --2- 172.21.15.19:0/1390675940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c108650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f871c009fb0 tx=0x7f871c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.001+0000 7f87297fa700 1 -- 172.21.15.19:0/1390675940 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f871c011bc0 con 0x7f872c10a0d0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.001+0000 7f8731b6a700 1 -- 172.21.15.19:0/1390675940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872c1a0ee0 con 0x7f872c10a0d0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.001+0000 7f87297fa700 1 -- 172.21.15.19:0/1390675940 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f871c024440 con 0x7f872c10a0d0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.001+0000 7f87297fa700 1 -- 172.21.15.19:0/1390675940 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f871c01a5c0 con 0x7f872c10a0d0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.001+0000 7f8731b6a700 1 -- 172.21.15.19:0/1390675940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872c1a1360 con 0x7f872c10a0d0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.001+0000 7f87297fa700 1 -- 172.21.15.19:0/1390675940 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f871c01ac10 con 0x7f872c10a0d0 2024-05-09T01:14:24.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.001+0000 7f87297fa700 1 -- 172.21.15.19:0/1390675940 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f871c019700 con 0x7f872c10a0d0 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.002+0000 7f8731b6a700 1 -- 172.21.15.19:0/1390675940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872c064e20 con 0x7f872c10a0d0 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.005+0000 7f87297fa700 1 -- 172.21.15.19:0/1390675940 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f871c02ddf0 con 0x7f872c10a0d0 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.075+0000 7f8731b6a700 1 -- 172.21.15.19:0/1390675940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f872c1a15e0 con 0x7f872c10a0d0 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.077+0000 7f87297fa700 1 -- 172.21.15.19:0/1390675940 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f871c0245f0 con 0x7f872c10a0d0 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.077+0000 7f8731b6a700 1 -- 172.21.15.19:0/1390675940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 msgr2=0x7f872c108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.078+0000 7f8731b6a700 1 --2- 172.21.15.19:0/1390675940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c108650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f871c009fb0 tx=0x7f871c011890 comp rx=0 tx=0).stop 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.078+0000 7f8731b6a700 1 -- 172.21.15.19:0/1390675940 shutdown_connections 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.078+0000 7f8731b6a700 1 --2- 172.21.15.19:0/1390675940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f872c10a0d0 0x7f872c108650 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.078+0000 7f8731b6a700 1 -- 172.21.15.19:0/1390675940 >> 172.21.15.19:0/1390675940 conn(0x7f872c103900 msgr2=0x7f872c104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.078+0000 7f8731b6a700 1 -- 172.21.15.19:0/1390675940 shutdown_connections 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:24.078+0000 7f8731b6a700 1 -- 172.21.15.19:0/1390675940 wait complete. 2024-05-09T01:14:24.670 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T01:14:24.691 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:14:24.691 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-05-09T01:14:24.692 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T01:14:24.692 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T01:14:24.973 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.x 2024-05-09T01:14:24.973 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-09T01:14:25.127 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968.target.wants/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@.service. 2024-05-09T01:14:25.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:25 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1390675940' entity='client.admin' 2024-05-09T01:14:25.353 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:25 smithi019 systemd[1]: Starting Ceph mgr.x for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:14:25.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:25 smithi019 podman[30377]: 2024-05-09 01:14:25.352226043 +0000 UTC m=+0.099881516 container create d07233fe87ec017c0fd98c86e3499ae1055e1438a4af20b7cc8048469baad868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:14:25.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:25 smithi019 podman[30377]: 2024-05-09 01:14:25.269913442 +0000 UTC m=+0.017568889 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:14:25.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:25 smithi019 podman[30377]: 2024-05-09 01:14:25.538586293 +0000 UTC m=+0.286241730 container init d07233fe87ec017c0fd98c86e3499ae1055e1438a4af20b7cc8048469baad868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T01:14:25.622 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:25 smithi019 podman[30377]: 2024-05-09 01:14:25.541709124 +0000 UTC m=+0.289364556 container start d07233fe87ec017c0fd98c86e3499ae1055e1438a4af20b7cc8048469baad868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-09T01:14:25.622 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:25 smithi019 bash[30377]: d07233fe87ec017c0fd98c86e3499ae1055e1438a4af20b7cc8048469baad868 2024-05-09T01:14:25.635 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T01:14:25.635 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-09T01:14:25.643 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T01:14:25.643 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:14:25.643 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-09T01:14:25.643 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T01:14:25.653 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T01:14:25.653 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-09T01:14:25.661 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T01:14:25.662 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:14:25.662 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-09T01:14:25.662 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T01:14:25.662 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-05-09T01:14:25.662 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-05-09T01:14:25.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:25 smithi019 systemd[1]: Started Ceph mgr.x for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:14:25.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:25 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:25.719+0000 7ffb4bbf0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T01:14:26.158 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:25 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:25.913+0000 7ffb4bbf0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:14:26.159 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:26 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:26.020+0000 7ffb4bbf0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T01:14:26.449 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:26 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2331154970' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T01:14:26.449 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:26 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:26.358+0000 7ffb4bbf0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T01:14:26.735 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:26 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:26.448+0000 7ffb4bbf0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T01:14:26.738 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T01:14:26.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T01:14:26.749 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:14:26.749 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-09T01:14:26.749 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T01:14:26.749 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:26.749 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T01:14:26.750 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T01:14:26.751 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T01:14:18.496181+0000", 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:14:26.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.147+0000 7f27bca59700 1 Processor -- start 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.147+0000 7f27bca59700 1 -- start start 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.147+0000 7f27bca59700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b810cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.147+0000 7f27bca59700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27b80770b0 con 0x7f27b810a820 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.148+0000 7f27b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b810cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.148+0000 7f27b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b810cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36852/0 (socket says 172.21.15.19:36852) 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.148+0000 7f27b659c700 1 -- 172.21.15.19:0/466516376 learned_addr learned my addr 172.21.15.19:0/466516376 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.148+0000 7f27b659c700 1 -- 172.21.15.19:0/466516376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b810d180 con 0x7f27b810a820 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.148+0000 7f27b659c700 1 --2- 172.21.15.19:0/466516376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b810cc40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098a0 tx=0x7f27a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d96c026b20e15f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.149+0000 7f27b559a700 1 -- 172.21.15.19:0/466516376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a0004160 con 0x7f27b810a820 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.149+0000 7f27b559a700 1 -- 172.21.15.19:0/466516376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f27a0004300 con 0x7f27b810a820 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.149+0000 7f27b559a700 1 -- 172.21.15.19:0/466516376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a0011af0 con 0x7f27b810a820 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.149+0000 7f27bca59700 1 -- 172.21.15.19:0/466516376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 msgr2=0x7f27b810cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.149+0000 7f27bca59700 1 --2- 172.21.15.19:0/466516376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b810cc40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098a0 tx=0x7f27a0009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:26.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.150+0000 7f27bca59700 1 -- 172.21.15.19:0/466516376 shutdown_connections 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.150+0000 7f27bca59700 1 --2- 172.21.15.19:0/466516376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b810cc40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.150+0000 7f27bca59700 1 -- 172.21.15.19:0/466516376 >> 172.21.15.19:0/466516376 conn(0x7f27b8104240 msgr2=0x7f27b8106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.150+0000 7f27bca59700 1 -- 172.21.15.19:0/466516376 shutdown_connections 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.150+0000 7f27bca59700 1 -- 172.21.15.19:0/466516376 wait complete. 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.151+0000 7f27bca59700 1 Processor -- start 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.151+0000 7f27bca59700 1 -- start start 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.151+0000 7f27bca59700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.151+0000 7f27bca59700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27b810e080 con 0x7f27b810a820 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.151+0000 7f27b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.151+0000 7f27b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36868/0 (socket says 172.21.15.19:36868) 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.151+0000 7f27b659c700 1 -- 172.21.15.19:0/2331154970 learned_addr learned my addr 172.21.15.19:0/2331154970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.151+0000 7f27b659c700 1 -- 172.21.15.19:0/2331154970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b819e560 con 0x7f27b810a820 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.152+0000 7f27b659c700 1 --2- 172.21.15.19:0/2331154970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b819e020 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009fb0 tx=0x7f27a0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:26.754 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.152+0000 7f27af7fe700 1 -- 172.21.15.19:0/2331154970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a00036a0 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.152+0000 7f27bca59700 1 -- 172.21.15.19:0/2331154970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b819e7f0 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.152+0000 7f27af7fe700 1 -- 172.21.15.19:0/2331154970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f27a001a430 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.152+0000 7f27af7fe700 1 -- 172.21.15.19:0/2331154970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a0019550 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.152+0000 7f27bca59700 1 -- 172.21.15.19:0/2331154970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b81a1400 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.152+0000 7f27af7fe700 1 -- 172.21.15.19:0/2331154970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f27a0019b90 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.152+0000 7f27af7fe700 1 -- 172.21.15.19:0/2331154970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f27a0024730 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.153+0000 7f27bca59700 1 -- 172.21.15.19:0/2331154970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b8064e20 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.157+0000 7f27af7fe700 1 -- 172.21.15.19:0/2331154970 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f27a0043ec0 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.231+0000 7f27bca59700 1 -- 172.21.15.19:0/2331154970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f27b81a1620 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.231+0000 7f27af7fe700 1 -- 172.21.15.19:0/2331154970 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f27a0032020 con 0x7f27b810a820 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.233+0000 7f27bca59700 1 -- 172.21.15.19:0/2331154970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 msgr2=0x7f27b819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.233+0000 7f27bca59700 1 --2- 172.21.15.19:0/2331154970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b819e020 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009fb0 tx=0x7f27a0011940 comp rx=0 tx=0).stop 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.233+0000 7f27bca59700 1 -- 172.21.15.19:0/2331154970 shutdown_connections 2024-05-09T01:14:26.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.233+0000 7f27bca59700 1 --2- 172.21.15.19:0/2331154970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27b810a820 0x7f27b819e020 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:26.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.233+0000 7f27bca59700 1 -- 172.21.15.19:0/2331154970 >> 172.21.15.19:0/2331154970 conn(0x7f27b8104240 msgr2=0x7f27b8105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:26.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.233+0000 7f27bca59700 1 -- 172.21.15.19:0/2331154970 shutdown_connections 2024-05-09T01:14:26.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:26.233+0000 7f27bca59700 1 -- 172.21.15.19:0/2331154970 wait complete. 2024-05-09T01:14:26.756 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-05-09T01:14:28.001 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:27 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:27.716+0000 7ffb4bbf0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T01:14:28.001 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:27 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:27.892+0000 7ffb4bbf0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T01:14:28.371 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:28.072+0000 7ffb4bbf0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T01:14:28.371 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:28.370+0000 7ffb4bbf0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T01:14:28.648 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:28.466+0000 7ffb4bbf0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T01:14:28.962 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:28.681+0000 7ffb4bbf0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T01:14:28.962 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:28.884+0000 7ffb4bbf0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:14:29.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:29 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1180533394' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T01:14:29.800 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:29 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:29.531+0000 7ffb4bbf0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T01:14:29.800 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:29 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:29.763+0000 7ffb4bbf0200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T01:14:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T01:14:29.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-09T01:14:29.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:14:29.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T01:14:29.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T01:14:29.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:14:29.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-09T01:14:29.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T01:14:29.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:29.818 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T01:14:29.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T01:14:29.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:29.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T01:14:29.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:29.819 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T01:14:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T01:14:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T01:14:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T01:14:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T01:14:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T01:14:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T01:14:29.821 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:29.822 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T01:14:18.496181+0000", 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.160+0000 7fd2d480a700 1 Processor -- start 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.160+0000 7fd2d480a700 1 -- start start 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.160+0000 7fd2d480a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d010cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.160+0000 7fd2d480a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d007ce10 con 0x7fd2d010a750 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.161+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d010cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.161+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d010cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36874/0 (socket says 172.21.15.19:36874) 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.161+0000 7fd2ce59c700 1 -- 172.21.15.19:0/871556511 learned_addr learned my addr 172.21.15.19:0/871556511 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.161+0000 7fd2ce59c700 1 -- 172.21.15.19:0/871556511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d010d0b0 con 0x7fd2d010a750 2024-05-09T01:14:29.823 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.161+0000 7fd2ce59c700 1 --2- 172.21.15.19:0/871556511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d010cb70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00098a0 tx=0x7fd2c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a9233545ec5b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.161+0000 7fd2cd59a700 1 -- 172.21.15.19:0/871556511 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c0004160 con 0x7fd2d010a750 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.161+0000 7fd2cd59a700 1 -- 172.21.15.19:0/871556511 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd2c0004300 con 0x7fd2d010a750 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.161+0000 7fd2cd59a700 1 -- 172.21.15.19:0/871556511 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c0011af0 con 0x7fd2d010a750 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.163+0000 7fd2d480a700 1 -- 172.21.15.19:0/871556511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 msgr2=0x7fd2d010cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.163+0000 7fd2d480a700 1 --2- 172.21.15.19:0/871556511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d010cb70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00098a0 tx=0x7fd2c0009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.163+0000 7fd2d480a700 1 -- 172.21.15.19:0/871556511 shutdown_connections 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.163+0000 7fd2d480a700 1 --2- 172.21.15.19:0/871556511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d010cb70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.163+0000 7fd2d480a700 1 -- 172.21.15.19:0/871556511 >> 172.21.15.19:0/871556511 conn(0x7fd2d0104190 msgr2=0x7fd2d0106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.163+0000 7fd2d480a700 1 -- 172.21.15.19:0/871556511 shutdown_connections 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.163+0000 7fd2d480a700 1 -- 172.21.15.19:0/871556511 wait complete. 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.164+0000 7fd2d480a700 1 Processor -- start 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.164+0000 7fd2d480a700 1 -- start start 2024-05-09T01:14:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.164+0000 7fd2d480a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:29.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.164+0000 7fd2d480a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d010e010 con 0x7fd2d010a750 2024-05-09T01:14:29.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.164+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:29.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.164+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d019dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36888/0 (socket says 172.21.15.19:36888) 2024-05-09T01:14:29.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.164+0000 7fd2ce59c700 1 -- 172.21.15.19:0/1180533394 learned_addr learned my addr 172.21.15.19:0/1180533394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:29.825 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.164+0000 7fd2ce59c700 1 -- 172.21.15.19:0/1180533394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d019e530 con 0x7fd2d010a750 2024-05-09T01:14:29.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.165+0000 7fd2ce59c700 1 --2- 172.21.15.19:0/1180533394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d019dff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00117b0 tx=0x7fd2c0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:29.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.165+0000 7fd2c77fe700 1 -- 172.21.15.19:0/1180533394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c0003af0 con 0x7fd2d010a750 2024-05-09T01:14:29.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.165+0000 7fd2c77fe700 1 -- 172.21.15.19:0/1180533394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd2c0020070 con 0x7fd2d010a750 2024-05-09T01:14:29.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.165+0000 7fd2d480a700 1 -- 172.21.15.19:0/1180533394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d019e7c0 con 0x7fd2d010a750 2024-05-09T01:14:29.826 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.165+0000 7fd2c77fe700 1 -- 172.21.15.19:0/1180533394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c0019630 con 0x7fd2d010a750 2024-05-09T01:14:29.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.165+0000 7fd2d480a700 1 -- 172.21.15.19:0/1180533394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d01a13d0 con 0x7fd2d010a750 2024-05-09T01:14:29.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.165+0000 7fd2c77fe700 1 -- 172.21.15.19:0/1180533394 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd2c0019c30 con 0x7fd2d010a750 2024-05-09T01:14:29.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.165+0000 7fd2c77fe700 1 -- 172.21.15.19:0/1180533394 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd2c000b720 con 0x7fd2d010a750 2024-05-09T01:14:29.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.166+0000 7fd2d480a700 1 -- 172.21.15.19:0/1180533394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d0064e20 con 0x7fd2d010a750 2024-05-09T01:14:29.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.168+0000 7fd2c77fe700 1 -- 172.21.15.19:0/1180533394 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd2c001a9c0 con 0x7fd2d010a750 2024-05-09T01:14:29.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.242+0000 7fd2d480a700 1 -- 172.21.15.19:0/1180533394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd2d01a15a0 con 0x7fd2d010a750 2024-05-09T01:14:29.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.243+0000 7fd2c77fe700 1 -- 172.21.15.19:0/1180533394 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd2c001a430 con 0x7fd2d010a750 2024-05-09T01:14:29.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.244+0000 7fd2d480a700 1 -- 172.21.15.19:0/1180533394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 msgr2=0x7fd2d019dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:29.827 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.244+0000 7fd2d480a700 1 --2- 172.21.15.19:0/1180533394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d019dff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00117b0 tx=0x7fd2c0011b70 comp rx=0 tx=0).stop 2024-05-09T01:14:29.828 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.244+0000 7fd2d480a700 1 -- 172.21.15.19:0/1180533394 shutdown_connections 2024-05-09T01:14:29.828 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.244+0000 7fd2d480a700 1 --2- 172.21.15.19:0/1180533394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d010a750 0x7fd2d019dff0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:29.828 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.244+0000 7fd2d480a700 1 -- 172.21.15.19:0/1180533394 >> 172.21.15.19:0/1180533394 conn(0x7fd2d0104190 msgr2=0x7fd2d0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:29.828 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.244+0000 7fd2d480a700 1 -- 172.21.15.19:0/1180533394 shutdown_connections 2024-05-09T01:14:29.828 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:29.245+0000 7fd2d480a700 1 -- 172.21.15.19:0/1180533394 wait complete. 2024-05-09T01:14:29.828 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-05-09T01:14:30.648 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:30 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:30.206+0000 7ffb4bbf0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T01:14:30.648 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:30 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:30.307+0000 7ffb4bbf0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T01:14:31.067 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:30 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:30.694+0000 7ffb4bbf0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T01:14:31.067 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:31.066+0000 7ffb4bbf0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T01:14:31.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:31.162+0000 7ffb4bbf0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T01:14:31.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:31.418+0000 7ffb4bbf0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T01:14:31.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:31.558+0000 7ffb4bbf0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T01:14:31.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:31.658+0000 7ffb4bbf0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T01:14:32.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:32.005+0000 7ffb4bbf0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T01:14:32.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:32.102+0000 7ffb4bbf0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T01:14:32.996 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:32.742+0000 7ffb4bbf0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T01:14:32.996 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:32.972+0000 7ffb4bbf0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T01:14:33.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: Activating manager daemon x 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: mgrmap e2: x(active, starting, since 0.00400163s) 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: Manager daemon x is now available 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' 2024-05-09T01:14:33.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:33 smithi019 ceph-mon[30158]: from='mgr.14100 172.21.15.19:0/584004058' entity='mgr.x' 2024-05-09T01:14:35.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:34 smithi019 ceph-mon[30158]: mgrmap e3: x(active, since 1.00926s) 2024-05-09T01:14:36.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:35 smithi019 ceph-mon[30158]: mgrmap e4: x(active, since 2s) 2024-05-09T01:14:36.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:35 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2802439250' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T01:14:41.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:41.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:14:41.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-09T01:14:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T01:14:41.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:41.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T01:14:41.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T01:14:41.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:41.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T01:14:41.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:41.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T01:14:41.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T01:14:41.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T01:14:41.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T01:14:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T01:14:18.496181+0000", 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.000+0000 7f0fefe96700 1 Processor -- start 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.000+0000 7f0fefe96700 1 -- start start 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fefe96700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe810cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:41.449 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fefe96700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe807ce10 con 0x7f0fe810a740 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fedc32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe810cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fedc32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe810cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47526/0 (socket says 172.21.15.19:47526) 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fedc32700 1 -- 172.21.15.19:0/3873956004 learned_addr learned my addr 172.21.15.19:0/3873956004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fedc32700 1 -- 172.21.15.19:0/3873956004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe810d0a0 con 0x7f0fe810a740 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fedc32700 1 --2- 172.21.15.19:0/3873956004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe810cb60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009e80 tx=0x7f0fe400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e10f6537ef4a130 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fecc30700 1 -- 172.21.15.19:0/3873956004 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fe4017070 con 0x7f0fe810a740 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fecc30700 1 -- 172.21.15.19:0/3873956004 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0fe400b8c0 con 0x7f0fe810a740 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.001+0000 7f0fecc30700 1 -- 172.21.15.19:0/3873956004 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fe4004490 con 0x7f0fe810a740 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.002+0000 7f0fefe96700 1 -- 172.21.15.19:0/3873956004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 msgr2=0x7f0fe810cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.002+0000 7f0fefe96700 1 --2- 172.21.15.19:0/3873956004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe810cb60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009e80 tx=0x7f0fe400b330 comp rx=0 tx=0).stop 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.002+0000 7f0fefe96700 1 -- 172.21.15.19:0/3873956004 shutdown_connections 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.002+0000 7f0fefe96700 1 --2- 172.21.15.19:0/3873956004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe810cb60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.002+0000 7f0fefe96700 1 -- 172.21.15.19:0/3873956004 >> 172.21.15.19:0/3873956004 conn(0x7f0fe81041e0 msgr2=0x7f0fe8106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:41.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.002+0000 7f0fefe96700 1 -- 172.21.15.19:0/3873956004 shutdown_connections 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.002+0000 7f0fefe96700 1 -- 172.21.15.19:0/3873956004 wait complete. 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.003+0000 7f0fefe96700 1 Processor -- start 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.003+0000 7f0fefe96700 1 -- start start 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.003+0000 7f0fefe96700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.003+0000 7f0fefe96700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe810e000 con 0x7f0fe810a740 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.003+0000 7f0fedc32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.003+0000 7f0fedc32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe819e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47534/0 (socket says 172.21.15.19:47534) 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.003+0000 7f0fedc32700 1 -- 172.21.15.19:0/2802439250 learned_addr learned my addr 172.21.15.19:0/2802439250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.004+0000 7f0fedc32700 1 -- 172.21.15.19:0/2802439250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe819e5a0 con 0x7f0fe810a740 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.004+0000 7f0fedc32700 1 --2- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe819e060 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400b7b0 tx=0x7f0fe400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.004+0000 7f0fdeffd700 1 -- 172.21.15.19:0/2802439250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fe4020020 con 0x7f0fe810a740 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.004+0000 7f0fdeffd700 1 -- 172.21.15.19:0/2802439250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0fe4012430 con 0x7f0fe810a740 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.004+0000 7f0fdeffd700 1 -- 172.21.15.19:0/2802439250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fe4017040 con 0x7f0fe810a740 2024-05-09T01:14:41.451 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.004+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe819e830 con 0x7f0fe810a740 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.004+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe81a1440 con 0x7f0fe810a740 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.005+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe8064e20 con 0x7f0fe810a740 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.005+0000 7f0fdeffd700 1 -- 172.21.15.19:0/2802439250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f0fe4023020 con 0x7f0fe810a740 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.005+0000 7f0fdeffd700 1 --2- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f0fd403e420 0x7f0fd4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.006+0000 7f0fdeffd700 1 -- 172.21.15.19:0/2802439250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0fe400f050 con 0x7f0fe810a740 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.006+0000 7f0fed431700 1 --2- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f0fd403e420 0x7f0fd4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.006+0000 7f0fed431700 1 --2- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f0fd403e420 0x7f0fd4040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8006fd0 tx=0x7f0fd8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.011+0000 7f0fdeffd700 1 -- 172.21.15.19:0/2802439250 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0fe402a020 con 0x7f0fe810a740 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.319+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0fe81a1ac0 con 0x7f0fe810a740 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.320+0000 7f0fdeffd700 1 -- 172.21.15.19:0/2802439250 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0fe40102f0 con 0x7f0fe810a740 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f0fd403e420 msgr2=0x7f0fd4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:41.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 --2- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f0fd403e420 0x7f0fd4040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8006fd0 tx=0x7f0fd8006e40 comp rx=0 tx=0).stop 2024-05-09T01:14:41.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 msgr2=0x7f0fe819e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:41.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 --2- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe819e060 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400b7b0 tx=0x7f0fe400bd50 comp rx=0 tx=0).stop 2024-05-09T01:14:41.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 shutdown_connections 2024-05-09T01:14:41.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 --2- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f0fd403e420 0x7f0fd4040930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:41.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 --2- 172.21.15.19:0/2802439250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fe810a740 0x7f0fe819e060 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:41.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 >> 172.21.15.19:0/2802439250 conn(0x7f0fe81041e0 msgr2=0x7f0fe8105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:41.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 shutdown_connections 2024-05-09T01:14:41.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:35.323+0000 7f0fefe96700 1 -- 172.21.15.19:0/2802439250 wait complete. 2024-05-09T01:14:41.453 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-05-09T01:14:44.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:44 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2142767234' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T01:14:45.023 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:14:45.024 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.094+0000 7f63af4d6700 1 Processor -- start 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.095+0000 7f63af4d6700 1 -- start start 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.095+0000 7f63af4d6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a810a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.095+0000 7f63af4d6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63a810a640 con 0x7f63a8109c40 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.095+0000 7f63ad272700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a810a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.095+0000 7f63ad272700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a810a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57864/0 (socket says 172.21.15.19:57864) 2024-05-09T01:14:45.025 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.095+0000 7f63ad272700 1 -- 172.21.15.19:0/1215503647 learned_addr learned my addr 172.21.15.19:0/1215503647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.096+0000 7f63ad272700 1 -- 172.21.15.19:0/1215503647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a810ae10 con 0x7f63a8109c40 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.096+0000 7f63ad272700 1 --2- 172.21.15.19:0/1215503647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a810a050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f63a40098a0 tx=0x7f63a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b01c06cb417417 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.096+0000 7f639bfff700 1 -- 172.21.15.19:0/1215503647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a4004160 con 0x7f63a8109c40 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.096+0000 7f639bfff700 1 -- 172.21.15.19:0/1215503647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f63a4004300 con 0x7f63a8109c40 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.096+0000 7f639bfff700 1 -- 172.21.15.19:0/1215503647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a4011af0 con 0x7f63a8109c40 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.097+0000 7f63af4d6700 1 -- 172.21.15.19:0/1215503647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 msgr2=0x7f63a810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.097+0000 7f63af4d6700 1 --2- 172.21.15.19:0/1215503647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a810a050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f63a40098a0 tx=0x7f63a4009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.097+0000 7f63af4d6700 1 -- 172.21.15.19:0/1215503647 shutdown_connections 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.097+0000 7f63af4d6700 1 --2- 172.21.15.19:0/1215503647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a810a050 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.097+0000 7f63af4d6700 1 -- 172.21.15.19:0/1215503647 >> 172.21.15.19:0/1215503647 conn(0x7f63a81052c0 msgr2=0x7f63a8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.097+0000 7f63af4d6700 1 -- 172.21.15.19:0/1215503647 shutdown_connections 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.097+0000 7f63af4d6700 1 -- 172.21.15.19:0/1215503647 wait complete. 2024-05-09T01:14:45.026 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.098+0000 7f63af4d6700 1 Processor -- start 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.098+0000 7f63af4d6700 1 -- start start 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.098+0000 7f63af4d6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.098+0000 7f63af4d6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63a810e1c0 con 0x7f63a8109c40 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.098+0000 7f63ad272700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.098+0000 7f63ad272700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a819e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57866/0 (socket says 172.21.15.19:57866) 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.098+0000 7f63ad272700 1 -- 172.21.15.19:0/2142767234 learned_addr learned my addr 172.21.15.19:0/2142767234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.098+0000 7f63ad272700 1 -- 172.21.15.19:0/2142767234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a819e690 con 0x7f63a8109c40 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.099+0000 7f63ad272700 1 --2- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a819e150 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f63a4011670 tx=0x7f63a4004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.099+0000 7f639a7fc700 1 -- 172.21.15.19:0/2142767234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a40036a0 con 0x7f63a8109c40 2024-05-09T01:14:45.027 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.099+0000 7f639a7fc700 1 -- 172.21.15.19:0/2142767234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f63a4003cd0 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.099+0000 7f639a7fc700 1 -- 172.21.15.19:0/2142767234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a40193c0 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.099+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a819e920 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.099+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81a1530 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.100+0000 7f639a7fc700 1 -- 172.21.15.19:0/2142767234 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f63a4019630 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.100+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a8064e20 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.100+0000 7f639a7fc700 1 --2- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f639403e420 0x7f6394040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.100+0000 7f639a7fc700 1 -- 172.21.15.19:0/2142767234 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f63a4053780 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.100+0000 7f63aca71700 1 --2- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f639403e420 0x7f6394040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.101+0000 7f63aca71700 1 --2- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f639403e420 0x7f6394040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f639c006fd0 tx=0x7f639c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.106+0000 7f639a7fc700 1 -- 172.21.15.19:0/2142767234 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63a40199d0 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.324+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f63a81a2d30 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.325+0000 7f639a7fc700 1 -- 172.21.15.19:0/2142767234 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f63a40531a0 con 0x7f63a8109c40 2024-05-09T01:14:45.028 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f639403e420 msgr2=0x7f6394040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 --2- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f639403e420 0x7f6394040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f639c006fd0 tx=0x7f639c006e40 comp rx=0 tx=0).stop 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 msgr2=0x7f63a819e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 --2- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a819e150 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f63a4011670 tx=0x7f63a4004280 comp rx=0 tx=0).stop 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 shutdown_connections 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 --2- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f639403e420 0x7f6394040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 --2- 172.21.15.19:0/2142767234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a8109c40 0x7f63a819e150 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 >> 172.21.15.19:0/2142767234 conn(0x7f63a81052c0 msgr2=0x7f63a8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 shutdown_connections 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:44.328+0000 7f63af4d6700 1 -- 172.21.15.19:0/2142767234 wait complete. 2024-05-09T01:14:45.029 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-05-09T01:14:46.058 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:45 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2166813264' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T01:14:46.058 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:45 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: ignoring --setuser ceph since I am not root 2024-05-09T01:14:46.058 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:45 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: ignoring --setgroup ceph since I am not root 2024-05-09T01:14:46.058 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:46.057+0000 7f2daaad9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T01:14:46.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:46.253+0000 7f2daaad9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:14:46.363 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:46.361+0000 7f2daaad9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T01:14:47.029 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:46 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2166813264' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T01:14:47.029 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:46 smithi019 ceph-mon[30158]: mgrmap e5: x(active, since 12s) 2024-05-09T01:14:47.029 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:46.688+0000 7f2daaad9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T01:14:47.029 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:46.779+0000 7f2daaad9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T01:14:47.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.447+0000 7fd85a58c700 1 Processor -- start 2024-05-09T01:14:47.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.447+0000 7fd85a58c700 1 -- start start 2024-05-09T01:14:47.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.447+0000 7fd85a58c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd85410c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:47.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.447+0000 7fd85a58c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd85410cde0 con 0x7fd85407ccc0 2024-05-09T01:14:47.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.448+0000 7fd853fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd85410c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:47.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.448+0000 7fd853fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd85410c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57870/0 (socket says 172.21.15.19:57870) 2024-05-09T01:14:47.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.448+0000 7fd853fff700 1 -- 172.21.15.19:0/2806050843 learned_addr learned my addr 172.21.15.19:0/2806050843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.448+0000 7fd853fff700 1 -- 172.21.15.19:0/2806050843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85410cf60 con 0x7fd85407ccc0 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.448+0000 7fd853fff700 1 --2- 172.21.15.19:0/2806050843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd85410c7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8440098a0 tx=0x7fd844009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d384b7910e4fac5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.448+0000 7fd852ffd700 1 -- 172.21.15.19:0/2806050843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844004160 con 0x7fd85407ccc0 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.448+0000 7fd852ffd700 1 -- 172.21.15.19:0/2806050843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd844004300 con 0x7fd85407ccc0 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.448+0000 7fd852ffd700 1 -- 172.21.15.19:0/2806050843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844011af0 con 0x7fd85407ccc0 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.449+0000 7fd85a58c700 1 -- 172.21.15.19:0/2806050843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 msgr2=0x7fd85410c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.449+0000 7fd85a58c700 1 --2- 172.21.15.19:0/2806050843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd85410c7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8440098a0 tx=0x7fd844009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.449+0000 7fd85a58c700 1 -- 172.21.15.19:0/2806050843 shutdown_connections 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.449+0000 7fd85a58c700 1 --2- 172.21.15.19:0/2806050843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd85410c7d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.449+0000 7fd85a58c700 1 -- 172.21.15.19:0/2806050843 >> 172.21.15.19:0/2806050843 conn(0x7fd8541041e0 msgr2=0x7fd854106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:47.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.449+0000 7fd85a58c700 1 -- 172.21.15.19:0/2806050843 shutdown_connections 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.449+0000 7fd85a58c700 1 -- 172.21.15.19:0/2806050843 wait complete. 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.450+0000 7fd85a58c700 1 Processor -- start 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.450+0000 7fd85a58c700 1 -- start start 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.450+0000 7fd85a58c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd854109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.450+0000 7fd85a58c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd85410dfe0 con 0x7fd85407ccc0 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.450+0000 7fd853fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd854109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.450+0000 7fd853fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd854109f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57882/0 (socket says 172.21.15.19:57882) 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.450+0000 7fd853fff700 1 -- 172.21.15.19:0/2166813264 learned_addr learned my addr 172.21.15.19:0/2166813264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.451+0000 7fd853fff700 1 -- 172.21.15.19:0/2166813264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85410a480 con 0x7fd85407ccc0 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.451+0000 7fd853fff700 1 --2- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd854109f40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd844009fb0 tx=0x7fd844011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.451+0000 7fd8517fa700 1 -- 172.21.15.19:0/2166813264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844011bc0 con 0x7fd85407ccc0 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.451+0000 7fd8517fa700 1 -- 172.21.15.19:0/2166813264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd844024440 con 0x7fd85407ccc0 2024-05-09T01:14:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.451+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8541086d0 con 0x7fd85407ccc0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.451+0000 7fd8517fa700 1 -- 172.21.15.19:0/2166813264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd84401a5c0 con 0x7fd85407ccc0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.451+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd854108bb0 con 0x7fd85407ccc0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.452+0000 7fd8517fa700 1 -- 172.21.15.19:0/2166813264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fd84401a830 con 0x7fd85407ccc0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.452+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd854064e20 con 0x7fd85407ccc0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.452+0000 7fd8517fa700 1 --2- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7fd83c03e420 0x7fd83c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.452+0000 7fd8517fa700 1 -- 172.21.15.19:0/2166813264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd844053760 con 0x7fd85407ccc0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.452+0000 7fd8537fe700 1 --2- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7fd83c03e420 0x7fd83c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.453+0000 7fd8537fe700 1 --2- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7fd83c03e420 0x7fd83c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd848006fd0 tx=0x7fd848006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.458+0000 7fd8517fa700 1 -- 172.21.15.19:0/2166813264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd84400b3e0 con 0x7fd85407ccc0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.738+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd854109330 con 0x7fd85407ccc0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.790+0000 7fd8517fa700 1 -- 172.21.15.19:0/2166813264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd844024730 con 0x7fd85407ccc0 2024-05-09T01:14:47.040 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.792+0000 7fd8517fa700 1 -- 172.21.15.19:0/2166813264 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fd84402d3f0 con 0x7fd85407ccc0 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7fd83c03e420 msgr2=0x7fd83c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 --2- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7fd83c03e420 0x7fd83c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd848006fd0 tx=0x7fd848006e40 comp rx=0 tx=0).stop 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 msgr2=0x7fd854109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 --2- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd854109f40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd844009fb0 tx=0x7fd844011890 comp rx=0 tx=0).stop 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 shutdown_connections 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 --2- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7fd83c03e420 0x7fd83c040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 --2- 172.21.15.19:0/2166813264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85407ccc0 0x7fd854109f40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 >> 172.21.15.19:0/2166813264 conn(0x7fd8541041e0 msgr2=0x7fd854105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 shutdown_connections 2024-05-09T01:14:47.041 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:45.793+0000 7fd85a58c700 1 -- 172.21.15.19:0/2166813264 wait complete. 2024-05-09T01:14:48.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:47 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/876733340' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T01:14:48.386 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:48 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:48.067+0000 7f2daaad9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T01:14:48.386 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:48 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:48.251+0000 7f2daaad9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T01:14:48.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:14:48.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T01:14:48.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T01:14:48.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.487+0000 7f1f987fc700 1 Processor -- start 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.487+0000 7f1f987fc700 1 -- start start 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.487+0000 7f1f987fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f9010cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.487+0000 7f1f987fc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f900770f0 con 0x7f1f9010a7e0 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.487+0000 7f1f96598700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f9010cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.487+0000 7f1f96598700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f9010cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57898/0 (socket says 172.21.15.19:57898) 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.487+0000 7f1f96598700 1 -- 172.21.15.19:0/2207173958 learned_addr learned my addr 172.21.15.19:0/2207173958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.488+0000 7f1f96598700 1 -- 172.21.15.19:0/2207173958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9010d140 con 0x7f1f9010a7e0 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.488+0000 7f1f96598700 1 --2- 172.21.15.19:0/2207173958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f9010cc00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009e80 tx=0x7f1f8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f6e1c876e89265 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.488+0000 7f1f95596700 1 -- 172.21.15.19:0/2207173958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f8c017070 con 0x7f1f9010a7e0 2024-05-09T01:14:48.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.488+0000 7f1f95596700 1 -- 172.21.15.19:0/2207173958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f8c00b8c0 con 0x7f1f9010a7e0 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.488+0000 7f1f95596700 1 -- 172.21.15.19:0/2207173958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f8c004490 con 0x7f1f9010a7e0 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.489+0000 7f1f987fc700 1 -- 172.21.15.19:0/2207173958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 msgr2=0x7f1f9010cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.489+0000 7f1f987fc700 1 --2- 172.21.15.19:0/2207173958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f9010cc00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009e80 tx=0x7f1f8c00b330 comp rx=0 tx=0).stop 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.489+0000 7f1f987fc700 1 -- 172.21.15.19:0/2207173958 shutdown_connections 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.489+0000 7f1f987fc700 1 --2- 172.21.15.19:0/2207173958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f9010cc00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.489+0000 7f1f987fc700 1 -- 172.21.15.19:0/2207173958 >> 172.21.15.19:0/2207173958 conn(0x7f1f90104240 msgr2=0x7f1f90106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.489+0000 7f1f987fc700 1 -- 172.21.15.19:0/2207173958 shutdown_connections 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.489+0000 7f1f987fc700 1 -- 172.21.15.19:0/2207173958 wait complete. 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.490+0000 7f1f987fc700 1 Processor -- start 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.490+0000 7f1f987fc700 1 -- start start 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.490+0000 7f1f987fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f90109f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.490+0000 7f1f987fc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9010e0a0 con 0x7f1f9010a7e0 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.490+0000 7f1f96598700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f90109f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.490+0000 7f1f96598700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f90109f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57914/0 (socket says 172.21.15.19:57914) 2024-05-09T01:14:48.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.490+0000 7f1f96598700 1 -- 172.21.15.19:0/876733340 learned_addr learned my addr 172.21.15.19:0/876733340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.490+0000 7f1f96598700 1 -- 172.21.15.19:0/876733340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9010a4a0 con 0x7f1f9010a7e0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.490+0000 7f1f96598700 1 --2- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f90109f60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c004220 tx=0x7f1f8c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.491+0000 7f1f877fe700 1 -- 172.21.15.19:0/876733340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f8c020020 con 0x7f1f9010a7e0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.491+0000 7f1f877fe700 1 -- 172.21.15.19:0/876733340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f8c012430 con 0x7f1f9010a7e0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.491+0000 7f1f877fe700 1 -- 172.21.15.19:0/876733340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f8c017040 con 0x7f1f9010a7e0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.491+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f901086f0 con 0x7f1f9010a7e0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.491+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f90108c30 con 0x7f1f9010a7e0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.492+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f90064e20 con 0x7f1f9010a7e0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.492+0000 7f1f877fe700 1 -- 172.21.15.19:0/876733340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f1f8c023020 con 0x7f1f9010a7e0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.492+0000 7f1f877fe700 1 --2- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f1f7c03e470 0x7f1f7c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.492+0000 7f1f877fe700 1 -- 172.21.15.19:0/876733340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f8c00f050 con 0x7f1f9010a7e0 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.492+0000 7f1f95d97700 1 -- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f1f7c03e470 msgr2=0x7f1f7c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3332485717 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.492+0000 7f1f95d97700 1 --2- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f1f7c03e470 0x7f1f7c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T01:14:48.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.498+0000 7f1f877fe700 1 -- 172.21.15.19:0/876733340 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f8c02a020 con 0x7f1f9010a7e0 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.693+0000 7f1f95d97700 1 -- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f1f7c03e470 msgr2=0x7f1f7c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3332485717 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.693+0000 7f1f95d97700 1 --2- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f1f7c03e470 0x7f1f7c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.772+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1f90108e00 con 0x7f1f9010a7e0 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.772+0000 7f1f877fe700 1 -- 172.21.15.19:0/876733340 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1f8c02ade0 con 0x7f1f9010a7e0 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f1f7c03e470 msgr2=0x7f1f7c040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 --2- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f1f7c03e470 0x7f1f7c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 msgr2=0x7f1f90109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 --2- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f90109f60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c004220 tx=0x7f1f8c00bd50 comp rx=0 tx=0).stop 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 shutdown_connections 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 --2- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f1f7c03e470 0x7f1f7c040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 --2- 172.21.15.19:0/876733340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010a7e0 0x7f1f90109f60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 >> 172.21.15.19:0/876733340 conn(0x7f1f90104240 msgr2=0x7f1f90105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:48.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 shutdown_connections 2024-05-09T01:14:48.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:47.775+0000 7f1f987fc700 1 -- 172.21.15.19:0/876733340 wait complete. 2024-05-09T01:14:48.399 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-05-09T01:14:48.399 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-05-09T01:14:48.639 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:48 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:48.435+0000 7f2daaad9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T01:14:48.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:48 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:48.750+0000 7f2daaad9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T01:14:48.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:48 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:48.854+0000 7f2daaad9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T01:14:49.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:49 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:49.069+0000 7f2daaad9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T01:14:49.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:49 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:49.274+0000 7f2daaad9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:14:50.278 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:49 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:49.912+0000 7f2daaad9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T01:14:50.278 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:50 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:50.146+0000 7f2daaad9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T01:14:50.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:50 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:50.595+0000 7f2daaad9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T01:14:50.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:50 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:50.705+0000 7f2daaad9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T01:14:51.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:51 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:51.093+0000 7f2daaad9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T01:14:51.836 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:51 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:51.477+0000 7f2daaad9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T01:14:51.837 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:51 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:51.576+0000 7f2daaad9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T01:14:51.837 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:51 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:51.835+0000 7f2daaad9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T01:14:52.148 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:51 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:51.978+0000 7f2daaad9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T01:14:52.148 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:52 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:52.080+0000 7f2daaad9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T01:14:52.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:52 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:52.422+0000 7f2daaad9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T01:14:52.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:52 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:52.520+0000 7f2daaad9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T01:14:53.444 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:53 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:53.158+0000 7f2daaad9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T01:14:53.444 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:53 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:14:53.390+0000 7f2daaad9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T01:14:53.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: Active manager daemon x restarted 2024-05-09T01:14:53.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: Activating manager daemon x 2024-05-09T01:14:53.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T01:14:53.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: mgrmap e6: x(active, starting, since 0.00405734s) 2024-05-09T01:14:53.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:14:53.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T01:14:53.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T01:14:53.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T01:14:53.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T01:14:53.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: Manager daemon x is now available 2024-05-09T01:14:53.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:14:53.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:14:53.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:14:53.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:53 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:14:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:54 smithi019 ceph-mon[30158]: Found migration_current of "None". Setting to last migration. 2024-05-09T01:14:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:54 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T01:14:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:54 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T01:14:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:54 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:14:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:54 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:14:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:54 smithi019 ceph-mon[30158]: mgrmap e7: x(active, since 1.00689s) 2024-05-09T01:14:54.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:14:54.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T01:14:54.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T01:14:54.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:14:54.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.868+0000 7f5d874cb700 1 Processor -- start 2024-05-09T01:14:54.994 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.869+0000 7f5d874cb700 1 -- start start 2024-05-09T01:14:54.994 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.869+0000 7f5d874cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8010cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:54.994 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.869+0000 7f5d874cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80077110 con 0x7f5d8010a7c0 2024-05-09T01:14:54.994 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.869+0000 7f5d85267700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8010cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:54.994 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.869+0000 7f5d85267700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8010cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57920/0 (socket says 172.21.15.19:57920) 2024-05-09T01:14:54.994 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.869+0000 7f5d85267700 1 -- 172.21.15.19:0/1501454756 learned_addr learned my addr 172.21.15.19:0/1501454756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:54.994 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.869+0000 7f5d85267700 1 -- 172.21.15.19:0/1501454756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8010d120 con 0x7f5d8010a7c0 2024-05-09T01:14:54.994 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.870+0000 7f5d85267700 1 --2- 172.21.15.19:0/1501454756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8010cbe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0098a0 tx=0x7f5d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2d8a489cf07568 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:54.994 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.870+0000 7f5d77fff700 1 -- 172.21.15.19:0/1501454756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d7c004160 con 0x7f5d8010a7c0 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.870+0000 7f5d77fff700 1 -- 172.21.15.19:0/1501454756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5d7c004300 con 0x7f5d8010a7c0 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.870+0000 7f5d77fff700 1 -- 172.21.15.19:0/1501454756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d7c011af0 con 0x7f5d8010a7c0 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.871+0000 7f5d874cb700 1 -- 172.21.15.19:0/1501454756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 msgr2=0x7f5d8010cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.871+0000 7f5d874cb700 1 --2- 172.21.15.19:0/1501454756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8010cbe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0098a0 tx=0x7f5d7c009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.871+0000 7f5d874cb700 1 -- 172.21.15.19:0/1501454756 shutdown_connections 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.871+0000 7f5d874cb700 1 --2- 172.21.15.19:0/1501454756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8010cbe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.871+0000 7f5d874cb700 1 -- 172.21.15.19:0/1501454756 >> 172.21.15.19:0/1501454756 conn(0x7f5d80104240 msgr2=0x7f5d80106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.871+0000 7f5d874cb700 1 -- 172.21.15.19:0/1501454756 shutdown_connections 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.871+0000 7f5d874cb700 1 -- 172.21.15.19:0/1501454756 wait complete. 2024-05-09T01:14:54.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.871+0000 7f5d874cb700 1 Processor -- start 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.872+0000 7f5d874cb700 1 -- start start 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.872+0000 7f5d874cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.872+0000 7f5d874cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8010e080 con 0x7f5d8010a7c0 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.872+0000 7f5d85267700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.872+0000 7f5d85267700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57936/0 (socket says 172.21.15.19:57936) 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.872+0000 7f5d85267700 1 -- 172.21.15.19:0/2389272661 learned_addr learned my addr 172.21.15.19:0/2389272661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.872+0000 7f5d85267700 1 -- 172.21.15.19:0/2389272661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8019e5a0 con 0x7f5d8010a7c0 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.872+0000 7f5d85267700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8019e060 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0117b0 tx=0x7f5d7c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.873+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d7c003af0 con 0x7f5d8010a7c0 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.873+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5d7c020070 con 0x7f5d8010a7c0 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.873+0000 7f5d874cb700 1 -- 172.21.15.19:0/2389272661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8019e830 con 0x7f5d8010a7c0 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.873+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d7c019630 con 0x7f5d8010a7c0 2024-05-09T01:14:54.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.873+0000 7f5d874cb700 1 -- 172.21.15.19:0/2389272661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801a1440 con 0x7f5d8010a7c0 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.874+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f5d7c0198f0 con 0x7f5d8010a7c0 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.874+0000 7f5d767fc700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 0x7f5d6c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.874+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 --> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5d6c041090 con 0x7f5d6c03e470 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.874+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5d7c053ad0 con 0x7f5d8010a7c0 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.874+0000 7f5d84a66700 1 -- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 msgr2=0x7f5d6c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3332485717 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:48.874+0000 7f5d84a66700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 0x7f5d6c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:49.075+0000 7f5d84a66700 1 -- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 msgr2=0x7f5d6c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3332485717 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:49.075+0000 7f5d84a66700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 0x7f5d6c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:49.476+0000 7f5d84a66700 1 -- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 msgr2=0x7f5d6c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3332485717 2024-05-09T01:14:54.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:49.476+0000 7f5d84a66700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 0x7f5d6c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:50.277+0000 7f5d84a66700 1 -- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 msgr2=0x7f5d6c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3332485717 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:50.277+0000 7f5d84a66700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 0x7f5d6c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:51.878+0000 7f5d84a66700 1 -- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 msgr2=0x7f5d6c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3332485717 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:51.878+0000 7f5d84a66700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 0x7f5d6c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:53.394+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f5d7c019c30 con 0x7f5d8010a7c0 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:53.394+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 msgr2=0x7f5d6c040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:53.394+0000 7f5d767fc700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 0x7f5d6c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.395+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f5d7c055b30 con 0x7f5d8010a7c0 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.395+0000 7f5d767fc700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f5d6c041db0 0x7f5d6c0441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.395+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5d6c041090 con 0x7f5d6c041db0 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.396+0000 7f5d84a66700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f5d6c041db0 0x7f5d6c0441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:54.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.397+0000 7f5d84a66700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f5d6c041db0 0x7f5d6c0441d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d70003a10 tx=0x7f5d70009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.397+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 <== mgr.14118 v2:172.21.15.19:6800/112627556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5d6c041090 con 0x7f5d6c041db0 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 -- 172.21.15.19:0/2389272661 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5d80064e20 con 0x7f5d6c041db0 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d767fc700 1 -- 172.21.15.19:0/2389272661 <== mgr.14118 v2:172.21.15.19:6800/112627556 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5d80064e20 con 0x7f5d6c041db0 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 -- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f5d6c041db0 msgr2=0x7f5d6c0441d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f5d6c041db0 0x7f5d6c0441d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d70003a10 tx=0x7f5d70009290 comp rx=0 tx=0).stop 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 -- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 msgr2=0x7f5d8019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8019e060 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0117b0 tx=0x7f5d7c011b70 comp rx=0 tx=0).stop 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 -- 172.21.15.19:0/2389272661 shutdown_connections 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f5d6c041db0 0x7f5d6c0441d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8010a7c0 0x7f5d8019e060 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 --2- 172.21.15.19:0/2389272661 >> [v2:172.21.15.19:6800/3332485717,v1:172.21.15.19:6801/3332485717] conn(0x7f5d6c03e470 0x7f5d6c040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 -- 172.21.15.19:0/2389272661 >> 172.21.15.19:0/2389272661 conn(0x7f5d80104240 msgr2=0x7f5d80105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:54.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 -- 172.21.15.19:0/2389272661 shutdown_connections 2024-05-09T01:14:55.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:54.406+0000 7f5d874cb700 1 -- 172.21.15.19:0/2389272661 wait complete. 2024-05-09T01:14:55.000 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-05-09T01:14:55.000 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-05-09T01:14:55.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:55 smithi019 ceph-mon[30158]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T01:14:55.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:55 smithi019 ceph-mon[30158]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T01:14:55.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:55 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:14:56.205 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.393+0000 7f72a5431700 1 Processor -- start 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.393+0000 7f72a5431700 1 -- start start 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.393+0000 7f72a5431700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a010cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.393+0000 7f72a5431700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0077170 con 0x7f72a010a760 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.394+0000 7f729effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a010cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.394+0000 7f729effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a010cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42324/0 (socket says 172.21.15.19:42324) 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.394+0000 7f729effd700 1 -- 172.21.15.19:0/4271181988 learned_addr learned my addr 172.21.15.19:0/4271181988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.394+0000 7f729effd700 1 -- 172.21.15.19:0/4271181988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a010d0c0 con 0x7f72a010a760 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.394+0000 7f729effd700 1 --2- 172.21.15.19:0/4271181988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a010cb80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f72900098a0 tx=0x7f7290009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116dd1ce3c1ba4b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.394+0000 7f729e7fc700 1 -- 172.21.15.19:0/4271181988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7290004160 con 0x7f72a010a760 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.394+0000 7f729e7fc700 1 -- 172.21.15.19:0/4271181988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7290004300 con 0x7f72a010a760 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.394+0000 7f729e7fc700 1 -- 172.21.15.19:0/4271181988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7290011af0 con 0x7f72a010a760 2024-05-09T01:14:56.206 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.395+0000 7f72a5431700 1 -- 172.21.15.19:0/4271181988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 msgr2=0x7f72a010cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.395+0000 7f72a5431700 1 --2- 172.21.15.19:0/4271181988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a010cb80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f72900098a0 tx=0x7f7290009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.395+0000 7f72a5431700 1 -- 172.21.15.19:0/4271181988 shutdown_connections 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.395+0000 7f72a5431700 1 --2- 172.21.15.19:0/4271181988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a010cb80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.395+0000 7f72a5431700 1 -- 172.21.15.19:0/4271181988 >> 172.21.15.19:0/4271181988 conn(0x7f72a0104240 msgr2=0x7f72a0106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.395+0000 7f72a5431700 1 -- 172.21.15.19:0/4271181988 shutdown_connections 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.395+0000 7f72a5431700 1 -- 172.21.15.19:0/4271181988 wait complete. 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.396+0000 7f72a5431700 1 Processor -- start 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.396+0000 7f72a5431700 1 -- start start 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.396+0000 7f72a5431700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.396+0000 7f72a5431700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72a010e080 con 0x7f72a010a760 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.396+0000 7f729effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.396+0000 7f729effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a019e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42334/0 (socket says 172.21.15.19:42334) 2024-05-09T01:14:56.207 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.396+0000 7f729effd700 1 -- 172.21.15.19:0/1899887384 learned_addr learned my addr 172.21.15.19:0/1899887384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.397+0000 7f729effd700 1 -- 172.21.15.19:0/1899887384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a019e570 con 0x7f72a010a760 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.397+0000 7f729effd700 1 --2- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a019e030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f72900117b0 tx=0x7f7290011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.397+0000 7f729cff9700 1 -- 172.21.15.19:0/1899887384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7290003af0 con 0x7f72a010a760 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.397+0000 7f729cff9700 1 -- 172.21.15.19:0/1899887384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7290020070 con 0x7f72a010a760 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.397+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a019e800 con 0x7f72a010a760 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.397+0000 7f729cff9700 1 -- 172.21.15.19:0/1899887384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7290019630 con 0x7f72a010a760 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.397+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a01a1410 con 0x7f72a010a760 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.398+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a0064e20 con 0x7f72a010a760 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.398+0000 7f729cff9700 1 -- 172.21.15.19:0/1899887384 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f72900198a0 con 0x7f72a010a760 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.398+0000 7f729cff9700 1 --2- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f728c03e3a0 0x7f728c0408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.398+0000 7f729cff9700 1 -- 172.21.15.19:0/1899887384 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7290053b00 con 0x7f72a010a760 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.399+0000 7f72967fc700 1 --2- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f728c03e3a0 0x7f728c0408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.399+0000 7f72967fc700 1 --2- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f728c03e3a0 0x7f728c0408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:56.208 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.404+0000 7f729cff9700 1 -- 172.21.15.19:0/1899887384 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7290018d60 con 0x7f72a010a760 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.447+0000 7f729cff9700 1 -- 172.21.15.19:0/1899887384 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f729002d350 con 0x7f72a010a760 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.641+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f72a0106250 con 0x7f728c03e3a0 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.646+0000 7f729cff9700 1 -- 172.21.15.19:0/1899887384 <== mgr.14118 v2:172.21.15.19:6800/112627556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f72a0106250 con 0x7f728c03e3a0 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f728c03e3a0 msgr2=0x7f728c0408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 --2- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f728c03e3a0 0x7f728c0408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288006e40 comp rx=0 tx=0).stop 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 msgr2=0x7f72a019e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 --2- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a019e030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f72900117b0 tx=0x7f7290011b70 comp rx=0 tx=0).stop 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 shutdown_connections 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 --2- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f728c03e3a0 0x7f728c0408b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 --2- 172.21.15.19:0/1899887384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a010a760 0x7f72a019e030 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 >> 172.21.15.19:0/1899887384 conn(0x7f72a0104240 msgr2=0x7f72a0105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 shutdown_connections 2024-05-09T01:14:56.209 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:55.649+0000 7f72a5431700 1 -- 172.21.15.19:0/1899887384 wait complete. 2024-05-09T01:14:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:56 smithi019 ceph-mon[30158]: [09/May/2024:01:14:55] ENGINE Bus STARTING 2024-05-09T01:14:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:56 smithi019 ceph-mon[30158]: [09/May/2024:01:14:55] ENGINE Serving on http://172.21.15.19:8765 2024-05-09T01:14:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:56 smithi019 ceph-mon[30158]: [09/May/2024:01:14:55] ENGINE Serving on https://172.21.15.19:7150 2024-05-09T01:14:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:56 smithi019 ceph-mon[30158]: [09/May/2024:01:14:55] ENGINE Bus STARTED 2024-05-09T01:14:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:56 smithi019 ceph-mon[30158]: mgrmap e8: x(active, since 2s) 2024-05-09T01:14:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:56 smithi019 ceph-mon[30158]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:14:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:56 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:14:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:56 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:14:57.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T01:14:57.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.611+0000 7f6c020f9700 1 Processor -- start 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.612+0000 7f6c020f9700 1 -- start start 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.612+0000 7f6c020f9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.612+0000 7f6c020f9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc10bc30 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.612+0000 7f6bfb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.612+0000 7f6bfb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42342/0 (socket says 172.21.15.19:42342) 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.612+0000 7f6bfb7fe700 1 -- 172.21.15.19:0/418652060 learned_addr learned my addr 172.21.15.19:0/418652060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.612+0000 7f6bfb7fe700 1 -- 172.21.15.19:0/418652060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc107c40 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.612+0000 7f6bfb7fe700 1 --2- 172.21.15.19:0/418652060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10b620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0098a0 tx=0x7f6bec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405ca0f501252e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.613+0000 7f6bfaffd700 1 -- 172.21.15.19:0/418652060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bec004160 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.613+0000 7f6bfaffd700 1 -- 172.21.15.19:0/418652060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6bec004300 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.613+0000 7f6bfaffd700 1 -- 172.21.15.19:0/418652060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bec011af0 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.613+0000 7f6c020f9700 1 -- 172.21.15.19:0/418652060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 msgr2=0x7f6bfc10b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.613+0000 7f6c020f9700 1 --2- 172.21.15.19:0/418652060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10b620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0098a0 tx=0x7f6bec009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.613+0000 7f6c020f9700 1 -- 172.21.15.19:0/418652060 shutdown_connections 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.613+0000 7f6c020f9700 1 --2- 172.21.15.19:0/418652060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10b620 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.613+0000 7f6c020f9700 1 -- 172.21.15.19:0/418652060 >> 172.21.15.19:0/418652060 conn(0x7f6bfc103930 msgr2=0x7f6bfc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.614+0000 7f6c020f9700 1 -- 172.21.15.19:0/418652060 shutdown_connections 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.614+0000 7f6c020f9700 1 -- 172.21.15.19:0/418652060 wait complete. 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.614+0000 7f6c020f9700 1 Processor -- start 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.614+0000 7f6c020f9700 1 -- start start 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.614+0000 7f6c020f9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.614+0000 7f6c020f9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc10e190 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6bfb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:57.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6bfb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42352/0 (socket says 172.21.15.19:42352) 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6bfb7fe700 1 -- 172.21.15.19:0/2518362102 learned_addr learned my addr 172.21.15.19:0/2518362102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6bfb7fe700 1 -- 172.21.15.19:0/2518362102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc10ac20 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6bfb7fe700 1 --2- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10a6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0117b0 tx=0x7f6bec011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6bf97fa700 1 -- 172.21.15.19:0/2518362102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bec003af0 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6bf97fa700 1 -- 172.21.15.19:0/2518362102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6bec020070 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc108e70 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6bf97fa700 1 -- 172.21.15.19:0/2518362102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bec019630 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.615+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc109350 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.616+0000 7f6bf97fa700 1 -- 172.21.15.19:0/2518362102 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6bec0198a0 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.616+0000 7f6bf97fa700 1 --2- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f6be803e4c0 0x7f6be80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.616+0000 7f6bf97fa700 1 -- 172.21.15.19:0/2518362102 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6bec054980 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.616+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bfc064e20 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.617+0000 7f6bf3fff700 1 --2- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f6be803e4c0 0x7f6be80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.617+0000 7f6bf3fff700 1 --2- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f6be803e4c0 0x7f6be80409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6be4006fd0 tx=0x7f6be4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.622+0000 7f6bf97fa700 1 -- 172.21.15.19:0/2518362102 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6bec016260 con 0x7f6bfc10b1f0 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.839+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6bfc109a80 con 0x7f6be803e4c0 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.840+0000 7f6bf97fa700 1 -- 172.21.15.19:0/2518362102 <== mgr.14118 v2:172.21.15.19:6800/112627556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6bfc109a80 con 0x7f6be803e4c0 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.842+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f6be803e4c0 msgr2=0x7f6be80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.842+0000 7f6c020f9700 1 --2- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f6be803e4c0 0x7f6be80409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6be4006fd0 tx=0x7f6be4006e40 comp rx=0 tx=0).stop 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.842+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 msgr2=0x7f6bfc10a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.842+0000 7f6c020f9700 1 --2- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10a6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0117b0 tx=0x7f6bec011b70 comp rx=0 tx=0).stop 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.843+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 shutdown_connections 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.843+0000 7f6c020f9700 1 --2- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f6be803e4c0 0x7f6be80409d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.843+0000 7f6c020f9700 1 --2- 172.21.15.19:0/2518362102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc10b1f0 0x7f6bfc10a6e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.843+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 >> 172.21.15.19:0/2518362102 conn(0x7f6bfc103930 msgr2=0x7f6bfc104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.843+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 shutdown_connections 2024-05-09T01:14:57.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:56.843+0000 7f6c020f9700 1 -- 172.21.15.19:0/2518362102 wait complete. 2024-05-09T01:14:57.394 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-05-09T01:14:57.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:57 smithi019 ceph-mon[30158]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:14:58.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: Generating public/private rsa key pair. 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: Your identification has been saved in /tmp/tmpvt2pzj5s/key. 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: Your public key has been saved in /tmp/tmpvt2pzj5s/key.pub. 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: The key fingerprint is: 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: SHA256:nQsrWZNSCwQnr7Ims7LnlTtnnvlEUYhmwXZiTDb2Ccs ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: The key's randomart image is: 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: +---[RSA 3072]----+ 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: | o=Xo .. | 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: | OX=oo | 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: | =Eo= | 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: | . o = . | 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: | . . . S o | 2024-05-09T01:14:58.399 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: | o . = + . | 2024-05-09T01:14:58.400 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: |o o o o o . | 2024-05-09T01:14:58.400 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: |.=....o= | 2024-05-09T01:14:58.400 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: |++. .=+.. | 2024-05-09T01:14:58.400 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:14:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: +----[SHA256]-----+ 2024-05-09T01:14:58.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.834+0000 7fa481be8700 1 Processor -- start 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.835+0000 7fa481be8700 1 -- start start 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.835+0000 7fa481be8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.835+0000 7fa481be8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c109580 con 0x7fa47c108b80 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.835+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.835+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42358/0 (socket says 172.21.15.19:42358) 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.835+0000 7fa47b7fe700 1 -- 172.21.15.19:0/1253279140 learned_addr learned my addr 172.21.15.19:0/1253279140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.835+0000 7fa47b7fe700 1 -- 172.21.15.19:0/1253279140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c109d50 con 0x7fa47c108b80 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.836+0000 7fa47b7fe700 1 --2- 172.21.15.19:0/1253279140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c108f90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0098a0 tx=0x7fa46c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55d4e418c06da3f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.836+0000 7fa47affd700 1 -- 172.21.15.19:0/1253279140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa46c004160 con 0x7fa47c108b80 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.836+0000 7fa47affd700 1 -- 172.21.15.19:0/1253279140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa46c004300 con 0x7fa47c108b80 2024-05-09T01:14:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.836+0000 7fa47affd700 1 -- 172.21.15.19:0/1253279140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa46c011af0 con 0x7fa47c108b80 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.837+0000 7fa481be8700 1 -- 172.21.15.19:0/1253279140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 msgr2=0x7fa47c108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.837+0000 7fa481be8700 1 --2- 172.21.15.19:0/1253279140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c108f90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0098a0 tx=0x7fa46c009bb0 comp rx=0 tx=0).stop 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.837+0000 7fa481be8700 1 -- 172.21.15.19:0/1253279140 shutdown_connections 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.837+0000 7fa481be8700 1 --2- 172.21.15.19:0/1253279140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c108f90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.837+0000 7fa481be8700 1 -- 172.21.15.19:0/1253279140 >> 172.21.15.19:0/1253279140 conn(0x7fa47c104240 msgr2=0x7fa47c106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.837+0000 7fa481be8700 1 -- 172.21.15.19:0/1253279140 shutdown_connections 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.837+0000 7fa481be8700 1 -- 172.21.15.19:0/1253279140 wait complete. 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.837+0000 7fa481be8700 1 Processor -- start 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.838+0000 7fa481be8700 1 -- start start 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.838+0000 7fa481be8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:58.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.838+0000 7fa481be8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c10d100 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.838+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.838+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c19e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42360/0 (socket says 172.21.15.19:42360) 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.838+0000 7fa47b7fe700 1 -- 172.21.15.19:0/3916926284 learned_addr learned my addr 172.21.15.19:0/3916926284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.838+0000 7fa47b7fe700 1 -- 172.21.15.19:0/3916926284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c19e630 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.839+0000 7fa47b7fe700 1 --2- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c19e0f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa46c011890 tx=0x7fa46c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.839+0000 7fa4797fa700 1 -- 172.21.15.19:0/3916926284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa46c0039e0 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.839+0000 7fa4797fa700 1 -- 172.21.15.19:0/3916926284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa46c003b80 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.839+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47c19e8c0 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.839+0000 7fa4797fa700 1 -- 172.21.15.19:0/3916926284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa46c0246e0 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.839+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47c1a14d0 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.840+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47c064e20 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.840+0000 7fa4797fa700 1 -- 172.21.15.19:0/3916926284 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa46c022020 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.840+0000 7fa4797fa700 1 --2- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fa46803e4c0 0x7fa4680409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.841+0000 7fa4797fa700 1 -- 172.21.15.19:0/3916926284 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa46c0536f0 con 0x7fa47c108b80 2024-05-09T01:14:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.841+0000 7fa472ffd700 1 --2- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fa46803e4c0 0x7fa4680409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.841+0000 7fa472ffd700 1 --2- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fa46803e4c0 0x7fa4680409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa464006fd0 tx=0x7fa464006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:57.846+0000 7fa4797fa700 1 -- 172.21.15.19:0/3916926284 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa46c020e00 con 0x7fa47c108b80 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.061+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa47c063c00 con 0x7fa46803e4c0 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.223+0000 7fa4797fa700 1 -- 172.21.15.19:0/3916926284 <== mgr.14118 v2:172.21.15.19:6800/112627556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa47c063c00 con 0x7fa46803e4c0 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.225+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fa46803e4c0 msgr2=0x7fa4680409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.225+0000 7fa481be8700 1 --2- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fa46803e4c0 0x7fa4680409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa464006fd0 tx=0x7fa464006e40 comp rx=0 tx=0).stop 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.225+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 msgr2=0x7fa47c19e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.225+0000 7fa481be8700 1 --2- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c19e0f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa46c011890 tx=0x7fa46c011970 comp rx=0 tx=0).stop 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.225+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 shutdown_connections 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.225+0000 7fa481be8700 1 --2- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fa46803e4c0 0x7fa4680409d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.225+0000 7fa481be8700 1 --2- 172.21.15.19:0/3916926284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c108b80 0x7fa47c19e0f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.225+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 >> 172.21.15.19:0/3916926284 conn(0x7fa47c104240 msgr2=0x7fa47c194680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.226+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 shutdown_connections 2024-05-09T01:14:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:58.226+0000 7fa481be8700 1 -- 172.21.15.19:0/3916926284 wait complete. 2024-05-09T01:14:59.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:59 smithi019 ceph-mon[30158]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:14:59.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:59 smithi019 ceph-mon[30158]: Generating ssh key... 2024-05-09T01:14:59.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:59 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:14:59.449 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:14:59 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:15:00.018 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCek/d7Cv3G0HaQRvk2oFaITO+1xWOHbqX2no2KAQHesROhGeBH7FdUITDfc/irK2SDlnYmwaS8X+PDx69L634pZzB11EuUd6zIJYJLXV2U57HCPNijt8j4HMFNPfKLu2JH9YYG33yX7qJsnGBmwfOMTzCveXjuQAeG9X+GOAnh0uxQCAPYWCF8tMJzACdf6DUlALRozoml9DpzUmuBf2ynyUz7A38f+1NE2SCEFaVkqwHIUGY1LrKf3jKc2/iuTgLC8rc8vTu5hoAlx8hXsdaYorjKJYCtPCR86952X94sAQrcwufc6Ewf4mHXALVl0/7sT2w27ur9hx/B+zUr14hIjbEwpQVYXgmIc11aix7WrrWsA2MLI1fctcuVKSOua5JlJ4q5+cZlsy96Zld0W+8HxURS6RGIb9/6wES1dvWY23xclGOPazQcRzkaFdK1TQ2BEPr1eazdlmMT1CuR4CYmAXtjkkETeQBV04ZTIx29u7KjMqysJsXHUX0dBRQ4/nc= ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.219+0000 7f28b2032700 1 Processor -- start 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.219+0000 7f28b2032700 1 -- start start 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.219+0000 7f28b2032700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac10a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.219+0000 7f28b2032700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10a6d0 con 0x7f28ac109cd0 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.219+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac10a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.219+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac10a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42366/0 (socket says 172.21.15.19:42366) 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.219+0000 7f28ab7fe700 1 -- 172.21.15.19:0/364735402 learned_addr learned my addr 172.21.15.19:0/364735402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.220+0000 7f28ab7fe700 1 -- 172.21.15.19:0/364735402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac10aea0 con 0x7f28ac109cd0 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.220+0000 7f28ab7fe700 1 --2- 172.21.15.19:0/364735402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac10a0e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f289c0098a0 tx=0x7f289c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d7dff14a01606b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.221+0000 7f28aa7fc700 1 -- 172.21.15.19:0/364735402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f289c004160 con 0x7f28ac109cd0 2024-05-09T01:15:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.221+0000 7f28aa7fc700 1 -- 172.21.15.19:0/364735402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f289c004300 con 0x7f28ac109cd0 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.221+0000 7f28aa7fc700 1 -- 172.21.15.19:0/364735402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f289c011af0 con 0x7f28ac109cd0 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.222+0000 7f28b2032700 1 -- 172.21.15.19:0/364735402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 msgr2=0x7f28ac10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.222+0000 7f28b2032700 1 --2- 172.21.15.19:0/364735402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac10a0e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f289c0098a0 tx=0x7f289c009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.222+0000 7f28b2032700 1 -- 172.21.15.19:0/364735402 shutdown_connections 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.222+0000 7f28b2032700 1 --2- 172.21.15.19:0/364735402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac10a0e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.222+0000 7f28b2032700 1 -- 172.21.15.19:0/364735402 >> 172.21.15.19:0/364735402 conn(0x7f28ac105330 msgr2=0x7f28ac1077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.222+0000 7f28b2032700 1 -- 172.21.15.19:0/364735402 shutdown_connections 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.222+0000 7f28b2032700 1 -- 172.21.15.19:0/364735402 wait complete. 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.222+0000 7f28b2032700 1 Processor -- start 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.223+0000 7f28b2032700 1 -- start start 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.223+0000 7f28b2032700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac19e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.223+0000 7f28b2032700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10e0d0 con 0x7f28ac109cd0 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.223+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac19e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:00.020 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.223+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac19e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42374/0 (socket says 172.21.15.19:42374) 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.223+0000 7f28ab7fe700 1 -- 172.21.15.19:0/1454552966 learned_addr learned my addr 172.21.15.19:0/1454552966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.223+0000 7f28ab7fe700 1 -- 172.21.15.19:0/1454552966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac19e5e0 con 0x7f28ac109cd0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.223+0000 7f28ab7fe700 1 --2- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac19e0a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f289c011890 tx=0x7f289c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.223+0000 7f28a8ff9700 1 -- 172.21.15.19:0/1454552966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f289c0039e0 con 0x7f28ac109cd0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.224+0000 7f28a8ff9700 1 -- 172.21.15.19:0/1454552966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f289c003b80 con 0x7f28ac109cd0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.224+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ac19e870 con 0x7f28ac109cd0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.224+0000 7f28a8ff9700 1 -- 172.21.15.19:0/1454552966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f289c0246e0 con 0x7f28ac109cd0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.224+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ac1a1480 con 0x7f28ac109cd0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.224+0000 7f28a8ff9700 1 -- 172.21.15.19:0/1454552966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f289c022020 con 0x7f28ac109cd0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.225+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ac064e20 con 0x7f28ac109cd0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.225+0000 7f28a8ff9700 1 --2- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f289403e4c0 0x7f28940409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.225+0000 7f28a8ff9700 1 -- 172.21.15.19:0/1454552966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f289c053770 con 0x7f28ac109cd0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.225+0000 7f28aaffd700 1 --2- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f289403e4c0 0x7f28940409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:00.021 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.225+0000 7f28aaffd700 1 --2- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f289403e4c0 0x7f28940409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f28a0006fd0 tx=0x7f28a0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.230+0000 7f28a8ff9700 1 -- 172.21.15.19:0/1454552966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f289c02c350 con 0x7f28ac109cd0 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.448+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f28ac1063c0 con 0x7f289403e4c0 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.448+0000 7f28a8ff9700 1 -- 172.21.15.19:0/1454552966 <== mgr.14118 v2:172.21.15.19:6800/112627556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f28ac1063c0 con 0x7f289403e4c0 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.451+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f289403e4c0 msgr2=0x7f28940409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.451+0000 7f28b2032700 1 --2- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f289403e4c0 0x7f28940409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f28a0006fd0 tx=0x7f28a0006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.451+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 msgr2=0x7f28ac19e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.451+0000 7f28b2032700 1 --2- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac19e0a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f289c011890 tx=0x7f289c011970 comp rx=0 tx=0).stop 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.452+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 shutdown_connections 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.452+0000 7f28b2032700 1 --2- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f289403e4c0 0x7f28940409d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.452+0000 7f28b2032700 1 --2- 172.21.15.19:0/1454552966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ac109cd0 0x7f28ac19e0a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.452+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 >> 172.21.15.19:0/1454552966 conn(0x7f28ac105330 msgr2=0x7f28ac105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.452+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 shutdown_connections 2024-05-09T01:15:00.022 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:14:59.452+0000 7f28b2032700 1 -- 172.21.15.19:0/1454552966 wait complete. 2024-05-09T01:15:00.023 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T01:15:00.023 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T01:15:00.023 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-05-09T01:15:00.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:00 smithi019 ceph-mon[30158]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:15:01.470 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:01 smithi019 ceph-mon[30158]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:15:02.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:02 smithi019 ceph-mon[30158]: Deploying cephadm binary to smithi019 2024-05-09T01:15:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-05-09T01:15:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.423+0000 7fbc76910700 1 Processor -- start 2024-05-09T01:15:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.423+0000 7fbc76910700 1 -- start start 2024-05-09T01:15:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.423+0000 7fbc76910700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7010cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.423+0000 7fbc76910700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7007ce70 con 0x7fbc7010a750 2024-05-09T01:15:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.424+0000 7fbc6ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7010cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.424+0000 7fbc6ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7010cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33152/0 (socket says 172.21.15.19:33152) 2024-05-09T01:15:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.424+0000 7fbc6ffff700 1 -- 172.21.15.19:0/3811069812 learned_addr learned my addr 172.21.15.19:0/3811069812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:05.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.424+0000 7fbc6ffff700 1 -- 172.21.15.19:0/3811069812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7010d0b0 con 0x7fbc7010a750 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.424+0000 7fbc6ffff700 1 --2- 172.21.15.19:0/3811069812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7010cb70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc600098a0 tx=0x7fbc60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ff79c12f2abf27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.424+0000 7fbc6effd700 1 -- 172.21.15.19:0/3811069812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc60004160 con 0x7fbc7010a750 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.424+0000 7fbc6effd700 1 -- 172.21.15.19:0/3811069812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbc60004300 con 0x7fbc7010a750 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.424+0000 7fbc6effd700 1 -- 172.21.15.19:0/3811069812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc60011af0 con 0x7fbc7010a750 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.425+0000 7fbc76910700 1 -- 172.21.15.19:0/3811069812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 msgr2=0x7fbc7010cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.425+0000 7fbc76910700 1 --2- 172.21.15.19:0/3811069812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7010cb70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc600098a0 tx=0x7fbc60009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.425+0000 7fbc76910700 1 -- 172.21.15.19:0/3811069812 shutdown_connections 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.425+0000 7fbc76910700 1 --2- 172.21.15.19:0/3811069812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7010cb70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.425+0000 7fbc76910700 1 -- 172.21.15.19:0/3811069812 >> 172.21.15.19:0/3811069812 conn(0x7fbc70104250 msgr2=0x7fbc701066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.425+0000 7fbc76910700 1 -- 172.21.15.19:0/3811069812 shutdown_connections 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.425+0000 7fbc76910700 1 -- 172.21.15.19:0/3811069812 wait complete. 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.426+0000 7fbc76910700 1 Processor -- start 2024-05-09T01:15:05.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.426+0000 7fbc76910700 1 -- start start 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.426+0000 7fbc76910700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.426+0000 7fbc76910700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7010e000 con 0x7fbc7010a750 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc6ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc6ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7019dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33158/0 (socket says 172.21.15.19:33158) 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc6ffff700 1 -- 172.21.15.19:0/1916341759 learned_addr learned my addr 172.21.15.19:0/1916341759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc6ffff700 1 -- 172.21.15.19:0/1916341759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7019e520 con 0x7fbc7010a750 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc6ffff700 1 --2- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7019dfe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc600117b0 tx=0x7fbc60011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc6d7fa700 1 -- 172.21.15.19:0/1916341759 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc60003af0 con 0x7fbc7010a750 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc6d7fa700 1 -- 172.21.15.19:0/1916341759 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbc60020070 con 0x7fbc7010a750 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc6d7fa700 1 -- 172.21.15.19:0/1916341759 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc60019630 con 0x7fbc7010a750 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7019e7b0 con 0x7fbc7010a750 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.427+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc701a13c0 con 0x7fbc7010a750 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.428+0000 7fbc6d7fa700 1 -- 172.21.15.19:0/1916341759 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fbc600198f0 con 0x7fbc7010a750 2024-05-09T01:15:05.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.428+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc70064e20 con 0x7fbc7010a750 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.429+0000 7fbc6d7fa700 1 --2- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbc5c03e470 0x7fbc5c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.429+0000 7fbc6d7fa700 1 -- 172.21.15.19:0/1916341759 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc60053af0 con 0x7fbc7010a750 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.429+0000 7fbc6f7fe700 1 --2- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbc5c03e470 0x7fbc5c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.429+0000 7fbc6f7fe700 1 --2- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbc5c03e470 0x7fbc5c040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc58006fd0 tx=0x7fbc58006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.434+0000 7fbc6d7fa700 1 -- 172.21.15.19:0/1916341759 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc60016260 con 0x7fbc7010a750 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:00.655+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7fbc701062f0 con 0x7fbc5c03e470 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.916+0000 7fbc6d7fa700 1 -- 172.21.15.19:0/1916341759 <== mgr.14118 v2:172.21.15.19:6800/112627556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fbc701062f0 con 0x7fbc5c03e470 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbc5c03e470 msgr2=0x7fbc5c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 --2- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbc5c03e470 0x7fbc5c040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc58006fd0 tx=0x7fbc58006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 msgr2=0x7fbc7019dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 --2- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7019dfe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc600117b0 tx=0x7fbc60011b70 comp rx=0 tx=0).stop 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 shutdown_connections 2024-05-09T01:15:05.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 --2- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbc5c03e470 0x7fbc5c040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:05.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 --2- 172.21.15.19:0/1916341759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7010a750 0x7fbc7019dfe0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:05.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 >> 172.21.15.19:0/1916341759 conn(0x7fbc70104250 msgr2=0x7fbc70105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:05.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 shutdown_connections 2024-05-09T01:15:05.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:04.919+0000 7fbc76910700 1 -- 172.21.15.19:0/1916341759 wait complete. 2024-05-09T01:15:05.497 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-05-09T01:15:06.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:05 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:15:06.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:05 smithi019 ceph-mon[30158]: Added host smithi019 2024-05-09T01:15:06.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:05 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:15:07.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T01:15:07.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.965+0000 7fd02f700700 1 Processor -- start 2024-05-09T01:15:07.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.965+0000 7fd02f700700 1 -- start start 2024-05-09T01:15:07.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.965+0000 7fd02f700700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd028108f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:07.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.965+0000 7fd02f700700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd028109570 con 0x7fd028108b70 2024-05-09T01:15:07.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.965+0000 7fd02d49c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd028108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.965+0000 7fd02d49c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd028108f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33170/0 (socket says 172.21.15.19:33170) 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.965+0000 7fd02d49c700 1 -- 172.21.15.19:0/2686517500 learned_addr learned my addr 172.21.15.19:0/2686517500 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.966+0000 7fd02d49c700 1 -- 172.21.15.19:0/2686517500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd028109d40 con 0x7fd028108b70 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.966+0000 7fd02d49c700 1 --2- 172.21.15.19:0/2686517500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd028108f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd02400bb40 tx=0x7fd02400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd11653bcdee786 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.966+0000 7fd01ffff700 1 -- 172.21.15.19:0/2686517500 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd02400dbe0 con 0x7fd028108b70 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.966+0000 7fd01ffff700 1 -- 172.21.15.19:0/2686517500 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd0240044c0 con 0x7fd028108b70 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.966+0000 7fd01ffff700 1 -- 172.21.15.19:0/2686517500 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd024010470 con 0x7fd028108b70 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.967+0000 7fd02f700700 1 -- 172.21.15.19:0/2686517500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 msgr2=0x7fd028108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.967+0000 7fd02f700700 1 --2- 172.21.15.19:0/2686517500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd028108f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd02400bb40 tx=0x7fd02400be50 comp rx=0 tx=0).stop 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.967+0000 7fd02f700700 1 -- 172.21.15.19:0/2686517500 shutdown_connections 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.967+0000 7fd02f700700 1 --2- 172.21.15.19:0/2686517500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd028108f80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.967+0000 7fd02f700700 1 -- 172.21.15.19:0/2686517500 >> 172.21.15.19:0/2686517500 conn(0x7fd028104250 msgr2=0x7fd0281066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.967+0000 7fd02f700700 1 -- 172.21.15.19:0/2686517500 shutdown_connections 2024-05-09T01:15:07.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.967+0000 7fd02f700700 1 -- 172.21.15.19:0/2686517500 wait complete. 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd02f700700 1 Processor -- start 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd02f700700 1 -- start start 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd02f700700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd02819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd02f700700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd02807bfc0 con 0x7fd028108b70 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd02d49c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd02819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd02d49c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd02819bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33180/0 (socket says 172.21.15.19:33180) 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd02d49c700 1 -- 172.21.15.19:0/1366794421 learned_addr learned my addr 172.21.15.19:0/1366794421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd02d49c700 1 -- 172.21.15.19:0/1366794421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02819c310 con 0x7fd028108b70 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd02d49c700 1 --2- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd02819bdd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd024003c30 tx=0x7fd024003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd01e7fc700 1 -- 172.21.15.19:0/1366794421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd024027070 con 0x7fd028108b70 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.968+0000 7fd01e7fc700 1 -- 172.21.15.19:0/1366794421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd024004300 con 0x7fd028108b70 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.969+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02819c5a0 con 0x7fd028108b70 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.969+0000 7fd01e7fc700 1 -- 172.21.15.19:0/1366794421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0240229d0 con 0x7fd028108b70 2024-05-09T01:15:07.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.969+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02819f170 con 0x7fd028108b70 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.970+0000 7fd01e7fc700 1 -- 172.21.15.19:0/1366794421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd02402a070 con 0x7fd028108b70 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.970+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd028064e20 con 0x7fd028108b70 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.970+0000 7fd01e7fc700 1 --2- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fd01403e4c0 0x7fd0140409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.970+0000 7fd01e7fc700 1 -- 172.21.15.19:0/1366794421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd02401f070 con 0x7fd028108b70 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.970+0000 7fd02cc9b700 1 --2- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fd01403e4c0 0x7fd0140409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.970+0000 7fd02cc9b700 1 --2- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fd01403e4c0 0x7fd0140409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd018006fd0 tx=0x7fd018006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:05.975+0000 7fd01e7fc700 1 -- 172.21.15.19:0/1366794421 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd024015070 con 0x7fd028108b70 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.214+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd028063c00 con 0x7fd01403e4c0 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.219+0000 7fd01e7fc700 1 -- 172.21.15.19:0/1366794421 <== mgr.14118 v2:172.21.15.19:6800/112627556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd028063c00 con 0x7fd01403e4c0 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.221+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fd01403e4c0 msgr2=0x7fd0140409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.221+0000 7fd02f700700 1 --2- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fd01403e4c0 0x7fd0140409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd018006fd0 tx=0x7fd018006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.221+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 msgr2=0x7fd02819bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.221+0000 7fd02f700700 1 --2- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd02819bdd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd024003c30 tx=0x7fd024003d10 comp rx=0 tx=0).stop 2024-05-09T01:15:07.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.221+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 shutdown_connections 2024-05-09T01:15:07.008 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.221+0000 7fd02f700700 1 --2- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fd01403e4c0 0x7fd0140409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:07.008 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.221+0000 7fd02f700700 1 --2- 172.21.15.19:0/1366794421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd028108b70 0x7fd02819bdd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:07.008 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.221+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 >> 172.21.15.19:0/1366794421 conn(0x7fd028104250 msgr2=0x7fd028192080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:07.008 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.221+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 shutdown_connections 2024-05-09T01:15:07.008 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:06.222+0000 7fd02f700700 1 -- 172.21.15.19:0/1366794421 wait complete. 2024-05-09T01:15:07.008 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-05-09T01:15:07.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:07 smithi019 ceph-mon[30158]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:15:07.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:07 smithi019 ceph-mon[30158]: Saving service mon spec with placement count:5 2024-05-09T01:15:07.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:07 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:15:08.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.104+0000 7fbfb81ea700 1 Processor -- start 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.104+0000 7fbfb81ea700 1 -- start start 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.104+0000 7fbfb81ea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb010cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.104+0000 7fbfb81ea700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb007ce30 con 0x7fbfb010a730 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.104+0000 7fbfb5f86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb010cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.104+0000 7fbfb5f86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb010cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33190/0 (socket says 172.21.15.19:33190) 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.104+0000 7fbfb5f86700 1 -- 172.21.15.19:0/2953063593 learned_addr learned my addr 172.21.15.19:0/2953063593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.105+0000 7fbfb5f86700 1 -- 172.21.15.19:0/2953063593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb010d090 con 0x7fbfb010a730 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.105+0000 7fbfb5f86700 1 --2- 172.21.15.19:0/2953063593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb010cb50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0098a0 tx=0x7fbfac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c6d7a181d2b29e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.105+0000 7fbfb4f84700 1 -- 172.21.15.19:0/2953063593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfac004160 con 0x7fbfb010a730 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.105+0000 7fbfb4f84700 1 -- 172.21.15.19:0/2953063593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac004300 con 0x7fbfb010a730 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.105+0000 7fbfb4f84700 1 -- 172.21.15.19:0/2953063593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfac011a20 con 0x7fbfb010a730 2024-05-09T01:15:08.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.106+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2953063593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 msgr2=0x7fbfb010cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.106+0000 7fbfb81ea700 1 --2- 172.21.15.19:0/2953063593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb010cb50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0098a0 tx=0x7fbfac009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.106+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2953063593 shutdown_connections 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.106+0000 7fbfb81ea700 1 --2- 172.21.15.19:0/2953063593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb010cb50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.106+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2953063593 >> 172.21.15.19:0/2953063593 conn(0x7fbfb0104190 msgr2=0x7fbfb0106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.106+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2953063593 shutdown_connections 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.106+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2953063593 wait complete. 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.107+0000 7fbfb81ea700 1 Processor -- start 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.107+0000 7fbfb81ea700 1 -- start start 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.107+0000 7fbfb81ea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.107+0000 7fbfb81ea700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010e030 con 0x7fbfb010a730 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.107+0000 7fbfb5f86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.107+0000 7fbfb5f86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb019df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33196/0 (socket says 172.21.15.19:33196) 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.107+0000 7fbfb5f86700 1 -- 172.21.15.19:0/2351782896 learned_addr learned my addr 172.21.15.19:0/2351782896 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.108+0000 7fbfb5f86700 1 -- 172.21.15.19:0/2351782896 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb019e490 con 0x7fbfb010a730 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.108+0000 7fbfb5f86700 1 --2- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb019df50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0119f0 tx=0x7fbfac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.108+0000 7fbfa6ffd700 1 -- 172.21.15.19:0/2351782896 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfac0039e0 con 0x7fbfb010a730 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.108+0000 7fbfa6ffd700 1 -- 172.21.15.19:0/2351782896 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac01b8f0 con 0x7fbfb010a730 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.108+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb019e720 con 0x7fbfb010a730 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.108+0000 7fbfa6ffd700 1 -- 172.21.15.19:0/2351782896 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfac0237a0 con 0x7fbfb010a730 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.108+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb01a1330 con 0x7fbfb010a730 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.109+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb0064e20 con 0x7fbfb010a730 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.109+0000 7fbfa6ffd700 1 -- 172.21.15.19:0/2351782896 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fbfac0193d0 con 0x7fbfb010a730 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.110+0000 7fbfa6ffd700 1 --2- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbf9c03e420 0x7fbf9c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.110+0000 7fbfa6ffd700 1 -- 172.21.15.19:0/2351782896 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbfac054140 con 0x7fbfb010a730 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.110+0000 7fbfb5785700 1 --2- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbf9c03e420 0x7fbf9c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.110+0000 7fbfb5785700 1 --2- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbf9c03e420 0x7fbf9c040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0006fd0 tx=0x7fbfa0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.115+0000 7fbfa6ffd700 1 -- 172.21.15.19:0/2351782896 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbfac01f030 con 0x7fbfb010a730 2024-05-09T01:15:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.350+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbfb01a15c0 con 0x7fbf9c03e420 2024-05-09T01:15:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.358+0000 7fbfa6ffd700 1 -- 172.21.15.19:0/2351782896 <== mgr.14118 v2:172.21.15.19:6800/112627556 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbfb01a15c0 con 0x7fbf9c03e420 2024-05-09T01:15:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbf9c03e420 msgr2=0x7fbf9c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 --2- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbf9c03e420 0x7fbf9c040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0006fd0 tx=0x7fbfa0006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 msgr2=0x7fbfb019df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 --2- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb019df50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0119f0 tx=0x7fbfac011ad0 comp rx=0 tx=0).stop 2024-05-09T01:15:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 shutdown_connections 2024-05-09T01:15:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 --2- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7fbf9c03e420 0x7fbf9c040930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 --2- 172.21.15.19:0/2351782896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb010a730 0x7fbfb019df50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 >> 172.21.15.19:0/2351782896 conn(0x7fbfb0104190 msgr2=0x7fbfb01049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 shutdown_connections 2024-05-09T01:15:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:08.361+0000 7fbfb81ea700 1 -- 172.21.15.19:0/2351782896 wait complete. 2024-05-09T01:15:09.026 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:08 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:15:09.026 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:08 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:15:09.026 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:08 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:15:09.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:09 smithi019 ceph-mon[30158]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:15:09.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:09 smithi019 ceph-mon[30158]: Saving service mgr spec with placement count:2 2024-05-09T01:15:09.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:09 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/626516783' entity='client.admin' 2024-05-09T01:15:10.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.347+0000 7f7111f3e700 1 Processor -- start 2024-05-09T01:15:10.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.347+0000 7f7111f3e700 1 -- start start 2024-05-09T01:15:10.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.347+0000 7f7111f3e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:10.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.347+0000 7f7111f3e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f710c109580 con 0x7f710c108b80 2024-05-09T01:15:10.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.348+0000 7f710b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:10.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.348+0000 7f710b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33202/0 (socket says 172.21.15.19:33202) 2024-05-09T01:15:10.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.348+0000 7f710b7fe700 1 -- 172.21.15.19:0/4240669156 learned_addr learned my addr 172.21.15.19:0/4240669156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.348+0000 7f710b7fe700 1 -- 172.21.15.19:0/4240669156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c109d50 con 0x7f710c108b80 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.349+0000 7f710b7fe700 1 --2- 172.21.15.19:0/4240669156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c108f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0098a0 tx=0x7f70fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14109ff64ea9b6fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.349+0000 7f710affd700 1 -- 172.21.15.19:0/4240669156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70fc004160 con 0x7f710c108b80 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.349+0000 7f710affd700 1 -- 172.21.15.19:0/4240669156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc004300 con 0x7f710c108b80 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.349+0000 7f710affd700 1 -- 172.21.15.19:0/4240669156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70fc011a20 con 0x7f710c108b80 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.350+0000 7f7111f3e700 1 -- 172.21.15.19:0/4240669156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 msgr2=0x7f710c108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.350+0000 7f7111f3e700 1 --2- 172.21.15.19:0/4240669156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c108f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0098a0 tx=0x7f70fc009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.350+0000 7f7111f3e700 1 -- 172.21.15.19:0/4240669156 shutdown_connections 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.350+0000 7f7111f3e700 1 --2- 172.21.15.19:0/4240669156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c108f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.350+0000 7f7111f3e700 1 -- 172.21.15.19:0/4240669156 >> 172.21.15.19:0/4240669156 conn(0x7f710c104240 msgr2=0x7f710c106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.351+0000 7f7111f3e700 1 -- 172.21.15.19:0/4240669156 shutdown_connections 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.351+0000 7f7111f3e700 1 -- 172.21.15.19:0/4240669156 wait complete. 2024-05-09T01:15:10.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.352+0000 7f7111f3e700 1 Processor -- start 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.352+0000 7f7111f3e700 1 -- start start 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.352+0000 7f7111f3e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.352+0000 7f7111f3e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f710c10d100 con 0x7f710c108b80 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.353+0000 7f710b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.353+0000 7f710b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c19cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33218/0 (socket says 172.21.15.19:33218) 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.353+0000 7f710b7fe700 1 -- 172.21.15.19:0/626516783 learned_addr learned my addr 172.21.15.19:0/626516783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.353+0000 7f710b7fe700 1 -- 172.21.15.19:0/626516783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c19d4a0 con 0x7f710c108b80 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.353+0000 7f710b7fe700 1 --2- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c19cf60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70fc011a90 tx=0x7f70fc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.353+0000 7f71097fa700 1 -- 172.21.15.19:0/626516783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70fc003950 con 0x7f710c108b80 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.354+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c19d730 con 0x7f710c108b80 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.354+0000 7f71097fa700 1 -- 172.21.15.19:0/626516783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc003af0 con 0x7f710c108b80 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.354+0000 7f71097fa700 1 -- 172.21.15.19:0/626516783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70fc01b7c0 con 0x7f710c108b80 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.354+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c0762c0 con 0x7f710c108b80 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.355+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710c064e20 con 0x7f710c108b80 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.355+0000 7f71097fa700 1 -- 172.21.15.19:0/626516783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f70fc01bac0 con 0x7f710c108b80 2024-05-09T01:15:10.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.356+0000 7f71097fa700 1 --2- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f70f803e420 0x7f70f8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.356+0000 7f71097fa700 1 -- 172.21.15.19:0/626516783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f70fc056310 con 0x7f710c108b80 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.356+0000 7f7102ffd700 1 --2- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f70f803e420 0x7f70f8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.357+0000 7f7102ffd700 1 --2- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f70f803e420 0x7f70f8040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006fd0 tx=0x7f70f4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.362+0000 7f71097fa700 1 -- 172.21.15.19:0/626516783 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70fc02b2b0 con 0x7f710c108b80 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.616+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f710c0768d0 con 0x7f710c108b80 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.620+0000 7f71097fa700 1 -- 172.21.15.19:0/626516783 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f70fc056070 con 0x7f710c108b80 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f70f803e420 msgr2=0x7f70f8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 --2- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f70f803e420 0x7f70f8040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006fd0 tx=0x7f70f4006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 msgr2=0x7f710c19cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 --2- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c19cf60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70fc011a90 tx=0x7f70fc011b70 comp rx=0 tx=0).stop 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 shutdown_connections 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 --2- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f70f803e420 0x7f70f8040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 --2- 172.21.15.19:0/626516783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710c108b80 0x7f710c19cf60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:10.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 >> 172.21.15.19:0/626516783 conn(0x7f710c104240 msgr2=0x7f710c105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:10.645 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 shutdown_connections 2024-05-09T01:15:10.645 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:09.623+0000 7f7111f3e700 1 -- 172.21.15.19:0/626516783 wait complete. 2024-05-09T01:15:11.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:11 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.124+0000 7fef11aae700 1 Processor -- start 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.124+0000 7fef11aae700 1 -- start start 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.124+0000 7fef11aae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c10a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.124+0000 7fef11aae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10a6d0 con 0x7fef0c109cd0 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.124+0000 7fef0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c10a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.124+0000 7fef0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c10a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50308/0 (socket says 172.21.15.19:50308) 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.124+0000 7fef0b7fe700 1 -- 172.21.15.19:0/1086634078 learned_addr learned my addr 172.21.15.19:0/1086634078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.125+0000 7fef0b7fe700 1 -- 172.21.15.19:0/1086634078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c10aea0 con 0x7fef0c109cd0 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.125+0000 7fef0b7fe700 1 --2- 172.21.15.19:0/1086634078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c10a0e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feefc0098a0 tx=0x7feefc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354c8af8933e0f4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.125+0000 7fef0a7fc700 1 -- 172.21.15.19:0/1086634078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feefc004160 con 0x7fef0c109cd0 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.125+0000 7fef0a7fc700 1 -- 172.21.15.19:0/1086634078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feefc004300 con 0x7fef0c109cd0 2024-05-09T01:15:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.125+0000 7fef0a7fc700 1 -- 172.21.15.19:0/1086634078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feefc011a20 con 0x7fef0c109cd0 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.126+0000 7fef11aae700 1 -- 172.21.15.19:0/1086634078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 msgr2=0x7fef0c10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.126+0000 7fef11aae700 1 --2- 172.21.15.19:0/1086634078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c10a0e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feefc0098a0 tx=0x7feefc009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.126+0000 7fef11aae700 1 -- 172.21.15.19:0/1086634078 shutdown_connections 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.126+0000 7fef11aae700 1 --2- 172.21.15.19:0/1086634078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c10a0e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.126+0000 7fef11aae700 1 -- 172.21.15.19:0/1086634078 >> 172.21.15.19:0/1086634078 conn(0x7fef0c105330 msgr2=0x7fef0c1077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.126+0000 7fef11aae700 1 -- 172.21.15.19:0/1086634078 shutdown_connections 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.126+0000 7fef11aae700 1 -- 172.21.15.19:0/1086634078 wait complete. 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.127+0000 7fef11aae700 1 Processor -- start 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.127+0000 7fef11aae700 1 -- start start 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.128+0000 7fef11aae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.128+0000 7fef11aae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10e250 con 0x7fef0c109cd0 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.128+0000 7fef0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.128+0000 7fef0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c1a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50320/0 (socket says 172.21.15.19:50320) 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.128+0000 7fef0b7fe700 1 -- 172.21.15.19:0/2895832098 learned_addr learned my addr 172.21.15.19:0/2895832098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.128+0000 7fef0b7fe700 1 -- 172.21.15.19:0/2895832098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c1a07c0 con 0x7fef0c109cd0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.128+0000 7fef0b7fe700 1 --2- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c1a0280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feefc003e70 tx=0x7feefc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.128+0000 7fef08ff9700 1 -- 172.21.15.19:0/2895832098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feefc003af0 con 0x7fef0c109cd0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.129+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c1a0a50 con 0x7fef0c109cd0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.129+0000 7fef08ff9700 1 -- 172.21.15.19:0/2895832098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feefc01a8e0 con 0x7fef0c109cd0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.129+0000 7fef08ff9700 1 -- 172.21.15.19:0/2895832098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feefc024870 con 0x7fef0c109cd0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.129+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c1a3660 con 0x7fef0c109cd0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.130+0000 7fef08ff9700 1 -- 172.21.15.19:0/2895832098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7feefc022020 con 0x7fef0c109cd0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.130+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c064e20 con 0x7fef0c109cd0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.130+0000 7fef08ff9700 1 --2- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7feef403e420 0x7feef4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.130+0000 7fef08ff9700 1 -- 172.21.15.19:0/2895832098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feefc053920 con 0x7fef0c109cd0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.130+0000 7fef0affd700 1 --2- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7feef403e420 0x7feef4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.131+0000 7fef0affd700 1 --2- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7feef403e420 0x7feef4040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef00006fd0 tx=0x7fef00006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.137+0000 7fef08ff9700 1 -- 172.21.15.19:0/2895832098 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feefc00e260 con 0x7fef0c109cd0 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.356+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fef0c1a3d40 con 0x7fef0c109cd0 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.360+0000 7fef08ff9700 1 -- 172.21.15.19:0/2895832098 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7feefc01ad90 con 0x7fef0c109cd0 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.363+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7feef403e420 msgr2=0x7feef4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.363+0000 7fef11aae700 1 --2- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7feef403e420 0x7feef4040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef00006fd0 tx=0x7fef00006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.363+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 msgr2=0x7fef0c1a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.363+0000 7fef11aae700 1 --2- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c1a0280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feefc003e70 tx=0x7feefc011aa0 comp rx=0 tx=0).stop 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.364+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 shutdown_connections 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.364+0000 7fef11aae700 1 --2- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7feef403e420 0x7feef4040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.364+0000 7fef11aae700 1 --2- 172.21.15.19:0/2895832098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef0c109cd0 0x7fef0c1a0280 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.364+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 >> 172.21.15.19:0/2895832098 conn(0x7fef0c105330 msgr2=0x7fef0c07d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.364+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 shutdown_connections 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:11.364+0000 7fef11aae700 1 -- 172.21.15.19:0/2895832098 wait complete. 2024-05-09T01:15:11.960 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-05-09T01:15:12.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:12 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2895832098' entity='client.admin' 2024-05-09T01:15:12.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:12 smithi019 ceph-mon[30158]: from='mgr.14118 172.21.15.19:0/3467415941' entity='mgr.x' 2024-05-09T01:15:13.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:13 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3950514326' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T01:15:13.589 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:13 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: ignoring --setuser ceph since I am not root 2024-05-09T01:15:13.590 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:13 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: ignoring --setgroup ceph since I am not root 2024-05-09T01:15:13.590 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:13 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:13.589+0000 7f92e6934200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T01:15:13.897 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:13 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:13.788+0000 7f92e6934200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:15:13.897 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:13 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:13.897+0000 7f92e6934200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T01:15:13.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.381+0000 7f4490f00700 1 Processor -- start 2024-05-09T01:15:13.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.381+0000 7f4490f00700 1 -- start start 2024-05-09T01:15:13.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.381+0000 7f4490f00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c108f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:13.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.381+0000 7f4490f00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f448c109500 con 0x7f448c108b00 2024-05-09T01:15:13.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.381+0000 7f448a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:13.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.381+0000 7f448a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c108f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50332/0 (socket says 172.21.15.19:50332) 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.381+0000 7f448a59c700 1 -- 172.21.15.19:0/1507756924 learned_addr learned my addr 172.21.15.19:0/1507756924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.382+0000 7f448a59c700 1 -- 172.21.15.19:0/1507756924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c109cd0 con 0x7f448c108b00 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.382+0000 7f448a59c700 1 --2- 172.21.15.19:0/1507756924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c108f10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f447c0098a0 tx=0x7f447c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a8fdd752fb30a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.382+0000 7f448959a700 1 -- 172.21.15.19:0/1507756924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f447c004160 con 0x7f448c108b00 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.382+0000 7f448959a700 1 -- 172.21.15.19:0/1507756924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f447c004300 con 0x7f448c108b00 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.382+0000 7f448959a700 1 -- 172.21.15.19:0/1507756924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f447c011a20 con 0x7f448c108b00 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.383+0000 7f4490f00700 1 -- 172.21.15.19:0/1507756924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 msgr2=0x7f448c108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.383+0000 7f4490f00700 1 --2- 172.21.15.19:0/1507756924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c108f10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f447c0098a0 tx=0x7f447c009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.383+0000 7f4490f00700 1 -- 172.21.15.19:0/1507756924 shutdown_connections 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.383+0000 7f4490f00700 1 --2- 172.21.15.19:0/1507756924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c108f10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.383+0000 7f4490f00700 1 -- 172.21.15.19:0/1507756924 >> 172.21.15.19:0/1507756924 conn(0x7f448c1041e0 msgr2=0x7f448c106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.383+0000 7f4490f00700 1 -- 172.21.15.19:0/1507756924 shutdown_connections 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.383+0000 7f4490f00700 1 -- 172.21.15.19:0/1507756924 wait complete. 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.384+0000 7f4490f00700 1 Processor -- start 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.384+0000 7f4490f00700 1 -- start start 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.384+0000 7f4490f00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.384+0000 7f4490f00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f448c07c140 con 0x7f448c108b00 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f448a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f448a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c19be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50342/0 (socket says 172.21.15.19:50342) 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f448a59c700 1 -- 172.21.15.19:0/3950514326 learned_addr learned my addr 172.21.15.19:0/3950514326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f448a59c700 1 -- 172.21.15.19:0/3950514326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c19c3c0 con 0x7f448c108b00 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f448a59c700 1 --2- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c19be80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f447c0119f0 tx=0x7f447c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f44837fe700 1 -- 172.21.15.19:0/3950514326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f447c003a60 con 0x7f448c108b00 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f44837fe700 1 -- 172.21.15.19:0/3950514326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f447c01b8f0 con 0x7f448c108b00 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f4490f00700 1 -- 172.21.15.19:0/3950514326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c19c650 con 0x7f448c108b00 2024-05-09T01:15:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f44837fe700 1 -- 172.21.15.19:0/3950514326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f447c023880 con 0x7f448c108b00 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.385+0000 7f4490f00700 1 -- 172.21.15.19:0/3950514326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c19f260 con 0x7f448c108b00 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.386+0000 7f44837fe700 1 -- 172.21.15.19:0/3950514326 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f447c02c3f0 con 0x7f448c108b00 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.387+0000 7f44837fe700 1 --2- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f447803e4d0 0x7f44780409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.387+0000 7f44817fa700 1 -- 172.21.15.19:0/3950514326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448c064e20 con 0x7f448c108b00 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.387+0000 7f44837fe700 1 -- 172.21.15.19:0/3950514326 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f447c0551c0 con 0x7f448c108b00 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.387+0000 7f4489d9b700 1 --2- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f447803e4d0 0x7f44780409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.388+0000 7f4489d9b700 1 --2- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f447803e4d0 0x7f44780409e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4474006fd0 tx=0x7f4474006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.393+0000 7f44837fe700 1 -- 172.21.15.19:0/3950514326 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f447c01f030 con 0x7f448c108b00 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:12.677+0000 7f44817fa700 1 -- 172.21.15.19:0/3950514326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f448c0643d0 con 0x7f448c108b00 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.364+0000 7f44837fe700 1 -- 172.21.15.19:0/3950514326 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f447c017e90 con 0x7f448c108b00 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.367+0000 7f44837fe700 1 -- 172.21.15.19:0/3950514326 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f447c02cca0 con 0x7f448c108b00 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 -- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f447803e4d0 msgr2=0x7f44780409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:13.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 --2- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f447803e4d0 0x7f44780409e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4474006fd0 tx=0x7f4474006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:13.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 -- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 msgr2=0x7f448c19be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:13.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 --2- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c19be80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f447c0119f0 tx=0x7f447c011ad0 comp rx=0 tx=0).stop 2024-05-09T01:15:13.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 -- 172.21.15.19:0/3950514326 shutdown_connections 2024-05-09T01:15:13.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 --2- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f447803e4d0 0x7f44780409e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:13.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 --2- 172.21.15.19:0/3950514326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c108b00 0x7f448c19be80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:13.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 -- 172.21.15.19:0/3950514326 >> 172.21.15.19:0/3950514326 conn(0x7f448c1041e0 msgr2=0x7f448c192150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:13.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 -- 172.21.15.19:0/3950514326 shutdown_connections 2024-05-09T01:15:13.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:13.368+0000 7f44817fa700 1 -- 172.21.15.19:0/3950514326 wait complete. 2024-05-09T01:15:14.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:14 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3950514326' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T01:15:14.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:14 smithi019 ceph-mon[30158]: mgrmap e9: x(active, since 19s) 2024-05-09T01:15:14.648 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:14 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:14.236+0000 7f92e6934200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T01:15:14.648 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:14 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:14.332+0000 7f92e6934200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T01:15:15.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:15:15.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T01:15:15.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T01:15:15.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T01:15:15.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T01:15:15.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:15:15.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.463+0000 7f4d2c2c7700 1 Processor -- start 2024-05-09T01:15:15.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.463+0000 7f4d2c2c7700 1 -- start start 2024-05-09T01:15:15.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.464+0000 7f4d2c2c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d2410cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:15.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.464+0000 7f4d2c2c7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d24077110 con 0x7f4d2410a7c0 2024-05-09T01:15:15.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.464+0000 7f4d2a063700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d2410cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:15.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.464+0000 7f4d2a063700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d2410cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50368/0 (socket says 172.21.15.19:50368) 2024-05-09T01:15:15.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.464+0000 7f4d2a063700 1 -- 172.21.15.19:0/2484985387 learned_addr learned my addr 172.21.15.19:0/2484985387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:15.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.464+0000 7f4d2a063700 1 -- 172.21.15.19:0/2484985387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2410d120 con 0x7f4d2410a7c0 2024-05-09T01:15:15.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.464+0000 7f4d2a063700 1 --2- 172.21.15.19:0/2484985387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d2410cbe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d180098a0 tx=0x7f4d18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb8e65af555f71a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:15.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.465+0000 7f4d29061700 1 -- 172.21.15.19:0/2484985387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d18004160 con 0x7f4d2410a7c0 2024-05-09T01:15:15.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.465+0000 7f4d29061700 1 -- 172.21.15.19:0/2484985387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d18004300 con 0x7f4d2410a7c0 2024-05-09T01:15:15.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.465+0000 7f4d29061700 1 -- 172.21.15.19:0/2484985387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d18011a20 con 0x7f4d2410a7c0 2024-05-09T01:15:15.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.465+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/2484985387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 msgr2=0x7f4d2410cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:15.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.465+0000 7f4d2c2c7700 1 --2- 172.21.15.19:0/2484985387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d2410cbe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d180098a0 tx=0x7f4d18009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:15.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.465+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/2484985387 shutdown_connections 2024-05-09T01:15:15.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.465+0000 7f4d2c2c7700 1 --2- 172.21.15.19:0/2484985387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d2410cbe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:15.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.465+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/2484985387 >> 172.21.15.19:0/2484985387 conn(0x7f4d24104240 msgr2=0x7f4d24106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:15.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.466+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/2484985387 shutdown_connections 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.466+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/2484985387 wait complete. 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.466+0000 7f4d2c2c7700 1 Processor -- start 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.466+0000 7f4d2c2c7700 1 -- start start 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d2c2c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d24109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d2c2c7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2410e0c0 con 0x7f4d2410a7c0 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d2a063700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d24109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d2a063700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d24109f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50380/0 (socket says 172.21.15.19:50380) 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d2a063700 1 -- 172.21.15.19:0/955623329 learned_addr learned my addr 172.21.15.19:0/955623329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d2a063700 1 -- 172.21.15.19:0/955623329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2410a480 con 0x7f4d2410a7c0 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d2a063700 1 --2- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d24109f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000c00 tx=0x7f4d18011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d177fe700 1 -- 172.21.15.19:0/955623329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d18003aa0 con 0x7f4d2410a7c0 2024-05-09T01:15:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d177fe700 1 -- 172.21.15.19:0/955623329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d1801b8f0 con 0x7f4d2410a7c0 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.467+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d241086d0 con 0x7f4d2410a7c0 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.468+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d24108bb0 con 0x7f4d2410a7c0 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.468+0000 7f4d177fe700 1 -- 172.21.15.19:0/955623329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d18023890 con 0x7f4d2410a7c0 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.468+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d24064e20 con 0x7f4d2410a7c0 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.469+0000 7f4d177fe700 1 -- 172.21.15.19:0/955623329 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f4d1802c3f0 con 0x7f4d2410a7c0 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.469+0000 7f4d177fe700 1 --2- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f4d1003e520 0x7f4d10040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.469+0000 7f4d177fe700 1 -- 172.21.15.19:0/955623329 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d18053e00 con 0x7f4d2410a7c0 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.469+0000 7f4d29862700 1 -- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f4d1003e520 msgr2=0x7f4d10040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/112627556 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.469+0000 7f4d29862700 1 --2- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f4d1003e520 0x7f4d10040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T01:15:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.474+0000 7f4d177fe700 1 -- 172.21.15.19:0/955623329 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d1801f030 con 0x7f4d2410a7c0 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.670+0000 7f4d29862700 1 -- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f4d1003e520 msgr2=0x7f4d10040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/112627556 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.670+0000 7f4d29862700 1 --2- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f4d1003e520 0x7f4d10040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.777+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4d24109280 con 0x7f4d2410a7c0 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.778+0000 7f4d177fe700 1 -- 172.21.15.19:0/955623329 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f4d1800f2e0 con 0x7f4d2410a7c0 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.782+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f4d1003e520 msgr2=0x7f4d10040a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.782+0000 7f4d2c2c7700 1 --2- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f4d1003e520 0x7f4d10040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.782+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 msgr2=0x7f4d24109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.782+0000 7f4d2c2c7700 1 --2- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d24109f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000c00 tx=0x7f4d18011aa0 comp rx=0 tx=0).stop 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.782+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 shutdown_connections 2024-05-09T01:15:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.782+0000 7f4d2c2c7700 1 --2- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f4d1003e520 0x7f4d10040a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:15.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.782+0000 7f4d2c2c7700 1 --2- 172.21.15.19:0/955623329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d2410a7c0 0x7f4d24109f40 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000c00 tx=0x7f4d18011aa0 comp rx=0 tx=0).stop 2024-05-09T01:15:15.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.783+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 >> 172.21.15.19:0/955623329 conn(0x7f4d24104240 msgr2=0x7f4d24105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:15.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.783+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 shutdown_connections 2024-05-09T01:15:15.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:14.783+0000 7f4d2c2c7700 1 -- 172.21.15.19:0/955623329 wait complete. 2024-05-09T01:15:15.400 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-05-09T01:15:15.400 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-05-09T01:15:15.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:15 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/955623329' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T01:15:15.874 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:15 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:15.624+0000 7f92e6934200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T01:15:15.874 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:15 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:15.818+0000 7f92e6934200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T01:15:16.147 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:16 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:16.003+0000 7f92e6934200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T01:15:16.629 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:16 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:16.315+0000 7f92e6934200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T01:15:16.629 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:16 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:16.415+0000 7f92e6934200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T01:15:16.897 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:16 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:16.630+0000 7f92e6934200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T01:15:16.898 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:16 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:16.837+0000 7f92e6934200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:15:17.648 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:17 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:17.496+0000 7f92e6934200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T01:15:18.147 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:17 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:17.734+0000 7f92e6934200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T01:15:18.648 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:18 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:18.190+0000 7f92e6934200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T01:15:18.648 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:18 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:18.295+0000 7f92e6934200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T01:15:19.074 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:18 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:18.689+0000 7f92e6934200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T01:15:19.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:19 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:19.074+0000 7f92e6934200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T01:15:19.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:19 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:19.178+0000 7f92e6934200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T01:15:19.752 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:19 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:19.444+0000 7f92e6934200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T01:15:19.752 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:19 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:19.587+0000 7f92e6934200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T01:15:19.752 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:19 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:19.693+0000 7f92e6934200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T01:15:20.397 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:20 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:20.039+0000 7f92e6934200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T01:15:20.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:20 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:20.142+0000 7f92e6934200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T01:15:21.079 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:20 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:20.790+0000 7f92e6934200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T01:15:21.079 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:15:21 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:15:21.024+0000 7f92e6934200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: Active manager daemon x restarted 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: Activating manager daemon x 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: mgrmap e10: x(active, starting, since 0.00455394s) 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: Manager daemon x is now available 2024-05-09T01:15:21.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:15:22.316 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T01:15:22.317 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T01:15:22.317 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:22 smithi019 ceph-mon[30158]: mgrmap e11: x(active, since 1.00677s) 2024-05-09T01:15:23.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.871+0000 7f1ae3b7f700 1 Processor -- start 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.871+0000 7f1ae3b7f700 1 -- start start 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.872+0000 7f1ae3b7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc10cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.872+0000 7f1ae3b7f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc07ce10 con 0x7f1adc10a750 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.872+0000 7f1ae191b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc10cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.872+0000 7f1ae191b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc10cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50384/0 (socket says 172.21.15.19:50384) 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.872+0000 7f1ae191b700 1 -- 172.21.15.19:0/3027519619 learned_addr learned my addr 172.21.15.19:0/3027519619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.872+0000 7f1ae191b700 1 -- 172.21.15.19:0/3027519619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc10d0b0 con 0x7f1adc10a750 2024-05-09T01:15:23.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.872+0000 7f1ae191b700 1 --2- 172.21.15.19:0/3027519619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc10cb70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80098a0 tx=0x7f1ad8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef6f966150762cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.872+0000 7f1ae0919700 1 -- 172.21.15.19:0/3027519619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad8004160 con 0x7f1adc10a750 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.873+0000 7f1ae0919700 1 -- 172.21.15.19:0/3027519619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad8004300 con 0x7f1adc10a750 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.873+0000 7f1ae0919700 1 -- 172.21.15.19:0/3027519619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad8011a20 con 0x7f1adc10a750 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.873+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3027519619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 msgr2=0x7f1adc10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.873+0000 7f1ae3b7f700 1 --2- 172.21.15.19:0/3027519619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc10cb70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80098a0 tx=0x7f1ad8009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.874+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3027519619 shutdown_connections 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.874+0000 7f1ae3b7f700 1 --2- 172.21.15.19:0/3027519619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc10cb70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.874+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3027519619 >> 172.21.15.19:0/3027519619 conn(0x7f1adc104190 msgr2=0x7f1adc106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.874+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3027519619 shutdown_connections 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.874+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3027519619 wait complete. 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.874+0000 7f1ae3b7f700 1 Processor -- start 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.874+0000 7f1ae3b7f700 1 -- start start 2024-05-09T01:15:23.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.875+0000 7f1ae3b7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc109ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.875+0000 7f1ae3b7f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc10e050 con 0x7f1adc10a750 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.875+0000 7f1ae191b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc109ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.875+0000 7f1ae191b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc109ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50400/0 (socket says 172.21.15.19:50400) 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.875+0000 7f1ae191b700 1 -- 172.21.15.19:0/3934618275 learned_addr learned my addr 172.21.15.19:0/3934618275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.875+0000 7f1ae191b700 1 -- 172.21.15.19:0/3934618275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc10a410 con 0x7f1adc10a750 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.875+0000 7f1ae191b700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc109ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80119f0 tx=0x7f1ad8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.876+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad80039e0 con 0x7f1adc10a750 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.876+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad801b8f0 con 0x7f1adc10a750 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.876+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3934618275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1adc108660 con 0x7f1adc10a750 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.876+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad80237a0 con 0x7f1adc10a750 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.876+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3934618275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1adc108b40 con 0x7f1adc10a750 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.877+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f1ad802c3f0 con 0x7f1adc10a750 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.877+0000 7f1ad2ffd700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 0x7f1ac8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:23.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.877+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 --> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1ac8041090 con 0x7f1ac803e470 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.877+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1ad8054e30 con 0x7f1adc10a750 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.877+0000 7f1ae111a700 1 -- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 msgr2=0x7f1ac8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/112627556 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:15.877+0000 7f1ae111a700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 0x7f1ac8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:16.078+0000 7f1ae111a700 1 -- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 msgr2=0x7f1ac8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/112627556 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:16.078+0000 7f1ae111a700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 0x7f1ac8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:16.479+0000 7f1ae111a700 1 -- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 msgr2=0x7f1ac8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/112627556 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:16.479+0000 7f1ae111a700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 0x7f1ac8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:17.280+0000 7f1ae111a700 1 -- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 msgr2=0x7f1ac8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/112627556 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:17.280+0000 7f1ae111a700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 0x7f1ac8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:18.880+0000 7f1ae111a700 1 -- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 msgr2=0x7f1ac8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/112627556 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:18.880+0000 7f1ae111a700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 0x7f1ac8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:21.029+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f1ad802cca0 con 0x7f1adc10a750 2024-05-09T01:15:23.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:21.029+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 msgr2=0x7f1ac8040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:21.029+0000 7f1ad2ffd700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 0x7f1ac8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.030+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f1ad8055e20 con 0x7f1adc10a750 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.030+0000 7f1ad2ffd700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1ad80549c0 0x7f1ac8043f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.030+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1ac8041090 con 0x7f1ad80549c0 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.030+0000 7f1ae111a700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1ad80549c0 0x7f1ac8043f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.031+0000 7f1ae111a700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1ad80549c0 0x7f1ac8043f70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1acc003a10 tx=0x7f1acc0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.032+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f1ac8041090 con 0x7f1ad80549c0 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.040+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3934618275 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1adc064e20 con 0x7f1ad80549c0 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ad2ffd700 1 -- 172.21.15.19:0/3934618275 <== mgr.14150 v2:172.21.15.19:6800/2331192605 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1adc064e20 con 0x7f1ad80549c0 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1ad80549c0 msgr2=0x7f1ac8043f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1ad80549c0 0x7f1ac8043f70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1acc003a10 tx=0x7f1acc0092b0 comp rx=0 tx=0).stop 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 msgr2=0x7f1adc109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc109ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80119f0 tx=0x7f1ad8011ad0 comp rx=0 tx=0).stop 2024-05-09T01:15:23.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3934618275 shutdown_connections 2024-05-09T01:15:23.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1ad80549c0 0x7f1ac8043f70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:23.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc10a750 0x7f1adc109ed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:23.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 --2- 172.21.15.19:0/3934618275 >> [v2:172.21.15.19:6800/112627556,v1:172.21.15.19:6801/112627556] conn(0x7f1ac803e470 0x7f1ac8040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:23.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3934618275 >> 172.21.15.19:0/3934618275 conn(0x7f1adc104190 msgr2=0x7f1adc104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:23.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3934618275 shutdown_connections 2024-05-09T01:15:23.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:22.041+0000 7f1ae3b7f700 1 -- 172.21.15.19:0/3934618275 wait complete. 2024-05-09T01:15:23.060 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-05-09T01:15:23.060 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-05-09T01:15:23.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:23 smithi019 ceph-mon[30158]: [09/May/2024:01:15:21] ENGINE Bus STARTING 2024-05-09T01:15:23.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:23 smithi019 ceph-mon[30158]: [09/May/2024:01:15:21] ENGINE Serving on http://172.21.15.19:8765 2024-05-09T01:15:23.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:23 smithi019 ceph-mon[30158]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T01:15:23.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:23 smithi019 ceph-mon[30158]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T01:15:23.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:23 smithi019 ceph-mon[30158]: [09/May/2024:01:15:22] ENGINE Serving on https://172.21.15.19:7150 2024-05-09T01:15:23.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:23 smithi019 ceph-mon[30158]: [09/May/2024:01:15:22] ENGINE Bus STARTED 2024-05-09T01:15:23.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:23 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.472+0000 7f8f39046700 1 Processor -- start 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.472+0000 7f8f39046700 1 -- start start 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.473+0000 7f8f39046700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.473+0000 7f8f39046700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34109580 con 0x7f8f34108b80 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.473+0000 7f8f32d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.473+0000 7f8f32d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58680/0 (socket says 172.21.15.19:58680) 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.473+0000 7f8f32d9d700 1 -- 172.21.15.19:0/3542893814 learned_addr learned my addr 172.21.15.19:0/3542893814 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.473+0000 7f8f32d9d700 1 -- 172.21.15.19:0/3542893814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f34109d50 con 0x7f8f34108b80 2024-05-09T01:15:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.474+0000 7f8f32d9d700 1 --2- 172.21.15.19:0/3542893814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34108f90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009e80 tx=0x7f8f2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcee51f1d0dca3f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.474+0000 7f8f31d9b700 1 -- 172.21.15.19:0/3542893814 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f2c017070 con 0x7f8f34108b80 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.474+0000 7f8f31d9b700 1 -- 172.21.15.19:0/3542893814 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c00b8c0 con 0x7f8f34108b80 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.474+0000 7f8f31d9b700 1 -- 172.21.15.19:0/3542893814 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f2c004490 con 0x7f8f34108b80 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.475+0000 7f8f39046700 1 -- 172.21.15.19:0/3542893814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 msgr2=0x7f8f34108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.475+0000 7f8f39046700 1 --2- 172.21.15.19:0/3542893814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34108f90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009e80 tx=0x7f8f2c00b330 comp rx=0 tx=0).stop 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.475+0000 7f8f39046700 1 -- 172.21.15.19:0/3542893814 shutdown_connections 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.475+0000 7f8f39046700 1 --2- 172.21.15.19:0/3542893814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34108f90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.475+0000 7f8f39046700 1 -- 172.21.15.19:0/3542893814 >> 172.21.15.19:0/3542893814 conn(0x7f8f34104240 msgr2=0x7f8f34106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.475+0000 7f8f39046700 1 -- 172.21.15.19:0/3542893814 shutdown_connections 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.475+0000 7f8f39046700 1 -- 172.21.15.19:0/3542893814 wait complete. 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.476+0000 7f8f39046700 1 Processor -- start 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.476+0000 7f8f39046700 1 -- start start 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.476+0000 7f8f39046700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34107340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:24.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.476+0000 7f8f39046700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3410d100 con 0x7f8f34108b80 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.476+0000 7f8f32d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.476+0000 7f8f32d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34107340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58684/0 (socket says 172.21.15.19:58684) 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.476+0000 7f8f32d9d700 1 -- 172.21.15.19:0/3309321421 learned_addr learned my addr 172.21.15.19:0/3309321421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.477+0000 7f8f32d9d700 1 -- 172.21.15.19:0/3309321421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f34107880 con 0x7f8f34108b80 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.477+0000 7f8f32d9d700 1 --2- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34107340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009fd0 tx=0x7f8f2c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.477+0000 7f8f1bfff700 1 -- 172.21.15.19:0/3309321421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f2c020020 con 0x7f8f34108b80 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.477+0000 7f8f1bfff700 1 -- 172.21.15.19:0/3309321421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c003b50 con 0x7f8f34108b80 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.477+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f34105ad0 con 0x7f8f34108b80 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.477+0000 7f8f1bfff700 1 -- 172.21.15.19:0/3309321421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f2c017040 con 0x7f8f34108b80 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.477+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f34106010 con 0x7f8f34108b80 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.478+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f34064e20 con 0x7f8f34108b80 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.479+0000 7f8f1bfff700 1 -- 172.21.15.19:0/3309321421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f8f2c023020 con 0x7f8f34108b80 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.479+0000 7f8f1bfff700 1 --2- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8f1c03e2f0 0x7f8f1c040800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:24.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.479+0000 7f8f1bfff700 1 -- 172.21.15.19:0/3309321421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f2c00f050 con 0x7f8f34108b80 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.479+0000 7f8f3259c700 1 --2- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8f1c03e2f0 0x7f8f1c040800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.480+0000 7f8f3259c700 1 --2- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8f1c03e2f0 0x7f8f1c040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006fd0 tx=0x7f8f28006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.485+0000 7f8f1bfff700 1 -- 172.21.15.19:0/3309321421 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f2c02a020 con 0x7f8f34108b80 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.689+0000 7f8f1bfff700 1 -- 172.21.15.19:0/3309321421 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f8f2c025400 con 0x7f8f34108b80 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.706+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f8f34197290 con 0x7f8f1c03e2f0 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.817+0000 7f8f1bfff700 1 -- 172.21.15.19:0/3309321421 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f8f34197290 con 0x7f8f1c03e2f0 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8f1c03e2f0 msgr2=0x7f8f1c040800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 --2- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8f1c03e2f0 0x7f8f1c040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006fd0 tx=0x7f8f28006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 msgr2=0x7f8f34107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 --2- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34107340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009fd0 tx=0x7f8f2c00bd30 comp rx=0 tx=0).stop 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 shutdown_connections 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 --2- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8f1c03e2f0 0x7f8f1c040800 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 --2- 172.21.15.19:0/3309321421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f34108b80 0x7f8f34107340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:24.429 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 >> 172.21.15.19:0/3309321421 conn(0x7f8f34104240 msgr2=0x7f8f34194660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:24.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 shutdown_connections 2024-05-09T01:15:24.430 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:23.820+0000 7f8f39046700 1 -- 172.21.15.19:0/3309321421 wait complete. 2024-05-09T01:15:24.430 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-05-09T01:15:25.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:24 smithi019 ceph-mon[30158]: mgrmap e12: x(active, since 2s) 2024-05-09T01:15:25.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:24 smithi019 ceph-mon[30158]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:15:25.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:25.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:25.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:26.498 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:26 smithi019 ceph-mon[30158]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:15:26.498 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$KCNkKsQ0JKJEnyg4dTwwWup0m3RjtCFLbUH3qeWJ2W6hJKJ1mF/ue", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715217325, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.892+0000 7f6c39b74700 1 Processor -- start 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.892+0000 7f6c39b74700 1 -- start start 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.893+0000 7f6c39b74700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3410cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.893+0000 7f6c39b74700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c34077130 con 0x7f6c3410a7a0 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.893+0000 7f6c337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3410cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.893+0000 7f6c337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3410cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58698/0 (socket says 172.21.15.19:58698) 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.893+0000 7f6c337fe700 1 -- 172.21.15.19:0/3942275829 learned_addr learned my addr 172.21.15.19:0/3942275829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.893+0000 7f6c337fe700 1 -- 172.21.15.19:0/3942275829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3410d100 con 0x7f6c3410a7a0 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.894+0000 7f6c337fe700 1 --2- 172.21.15.19:0/3942275829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3410cbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c240098a0 tx=0x7f6c24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc63fc2c1f0f244 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.894+0000 7f6c327fc700 1 -- 172.21.15.19:0/3942275829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c24004160 con 0x7f6c3410a7a0 2024-05-09T01:15:26.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.894+0000 7f6c327fc700 1 -- 172.21.15.19:0/3942275829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c24004300 con 0x7f6c3410a7a0 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.894+0000 7f6c327fc700 1 -- 172.21.15.19:0/3942275829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c24011a20 con 0x7f6c3410a7a0 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.895+0000 7f6c39b74700 1 -- 172.21.15.19:0/3942275829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 msgr2=0x7f6c3410cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.895+0000 7f6c39b74700 1 --2- 172.21.15.19:0/3942275829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3410cbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c240098a0 tx=0x7f6c24009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.895+0000 7f6c39b74700 1 -- 172.21.15.19:0/3942275829 shutdown_connections 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.895+0000 7f6c39b74700 1 --2- 172.21.15.19:0/3942275829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3410cbc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.895+0000 7f6c39b74700 1 -- 172.21.15.19:0/3942275829 >> 172.21.15.19:0/3942275829 conn(0x7f6c34104240 msgr2=0x7f6c34106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.895+0000 7f6c39b74700 1 -- 172.21.15.19:0/3942275829 shutdown_connections 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.895+0000 7f6c39b74700 1 -- 172.21.15.19:0/3942275829 wait complete. 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.896+0000 7f6c39b74700 1 Processor -- start 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.896+0000 7f6c39b74700 1 -- start start 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.896+0000 7f6c39b74700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.897+0000 7f6c39b74700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3410e0a0 con 0x7f6c3410a7a0 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.897+0000 7f6c337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.897+0000 7f6c337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3419e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58710/0 (socket says 172.21.15.19:58710) 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.897+0000 7f6c337fe700 1 -- 172.21.15.19:0/3573411641 learned_addr learned my addr 172.21.15.19:0/3573411641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.897+0000 7f6c337fe700 1 -- 172.21.15.19:0/3573411641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3419e560 con 0x7f6c3410a7a0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.897+0000 7f6c337fe700 1 --2- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3419e020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c240119f0 tx=0x7f6c24011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.897+0000 7f6c30ff9700 1 -- 172.21.15.19:0/3573411641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c240039e0 con 0x7f6c3410a7a0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.897+0000 7f6c30ff9700 1 -- 172.21.15.19:0/3573411641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c2401b8f0 con 0x7f6c3410a7a0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.897+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3419e7f0 con 0x7f6c3410a7a0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.898+0000 7f6c30ff9700 1 -- 172.21.15.19:0/3573411641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c240237a0 con 0x7f6c3410a7a0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.898+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c341a1400 con 0x7f6c3410a7a0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.899+0000 7f6c30ff9700 1 -- 172.21.15.19:0/3573411641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f6c24023aa0 con 0x7f6c3410a7a0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.899+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c34064e20 con 0x7f6c3410a7a0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.899+0000 7f6c30ff9700 1 --2- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6c1c03e4d0 0x7f6c1c0409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.899+0000 7f6c30ff9700 1 -- 172.21.15.19:0/3573411641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c24054050 con 0x7f6c3410a7a0 2024-05-09T01:15:26.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.899+0000 7f6c32ffd700 1 --2- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6c1c03e4d0 0x7f6c1c0409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.900+0000 7f6c32ffd700 1 --2- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6c1c03e4d0 0x7f6c1c0409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c28006fd0 tx=0x7f6c28006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:24.906+0000 7f6c30ff9700 1 -- 172.21.15.19:0/3573411641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c2401f030 con 0x7f6c3410a7a0 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.143+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f6c341a15d0 con 0x7f6c1c03e4d0 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.390+0000 7f6c30ff9700 1 -- 172.21.15.19:0/3573411641 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f6c341a15d0 con 0x7f6c1c03e4d0 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.392+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6c1c03e4d0 msgr2=0x7f6c1c0409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.392+0000 7f6c39b74700 1 --2- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6c1c03e4d0 0x7f6c1c0409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c28006fd0 tx=0x7f6c28006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.392+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 msgr2=0x7f6c3419e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.392+0000 7f6c39b74700 1 --2- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3419e020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c240119f0 tx=0x7f6c24011ad0 comp rx=0 tx=0).stop 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.393+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 shutdown_connections 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.393+0000 7f6c39b74700 1 --2- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6c1c03e4d0 0x7f6c1c0409e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.393+0000 7f6c39b74700 1 --2- 172.21.15.19:0/3573411641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3410a7a0 0x7f6c3419e020 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.393+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 >> 172.21.15.19:0/3573411641 conn(0x7f6c34104240 msgr2=0x7f6c34105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.393+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 shutdown_connections 2024-05-09T01:15:26.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:25.393+0000 7f6c39b74700 1 -- 172.21.15.19:0/3573411641 wait complete. 2024-05-09T01:15:26.510 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-05-09T01:15:27.488 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:27 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1652489649' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T01:15:27.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T01:15:27.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.933+0000 7fd39a77b700 1 Processor -- start 2024-05-09T01:15:27.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.933+0000 7fd39a77b700 1 -- start start 2024-05-09T01:15:27.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.933+0000 7fd39a77b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39410cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:27.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.933+0000 7fd39a77b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd39407ce70 con 0x7fd39410a750 2024-05-09T01:15:27.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.933+0000 7fd393fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39410cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:27.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.933+0000 7fd393fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39410cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58714/0 (socket says 172.21.15.19:58714) 2024-05-09T01:15:27.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.933+0000 7fd393fff700 1 -- 172.21.15.19:0/1649434540 learned_addr learned my addr 172.21.15.19:0/1649434540 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:27.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.934+0000 7fd393fff700 1 -- 172.21.15.19:0/1649434540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39410d0b0 con 0x7fd39410a750 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.934+0000 7fd393fff700 1 --2- 172.21.15.19:0/1649434540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39410cb70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098a0 tx=0x7fd384009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffae73d62de2264 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.934+0000 7fd392ffd700 1 -- 172.21.15.19:0/1649434540 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384004160 con 0x7fd39410a750 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.934+0000 7fd392ffd700 1 -- 172.21.15.19:0/1649434540 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd384004300 con 0x7fd39410a750 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.934+0000 7fd392ffd700 1 -- 172.21.15.19:0/1649434540 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384011a20 con 0x7fd39410a750 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.935+0000 7fd39a77b700 1 -- 172.21.15.19:0/1649434540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 msgr2=0x7fd39410cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.935+0000 7fd39a77b700 1 --2- 172.21.15.19:0/1649434540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39410cb70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098a0 tx=0x7fd384009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.935+0000 7fd39a77b700 1 -- 172.21.15.19:0/1649434540 shutdown_connections 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.935+0000 7fd39a77b700 1 --2- 172.21.15.19:0/1649434540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39410cb70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.935+0000 7fd39a77b700 1 -- 172.21.15.19:0/1649434540 >> 172.21.15.19:0/1649434540 conn(0x7fd394104250 msgr2=0x7fd3941066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.935+0000 7fd39a77b700 1 -- 172.21.15.19:0/1649434540 shutdown_connections 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.935+0000 7fd39a77b700 1 -- 172.21.15.19:0/1649434540 wait complete. 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.936+0000 7fd39a77b700 1 Processor -- start 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.936+0000 7fd39a77b700 1 -- start start 2024-05-09T01:15:27.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.936+0000 7fd39a77b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.936+0000 7fd39a77b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd39410e0b0 con 0x7fd39410a750 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.936+0000 7fd393fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.936+0000 7fd393fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58716/0 (socket says 172.21.15.19:58716) 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.936+0000 7fd393fff700 1 -- 172.21.15.19:0/1652489649 learned_addr learned my addr 172.21.15.19:0/1652489649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.936+0000 7fd393fff700 1 -- 172.21.15.19:0/1652489649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39419e4e0 con 0x7fd39410a750 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.937+0000 7fd393fff700 1 --2- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39419dfa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd384000c00 tx=0x7fd384011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.937+0000 7fd3917fa700 1 -- 172.21.15.19:0/1652489649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384003aa0 con 0x7fd39410a750 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.937+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39419e770 con 0x7fd39410a750 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.937+0000 7fd3917fa700 1 -- 172.21.15.19:0/1652489649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd38401b8f0 con 0x7fd39410a750 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.937+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3941a1380 con 0x7fd39410a750 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.937+0000 7fd3917fa700 1 -- 172.21.15.19:0/1652489649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384023890 con 0x7fd39410a750 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.938+0000 7fd3917fa700 1 -- 172.21.15.19:0/1652489649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fd38402c3f0 con 0x7fd39410a750 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.938+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd380005360 con 0x7fd39410a750 2024-05-09T01:15:27.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.938+0000 7fd3917fa700 1 --2- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd37c03e520 0x7fd37c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.938+0000 7fd3917fa700 1 -- 172.21.15.19:0/1652489649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd384053ec0 con 0x7fd39410a750 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.938+0000 7fd3937fe700 1 --2- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd37c03e520 0x7fd37c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.939+0000 7fd3937fe700 1 --2- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd37c03e520 0x7fd37c040a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd388006fd0 tx=0x7fd388006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:26.944+0000 7fd3917fa700 1 -- 172.21.15.19:0/1652489649 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd38401f030 con 0x7fd39410a750 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.158+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd380005650 con 0x7fd39410a750 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.159+0000 7fd3917fa700 1 -- 172.21.15.19:0/1652489649 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd38400f2e0 con 0x7fd39410a750 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd37c03e520 msgr2=0x7fd37c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 --2- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd37c03e520 0x7fd37c040a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd388006fd0 tx=0x7fd388006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 msgr2=0x7fd39419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 --2- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39419dfa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd384000c00 tx=0x7fd384011aa0 comp rx=0 tx=0).stop 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 shutdown_connections 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 --2- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd37c03e520 0x7fd37c040a30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:27.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 --2- 172.21.15.19:0/1652489649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39410a750 0x7fd39419dfa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:27.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 >> 172.21.15.19:0/1652489649 conn(0x7fd394104250 msgr2=0x7fd394105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:27.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 shutdown_connections 2024-05-09T01:15:27.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:27.161+0000 7fd39a77b700 1 -- 172.21.15.19:0/1652489649 wait complete. 2024-05-09T01:15:27.781 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T01:15:27.781 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-09T01:15:27.790 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T01:15:27.790 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:15:27.790 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-09T01:15:27.790 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T01:15:27.794 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-05-09T01:15:27.794 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:27.794 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019.front.sepia.ceph.com:8443/ 2024-05-09T01:15:27.794 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-05-09T01:15:27.794 INFO:teuthology.orchestra.run.smithi019.stdout: Password: imuhamlktv 2024-05-09T01:15:27.794 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:27.794 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config directory 2024-05-09T01:15:27.794 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling autotune for osd_memory_target 2024-05-09T01:15:28.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:28 smithi019 ceph-mon[30158]: mgrmap e13: x(active, since 6s) 2024-05-09T01:15:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.241+0000 7fbdd28d5700 1 Processor -- start 2024-05-09T01:15:29.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.241+0000 7fbdd28d5700 1 -- start start 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.241+0000 7fbdd28d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc108f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.241+0000 7fbdd28d5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc109500 con 0x7fbdcc108b00 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.242+0000 7fbdcbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.242+0000 7fbdcbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc108f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58732/0 (socket says 172.21.15.19:58732) 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.242+0000 7fbdcbfff700 1 -- 172.21.15.19:0/2722012676 learned_addr learned my addr 172.21.15.19:0/2722012676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.242+0000 7fbdcbfff700 1 -- 172.21.15.19:0/2722012676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc109cd0 con 0x7fbdcc108b00 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.242+0000 7fbdcbfff700 1 --2- 172.21.15.19:0/2722012676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc108f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0098a0 tx=0x7fbdbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132b3725a7b676a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.242+0000 7fbdcaffd700 1 -- 172.21.15.19:0/2722012676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdbc004160 con 0x7fbdcc108b00 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.242+0000 7fbdcaffd700 1 -- 172.21.15.19:0/2722012676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdbc004300 con 0x7fbdcc108b00 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.242+0000 7fbdcaffd700 1 -- 172.21.15.19:0/2722012676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdbc011a20 con 0x7fbdcc108b00 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.243+0000 7fbdd28d5700 1 -- 172.21.15.19:0/2722012676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 msgr2=0x7fbdcc108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:29.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.243+0000 7fbdd28d5700 1 --2- 172.21.15.19:0/2722012676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc108f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0098a0 tx=0x7fbdbc009bb0 comp rx=0 tx=0).stop 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.243+0000 7fbdd28d5700 1 -- 172.21.15.19:0/2722012676 shutdown_connections 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.243+0000 7fbdd28d5700 1 --2- 172.21.15.19:0/2722012676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc108f10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.243+0000 7fbdd28d5700 1 -- 172.21.15.19:0/2722012676 >> 172.21.15.19:0/2722012676 conn(0x7fbdcc1041e0 msgr2=0x7fbdcc106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.243+0000 7fbdd28d5700 1 -- 172.21.15.19:0/2722012676 shutdown_connections 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.243+0000 7fbdd28d5700 1 -- 172.21.15.19:0/2722012676 wait complete. 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.244+0000 7fbdd28d5700 1 Processor -- start 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.244+0000 7fbdd28d5700 1 -- start start 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.244+0000 7fbdd28d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc1073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.244+0000 7fbdd28d5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc10d080 con 0x7fbdcc108b00 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.244+0000 7fbdcbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc1073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.244+0000 7fbdcbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc1073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58740/0 (socket says 172.21.15.19:58740) 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.244+0000 7fbdcbfff700 1 -- 172.21.15.19:0/3294911352 learned_addr learned my addr 172.21.15.19:0/3294911352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:29.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.245+0000 7fbdcbfff700 1 -- 172.21.15.19:0/3294911352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc107920 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.245+0000 7fbdcbfff700 1 --2- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc1073e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009130 tx=0x7fbdbc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.245+0000 7fbdc97fa700 1 -- 172.21.15.19:0/3294911352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdbc003aa0 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.245+0000 7fbdc97fa700 1 -- 172.21.15.19:0/3294911352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdbc01a8e0 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.245+0000 7fbdc97fa700 1 -- 172.21.15.19:0/3294911352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdbc0248f0 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.245+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdcc105b70 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.245+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdcc106050 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.246+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdcc064e20 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.246+0000 7fbdc97fa700 1 -- 172.21.15.19:0/3294911352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbdbc022020 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.246+0000 7fbdc97fa700 1 --2- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbdb403e560 0x7fbdb4040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.246+0000 7fbdc97fa700 1 -- 172.21.15.19:0/3294911352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbdbc053bd0 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.247+0000 7fbdcb7fe700 1 --2- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbdb403e560 0x7fbdb4040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.247+0000 7fbdcb7fe700 1 --2- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbdb403e560 0x7fbdb4040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0006fd0 tx=0x7fbdc0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.252+0000 7fbdc97fa700 1 -- 172.21.15.19:0/3294911352 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbdbc02c350 con 0x7fbdcc108b00 2024-05-09T01:15:29.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.464+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fbdcc197280 con 0x7fbdcc108b00 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.465+0000 7fbdc97fa700 1 -- 172.21.15.19:0/3294911352 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fbdbc010e70 con 0x7fbdcc108b00 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.467+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbdb403e560 msgr2=0x7fbdb4040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.467+0000 7fbdd28d5700 1 --2- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbdb403e560 0x7fbdb4040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0006fd0 tx=0x7fbdc0006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.467+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 msgr2=0x7fbdcc1073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.467+0000 7fbdd28d5700 1 --2- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc1073e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009130 tx=0x7fbdbc011aa0 comp rx=0 tx=0).stop 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.467+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 shutdown_connections 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.467+0000 7fbdd28d5700 1 --2- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbdb403e560 0x7fbdb4040a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.467+0000 7fbdd28d5700 1 --2- 172.21.15.19:0/3294911352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdcc108b00 0x7fbdcc1073e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.467+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 >> 172.21.15.19:0/3294911352 conn(0x7fbdcc1041e0 msgr2=0x7fbdcc194650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.468+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 shutdown_connections 2024-05-09T01:15:29.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:28.468+0000 7fbdd28d5700 1 -- 172.21.15.19:0/3294911352 wait complete. 2024-05-09T01:15:30.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.484+0000 7f7963b67700 1 Processor -- start 2024-05-09T01:15:30.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.484+0000 7f7963b67700 1 -- start start 2024-05-09T01:15:30.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.485+0000 7f7963b67700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:30.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.485+0000 7f7963b67700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f795c07cdf0 con 0x7f795c108550 2024-05-09T01:15:30.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.485+0000 7f7961903700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:30.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.485+0000 7f7961903700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58742/0 (socket says 172.21.15.19:58742) 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.485+0000 7f7961903700 1 -- 172.21.15.19:0/1008558398 learned_addr learned my addr 172.21.15.19:0/1008558398 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.485+0000 7f7961903700 1 -- 172.21.15.19:0/1008558398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c108ea0 con 0x7f795c108550 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.485+0000 7f7961903700 1 --2- 172.21.15.19:0/1008558398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c108960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7958009e80 tx=0x7f795800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eeb7a2d6f990835 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.485+0000 7f7960901700 1 -- 172.21.15.19:0/1008558398 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7958017070 con 0x7f795c108550 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.486+0000 7f7960901700 1 -- 172.21.15.19:0/1008558398 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795800b8c0 con 0x7f795c108550 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.486+0000 7f7960901700 1 -- 172.21.15.19:0/1008558398 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7958004490 con 0x7f795c108550 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.486+0000 7f7963b67700 1 -- 172.21.15.19:0/1008558398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 msgr2=0x7f795c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.486+0000 7f7963b67700 1 --2- 172.21.15.19:0/1008558398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c108960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7958009e80 tx=0x7f795800b330 comp rx=0 tx=0).stop 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.486+0000 7f7963b67700 1 -- 172.21.15.19:0/1008558398 shutdown_connections 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.486+0000 7f7963b67700 1 --2- 172.21.15.19:0/1008558398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c108960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.486+0000 7f7963b67700 1 -- 172.21.15.19:0/1008558398 >> 172.21.15.19:0/1008558398 conn(0x7f795c1041e0 msgr2=0x7f795c106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.486+0000 7f7963b67700 1 -- 172.21.15.19:0/1008558398 shutdown_connections 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.486+0000 7f7963b67700 1 -- 172.21.15.19:0/1008558398 wait complete. 2024-05-09T01:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.487+0000 7f7963b67700 1 Processor -- start 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.487+0000 7f7963b67700 1 -- start start 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.487+0000 7f7963b67700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.487+0000 7f7963b67700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f795c109f60 con 0x7f795c108550 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7961903700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7961903700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c1a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58748/0 (socket says 172.21.15.19:58748) 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7961903700 1 -- 172.21.15.19:0/3735901342 learned_addr learned my addr 172.21.15.19:0/3735901342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7961903700 1 -- 172.21.15.19:0/3735901342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c1a07e0 con 0x7f795c108550 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7961903700 1 --2- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c1a02a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f795800b7b0 tx=0x7f795800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7952ffd700 1 -- 172.21.15.19:0/3735901342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7958020020 con 0x7f795c108550 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7952ffd700 1 -- 172.21.15.19:0/3735901342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7958012430 con 0x7f795c108550 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c1a0a70 con 0x7f795c108550 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7952ffd700 1 -- 172.21.15.19:0/3735901342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7958017040 con 0x7f795c108550 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.488+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c1a3680 con 0x7f795c108550 2024-05-09T01:15:30.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.490+0000 7f7952ffd700 1 -- 172.21.15.19:0/3735901342 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7958023020 con 0x7f795c108550 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.490+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c064e20 con 0x7f795c108550 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.490+0000 7f7952ffd700 1 --2- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f794803e560 0x7f7948040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.490+0000 7f7952ffd700 1 -- 172.21.15.19:0/3735901342 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f795800f050 con 0x7f795c108550 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.490+0000 7f7961102700 1 --2- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f794803e560 0x7f7948040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.490+0000 7f7961102700 1 --2- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f794803e560 0x7f7948040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f794c006fd0 tx=0x7f794c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.496+0000 7f7952ffd700 1 -- 172.21.15.19:0/3735901342 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f795802a020 con 0x7f795c108550 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.826+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f795c02e8c0 con 0x7f795c108550 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.829+0000 7f7952ffd700 1 -- 172.21.15.19:0/3735901342 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f7958012650 con 0x7f795c108550 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f794803e560 msgr2=0x7f7948040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 --2- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f794803e560 0x7f7948040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f794c006fd0 tx=0x7f794c006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 msgr2=0x7f795c1a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 --2- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c1a02a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f795800b7b0 tx=0x7f795800bd30 comp rx=0 tx=0).stop 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 shutdown_connections 2024-05-09T01:15:30.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 --2- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f794803e560 0x7f7948040a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 --2- 172.21.15.19:0/3735901342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f795c108550 0x7f795c1a02a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 >> 172.21.15.19:0/3735901342 conn(0x7f795c1041e0 msgr2=0x7f795c194240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 shutdown_connections 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:15:29.832+0000 7f7963b67700 1 -- 172.21.15.19:0/3735901342 wait complete. 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:30.447 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T01:15:30.448 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:30.448 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-05-09T01:15:30.448 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:30.448 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-05-09T01:15:30.448 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:30.448 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T01:15:30.448 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:30.448 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-05-09T01:15:30.490 INFO:tasks.cephadm:Fetching config... 2024-05-09T01:15:30.490 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:15:30.490 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T01:15:30.508 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T01:15:30.508 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:15:30.508 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T01:15:30.565 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T01:15:30.565 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:15:30.565 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T01:15:30.632 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T01:15:30.632 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:15:30.632 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T01:15:30.688 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T01:15:30.688 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCek/d7Cv3G0HaQRvk2oFaITO+1xWOHbqX2no2KAQHesROhGeBH7FdUITDfc/irK2SDlnYmwaS8X+PDx69L634pZzB11EuUd6zIJYJLXV2U57HCPNijt8j4HMFNPfKLu2JH9YYG33yX7qJsnGBmwfOMTzCveXjuQAeG9X+GOAnh0uxQCAPYWCF8tMJzACdf6DUlALRozoml9DpzUmuBf2ynyUz7A38f+1NE2SCEFaVkqwHIUGY1LrKf3jKc2/iuTgLC8rc8vTu5hoAlx8hXsdaYorjKJYCtPCR86952X94sAQrcwufc6Ewf4mHXALVl0/7sT2w27ur9hx/B+zUr14hIjbEwpQVYXgmIc11aix7WrrWsA2MLI1fctcuVKSOua5JlJ4q5+cZlsy96Zld0W+8HxURS6RGIb9/6wES1dvWY23xclGOPazQcRzkaFdK1TQ2BEPr1eazdlmMT1CuR4CYmAXtjkkETeQBV04ZTIx29u7KjMqysJsXHUX0dBRQ4/nc= ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T01:15:30.769 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCek/d7Cv3G0HaQRvk2oFaITO+1xWOHbqX2no2KAQHesROhGeBH7FdUITDfc/irK2SDlnYmwaS8X+PDx69L634pZzB11EuUd6zIJYJLXV2U57HCPNijt8j4HMFNPfKLu2JH9YYG33yX7qJsnGBmwfOMTzCveXjuQAeG9X+GOAnh0uxQCAPYWCF8tMJzACdf6DUlALRozoml9DpzUmuBf2ynyUz7A38f+1NE2SCEFaVkqwHIUGY1LrKf3jKc2/iuTgLC8rc8vTu5hoAlx8hXsdaYorjKJYCtPCR86952X94sAQrcwufc6Ewf4mHXALVl0/7sT2w27ur9hx/B+zUr14hIjbEwpQVYXgmIc11aix7WrrWsA2MLI1fctcuVKSOua5JlJ4q5+cZlsy96Zld0W+8HxURS6RGIb9/6wES1dvWY23xclGOPazQcRzkaFdK1TQ2BEPr1eazdlmMT1CuR4CYmAXtjkkETeQBV04ZTIx29u7KjMqysJsXHUX0dBRQ4/nc= ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:15:30.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCek/d7Cv3G0HaQRvk2oFaITO+1xWOHbqX2no2KAQHesROhGeBH7FdUITDfc/irK2SDlnYmwaS8X+PDx69L634pZzB11EuUd6zIJYJLXV2U57HCPNijt8j4HMFNPfKLu2JH9YYG33yX7qJsnGBmwfOMTzCveXjuQAeG9X+GOAnh0uxQCAPYWCF8tMJzACdf6DUlALRozoml9DpzUmuBf2ynyUz7A38f+1NE2SCEFaVkqwHIUGY1LrKf3jKc2/iuTgLC8rc8vTu5hoAlx8hXsdaYorjKJYCtPCR86952X94sAQrcwufc6Ewf4mHXALVl0/7sT2w27ur9hx/B+zUr14hIjbEwpQVYXgmIc11aix7WrrWsA2MLI1fctcuVKSOua5JlJ4q5+cZlsy96Zld0W+8HxURS6RGIb9/6wES1dvWY23xclGOPazQcRzkaFdK1TQ2BEPr1eazdlmMT1CuR4CYmAXtjkkETeQBV04ZTIx29u7KjMqysJsXHUX0dBRQ4/nc= ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T01:15:30.821 INFO:teuthology.orchestra.run.smithi059.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCek/d7Cv3G0HaQRvk2oFaITO+1xWOHbqX2no2KAQHesROhGeBH7FdUITDfc/irK2SDlnYmwaS8X+PDx69L634pZzB11EuUd6zIJYJLXV2U57HCPNijt8j4HMFNPfKLu2JH9YYG33yX7qJsnGBmwfOMTzCveXjuQAeG9X+GOAnh0uxQCAPYWCF8tMJzACdf6DUlALRozoml9DpzUmuBf2ynyUz7A38f+1NE2SCEFaVkqwHIUGY1LrKf3jKc2/iuTgLC8rc8vTu5hoAlx8hXsdaYorjKJYCtPCR86952X94sAQrcwufc6Ewf4mHXALVl0/7sT2w27ur9hx/B+zUr14hIjbEwpQVYXgmIc11aix7WrrWsA2MLI1fctcuVKSOua5JlJ4q5+cZlsy96Zld0W+8HxURS6RGIb9/6wES1dvWY23xclGOPazQcRzkaFdK1TQ2BEPr1eazdlmMT1CuR4CYmAXtjkkETeQBV04ZTIx29u7KjMqysJsXHUX0dBRQ4/nc= ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:15:30.834 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCek/d7Cv3G0HaQRvk2oFaITO+1xWOHbqX2no2KAQHesROhGeBH7FdUITDfc/irK2SDlnYmwaS8X+PDx69L634pZzB11EuUd6zIJYJLXV2U57HCPNijt8j4HMFNPfKLu2JH9YYG33yX7qJsnGBmwfOMTzCveXjuQAeG9X+GOAnh0uxQCAPYWCF8tMJzACdf6DUlALRozoml9DpzUmuBf2ynyUz7A38f+1NE2SCEFaVkqwHIUGY1LrKf3jKc2/iuTgLC8rc8vTu5hoAlx8hXsdaYorjKJYCtPCR86952X94sAQrcwufc6Ewf4mHXALVl0/7sT2w27ur9hx/B+zUr14hIjbEwpQVYXgmIc11aix7WrrWsA2MLI1fctcuVKSOua5JlJ4q5+cZlsy96Zld0W+8HxURS6RGIb9/6wES1dvWY23xclGOPazQcRzkaFdK1TQ2BEPr1eazdlmMT1CuR4CYmAXtjkkETeQBV04ZTIx29u7KjMqysJsXHUX0dBRQ4/nc= ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T01:15:30.873 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCek/d7Cv3G0HaQRvk2oFaITO+1xWOHbqX2no2KAQHesROhGeBH7FdUITDfc/irK2SDlnYmwaS8X+PDx69L634pZzB11EuUd6zIJYJLXV2U57HCPNijt8j4HMFNPfKLu2JH9YYG33yX7qJsnGBmwfOMTzCveXjuQAeG9X+GOAnh0uxQCAPYWCF8tMJzACdf6DUlALRozoml9DpzUmuBf2ynyUz7A38f+1NE2SCEFaVkqwHIUGY1LrKf3jKc2/iuTgLC8rc8vTu5hoAlx8hXsdaYorjKJYCtPCR86952X94sAQrcwufc6Ewf4mHXALVl0/7sT2w27ur9hx/B+zUr14hIjbEwpQVYXgmIc11aix7WrrWsA2MLI1fctcuVKSOua5JlJ4q5+cZlsy96Zld0W+8HxURS6RGIb9/6wES1dvWY23xclGOPazQcRzkaFdK1TQ2BEPr1eazdlmMT1CuR4CYmAXtjkkETeQBV04ZTIx29u7KjMqysJsXHUX0dBRQ4/nc= ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:15:30.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T01:15:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:15:31.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:30 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3735901342' entity='client.admin' 2024-05-09T01:15:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.238+0000 7fea97422700 1 -- 172.21.15.19:0/977291276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 msgr2=0x7fea90103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.238+0000 7fea97422700 1 --2- 172.21.15.19:0/977291276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 0x7fea90103ff0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009910 tx=0x7fea8c009c20 comp rx=0 tx=0).stop 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.238+0000 7fea97422700 1 -- 172.21.15.19:0/977291276 shutdown_connections 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.238+0000 7fea97422700 1 --2- 172.21.15.19:0/977291276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 0x7fea90103ff0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.238+0000 7fea97422700 1 -- 172.21.15.19:0/977291276 >> 172.21.15.19:0/977291276 conn(0x7fea900ff220 msgr2=0x7fea901016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.238+0000 7fea97422700 1 -- 172.21.15.19:0/977291276 shutdown_connections 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.238+0000 7fea97422700 1 -- 172.21.15.19:0/977291276 wait complete. 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.239+0000 7fea97422700 1 Processor -- start 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.239+0000 7fea97422700 1 -- start start 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.239+0000 7fea97422700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 0x7fea90199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.239+0000 7fea97422700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea90109ba0 con 0x7fea90103be0 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.239+0000 7fea951be700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 0x7fea90199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.239+0000 7fea951be700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 0x7fea90199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51156/0 (socket says 172.21.15.19:51156) 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.239+0000 7fea951be700 1 -- 172.21.15.19:0/3607616288 learned_addr learned my addr 172.21.15.19:0/3607616288 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.239+0000 7fea951be700 1 -- 172.21.15.19:0/3607616288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea90199c60 con 0x7fea90103be0 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.240+0000 7fea951be700 1 --2- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 0x7fea90199720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0057d0 tx=0x7fea8c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.240+0000 7fea867fc700 1 -- 172.21.15.19:0/3607616288 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea8c027070 con 0x7fea90103be0 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.240+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea90199ef0 con 0x7fea90103be0 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.240+0000 7fea867fc700 1 -- 172.21.15.19:0/3607616288 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c00be10 con 0x7fea90103be0 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.240+0000 7fea867fc700 1 -- 172.21.15.19:0/3607616288 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea8c022dd0 con 0x7fea90103be0 2024-05-09T01:15:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.240+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9019cb00 con 0x7fea90103be0 2024-05-09T01:15:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.241+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9010a320 con 0x7fea90103be0 2024-05-09T01:15:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.241+0000 7fea867fc700 1 -- 172.21.15.19:0/3607616288 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fea8c02a070 con 0x7fea90103be0 2024-05-09T01:15:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.241+0000 7fea867fc700 1 --2- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fea7c03e570 0x7fea7c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.241+0000 7fea867fc700 1 -- 172.21.15.19:0/3607616288 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fea8c01e030 con 0x7fea90103be0 2024-05-09T01:15:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.242+0000 7fea949bd700 1 --2- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fea7c03e570 0x7fea7c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.242+0000 7fea949bd700 1 --2- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fea7c03e570 0x7fea7c040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea80006fd0 tx=0x7fea80006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:33.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.247+0000 7fea867fc700 1 -- 172.21.15.19:0/3607616288 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea8c013070 con 0x7fea90103be0 2024-05-09T01:15:33.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.464+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fea9002e700 con 0x7fea90103be0 2024-05-09T01:15:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.467+0000 7fea867fc700 1 -- 172.21.15.19:0/3607616288 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fea8c0310c0 con 0x7fea90103be0 2024-05-09T01:15:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fea7c03e570 msgr2=0x7fea7c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 --2- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fea7c03e570 0x7fea7c040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea80006fd0 tx=0x7fea80006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 msgr2=0x7fea90199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 --2- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 0x7fea90199720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0057d0 tx=0x7fea8c005800 comp rx=0 tx=0).stop 2024-05-09T01:15:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 shutdown_connections 2024-05-09T01:15:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 --2- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fea7c03e570 0x7fea7c040a80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 --2- 172.21.15.19:0/3607616288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea90103be0 0x7fea90199720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 >> 172.21.15.19:0/3607616288 conn(0x7fea900ff220 msgr2=0x7fea900fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 shutdown_connections 2024-05-09T01:15:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:33.470+0000 7fea97422700 1 -- 172.21.15.19:0/3607616288 wait complete. 2024-05-09T01:15:34.244 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T01:15:34.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T01:15:34.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:34 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3607616288' entity='client.admin' 2024-05-09T01:15:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:15:36.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:36.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:36.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:15:36.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:36.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:15:36.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.476+0000 7f1347af9700 1 -- 172.21.15.19:0/2488050697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 msgr2=0x7f1340106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.476+0000 7f1347af9700 1 --2- 172.21.15.19:0/2488050697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 0x7f1340106b30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f133800b0a0 tx=0x7f133800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:15:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.476+0000 7f1347af9700 1 -- 172.21.15.19:0/2488050697 shutdown_connections 2024-05-09T01:15:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.476+0000 7f1347af9700 1 --2- 172.21.15.19:0/2488050697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 0x7f1340106b30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.476+0000 7f1347af9700 1 -- 172.21.15.19:0/2488050697 >> 172.21.15.19:0/2488050697 conn(0x7f13400fe130 msgr2=0x7f1340100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.476+0000 7f1347af9700 1 -- 172.21.15.19:0/2488050697 shutdown_connections 2024-05-09T01:15:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.476+0000 7f1347af9700 1 -- 172.21.15.19:0/2488050697 wait complete. 2024-05-09T01:15:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.477+0000 7f1347af9700 1 Processor -- start 2024-05-09T01:15:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.477+0000 7f1347af9700 1 -- start start 2024-05-09T01:15:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.477+0000 7f1347af9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 0x7f1340199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f1347af9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1340109bd0 con 0x7f1340104710 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f1345895700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 0x7f1340199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f1345895700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 0x7f1340199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51172/0 (socket says 172.21.15.19:51172) 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f1345895700 1 -- 172.21.15.19:0/4221496474 learned_addr learned my addr 172.21.15.19:0/4221496474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f1345895700 1 -- 172.21.15.19:0/4221496474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1340199bb0 con 0x7f1340104710 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f1345895700 1 --2- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 0x7f1340199670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1338015040 tx=0x7f1338003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f133effd700 1 -- 172.21.15.19:0/4221496474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1338021020 con 0x7f1340104710 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f133effd700 1 -- 172.21.15.19:0/4221496474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13380044b0 con 0x7f1340104710 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1340199e40 con 0x7f1340104710 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f133effd700 1 -- 172.21.15.19:0/4221496474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1338017040 con 0x7f1340104710 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.478+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134019ca50 con 0x7f1340104710 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.479+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134010a350 con 0x7f1340104710 2024-05-09T01:15:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.479+0000 7f133effd700 1 -- 172.21.15.19:0/4221496474 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1338024020 con 0x7f1340104710 2024-05-09T01:15:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.480+0000 7f133effd700 1 --2- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f132803e510 0x7f1328040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.480+0000 7f133effd700 1 -- 172.21.15.19:0/4221496474 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f133800f050 con 0x7f1340104710 2024-05-09T01:15:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.480+0000 7f1345094700 1 --2- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f132803e510 0x7f1328040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.480+0000 7f1345094700 1 --2- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f132803e510 0x7f1328040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1334006fd0 tx=0x7f1334006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.485+0000 7f133effd700 1 -- 172.21.15.19:0/4221496474 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f133802b020 con 0x7f1340104710 2024-05-09T01:15:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.703+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f134002e700 con 0x7f132803e510 2024-05-09T01:15:36.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.706+0000 7f133effd700 1 -- 172.21.15.19:0/4221496474 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f134002e700 con 0x7f132803e510 2024-05-09T01:15:36.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f132803e510 msgr2=0x7f1328040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:36.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 --2- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f132803e510 0x7f1328040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1334006fd0 tx=0x7f1334006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:36.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 msgr2=0x7f1340199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:36.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 --2- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 0x7f1340199670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1338015040 tx=0x7f1338003ce0 comp rx=0 tx=0).stop 2024-05-09T01:15:36.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 shutdown_connections 2024-05-09T01:15:36.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 --2- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f132803e510 0x7f1328040a20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:36.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 --2- 172.21.15.19:0/4221496474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340104710 0x7f1340199670 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:36.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 >> 172.21.15.19:0/4221496474 conn(0x7f13400fe130 msgr2=0x7f13400fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:36.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 shutdown_connections 2024-05-09T01:15:36.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:36.709+0000 7f1347af9700 1 -- 172.21.15.19:0/4221496474 wait complete. 2024-05-09T01:15:37.433 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi059 2024-05-09T01:15:37.433 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:15:37.433 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/etc/ceph/ceph.conf 2024-05-09T01:15:37.452 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:15:37.452 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:15:37.510 INFO:tasks.cephadm:Adding host smithi059 to orchestrator... 2024-05-09T01:15:37.511 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch host add smithi059 2024-05-09T01:15:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:15:38.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:37 smithi019 ceph-mon[30158]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:15:38.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:38.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:15:38.007 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:15:38.007 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:15:38.007 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:37 smithi019 ceph-mon[30158]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T01:15:38.007 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:37 smithi019 ceph-mon[30158]: Updating smithi019:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:15:38.007 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:37 smithi019 ceph-mon[30158]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:15:39.008 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:38 smithi019 ceph-mon[30158]: Updating smithi019:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T01:15:39.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:39.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:39.009 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:39.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.257+0000 7f8b4be6d700 1 -- 172.21.15.19:0/1584357220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 msgr2=0x7f8b441050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.257+0000 7f8b4be6d700 1 --2- 172.21.15.19:0/1584357220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 0x7f8b441050e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009910 tx=0x7f8b40009c20 comp rx=0 tx=0).stop 2024-05-09T01:15:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.257+0000 7f8b4be6d700 1 -- 172.21.15.19:0/1584357220 shutdown_connections 2024-05-09T01:15:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.257+0000 7f8b4be6d700 1 --2- 172.21.15.19:0/1584357220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 0x7f8b441050e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.257+0000 7f8b4be6d700 1 -- 172.21.15.19:0/1584357220 >> 172.21.15.19:0/1584357220 conn(0x7f8b440fe560 msgr2=0x7f8b441009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.257+0000 7f8b4be6d700 1 -- 172.21.15.19:0/1584357220 shutdown_connections 2024-05-09T01:15:39.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.257+0000 7f8b4be6d700 1 -- 172.21.15.19:0/1584357220 wait complete. 2024-05-09T01:15:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.258+0000 7f8b4be6d700 1 Processor -- start 2024-05-09T01:15:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.258+0000 7f8b4be6d700 1 -- start start 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.258+0000 7f8b4be6d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 0x7f8b441029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.258+0000 7f8b4be6d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4410e130 con 0x7f8b44104cd0 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.258+0000 7f8b49c09700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 0x7f8b441029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.258+0000 7f8b49c09700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 0x7f8b441029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51190/0 (socket says 172.21.15.19:51190) 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.258+0000 7f8b49c09700 1 -- 172.21.15.19:0/3008388130 learned_addr learned my addr 172.21.15.19:0/3008388130 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.258+0000 7f8b49c09700 1 -- 172.21.15.19:0/3008388130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b441047c0 con 0x7f8b44104cd0 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.259+0000 7f8b49c09700 1 --2- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 0x7f8b441029c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b400056d0 tx=0x7f8b400057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.259+0000 7f8b36ffd700 1 -- 172.21.15.19:0/3008388130 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b40026070 con 0x7f8b44104cd0 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.259+0000 7f8b36ffd700 1 -- 172.21.15.19:0/3008388130 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b40004370 con 0x7f8b44104cd0 2024-05-09T01:15:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.259+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b44102f00 con 0x7f8b44104cd0 2024-05-09T01:15:39.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.259+0000 7f8b36ffd700 1 -- 172.21.15.19:0/3008388130 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b400039b0 con 0x7f8b44104cd0 2024-05-09T01:15:39.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.259+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b44103360 con 0x7f8b44104cd0 2024-05-09T01:15:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.260+0000 7f8b36ffd700 1 -- 172.21.15.19:0/3008388130 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8b40029070 con 0x7f8b44104cd0 2024-05-09T01:15:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.260+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4406c3c0 con 0x7f8b44104cd0 2024-05-09T01:15:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.260+0000 7f8b36ffd700 1 --2- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b3003e510 0x7f8b30040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.260+0000 7f8b36ffd700 1 -- 172.21.15.19:0/3008388130 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8b4001e070 con 0x7f8b44104cd0 2024-05-09T01:15:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.260+0000 7f8b49408700 1 --2- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b3003e510 0x7f8b30040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.261+0000 7f8b49408700 1 --2- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b3003e510 0x7f8b30040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8b38006fd0 tx=0x7f8b38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.266+0000 7f8b36ffd700 1 -- 172.21.15.19:0/3008388130 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b40013070 con 0x7f8b44104cd0 2024-05-09T01:15:39.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:39.486+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi059", "target": ["mon-mgr", ""]}) -- 0x7f8b44103f10 con 0x7f8b3003e510 2024-05-09T01:15:40.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:40 smithi019 ceph-mon[30158]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi059", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:15:40.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:40 smithi019 ceph-mon[30158]: Deploying cephadm binary to smithi059 2024-05-09T01:15:42.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:42 smithi019 ceph-mon[30158]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:15:43.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.828+0000 7f8b36ffd700 1 -- 172.21.15.19:0/3008388130 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8b44103f10 con 0x7f8b3003e510 2024-05-09T01:15:43.828 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi059' with addr '172.21.15.59' 2024-05-09T01:15:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b3003e510 msgr2=0x7f8b30040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 --2- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b3003e510 0x7f8b30040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8b38006fd0 tx=0x7f8b38006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 msgr2=0x7f8b441029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 --2- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 0x7f8b441029c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b400056d0 tx=0x7f8b400057b0 comp rx=0 tx=0).stop 2024-05-09T01:15:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 shutdown_connections 2024-05-09T01:15:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 --2- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b3003e510 0x7f8b30040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 --2- 172.21.15.19:0/3008388130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b44104cd0 0x7f8b441029c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 >> 172.21.15.19:0/3008388130 conn(0x7f8b440fe560 msgr2=0x7f8b44109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 shutdown_connections 2024-05-09T01:15:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:43.831+0000 7f8b4be6d700 1 -- 172.21.15.19:0/3008388130 wait complete. 2024-05-09T01:15:44.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:44 smithi019 ceph-mon[30158]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:15:44.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:44.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:15:44.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T01:15:44.630 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:15:45.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:45 smithi019 ceph-mon[30158]: Added host smithi059 2024-05-09T01:15:45.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:45.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:46.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.011+0000 7f408a3d8700 1 -- 172.21.15.19:0/715623671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 msgr2=0x7f4084108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:46.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.011+0000 7f408a3d8700 1 --2- 172.21.15.19:0/715623671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 0x7f4084108ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4074009910 tx=0x7f4074009c20 comp rx=0 tx=0).stop 2024-05-09T01:15:46.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.011+0000 7f408a3d8700 1 -- 172.21.15.19:0/715623671 shutdown_connections 2024-05-09T01:15:46.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.011+0000 7f408a3d8700 1 --2- 172.21.15.19:0/715623671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 0x7f4084108ea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:46.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.011+0000 7f408a3d8700 1 -- 172.21.15.19:0/715623671 >> 172.21.15.19:0/715623671 conn(0x7f40840785b0 msgr2=0x7f4084078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.011+0000 7f408a3d8700 1 -- 172.21.15.19:0/715623671 shutdown_connections 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.011+0000 7f408a3d8700 1 -- 172.21.15.19:0/715623671 wait complete. 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.011+0000 7f408a3d8700 1 Processor -- start 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.012+0000 7f408a3d8700 1 -- start start 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.012+0000 7f408a3d8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 0x7f408419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.012+0000 7f408a3d8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f408410cfe0 con 0x7f4084108a90 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.012+0000 7f4083fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 0x7f408419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.012+0000 7f4083fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 0x7f408419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54140/0 (socket says 172.21.15.19:54140) 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.012+0000 7f4083fff700 1 -- 172.21.15.19:0/1494432930 learned_addr learned my addr 172.21.15.19:0/1494432930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:46.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.012+0000 7f4083fff700 1 -- 172.21.15.19:0/1494432930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408406bd10 con 0x7f4084108a90 2024-05-09T01:15:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.012+0000 7f4083fff700 1 --2- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 0x7f408419a7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40740056d0 tx=0x7f40740057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.013+0000 7f4081ffb700 1 -- 172.21.15.19:0/1494432930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074027070 con 0x7f4084108a90 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.013+0000 7f4081ffb700 1 -- 172.21.15.19:0/1494432930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f407400be10 con 0x7f4084108a90 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.013+0000 7f4081ffb700 1 -- 172.21.15.19:0/1494432930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074022d20 con 0x7f4084108a90 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.013+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408419add0 con 0x7f4084108a90 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.013+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408419b290 con 0x7f4084108a90 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.014+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40841041a0 con 0x7f4084108a90 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.014+0000 7f4081ffb700 1 -- 172.21.15.19:0/1494432930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f407402a070 con 0x7f4084108a90 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.014+0000 7f4081ffb700 1 --2- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f407003e570 0x7f4070040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.014+0000 7f4081ffb700 1 -- 172.21.15.19:0/1494432930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f407401e030 con 0x7f4084108a90 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.014+0000 7f407bfff700 1 --2- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f407003e570 0x7f4070040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.015+0000 7f407bfff700 1 --2- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f407003e570 0x7f4070040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f406c006fd0 tx=0x7f406c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:46.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.019+0000 7f4081ffb700 1 -- 172.21.15.19:0/1494432930 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4074013070 con 0x7f4084108a90 2024-05-09T01:15:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.239+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f408419b570 con 0x7f407003e570 2024-05-09T01:15:46.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.240+0000 7f4081ffb700 1 -- 172.21.15.19:0/1494432930 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f408419b570 con 0x7f407003e570 2024-05-09T01:15:46.240 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:46.240 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}] 2024-05-09T01:15:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f407003e570 msgr2=0x7f4070040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 --2- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f407003e570 0x7f4070040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f406c006fd0 tx=0x7f406c006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 msgr2=0x7f408419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 --2- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 0x7f408419a7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40740056d0 tx=0x7f40740057b0 comp rx=0 tx=0).stop 2024-05-09T01:15:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 shutdown_connections 2024-05-09T01:15:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 --2- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f407003e570 0x7f4070040a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 --2- 172.21.15.19:0/1494432930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4084108a90 0x7f408419a7d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 >> 172.21.15.19:0/1494432930 conn(0x7f40840785b0 msgr2=0x7f40840ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 shutdown_connections 2024-05-09T01:15:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:46.242+0000 7f408a3d8700 1 -- 172.21.15.19:0/1494432930 wait complete. 2024-05-09T01:15:46.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:46 smithi019 ceph-mon[30158]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:15:46.867 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-05-09T01:15:46.868 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:15:46.868 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-05-09T01:15:46.887 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:15:46.887 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:15:46.943 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-05-09T01:15:46.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch host add smithi196 2024-05-09T01:15:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:15:47.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:47 smithi019 ceph-mon[30158]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:15:47.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.526+0000 7f0f4d470700 1 -- 172.21.15.19:0/1830370497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 msgr2=0x7f0f4810a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.526+0000 7f0f4d470700 1 --2- 172.21.15.19:0/1830370497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 0x7f0f4810a040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800b0a0 tx=0x7f0f3800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:15:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.526+0000 7f0f4d470700 1 -- 172.21.15.19:0/1830370497 shutdown_connections 2024-05-09T01:15:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.526+0000 7f0f4d470700 1 --2- 172.21.15.19:0/1830370497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 0x7f0f4810a040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.526+0000 7f0f4d470700 1 -- 172.21.15.19:0/1830370497 >> 172.21.15.19:0/1830370497 conn(0x7f0f480ff670 msgr2=0x7f0f48101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.526+0000 7f0f4d470700 1 -- 172.21.15.19:0/1830370497 shutdown_connections 2024-05-09T01:15:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.526+0000 7f0f4d470700 1 -- 172.21.15.19:0/1830370497 wait complete. 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.527+0000 7f0f4d470700 1 Processor -- start 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.527+0000 7f0f4d470700 1 -- start start 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.528+0000 7f0f4d470700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 0x7f0f4819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.528+0000 7f0f4d470700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4810e110 con 0x7f0f48109c30 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.528+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 0x7f0f4819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.528+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 0x7f0f4819f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54166/0 (socket says 172.21.15.19:54166) 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.528+0000 7f0f46ffd700 1 -- 172.21.15.19:0/1311591656 learned_addr learned my addr 172.21.15.19:0/1311591656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.528+0000 7f0f46ffd700 1 -- 172.21.15.19:0/1311591656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4819fa90 con 0x7f0f48109c30 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.528+0000 7f0f46ffd700 1 --2- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 0x7f0f4819f550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009cd0 tx=0x7f0f38003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.528+0000 7f0f3ffff700 1 -- 172.21.15.19:0/1311591656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f38021020 con 0x7f0f48109c30 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.529+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4819fd20 con 0x7f0f48109c30 2024-05-09T01:15:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.529+0000 7f0f3ffff700 1 -- 172.21.15.19:0/1311591656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f380096c0 con 0x7f0f48109c30 2024-05-09T01:15:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.529+0000 7f0f3ffff700 1 -- 172.21.15.19:0/1311591656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f38017040 con 0x7f0f48109c30 2024-05-09T01:15:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.529+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f481996e0 con 0x7f0f48109c30 2024-05-09T01:15:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.530+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f48105200 con 0x7f0f48109c30 2024-05-09T01:15:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.530+0000 7f0f3ffff700 1 -- 172.21.15.19:0/1311591656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0f38024020 con 0x7f0f48109c30 2024-05-09T01:15:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.530+0000 7f0f3ffff700 1 --2- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f3403e560 0x7f0f34040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.530+0000 7f0f3ffff700 1 -- 172.21.15.19:0/1311591656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0f3800f050 con 0x7f0f48109c30 2024-05-09T01:15:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.530+0000 7f0f467fc700 1 --2- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f3403e560 0x7f0f34040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.531+0000 7f0f467fc700 1 --2- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f3403e560 0x7f0f34040a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0f30006fd0 tx=0x7f0f30006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.536+0000 7f0f3ffff700 1 -- 172.21.15.19:0/1311591656 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f3802b020 con 0x7f0f48109c30 2024-05-09T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:48 smithi019 ceph-mon[30158]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:15:48.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:48.756+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7f0f4819abf0 con 0x7f0f3403e560 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: Deploying cephadm binary to smithi196 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:15:50.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:15:51.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:51 smithi019 ceph-mon[30158]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T01:15:52.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:52 smithi019 ceph-mon[30158]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:15:53.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.068+0000 7f0f3ffff700 1 -- 172.21.15.19:0/1311591656 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0f4819abf0 con 0x7f0f3403e560 2024-05-09T01:15:53.069 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-05-09T01:15:53.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.070+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f3403e560 msgr2=0x7f0f34040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.070+0000 7f0f4d470700 1 --2- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f3403e560 0x7f0f34040a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0f30006fd0 tx=0x7f0f30006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.070+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 msgr2=0x7f0f4819f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.070+0000 7f0f4d470700 1 --2- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 0x7f0f4819f550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009cd0 tx=0x7f0f38003ce0 comp rx=0 tx=0).stop 2024-05-09T01:15:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.070+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 shutdown_connections 2024-05-09T01:15:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.070+0000 7f0f4d470700 1 --2- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f3403e560 0x7f0f34040a70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.070+0000 7f0f4d470700 1 --2- 172.21.15.19:0/1311591656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109c30 0x7f0f4819f550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.070+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 >> 172.21.15.19:0/1311591656 conn(0x7f0f480ff670 msgr2=0x7f0f48100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.071+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 shutdown_connections 2024-05-09T01:15:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:53.071+0000 7f0f4d470700 1 -- 172.21.15.19:0/1311591656 wait complete. 2024-05-09T01:15:53.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T01:15:53.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:53 smithi019 ceph-mon[30158]: Updating smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:15:53.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:53.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:53.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:53.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:53.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:15:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:15:54.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:54 smithi019 ceph-mon[30158]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:15:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:54 smithi019 ceph-mon[30158]: Updating smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T01:15:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:54 smithi019 ceph-mon[30158]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:15:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:54 smithi019 ceph-mon[30158]: Added host smithi196 2024-05-09T01:15:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:54.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.542+0000 7fedc7e1d700 1 -- 172.21.15.19:0/4171015963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 msgr2=0x7fedc0108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.542+0000 7fedc7e1d700 1 --2- 172.21.15.19:0/4171015963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 0x7fedc0108e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00b0a0 tx=0x7fedbc00b3b0 comp rx=0 tx=0).stop 2024-05-09T01:15:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.542+0000 7fedc7e1d700 1 -- 172.21.15.19:0/4171015963 shutdown_connections 2024-05-09T01:15:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.542+0000 7fedc7e1d700 1 --2- 172.21.15.19:0/4171015963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 0x7fedc0108e80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.542+0000 7fedc7e1d700 1 -- 172.21.15.19:0/4171015963 >> 172.21.15.19:0/4171015963 conn(0x7fedc00fe5c0 msgr2=0x7fedc0100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.542+0000 7fedc7e1d700 1 -- 172.21.15.19:0/4171015963 shutdown_connections 2024-05-09T01:15:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.543+0000 7fedc7e1d700 1 -- 172.21.15.19:0/4171015963 wait complete. 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.543+0000 7fedc7e1d700 1 Processor -- start 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.543+0000 7fedc7e1d700 1 -- start start 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.543+0000 7fedc7e1d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 0x7fedc00ff490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedc7e1d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fedc010cfc0 con 0x7fedc0108a70 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedc5bb9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 0x7fedc00ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedc5bb9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 0x7fedc00ff490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39616/0 (socket says 172.21.15.19:39616) 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedc5bb9700 1 -- 172.21.15.19:0/3298487489 learned_addr learned my addr 172.21.15.19:0/3298487489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedc5bb9700 1 -- 172.21.15.19:0/3298487489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc01012e0 con 0x7fedc0108a70 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedc5bb9700 1 --2- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 0x7fedc00ff490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00d010 tx=0x7fedbc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedb6ffd700 1 -- 172.21.15.19:0/3298487489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedbc021020 con 0x7fedc0108a70 2024-05-09T01:15:55.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedb6ffd700 1 -- 172.21.15.19:0/3298487489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedbc007a40 con 0x7fedc0108a70 2024-05-09T01:15:55.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc00ff9d0 con 0x7fedc0108a70 2024-05-09T01:15:55.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedb6ffd700 1 -- 172.21.15.19:0/3298487489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedbc017040 con 0x7fedc0108a70 2024-05-09T01:15:55.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.544+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc00ffe30 con 0x7fedc0108a70 2024-05-09T01:15:55.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.545+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc0104180 con 0x7fedc0108a70 2024-05-09T01:15:55.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.545+0000 7fedb6ffd700 1 -- 172.21.15.19:0/3298487489 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fedbc024020 con 0x7fedc0108a70 2024-05-09T01:15:55.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.546+0000 7fedb6ffd700 1 --2- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fedac03e560 0x7fedac040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:55.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.546+0000 7fedb6ffd700 1 -- 172.21.15.19:0/3298487489 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fedbc00f050 con 0x7fedc0108a70 2024-05-09T01:15:55.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.546+0000 7fedc53b8700 1 --2- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fedac03e560 0x7fedac040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:55.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.546+0000 7fedc53b8700 1 --2- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fedac03e560 0x7fedac040a70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fedb0006fd0 tx=0x7fedb0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.551+0000 7fedb6ffd700 1 -- 172.21.15.19:0/3298487489 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fedbc02b020 con 0x7fedc0108a70 2024-05-09T01:15:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.772+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fedc01009c0 con 0x7fedac03e560 2024-05-09T01:15:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.774+0000 7fedb6ffd700 1 -- 172.21.15.19:0/3298487489 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fedc01009c0 con 0x7fedac03e560 2024-05-09T01:15:55.774 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:15:55.774 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.776+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fedac03e560 msgr2=0x7fedac040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.776+0000 7fedc7e1d700 1 --2- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fedac03e560 0x7fedac040a70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fedb0006fd0 tx=0x7fedb0006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.776+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 msgr2=0x7fedc00ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.776+0000 7fedc7e1d700 1 --2- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 0x7fedc00ff490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00d010 tx=0x7fedbc003ce0 comp rx=0 tx=0).stop 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.776+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 shutdown_connections 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.776+0000 7fedc7e1d700 1 --2- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fedac03e560 0x7fedac040a70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.776+0000 7fedc7e1d700 1 --2- 172.21.15.19:0/3298487489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedc0108a70 0x7fedc00ff490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.776+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 >> 172.21.15.19:0/3298487489 conn(0x7fedc00fe5c0 msgr2=0x7fedc0106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.777+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 shutdown_connections 2024-05-09T01:15:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:55.777+0000 7fedc7e1d700 1 -- 172.21.15.19:0/3298487489 wait complete. 2024-05-09T01:15:56.376 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T01:15:56.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-09T01:15:56.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:56 smithi019 ceph-mon[30158]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:15:56.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:56.662 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:15:57.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:57 smithi019 ceph-mon[30158]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:15:58.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.115+0000 7f2447a92700 1 -- 172.21.15.19:0/3650264427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 msgr2=0x7f24401041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:58.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.115+0000 7f2447a92700 1 --2- 172.21.15.19:0/3650264427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 0x7f24401041e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f243c009910 tx=0x7f243c009c20 comp rx=0 tx=0).stop 2024-05-09T01:15:58.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.115+0000 7f2447a92700 1 -- 172.21.15.19:0/3650264427 shutdown_connections 2024-05-09T01:15:58.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.115+0000 7f2447a92700 1 --2- 172.21.15.19:0/3650264427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 0x7f24401041e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:58.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.115+0000 7f2447a92700 1 -- 172.21.15.19:0/3650264427 >> 172.21.15.19:0/3650264427 conn(0x7f24400ff660 msgr2=0x7f2440101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:58.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.115+0000 7f2447a92700 1 -- 172.21.15.19:0/3650264427 shutdown_connections 2024-05-09T01:15:58.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.116+0000 7f2447a92700 1 -- 172.21.15.19:0/3650264427 wait complete. 2024-05-09T01:15:58.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.116+0000 7f2447a92700 1 Processor -- start 2024-05-09T01:15:58.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.116+0000 7f2447a92700 1 -- start start 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f2447a92700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 0x7f244019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f2447a92700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f244010e0c0 con 0x7f2440103dd0 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f244582e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 0x7f244019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f244582e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 0x7f244019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39626/0 (socket says 172.21.15.19:39626) 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f244582e700 1 -- 172.21.15.19:0/271277691 learned_addr learned my addr 172.21.15.19:0/271277691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f244582e700 1 -- 172.21.15.19:0/271277691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244019fb40 con 0x7f2440103dd0 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f244582e700 1 --2- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 0x7f244019f600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f243c0056d0 tx=0x7f243c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f2436ffd700 1 -- 172.21.15.19:0/271277691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f243c026070 con 0x7f2440103dd0 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244019fe30 con 0x7f2440103dd0 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f2436ffd700 1 -- 172.21.15.19:0/271277691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f243c00bba0 con 0x7f2440103dd0 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f2436ffd700 1 -- 172.21.15.19:0/271277691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f243c003920 con 0x7f2440103dd0 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.117+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2440199960 con 0x7f2440103dd0 2024-05-09T01:15:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.118+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2440105280 con 0x7f2440103dd0 2024-05-09T01:15:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.118+0000 7f2436ffd700 1 -- 172.21.15.19:0/271277691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f243c029070 con 0x7f2440103dd0 2024-05-09T01:15:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.119+0000 7f2436ffd700 1 --2- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f242c03e510 0x7f242c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:15:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.119+0000 7f2436ffd700 1 -- 172.21.15.19:0/271277691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f243c01e070 con 0x7f2440103dd0 2024-05-09T01:15:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.119+0000 7f244502d700 1 --2- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f242c03e510 0x7f242c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:15:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.119+0000 7f244502d700 1 --2- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f242c03e510 0x7f242c040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2430006fd0 tx=0x7f2430006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:15:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.124+0000 7f2436ffd700 1 -- 172.21.15.19:0/271277691 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f243c013070 con 0x7f2440103dd0 2024-05-09T01:15:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.345+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f244019aca0 con 0x7f2440103dd0 2024-05-09T01:15:58.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.411+0000 7f2436ffd700 1 -- 172.21.15.19:0/271277691 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f243c015350 con 0x7f2440103dd0 2024-05-09T01:15:58.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f242c03e510 msgr2=0x7f242c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:58.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 --2- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f242c03e510 0x7f242c040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2430006fd0 tx=0x7f2430006e40 comp rx=0 tx=0).stop 2024-05-09T01:15:58.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 msgr2=0x7f244019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:15:58.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 --2- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 0x7f244019f600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f243c0056d0 tx=0x7f243c0057b0 comp rx=0 tx=0).stop 2024-05-09T01:15:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 shutdown_connections 2024-05-09T01:15:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 --2- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f242c03e510 0x7f242c040a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 --2- 172.21.15.19:0/271277691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2440103dd0 0x7f244019f600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:15:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 >> 172.21.15.19:0/271277691 conn(0x7f24400ff660 msgr2=0x7f2440109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:15:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 shutdown_connections 2024-05-09T01:15:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:15:58.414+0000 7f2447a92700 1 -- 172.21.15.19:0/271277691 wait complete. 2024-05-09T01:15:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-05-09T01:15:58.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:58 smithi019 ceph-mon[30158]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:15:58.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:58 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/271277691' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T01:15:59.051 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-05-09T01:15:59.051 INFO:tasks.cephadm:Adding mon.b on smithi059 2024-05-09T01:15:59.051 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-05-09T01:15:59.052 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c' 2024-05-09T01:15:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:15:59.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:59 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/271277691' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T01:15:59.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:59 smithi019 ceph-mon[30158]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T01:15:59.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:59.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:59.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:59.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:15:59.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:15:59.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:15:59.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:15:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:00.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.821+0000 7f26f2f83700 1 -- 172.21.15.196:0/1824241913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 msgr2=0x7f26ec108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:00.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.821+0000 7f26f2f83700 1 --2- 172.21.15.196:0/1824241913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 0x7f26ec108e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009910 tx=0x7f26e0009c20 comp rx=0 tx=0).stop 2024-05-09T01:16:00.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.821+0000 7f26f2f83700 1 -- 172.21.15.196:0/1824241913 shutdown_connections 2024-05-09T01:16:00.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.821+0000 7f26f2f83700 1 --2- 172.21.15.196:0/1824241913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 0x7f26ec108e80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:00.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.821+0000 7f26f2f83700 1 -- 172.21.15.196:0/1824241913 >> 172.21.15.196:0/1824241913 conn(0x7f26ec0fe5c0 msgr2=0x7f26ec100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:00.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.821+0000 7f26f2f83700 1 -- 172.21.15.196:0/1824241913 shutdown_connections 2024-05-09T01:16:00.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.821+0000 7f26f2f83700 1 -- 172.21.15.196:0/1824241913 wait complete. 2024-05-09T01:16:00.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.821+0000 7f26f2f83700 1 Processor -- start 2024-05-09T01:16:00.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.821+0000 7f26f2f83700 1 -- start start 2024-05-09T01:16:00.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.822+0000 7f26f2f83700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 0x7f26ec19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:00.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.822+0000 7f26f2f83700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec10cfc0 con 0x7f26ec108a70 2024-05-09T01:16:00.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.822+0000 7f26f0d1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 0x7f26ec19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:00.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.822+0000 7f26f0d1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 0x7f26ec19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.196:53504/0 (socket says 172.21.15.196:53504) 2024-05-09T01:16:00.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.822+0000 7f26f0d1f700 1 -- 172.21.15.196:0/214681674 learned_addr learned my addr 172.21.15.196:0/214681674 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:16:00.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.822+0000 7f26f0d1f700 1 -- 172.21.15.196:0/214681674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec06ca70 con 0x7f26ec108a70 2024-05-09T01:16:00.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.822+0000 7f26f0d1f700 1 --2- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 0x7f26ec19a780 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f26e00056d0 tx=0x7f26e00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:00.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.823+0000 7f26e9ffb700 1 -- 172.21.15.196:0/214681674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26e0027070 con 0x7f26ec108a70 2024-05-09T01:16:00.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.823+0000 7f26e9ffb700 1 -- 172.21.15.196:0/214681674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e000be10 con 0x7f26ec108a70 2024-05-09T01:16:00.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.823+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ec19ad80 con 0x7f26ec108a70 2024-05-09T01:16:00.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.823+0000 7f26e9ffb700 1 -- 172.21.15.196:0/214681674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26e0022d20 con 0x7f26ec108a70 2024-05-09T01:16:00.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.823+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ec19b240 con 0x7f26ec108a70 2024-05-09T01:16:00.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.824+0000 7f26e9ffb700 1 -- 172.21.15.196:0/214681674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f26e002a070 con 0x7f26ec108a70 2024-05-09T01:16:00.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.824+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ec104180 con 0x7f26ec108a70 2024-05-09T01:16:00.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.824+0000 7f26e9ffb700 1 --2- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26dc03e510 0x7f26dc040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:00.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.824+0000 7f26e9ffb700 1 -- 172.21.15.196:0/214681674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f26e001e030 con 0x7f26ec108a70 2024-05-09T01:16:00.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.824+0000 7f26ebfff700 1 --2- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26dc03e510 0x7f26dc040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:00.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.825+0000 7f26ebfff700 1 --2- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26dc03e510 0x7f26dc040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:00.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:00.830+0000 7f26e9ffb700 1 -- 172.21.15.196:0/214681674 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26e0013070 con 0x7f26ec108a70 2024-05-09T01:16:00.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:00 smithi019 ceph-mon[30158]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T01:16:00.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:00 smithi019 ceph-mon[30158]: Updating smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:00.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:00 smithi019 ceph-mon[30158]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:00.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:00 smithi019 ceph-mon[30158]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:16:00.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:00 smithi019 ceph-mon[30158]: Updating smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T01:16:00.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:00 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:00.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:00 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:00.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:00 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:01.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.045+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7f26ec06c1c0 con 0x7f26dc03e510 2024-05-09T01:16:01.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.053+0000 7f26e9ffb700 1 -- 172.21.15.196:0/214681674 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f26ec06c1c0 con 0x7f26dc03e510 2024-05-09T01:16:01.053 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-05-09T01:16:01.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26dc03e510 msgr2=0x7f26dc040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:01.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 --2- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26dc03e510 0x7f26dc040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8006e40 comp rx=0 tx=0).stop 2024-05-09T01:16:01.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 msgr2=0x7f26ec19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:01.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 --2- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 0x7f26ec19a780 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f26e00056d0 tx=0x7f26e00057b0 comp rx=0 tx=0).stop 2024-05-09T01:16:01.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 shutdown_connections 2024-05-09T01:16:01.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 --2- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26dc03e510 0x7f26dc040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:01.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 --2- 172.21.15.196:0/214681674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ec108a70 0x7f26ec19a780 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:01.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 >> 172.21.15.196:0/214681674 conn(0x7f26ec0fe5c0 msgr2=0x7f26ec0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:01.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 shutdown_connections 2024-05-09T01:16:01.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:01.055+0000 7f26f2f83700 1 -- 172.21.15.196:0/214681674 wait complete. 2024-05-09T01:16:01.667 DEBUG:teuthology.orchestra.run.smithi059:mon.b> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.b.service 2024-05-09T01:16:01.670 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.c.service 2024-05-09T01:16:01.672 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T01:16:01.672 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c;count:3 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:02.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:02 smithi019 ceph-mon[30158]: Deploying daemon mon.c on smithi196 2024-05-09T01:16:02.654 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:04.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.287+0000 7f4cfe651700 1 -- 172.21.15.196:0/3811289199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 msgr2=0x7f4cf81041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:04.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.287+0000 7f4cfe651700 1 --2- 172.21.15.196:0/3811289199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 0x7f4cf81041e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009910 tx=0x7f4cec009c20 comp rx=0 tx=0).stop 2024-05-09T01:16:04.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.287+0000 7f4cfe651700 1 -- 172.21.15.196:0/3811289199 shutdown_connections 2024-05-09T01:16:04.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.287+0000 7f4cfe651700 1 --2- 172.21.15.196:0/3811289199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 0x7f4cf81041e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:04.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.287+0000 7f4cfe651700 1 -- 172.21.15.196:0/3811289199 >> 172.21.15.196:0/3811289199 conn(0x7f4cf80ff660 msgr2=0x7f4cf8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:04.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.287+0000 7f4cfe651700 1 -- 172.21.15.196:0/3811289199 shutdown_connections 2024-05-09T01:16:04.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.287+0000 7f4cfe651700 1 -- 172.21.15.196:0/3811289199 wait complete. 2024-05-09T01:16:04.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.288+0000 7f4cfe651700 1 Processor -- start 2024-05-09T01:16:04.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.288+0000 7f4cfe651700 1 -- start start 2024-05-09T01:16:04.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.288+0000 7f4cfe651700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 0x7f4cf8075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:04.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.288+0000 7f4cfe651700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf810e0c0 con 0x7f4cf8103dd0 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.289+0000 7f4cf7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 0x7f4cf8075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.289+0000 7f4cf7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 0x7f4cf8075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.196:53514/0 (socket says 172.21.15.196:53514) 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.289+0000 7f4cf7fff700 1 -- 172.21.15.196:0/1835594462 learned_addr learned my addr 172.21.15.196:0/1835594462 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.289+0000 7f4cf7fff700 1 -- 172.21.15.196:0/1835594462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf8077910 con 0x7f4cf8103dd0 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.289+0000 7f4cf7fff700 1 --2- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 0x7f4cf8075ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0056d0 tx=0x7f4cec0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.289+0000 7f4cf57fa700 1 -- 172.21.15.196:0/1835594462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec026070 con 0x7f4cf8103dd0 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.289+0000 7f4cf57fa700 1 -- 172.21.15.196:0/1835594462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cec00bba0 con 0x7f4cf8103dd0 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.290+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf8076000 con 0x7f4cf8103dd0 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.290+0000 7f4cf57fa700 1 -- 172.21.15.196:0/1835594462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec003920 con 0x7f4cf8103dd0 2024-05-09T01:16:04.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.290+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf80764c0 con 0x7f4cf8103dd0 2024-05-09T01:16:04.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.291+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf8105280 con 0x7f4cf8103dd0 2024-05-09T01:16:04.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.291+0000 7f4cf57fa700 1 -- 172.21.15.196:0/1835594462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4cec029070 con 0x7f4cf8103dd0 2024-05-09T01:16:04.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.291+0000 7f4cf57fa700 1 --2- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ce003e510 0x7f4ce0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:04.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.291+0000 7f4cf57fa700 1 -- 172.21.15.196:0/1835594462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4cec01e070 con 0x7f4cf8103dd0 2024-05-09T01:16:04.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.291+0000 7f4cf77fe700 1 --2- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ce003e510 0x7f4ce0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:04.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.292+0000 7f4cf77fe700 1 --2- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ce003e510 0x7f4ce0040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8006fd0 tx=0x7f4ce8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:04.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.297+0000 7f4cf57fa700 1 -- 172.21.15.196:0/1835594462 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4cec013070 con 0x7f4cf8103dd0 2024-05-09T01:16:04.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:04 smithi019 ceph-mon[30158]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:04.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.593+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4cf8077020 con 0x7f4cf8103dd0 2024-05-09T01:16:04.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.593+0000 7f4cf57fa700 1 -- 172.21.15.196:0/1835594462 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f4cec00d340 con 0x7f4cf8103dd0 2024-05-09T01:16:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:16:04.594 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","modified":"2024-05-09T01:14:15.452380Z","created":"2024-05-09T01:14:15.452380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T01:16:04.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ce003e510 msgr2=0x7f4ce0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:04.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 --2- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ce003e510 0x7f4ce0040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8006fd0 tx=0x7f4ce8006e40 comp rx=0 tx=0).stop 2024-05-09T01:16:04.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 msgr2=0x7f4cf8075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:04.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 --2- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 0x7f4cf8075ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0056d0 tx=0x7f4cec0057b0 comp rx=0 tx=0).stop 2024-05-09T01:16:04.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 shutdown_connections 2024-05-09T01:16:04.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 --2- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ce003e510 0x7f4ce0040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:04.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 --2- 172.21.15.196:0/1835594462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8103dd0 0x7f4cf8075ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:04.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 >> 172.21.15.196:0/1835594462 conn(0x7f4cf80ff660 msgr2=0x7f4cf8109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:04.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 shutdown_connections 2024-05-09T01:16:04.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:04.596+0000 7f4cfe651700 1 -- 172.21.15.196:0/1835594462 wait complete. 2024-05-09T01:16:04.597 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-05-09T01:16:05.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:05 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/1835594462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T01:16:06.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 podman[35023]: 2024-05-09 01:16:06.033117135 +0000 UTC m=+0.017506151 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:16:06.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:06 smithi019 ceph-mon[30158]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.606 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T01:16:06.607 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T01:16:06.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 podman[35023]: 2024-05-09 01:16:06.301446125 +0000 UTC m=+0.285835141 container init 98589ef238fd1a4bfe2246cd2f2bebdae22af19b07f2741dc5af63cf8a31f8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux ) 2024-05-09T01:16:06.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 podman[35023]: 2024-05-09 01:16:06.304426739 +0000 UTC m=+0.288815759 container start 98589ef238fd1a4bfe2246cd2f2bebdae22af19b07f2741dc5af63cf8a31f8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-c, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T01:16:06.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T01:16:06.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T01:16:06.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pidfile_write: ignore empty --pid-file 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: load: jerasure load: lrc 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: RocksDB version: 7.9.2 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Git sha 0 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: DB SUMMARY 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: DB Session ID: 6SP219FCU2R4XN1JRLSL 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: CURRENT file: CURRENT 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-09T01:16:06.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.error_if_exists: 0 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.create_if_missing: 0 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.paranoid_checks: 1 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.env: 0x562e545fa500 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.info_log: 0x562e55f50020 2024-05-09T01:16:06.632 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T01:16:06.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.statistics: (nil) 2024-05-09T01:16:06.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.use_fsync: 0 2024-05-09T01:16:06.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_log_file_size: 0 2024-05-09T01:16:06.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T01:16:06.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T01:16:06.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T01:16:06.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T01:16:06.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.allow_fallocate: 1 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.use_direct_reads: 0 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.db_log_dir: 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.wal_dir: 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T01:16:06.635 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.write_buffer_manager: 0x562e55e1a8c0 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.unordered_write: 0 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T01:16:06.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.row_cache: None 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.wal_filter: None 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.two_write_queues: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.wal_compression: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.atomic_flush: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.log_readahead_size: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T01:16:06.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_background_jobs: 2 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_background_compactions: -1 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_subcompactions: 1 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T01:16:06.638 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_open_files: -1 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_background_flushes: -1 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Compression algorithms supported: 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: kZSTD supported: 0 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: kXpressCompression supported: 0 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T01:16:06.639 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: kZlibCompression supported: 1 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: kSnappyCompression supported: 1 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: kLZ4Compression supported: 1 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: kBZip2Compression supported: 0 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.merge_operator: 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_filter: None 2024-05-09T01:16:06.640 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e55f501e0) 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_type: 0 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_index_type: 0 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_shortening: 1 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: checksum: 4 2024-05-09T01:16:06.641 INFO:journalctl@ceph.mon.c.smithi196.stdout: no_block_cache: 0 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache: 0x562e55e4d1f0 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_options: 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: capacity : 536870912 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_shard_bits : 4 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: strict_capacity_limit : 0 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_compressed: (nil) 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: persistent_cache: (nil) 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size: 4096 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size_deviation: 10 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_restart_interval: 16 2024-05-09T01:16:06.642 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_block_restart_interval: 1 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: metadata_block_size: 4096 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: partition_filters: 0 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: use_delta_encoding: 1 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: filter_policy: bloomfilter 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: whole_key_filtering: 1 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: verify_compression: 0 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: format_version: 5 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_index_compression: 1 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_align: 0 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: max_auto_readahead_size: 262144 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: prepopulate_block_cache: 0 2024-05-09T01:16:06.643 INFO:journalctl@ceph.mon.c.smithi196.stdout: initial_auto_readahead_size: 8192 2024-05-09T01:16:06.644 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T01:16:06.644 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T01:16:06.644 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T01:16:06.644 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression: NoCompression 2024-05-09T01:16:06.644 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T01:16:06.644 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T01:16:06.644 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.num_levels: 7 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T01:16:06.645 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T01:16:06.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T01:16:06.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.inplace_update_support: 0 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T01:16:06.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.bloom_locality: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.max_successive_merges: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.ttl: 2592000 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.enable_blob_files: false 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.min_blob_size: 0 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T01:16:06.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 033de95c-9e39-4e59-9276-52a761fb27fc 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715217366355509, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715217366355928, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715217366, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "033de95c-9e39-4e59-9276-52a761fb27fc", "db_session_id": "6SP219FCU2R4XN1JRLSL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715217366356007, "job": 1, "event": "recovery_finished"} 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-09T01:16:06.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e55e31c00 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: DB pointer 0x562e5ac24000 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** DB Stats ** 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-05-09T01:16:06.651 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-05-09T01:16:06.652 INFO:journalctl@ceph.mon.c.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache BinnedLRUCache@0x562e55e4d1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T01:16:06.653 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: starting mon.c rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(???) e0 preinit fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).mds e1 new map 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout: e1 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout: legacy client fscid: -1 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout: No filesystems configured 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T01:16:06.654 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Deploying cephadm binary to smithi196 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Updating smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Updating smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Added host smithi196 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/271277691' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T01:16:06.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/271277691' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Updating smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Updating smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T01:16:06.658 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.659 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.659 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.659 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.659 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:06.659 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c;count:3 2024-05-09T01:16:06.659 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.659 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:06.659 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:06.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:06.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:06.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:06.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:06.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: Deploying daemon mon.c on smithi196 2024-05-09T01:16:06.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.661 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/1835594462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T01:16:06.661 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:06.661 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 ceph-mon[35041]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T01:16:06.661 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 bash[35023]: 98589ef238fd1a4bfe2246cd2f2bebdae22af19b07f2741dc5af63cf8a31f8a0 2024-05-09T01:16:06.661 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:06 smithi196 systemd[1]: Started Ceph mon.c for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:16:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.420+0000 7ffa93f73700 1 -- 172.21.15.196:0/3069504127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa780035c0 msgr2=0x7ffa78005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.420+0000 7ffa93f73700 1 --2- 172.21.15.196:0/3069504127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa780035c0 0x7ffa78005aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c1021f0 tx=0x7ffa8800de60 comp rx=0 tx=0).stop 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.420+0000 7ffa93f73700 1 -- 172.21.15.196:0/3069504127 shutdown_connections 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.420+0000 7ffa93f73700 1 --2- 172.21.15.196:0/3069504127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa780035c0 0x7ffa78005aa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.420+0000 7ffa93f73700 1 -- 172.21.15.196:0/3069504127 >> 172.21.15.196:0/3069504127 conn(0x7ffa8c077e90 msgr2=0x7ffa8c0782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.420+0000 7ffa93f73700 1 -- 172.21.15.196:0/3069504127 shutdown_connections 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.420+0000 7ffa93f73700 1 -- 172.21.15.196:0/3069504127 wait complete. 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.421+0000 7ffa93f73700 1 Processor -- start 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.421+0000 7ffa93f73700 1 -- start start 2024-05-09T01:16:11.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa93f73700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 0x7ffa8c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa93f73700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa8c19c280 0x7ffa8c1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa93f73700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c109a00 con 0x7ffa8c19c280 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa93f73700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c109880 con 0x7ffa780035c0 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa91d0f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 0x7ffa8c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa91d0f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 0x7ffa8c19bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53400/0 (socket says 172.21.15.196:53400) 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa91d0f700 1 -- 172.21.15.196:0/1752296444 learned_addr learned my addr 172.21.15.196:0/1752296444 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa91d0f700 1 -- 172.21.15.196:0/1752296444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 msgr2=0x7ffa8c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa91d0f700 1 -- 172.21.15.196:0/1752296444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 msgr2=0x7ffa8c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa91d0f700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 0x7ffa8c19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa91d0f700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 0x7ffa8c19bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa9150e700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa8c19c280 0x7ffa8c1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa9150e700 1 -- 172.21.15.196:0/1752296444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 msgr2=0x7ffa8c19bd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T01:16:11.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa9150e700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 0x7ffa8c19bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:11.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.422+0000 7ffa9150e700 1 -- 172.21.15.196:0/1752296444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c19c880 con 0x7ffa8c19c280 2024-05-09T01:16:11.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.423+0000 7ffa9150e700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa8c19c280 0x7ffa8c1a0570 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00b4d0 tx=0x7ffa7c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:11.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.423+0000 7ffa82ffd700 1 -- 172.21.15.196:0/1752296444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffa7c011630 con 0x7ffa8c19c280 2024-05-09T01:16:11.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.423+0000 7ffa82ffd700 1 -- 172.21.15.196:0/1752296444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa7c011c80 con 0x7ffa8c19c280 2024-05-09T01:16:11.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.423+0000 7ffa82ffd700 1 -- 172.21.15.196:0/1752296444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffa7c010da0 con 0x7ffa8c19c280 2024-05-09T01:16:11.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.423+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c1a0ab0 con 0x7ffa8c19c280 2024-05-09T01:16:11.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.423+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c1a1040 con 0x7ffa8c19c280 2024-05-09T01:16:11.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.424+0000 7ffa82ffd700 1 -- 172.21.15.196:0/1752296444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ffa7c018020 con 0x7ffa8c19c280 2024-05-09T01:16:11.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.424+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa8c103a00 con 0x7ffa8c19c280 2024-05-09T01:16:11.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.424+0000 7ffa82ffd700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffa7003e300 0x7ffa70040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:11.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.425+0000 7ffa82ffd700 1 -- 172.21.15.196:0/1752296444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffa7c052150 con 0x7ffa8c19c280 2024-05-09T01:16:11.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.425+0000 7ffa91d0f700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffa7003e300 0x7ffa70040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:11.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.425+0000 7ffa91d0f700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffa7003e300 0x7ffa70040810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800da10 tx=0x7ffa8800d4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:11.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.430+0000 7ffa82ffd700 1 -- 172.21.15.196:0/1752296444 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa7c01f020 con 0x7ffa8c19c280 2024-05-09T01:16:11.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:16:11.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: mon.a calling monitor election 2024-05-09T01:16:11.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:11.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: mon.c calling monitor election 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: monmap epoch 2 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: last_changed 2024-05-09T01:16:06.382132+0000 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: created 2024-05-09T01:14:15.452380+0000 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: min_mon_release 19 (squid) 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: election_strategy: 1 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T01:16:11.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: fsmap 2024-05-09T01:16:11.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T01:16:11.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: mgrmap e13: x(active, since 50s) 2024-05-09T01:16:11.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: overall HEALTH_OK 2024-05-09T01:16:11.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:11.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:11.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:11.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:11.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:11.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.724+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ffa8c19d0a0 con 0x7ffa8c19c280 2024-05-09T01:16:11.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.725+0000 7ffa82ffd700 1 -- 172.21.15.196:0/1752296444 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7ffa7c0187b0 con 0x7ffa8c19c280 2024-05-09T01:16:11.726 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:16:11.726 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","modified":"2024-05-09T01:16:06.382132Z","created":"2024-05-09T01:14:15.452380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T01:16:11.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.727+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffa7003e300 msgr2=0x7ffa70040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:11.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.727+0000 7ffa93f73700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffa7003e300 0x7ffa70040810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800da10 tx=0x7ffa8800d4c0 comp rx=0 tx=0).stop 2024-05-09T01:16:11.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.727+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa8c19c280 msgr2=0x7ffa8c1a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:11.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.727+0000 7ffa93f73700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa8c19c280 0x7ffa8c1a0570 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00b4d0 tx=0x7ffa7c00b890 comp rx=0 tx=0).stop 2024-05-09T01:16:11.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.727+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 shutdown_connections 2024-05-09T01:16:11.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.727+0000 7ffa93f73700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa780035c0 0x7ffa8c19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:11.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.727+0000 7ffa93f73700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffa7003e300 0x7ffa70040810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:11.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.727+0000 7ffa93f73700 1 --2- 172.21.15.196:0/1752296444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa8c19c280 0x7ffa8c1a0570 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:11.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.727+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 >> 172.21.15.196:0/1752296444 conn(0x7ffa8c077e90 msgr2=0x7ffa8c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:11.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.728+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 shutdown_connections 2024-05-09T01:16:11.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:11.728+0000 7ffa93f73700 1 -- 172.21.15.196:0/1752296444 wait complete. 2024-05-09T01:16:11.729 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-05-09T01:16:11.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:16:11.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: mon.a calling monitor election 2024-05-09T01:16:11.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:11.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: mon.c calling monitor election 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: monmap epoch 2 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: last_changed 2024-05-09T01:16:06.382132+0000 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: created 2024-05-09T01:14:15.452380+0000 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: min_mon_release 19 (squid) 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: election_strategy: 1 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T01:16:11.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: fsmap 2024-05-09T01:16:11.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T01:16:11.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: mgrmap e13: x(active, since 50s) 2024-05-09T01:16:11.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: overall HEALTH_OK 2024-05-09T01:16:11.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:11.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:11.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:11.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:11.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:12.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:12 smithi196 ceph-mon[35041]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:12.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:12 smithi196 ceph-mon[35041]: Deploying daemon mon.b on smithi059 2024-05-09T01:16:12.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:12 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/1752296444' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T01:16:12.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:12.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:12 smithi019 ceph-mon[30158]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:12.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:12 smithi019 ceph-mon[30158]: Deploying daemon mon.b on smithi059 2024-05-09T01:16:12.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:12 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/1752296444' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T01:16:12.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:13.314 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T01:16:13.315 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T01:16:13.555 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:16:14.720 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:14 smithi196 ceph-mon[35041]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:14.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:14 smithi019 ceph-mon[30158]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:14.973 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:14 smithi059 podman[34332]: 2024-05-09 01:16:14.784540436 +0000 UTC m=+0.018741937 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:16:15.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.023+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/3788367368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e58103510 msgr2=0x7f0e58103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:15.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.023+0000 7f0e5cb9e700 1 --2- 172.21.15.196:0/3788367368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e58103510 0x7f0e58103920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000b0a0 tx=0x7f0e5000b3b0 comp rx=0 tx=0).stop 2024-05-09T01:16:15.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.023+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/3788367368 shutdown_connections 2024-05-09T01:16:15.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.023+0000 7f0e5cb9e700 1 --2- 172.21.15.196:0/3788367368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e58103510 0x7f0e58103920 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:15.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.023+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/3788367368 >> 172.21.15.196:0/3788367368 conn(0x7f0e58077e10 msgr2=0x7f0e58078270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:15.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.023+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/3788367368 shutdown_connections 2024-05-09T01:16:15.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.023+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/3788367368 wait complete. 2024-05-09T01:16:15.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.024+0000 7f0e5cb9e700 1 Processor -- start 2024-05-09T01:16:15.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.024+0000 7f0e5cb9e700 1 -- start start 2024-05-09T01:16:15.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.024+0000 7f0e5cb9e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 0x7f0e5819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:15.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5cb9e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e5819b100 0x7f0e5819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:15.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5cb9e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5810d2e0 con 0x7f0e5819b100 2024-05-09T01:16:15.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5cb9e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5810d160 con 0x7f0e58103510 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 0x7f0e5819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 0x7f0e5819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53414/0 (socket says 172.21.15.196:53414) 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5659c700 1 -- 172.21.15.196:0/1339012211 learned_addr learned my addr 172.21.15.196:0/1339012211 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e55d9b700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e5819b100 0x7f0e5819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5659c700 1 -- 172.21.15.196:0/1339012211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 msgr2=0x7f0e5819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5659c700 1 -- 172.21.15.196:0/1339012211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 msgr2=0x7f0e5819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5659c700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 0x7f0e5819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e5659c700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 0x7f0e5819abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e55d9b700 1 -- 172.21.15.196:0/1339012211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 msgr2=0x7f0e5819abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e55d9b700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 0x7f0e5819abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e55d9b700 1 -- 172.21.15.196:0/1339012211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e581a3c30 con 0x7f0e5819b100 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.025+0000 7f0e55d9b700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e5819b100 0x7f0e5819b510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00b4d0 tx=0x7f0e4c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.026+0000 7f0e477fe700 1 -- 172.21.15.196:0/1339012211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0e4c011630 con 0x7f0e5819b100 2024-05-09T01:16:15.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.026+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e581a3e60 con 0x7f0e5819b100 2024-05-09T01:16:15.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.026+0000 7f0e477fe700 1 -- 172.21.15.196:0/1339012211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c011c80 con 0x7f0e5819b100 2024-05-09T01:16:15.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.026+0000 7f0e477fe700 1 -- 172.21.15.196:0/1339012211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0e4c010da0 con 0x7f0e5819b100 2024-05-09T01:16:15.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.026+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e581a43c0 con 0x7f0e5819b100 2024-05-09T01:16:15.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.026+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e58104530 con 0x7f0e5819b100 2024-05-09T01:16:15.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.027+0000 7f0e477fe700 1 -- 172.21.15.196:0/1339012211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0e4c018020 con 0x7f0e5819b100 2024-05-09T01:16:15.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.027+0000 7f0e477fe700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e4003e2b0 0x7f0e400407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:15.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.027+0000 7f0e477fe700 1 -- 172.21.15.196:0/1339012211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0e4c0521e0 con 0x7f0e5819b100 2024-05-09T01:16:15.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.027+0000 7f0e5659c700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e4003e2b0 0x7f0e400407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:15.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.028+0000 7f0e5659c700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e4003e2b0 0x7f0e400407c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009900 tx=0x7f0e500075c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:15.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.033+0000 7f0e477fe700 1 -- 172.21.15.196:0/1339012211 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e4c01f020 con 0x7f0e5819b100 2024-05-09T01:16:15.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.129+0000 7f0e477fe700 1 -- 172.21.15.196:0/1339012211 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0117d0 con 0x7f0e5819b100 2024-05-09T01:16:15.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:15.322+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0e5806baf0 con 0x7f0e5819b100 2024-05-09T01:16:15.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 podman[34332]: 2024-05-09 01:16:15.060732112 +0000 UTC m=+0.294933594 container init 9a88f306158aadb37780c72fc373a098022f32baccaa78e53895cb6c14ccd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-09T01:16:15.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 podman[34332]: 2024-05-09 01:16:15.063725096 +0000 UTC m=+0.297926571 container start 9a88f306158aadb37780c72fc373a098022f32baccaa78e53895cb6c14ccd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pidfile_write: ignore empty --pid-file 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: load: jerasure load: lrc 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: RocksDB version: 7.9.2 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Git sha 0 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: DB SUMMARY 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: DB Session ID: OTHG9GJD33Z38U52OEAD 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: CURRENT file: CURRENT 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T01:16:15.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.error_if_exists: 0 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.create_if_missing: 0 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.paranoid_checks: 1 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.env: 0x555d551ef500 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.info_log: 0x555d562a2020 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T01:16:15.351 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.statistics: (nil) 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.use_fsync: 0 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_log_file_size: 0 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.allow_fallocate: 1 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.use_direct_reads: 0 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.db_log_dir: 2024-05-09T01:16:15.352 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.wal_dir: 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.write_buffer_manager: 0x555d5616c8c0 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T01:16:15.353 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T01:16:15.354 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T01:16:15.354 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T01:16:15.354 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.unordered_write: 0 2024-05-09T01:16:15.354 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T01:16:15.354 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T01:16:15.354 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T01:16:15.354 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T01:16:15.354 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.row_cache: None 2024-05-09T01:16:15.354 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.wal_filter: None 2024-05-09T01:16:15.356 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T01:16:15.356 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T01:16:15.356 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.two_write_queues: 0 2024-05-09T01:16:15.356 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T01:16:15.356 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.wal_compression: 0 2024-05-09T01:16:15.356 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.atomic_flush: 0 2024-05-09T01:16:15.356 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T01:16:15.356 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.log_readahead_size: 0 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_background_jobs: 2 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_background_compactions: -1 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_subcompactions: 1 2024-05-09T01:16:15.357 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_open_files: -1 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T01:16:15.358 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_background_flushes: -1 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Compression algorithms supported: 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: kZSTD supported: 0 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: kXpressCompression supported: 0 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: kZlibCompression supported: 1 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: kSnappyCompression supported: 1 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: kLZ4Compression supported: 1 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: kBZip2Compression supported: 0 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T01:16:15.359 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.merge_operator: 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_filter: None 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555d562a21e0) 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout: cache_index_and_filter_blocks: 1 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout: pin_top_level_index_and_filter: 1 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout: index_type: 0 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout: data_block_index_type: 0 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout: index_shortening: 1 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T01:16:15.360 INFO:journalctl@ceph.mon.b.smithi059.stdout: checksum: 4 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: no_block_cache: 0 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_cache: 0x555d5619f1f0 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_cache_name: BinnedLRUCache 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_cache_options: 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: capacity : 536870912 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: num_shard_bits : 4 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: strict_capacity_limit : 0 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: high_pri_pool_ratio: 0.000 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_cache_compressed: (nil) 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: persistent_cache: (nil) 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_size: 4096 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_size_deviation: 10 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_restart_interval: 16 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: index_block_restart_interval: 1 2024-05-09T01:16:15.361 INFO:journalctl@ceph.mon.b.smithi059.stdout: metadata_block_size: 4096 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: partition_filters: 0 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: use_delta_encoding: 1 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: filter_policy: bloomfilter 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: whole_key_filtering: 1 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: verify_compression: 0 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: read_amp_bytes_per_bit: 0 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: format_version: 5 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: enable_index_compression: 1 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_align: 0 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: max_auto_readahead_size: 262144 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: prepopulate_block_cache: 0 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: initial_auto_readahead_size: 8192 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T01:16:15.362 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression: NoCompression 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.num_levels: 7 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T01:16:15.363 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T01:16:15.364 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T01:16:15.365 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T01:16:15.366 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.inplace_update_support: 0 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.bloom_locality: 0 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.max_successive_merges: 0 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T01:16:15.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.ttl: 2592000 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.enable_blob_files: false 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.min_blob_size: 0 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T01:16:15.368 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ca995c1c-c8d9-49c0-998b-ef44f1615c61 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715217375104072, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715217375104494, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715217375, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ca995c1c-c8d9-49c0-998b-ef44f1615c61", "db_session_id": "OTHG9GJD33Z38U52OEAD", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715217375104575, "job": 1, "event": "recovery_finished"} 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555d56183c00 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: DB pointer 0x555d5af76000 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T01:16:15.369 INFO:journalctl@ceph.mon.b.smithi059.stdout: ** DB Stats ** 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: ** Compaction Stats [default] ** 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T01:16:15.370 INFO:journalctl@ceph.mon.b.smithi059.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: ** Compaction Stats [default] ** 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T01:16:15.371 INFO:journalctl@ceph.mon.b.smithi059.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout: Block cache BinnedLRUCache@0x555d5619f1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: using public_addr v2:172.21.15.59:0/0 -> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: starting mon.b rank -1 at public addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] at bind addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(???) e0 preinit fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).mds e1 new map 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-09T01:16:15.372 INFO:journalctl@ceph.mon.b.smithi059.stdout: e1 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout: legacy client fscid: -1 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout: No filesystems configured 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:16:15.373 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Deploying cephadm binary to smithi196 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:15.374 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Updating smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Updating smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Added host smithi196 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.375 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/271277691' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/271277691' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:15.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Updating smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Updating smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c;count:3 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:15.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Deploying daemon mon.c on smithi196 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/1835594462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.a calling monitor election 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:15.378 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.c calling monitor election 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: monmap epoch 2 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: last_changed 2024-05-09T01:16:06.382132+0000 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: created 2024-05-09T01:14:15.452380+0000 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: min_mon_release 19 (squid) 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: election_strategy: 1 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T01:16:15.379 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: fsmap 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mgrmap e13: x(active, since 50s) 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: overall HEALTH_OK 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: Deploying daemon mon.b on smithi059 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/1752296444' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 ceph-mon[34351]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T01:16:15.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 bash[34332]: 9a88f306158aadb37780c72fc373a098022f32baccaa78e53895cb6c14ccd604 2024-05-09T01:16:15.381 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:15 smithi059 systemd[1]: Started Ceph mon.b for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:16:20.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.142+0000 7f0e477fe700 1 -- 172.21.15.196:0/1339012211 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f0e4c010410 con 0x7f0e5819b100 2024-05-09T01:16:20.144 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:16:20.144 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","modified":"2024-05-09T01:16:15.129683Z","created":"2024-05-09T01:14:15.452380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-09T01:16:20.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e4003e2b0 msgr2=0x7f0e400407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:20.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e4003e2b0 0x7f0e400407c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009900 tx=0x7f0e500075c0 comp rx=0 tx=0).stop 2024-05-09T01:16:20.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e5819b100 msgr2=0x7f0e5819b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:20.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e5819b100 0x7f0e5819b510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00b4d0 tx=0x7f0e4c00b890 comp rx=0 tx=0).stop 2024-05-09T01:16:20.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 shutdown_connections 2024-05-09T01:16:20.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e58103510 0x7f0e5819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:20.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e4003e2b0 0x7f0e400407c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:20.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 --2- 172.21.15.196:0/1339012211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e5819b100 0x7f0e5819b510 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:20.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 >> 172.21.15.196:0/1339012211 conn(0x7f0e58077e10 msgr2=0x7f0e58104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:20.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 shutdown_connections 2024-05-09T01:16:20.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:20.146+0000 7f0e5cb9e700 1 -- 172.21.15.196:0/1339012211 wait complete. 2024-05-09T01:16:20.148 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-05-09T01:16:20.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:20.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:16:20.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: mon.a calling monitor election 2024-05-09T01:16:20.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: mon.c calling monitor election 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/1339012211' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: mon.b calling monitor election 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: monmap epoch 3 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: last_changed 2024-05-09T01:16:15.129683+0000 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: created 2024-05-09T01:14:15.452380+0000 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: min_mon_release 19 (squid) 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: election_strategy: 1 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: 2: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-05-09T01:16:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: fsmap 2024-05-09T01:16:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T01:16:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: mgrmap e13: x(active, since 59s) 2024-05-09T01:16:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: overall HEALTH_OK 2024-05-09T01:16:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:20.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:16:20.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: mon.a calling monitor election 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: mon.c calling monitor election 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/1339012211' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: mon.b calling monitor election 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: monmap epoch 3 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: last_changed 2024-05-09T01:16:15.129683+0000 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: created 2024-05-09T01:14:15.452380+0000 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: min_mon_release 19 (squid) 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: election_strategy: 1 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:16:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T01:16:20.602 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: 2: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-05-09T01:16:20.602 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: fsmap 2024-05-09T01:16:20.602 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T01:16:20.602 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: mgrmap e13: x(active, since 59s) 2024-05-09T01:16:20.602 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: overall HEALTH_OK 2024-05-09T01:16:20.602 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.602 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.602 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.602 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:20.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:16:20.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: mon.a calling monitor election 2024-05-09T01:16:20.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: mon.c calling monitor election 2024-05-09T01:16:20.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:16:20.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/1339012211' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T01:16:20.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:20.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: mon.b calling monitor election 2024-05-09T01:16:20.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:20.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:20.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T01:16:20.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: monmap epoch 3 2024-05-09T01:16:20.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:20.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: last_changed 2024-05-09T01:16:15.129683+0000 2024-05-09T01:16:20.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: created 2024-05-09T01:14:15.452380+0000 2024-05-09T01:16:20.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: min_mon_release 19 (squid) 2024-05-09T01:16:20.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: election_strategy: 1 2024-05-09T01:16:20.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:16:20.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T01:16:20.651 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: 2: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-05-09T01:16:20.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: fsmap 2024-05-09T01:16:20.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T01:16:20.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: mgrmap e13: x(active, since 59s) 2024-05-09T01:16:20.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: overall HEALTH_OK 2024-05-09T01:16:20.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.652 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:20.686 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T01:16:20.686 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T01:16:20.966 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:16:21.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:21.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:21.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:21.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:21.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:21.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:21.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:21.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:21.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:16:22.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:22.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:22.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T01:16:22.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:22.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:22.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:22.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:22.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:22.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T01:16:22.699 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:22.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.797+0000 7fcc9b52f700 1 -- 172.21.15.19:0/2342765868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc94103410 msgr2=0x7fcc94103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:22.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.797+0000 7fcc9b52f700 1 --2- 172.21.15.19:0/2342765868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc94103410 0x7fcc94103820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009910 tx=0x7fcc90009c20 comp rx=0 tx=0).stop 2024-05-09T01:16:22.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.797+0000 7fcc9b52f700 1 -- 172.21.15.19:0/2342765868 shutdown_connections 2024-05-09T01:16:22.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.797+0000 7fcc9b52f700 1 --2- 172.21.15.19:0/2342765868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc94103410 0x7fcc94103820 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:22.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.797+0000 7fcc9b52f700 1 -- 172.21.15.19:0/2342765868 >> 172.21.15.19:0/2342765868 conn(0x7fcc940fde30 msgr2=0x7fcc94100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.797+0000 7fcc9b52f700 1 -- 172.21.15.19:0/2342765868 shutdown_connections 2024-05-09T01:16:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.797+0000 7fcc9b52f700 1 -- 172.21.15.19:0/2342765868 wait complete. 2024-05-09T01:16:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc9b52f700 1 Processor -- start 2024-05-09T01:16:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc9b52f700 1 -- start start 2024-05-09T01:16:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc9b52f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc94103410 0x7fcc94198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc9b52f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc941992c0 0x7fcc9419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc9b52f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc9419daf0 0x7fcc9419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:22.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc9b52f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9410d1e0 con 0x7fcc941992c0 2024-05-09T01:16:22.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc9b52f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9410d060 con 0x7fcc94103410 2024-05-09T01:16:22.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc9b52f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9410d360 con 0x7fcc9419daf0 2024-05-09T01:16:22.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc98aca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc941992c0 0x7fcc9419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:22.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc98aca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc941992c0 0x7fcc9419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48566/0 (socket says 172.21.15.19:48566) 2024-05-09T01:16:22.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc98aca700 1 -- 172.21.15.19:0/490548464 learned_addr learned my addr 172.21.15.19:0/490548464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:16:22.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc992cb700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc94103410 0x7fcc94198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc99acc700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc9419daf0 0x7fcc9419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc98aca700 1 -- 172.21.15.19:0/490548464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc9419daf0 msgr2=0x7fcc9419df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc98aca700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc9419daf0 0x7fcc9419df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc98aca700 1 -- 172.21.15.19:0/490548464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc94103410 msgr2=0x7fcc94198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc98aca700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc94103410 0x7fcc94198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.798+0000 7fcc98aca700 1 -- 172.21.15.19:0/490548464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc941a5860 con 0x7fcc941992c0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.799+0000 7fcc98aca700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc941992c0 0x7fcc9419d5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400b780 tx=0x7fcc8400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.799+0000 7fcc8a7fc700 1 -- 172.21.15.19:0/490548464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84004230 con 0x7fcc941992c0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.799+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc941a5a90 con 0x7fcc941992c0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.799+0000 7fcc8a7fc700 1 -- 172.21.15.19:0/490548464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc840043d0 con 0x7fcc941992c0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.799+0000 7fcc8a7fc700 1 -- 172.21.15.19:0/490548464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84011ad0 con 0x7fcc941992c0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.799+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc941a5fd0 con 0x7fcc941992c0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.800+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc94064e20 con 0x7fcc941992c0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.800+0000 7fcc8a7fc700 1 -- 172.21.15.19:0/490548464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fcc84011cb0 con 0x7fcc941992c0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.800+0000 7fcc8a7fc700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcc8003e9a0 0x7fcc80040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.800+0000 7fcc8a7fc700 1 -- 172.21.15.19:0/490548464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcc84052380 con 0x7fcc941992c0 2024-05-09T01:16:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.800+0000 7fcc992cb700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcc8003e9a0 0x7fcc80040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:22.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.801+0000 7fcc992cb700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcc8003e9a0 0x7fcc80040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009750 tx=0x7fcc9001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:22.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:22.806+0000 7fcc8a7fc700 1 -- 172.21.15.19:0/490548464 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc84020740 con 0x7fcc941992c0 2024-05-09T01:16:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:22.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T01:16:22.851 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.020+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fcc941a61a0 con 0x7fcc941992c0 2024-05-09T01:16:23.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.021+0000 7fcc8a7fc700 1 -- 172.21.15.19:0/490548464 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fcc84017d50 con 0x7fcc941992c0 2024-05-09T01:16:23.021 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:23.021 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-05-09T01:16:23.021 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:16:23.021 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-05-09T01:16:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcc8003e9a0 msgr2=0x7fcc80040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcc8003e9a0 0x7fcc80040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009750 tx=0x7fcc9001a040 comp rx=0 tx=0).stop 2024-05-09T01:16:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc941992c0 msgr2=0x7fcc9419d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc941992c0 0x7fcc9419d5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400b780 tx=0x7fcc8400ba90 comp rx=0 tx=0).stop 2024-05-09T01:16:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 shutdown_connections 2024-05-09T01:16:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc9419daf0 0x7fcc9419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc94103410 0x7fcc94198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcc8003e9a0 0x7fcc80040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 --2- 172.21.15.19:0/490548464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc941992c0 0x7fcc9419d5b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 >> 172.21.15.19:0/490548464 conn(0x7fcc940fde30 msgr2=0x7fcc94108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 shutdown_connections 2024-05-09T01:16:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:23.023+0000 7fcc9b52f700 1 -- 172.21.15.19:0/490548464 wait complete. 2024-05-09T01:16:23.634 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:23 smithi019 ceph-mon[30158]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T01:16:23.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:23 smithi019 ceph-mon[30158]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T01:16:23.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:23 smithi019 ceph-mon[30158]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T01:16:23.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:23 smithi019 ceph-mon[30158]: Updating smithi019:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:23.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:23 smithi019 ceph-mon[30158]: Updating smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:23.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:23 smithi019 ceph-mon[30158]: Updating smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:23.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:23 smithi019 ceph-mon[30158]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T01:16:23.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:23 smithi019 ceph-mon[30158]: Reconfiguring daemon mon.a on smithi019 2024-05-09T01:16:23.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:23 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/490548464' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:23.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:23 smithi196 ceph-mon[35041]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T01:16:23.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:23 smithi196 ceph-mon[35041]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T01:16:23.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:23 smithi196 ceph-mon[35041]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T01:16:23.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:23 smithi196 ceph-mon[35041]: Updating smithi019:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:23.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:23 smithi196 ceph-mon[35041]: Updating smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:23.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:23 smithi196 ceph-mon[35041]: Updating smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:23.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:23 smithi196 ceph-mon[35041]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T01:16:23.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:23 smithi196 ceph-mon[35041]: Reconfiguring daemon mon.a on smithi019 2024-05-09T01:16:23.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:23 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/490548464' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:23.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:23 smithi059 ceph-mon[34351]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T01:16:23.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:23 smithi059 ceph-mon[34351]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T01:16:23.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:23 smithi059 ceph-mon[34351]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T01:16:23.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:23 smithi059 ceph-mon[34351]: Updating smithi019:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:23.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:23 smithi059 ceph-mon[34351]: Updating smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:23.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:23 smithi059 ceph-mon[34351]: Updating smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T01:16:23.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:23 smithi059 ceph-mon[34351]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T01:16:23.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:23 smithi059 ceph-mon[34351]: Reconfiguring daemon mon.a on smithi019 2024-05-09T01:16:23.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:23 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/490548464' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:24.497 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:24 smithi019 ceph-mon[30158]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:24.497 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:24.497 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:24.497 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:24.497 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T01:16:24.498 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:24.526 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T01:16:24.526 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:16:24.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T01:16:24.556 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:16:24.556 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:16:24.626 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:16:24.626 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T01:16:24.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:24 smithi059 ceph-mon[34351]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:24.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:24.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:24.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:24.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T01:16:24.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:24.653 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:16:24.653 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:16:24.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:24 smithi196 ceph-mon[35041]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:24.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:24.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:24.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:24.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T01:16:24.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:24.719 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:16:24.719 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T01:16:24.751 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:16:24.751 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T01:16:24.820 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-05-09T01:16:24.820 INFO:tasks.cephadm:Adding mgr.y on smithi059 2024-05-09T01:16:24.820 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-05-09T01:16:24.820 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi059=y;smithi196=z' 2024-05-09T01:16:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:16:25.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:25 smithi196 ceph-mon[35041]: Reconfiguring mon.b (monmap changed)... 2024-05-09T01:16:25.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:25 smithi196 ceph-mon[35041]: Reconfiguring daemon mon.b on smithi059 2024-05-09T01:16:25.643 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:25 smithi059 ceph-mon[34351]: Reconfiguring mon.b (monmap changed)... 2024-05-09T01:16:25.644 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:25 smithi059 ceph-mon[34351]: Reconfiguring daemon mon.b on smithi059 2024-05-09T01:16:25.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:25 smithi019 ceph-mon[30158]: Reconfiguring mon.b (monmap changed)... 2024-05-09T01:16:25.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:25 smithi019 ceph-mon[30158]: Reconfiguring daemon mon.b on smithi059 2024-05-09T01:16:26.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.552+0000 7f67095ee700 1 -- 172.21.15.196:0/3263868269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704104220 msgr2=0x7f6704104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:26.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.552+0000 7f67095ee700 1 --2- 172.21.15.196:0/3263868269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704104220 0x7f6704104630 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009910 tx=0x7f66f4009c20 comp rx=0 tx=0).stop 2024-05-09T01:16:26.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.552+0000 7f67095ee700 1 -- 172.21.15.196:0/3263868269 shutdown_connections 2024-05-09T01:16:26.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.552+0000 7f67095ee700 1 --2- 172.21.15.196:0/3263868269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704104220 0x7f6704104630 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:26.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.552+0000 7f67095ee700 1 -- 172.21.15.196:0/3263868269 >> 172.21.15.196:0/3263868269 conn(0x7f67040fd9d0 msgr2=0x7f67040ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:26.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.552+0000 7f67095ee700 1 -- 172.21.15.196:0/3263868269 shutdown_connections 2024-05-09T01:16:26.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.552+0000 7f67095ee700 1 -- 172.21.15.196:0/3263868269 wait complete. 2024-05-09T01:16:26.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67095ee700 1 Processor -- start 2024-05-09T01:16:26.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67095ee700 1 -- start start 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67095ee700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6704104220 0x7f67040ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67095ee700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67040fdf90 0x7f67040fe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67095ee700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67040fe8e0 0x7f67041a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67095ee700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6704108be0 con 0x7f67040fe8e0 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67095ee700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6704108a60 con 0x7f6704104220 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67095ee700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6704108d60 con 0x7f67040fdf90 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67027fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67040fdf90 0x7f67040fe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67027fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67040fdf90 0x7f67040fe3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54910/0 (socket says 172.21.15.196:54910) 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67027fc700 1 -- 172.21.15.196:0/3489008998 learned_addr learned my addr 172.21.15.196:0/3489008998 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:16:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f67037fe700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67040fe8e0 0x7f67041a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.553+0000 7f6702ffd700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6704104220 0x7f67040ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f67027fc700 1 -- 172.21.15.196:0/3489008998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6704104220 msgr2=0x7f67040ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f67027fc700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6704104220 0x7f67040ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f67027fc700 1 -- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67040fe8e0 msgr2=0x7f67041a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f67027fc700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67040fe8e0 0x7f67041a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f67027fc700 1 -- 172.21.15.196:0/3489008998 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67041a3fb0 con 0x7f67040fdf90 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f67037fe700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67040fe8e0 0x7f67041a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f67027fc700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67040fdf90 0x7f67040fe3a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f66f800b780 tx=0x7f66f800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f66ebfff700 1 -- 172.21.15.196:0/3489008998 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8016070 con 0x7f67040fdf90 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67041a4240 con 0x7f67040fdf90 2024-05-09T01:16:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f66ebfff700 1 -- 172.21.15.196:0/3489008998 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f80036a0 con 0x7f67040fdf90 2024-05-09T01:16:26.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f66ebfff700 1 -- 172.21.15.196:0/3489008998 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f80119c0 con 0x7f67040fdf90 2024-05-09T01:16:26.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.554+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67041a4820 con 0x7f67040fdf90 2024-05-09T01:16:26.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.555+0000 7f66ebfff700 1 -- 172.21.15.196:0/3489008998 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f66f8019070 con 0x7f67040fdf90 2024-05-09T01:16:26.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.555+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6704064e20 con 0x7f67040fdf90 2024-05-09T01:16:26.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.555+0000 7f66ebfff700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f66ec03e9a0 0x7f66ec040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:26.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.555+0000 7f66ebfff700 1 -- 172.21.15.196:0/3489008998 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f66f80512a0 con 0x7f67040fdf90 2024-05-09T01:16:26.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.556+0000 7f6702ffd700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f66ec03e9a0 0x7f66ec040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:26.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.556+0000 7f6702ffd700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f66ec03e9a0 0x7f66ec040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006010 tx=0x7f66f401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:26.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.561+0000 7f66ebfff700 1 -- 172.21.15.196:0/3489008998 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66f8019360 con 0x7f67040fdf90 2024-05-09T01:16:26.588 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:26 smithi196 ceph-mon[35041]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:26.588 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:26.588 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:26.588 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:26.588 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T01:16:26.588 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:26.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:26 smithi019 ceph-mon[30158]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:26.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:26.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:26.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:26.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T01:16:26.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:26.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.782+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi059=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7f6704063c00 con 0x7f66ec03e9a0 2024-05-09T01:16:26.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.789+0000 7f66ebfff700 1 -- 172.21.15.196:0/3489008998 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6704063c00 con 0x7f66ec03e9a0 2024-05-09T01:16:26.788 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-05-09T01:16:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f66ec03e9a0 msgr2=0x7f66ec040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f66ec03e9a0 0x7f66ec040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006010 tx=0x7f66f401a040 comp rx=0 tx=0).stop 2024-05-09T01:16:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67040fdf90 msgr2=0x7f67040fe3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67040fdf90 0x7f67040fe3a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f66f800b780 tx=0x7f66f800ba90 comp rx=0 tx=0).stop 2024-05-09T01:16:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 shutdown_connections 2024-05-09T01:16:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f66ec03e9a0 0x7f66ec040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67040fe8e0 0x7f67041a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6704104220 0x7f67040ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 --2- 172.21.15.196:0/3489008998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67040fdf90 0x7f67040fe3a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:26.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.791+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 >> 172.21.15.196:0/3489008998 conn(0x7f67040fd9d0 msgr2=0x7f6704102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:26.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.792+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 shutdown_connections 2024-05-09T01:16:26.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:16:26.792+0000 7f67095ee700 1 -- 172.21.15.196:0/3489008998 wait complete. 2024-05-09T01:16:26.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:26 smithi059 ceph-mon[34351]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:26.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:26.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:26.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T01:16:26.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T01:16:26.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:27.427 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:27 smithi196 ceph-mon[35041]: Reconfiguring mon.c (monmap changed)... 2024-05-09T01:16:27.428 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:27 smithi196 ceph-mon[35041]: Reconfiguring daemon mon.c on smithi196 2024-05-09T01:16:27.428 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:27.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:27 smithi019 ceph-mon[30158]: Reconfiguring mon.c (monmap changed)... 2024-05-09T01:16:27.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:27 smithi019 ceph-mon[30158]: Reconfiguring daemon mon.c on smithi196 2024-05-09T01:16:27.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:27.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:27 smithi059 ceph-mon[34351]: Reconfiguring mon.c (monmap changed)... 2024-05-09T01:16:27.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:27 smithi059 ceph-mon[34351]: Reconfiguring daemon mon.c on smithi196 2024-05-09T01:16:27.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:28.023 DEBUG:teuthology.orchestra.run.smithi059:mgr.y> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.y.service 2024-05-09T01:16:28.026 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.z.service 2024-05-09T01:16:28.029 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T01:16:28.029 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:16:28.029 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T01:16:28.047 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T01:16:28.047 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-05-09T01:16:28.103 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T01:16:28.103 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:28.103 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T01:16:28.103 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:28.103 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:28.103 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-09 01:15:34.055860951 +0000 2024-05-09T01:16:28.103 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-09 01:10:48.209613894 +0000 2024-05-09T01:16:28.104 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-09 01:10:48.209613894 +0000 2024-05-09T01:16:28.104 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-09 01:10:48.209613894 +0000 2024-05-09T01:16:28.104 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T01:16:28.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-09T01:16:28.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-09T01:16:28.169 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000211918 s, 2.4 MB/s 2024-05-09T01:16:28.170 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T01:16:28.227 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-05-09T01:16:28.283 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T01:16:28.284 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:28.284 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-09T01:16:28.284 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:28.284 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:28.284 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-09 01:15:34.091860652 +0000 2024-05-09T01:16:28.284 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-09 01:10:48.576611042 +0000 2024-05-09T01:16:28.284 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-09 01:10:48.576611042 +0000 2024-05-09T01:16:28.284 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-09 01:10:48.576611042 +0000 2024-05-09T01:16:28.284 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T01:16:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-09T01:16:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-09T01:16:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000247844 s, 2.1 MB/s 2024-05-09T01:16:28.350 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T01:16:28.407 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-05-09T01:16:28.464 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T01:16:28.464 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:28.464 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-09T01:16:28.464 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:28.464 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:28.464 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-09 01:15:34.125860369 +0000 2024-05-09T01:16:28.464 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-09 01:10:48.933608269 +0000 2024-05-09T01:16:28.464 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-09 01:10:48.933608269 +0000 2024-05-09T01:16:28.464 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-09 01:10:48.933608269 +0000 2024-05-09T01:16:28.465 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T01:16:28.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:28 smithi019 ceph-mon[30158]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi059=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:28.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:28 smithi019 ceph-mon[30158]: Saving service mgr spec with placement smithi019=x;smithi059=y;smithi196=z;count:3 2024-05-09T01:16:28.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:28 smithi019 ceph-mon[30158]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:28.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:28 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:28.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:28 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:28.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:28 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-09T01:16:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-09T01:16:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000185072 s, 2.8 MB/s 2024-05-09T01:16:28.531 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T01:16:28.588 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-05-09T01:16:28.645 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T01:16:28.645 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:28.645 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-09T01:16:28.645 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:28.645 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:28.645 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-09 01:15:34.160860078 +0000 2024-05-09T01:16:28.645 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-09 01:10:49.286605526 +0000 2024-05-09T01:16:28.645 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-09 01:10:49.286605526 +0000 2024-05-09T01:16:28.645 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-09 01:10:49.286605526 +0000 2024-05-09T01:16:28.645 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T01:16:28.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:28 smithi196 ceph-mon[35041]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi059=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:28.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:28 smithi196 ceph-mon[35041]: Saving service mgr spec with placement smithi019=x;smithi059=y;smithi196=z;count:3 2024-05-09T01:16:28.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:28 smithi196 ceph-mon[35041]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:28.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:28 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:28.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:28 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:28.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:28 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:28.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-09T01:16:28.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-09T01:16:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000189345 s, 2.7 MB/s 2024-05-09T01:16:28.711 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T01:16:28.767 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:16:28.767 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T01:16:28.787 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T01:16:28.787 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-05-09T01:16:28.845 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T01:16:28.845 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:28.845 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-09T01:16:28.845 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:28.846 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:28.846 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-09 01:16:14.572397803 +0000 2024-05-09T01:16:28.846 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-09 01:11:03.685283197 +0000 2024-05-09T01:16:28.846 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-09 01:11:03.685283197 +0000 2024-05-09T01:16:28.846 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-09 01:11:03.685283197 +0000 2024-05-09T01:16:28.846 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T01:16:28.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:28 smithi059 ceph-mon[34351]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi059=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:28.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:28 smithi059 ceph-mon[34351]: Saving service mgr spec with placement smithi019=x;smithi059=y;smithi196=z;count:3 2024-05-09T01:16:28.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:28 smithi059 ceph-mon[34351]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:28.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:28 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:28.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:28 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:28.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:28 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:28.874 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-09T01:16:28.875 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-09T01:16:28.875 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000288039 s, 1.8 MB/s 2024-05-09T01:16:28.876 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T01:16:28.934 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-05-09T01:16:28.991 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T01:16:28.991 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:28.991 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-05-09T01:16:28.991 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:28.991 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:28.991 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-09 01:16:14.573397791 +0000 2024-05-09T01:16:28.992 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-09 01:11:04.038278575 +0000 2024-05-09T01:16:28.992 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-09 01:11:04.038278575 +0000 2024-05-09T01:16:28.992 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-09 01:11:04.038278575 +0000 2024-05-09T01:16:28.992 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T01:16:29.056 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-09T01:16:29.057 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-09T01:16:29.057 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000235508 s, 2.2 MB/s 2024-05-09T01:16:29.058 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T01:16:29.115 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-05-09T01:16:29.172 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T01:16:29.172 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:29.172 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-09T01:16:29.172 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:29.172 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:29.172 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-09 01:16:14.573397791 +0000 2024-05-09T01:16:29.172 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-09 01:11:04.394273911 +0000 2024-05-09T01:16:29.172 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-09 01:11:04.394273911 +0000 2024-05-09T01:16:29.172 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-09 01:11:04.394273911 +0000 2024-05-09T01:16:29.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T01:16:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-09T01:16:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-09T01:16:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000181554 s, 2.8 MB/s 2024-05-09T01:16:29.238 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T01:16:29.295 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-05-09T01:16:29.351 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T01:16:29.351 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:29.351 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-09T01:16:29.352 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:29.352 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:29.352 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-09 01:16:14.574397779 +0000 2024-05-09T01:16:29.352 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-09 01:11:04.752269220 +0000 2024-05-09T01:16:29.352 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-09 01:11:04.752269220 +0000 2024-05-09T01:16:29.352 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-09 01:11:04.752269220 +0000 2024-05-09T01:16:29.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T01:16:29.415 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-09T01:16:29.415 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-09T01:16:29.415 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000184853 s, 2.8 MB/s 2024-05-09T01:16:29.416 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T01:16:29.472 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:16:29.472 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T01:16:29.489 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T01:16:29.489 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-05-09T01:16:29.544 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T01:16:29.544 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:29.545 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-09T01:16:29.545 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:29.545 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:29.545 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-09 01:16:05.833841605 +0000 2024-05-09T01:16:29.545 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-09 01:11:12.853097843 +0000 2024-05-09T01:16:29.545 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-09 01:11:12.853097843 +0000 2024-05-09T01:16:29.545 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-09 01:11:12.853097843 +0000 2024-05-09T01:16:29.546 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T01:16:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T01:16:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T01:16:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T01:16:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:29 smithi059 ceph-mon[34351]: Deploying daemon mgr.z on smithi196 2024-05-09T01:16:29.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:29.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:29.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:29.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T01:16:29.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T01:16:29.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T01:16:29.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:29.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:29 smithi196 ceph-mon[35041]: Deploying daemon mgr.z on smithi196 2024-05-09T01:16:29.608 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-09T01:16:29.608 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-09T01:16:29.608 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000266084 s, 1.9 MB/s 2024-05-09T01:16:29.609 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T01:16:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:29.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T01:16:29.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T01:16:29.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T01:16:29.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:29.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:29 smithi019 ceph-mon[30158]: Deploying daemon mgr.z on smithi196 2024-05-09T01:16:29.664 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-05-09T01:16:29.719 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T01:16:29.719 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:29.719 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-09T01:16:29.719 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:29.719 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:29.719 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-09 01:16:05.834841584 +0000 2024-05-09T01:16:29.719 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-09 01:11:13.178090825 +0000 2024-05-09T01:16:29.719 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-09 01:11:13.178090825 +0000 2024-05-09T01:16:29.719 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-09 01:11:13.178090825 +0000 2024-05-09T01:16:29.720 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T01:16:29.781 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-09T01:16:29.781 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-09T01:16:29.781 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000182863 s, 2.8 MB/s 2024-05-09T01:16:29.782 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T01:16:29.837 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-05-09T01:16:29.891 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T01:16:29.891 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:29.891 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-09T01:16:29.891 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:29.891 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:29.891 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-09 01:16:05.834841584 +0000 2024-05-09T01:16:29.892 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-09 01:11:13.521083418 +0000 2024-05-09T01:16:29.892 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-09 01:11:13.521083418 +0000 2024-05-09T01:16:29.892 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-09 01:11:13.521083418 +0000 2024-05-09T01:16:29.892 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T01:16:29.954 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-09T01:16:29.954 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-09T01:16:29.954 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000174312 s, 2.9 MB/s 2024-05-09T01:16:29.955 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T01:16:30.011 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-05-09T01:16:30.068 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T01:16:30.068 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T01:16:30.068 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T01:16:30.069 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:16:30.069 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T01:16:30.069 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-09 01:16:05.835841564 +0000 2024-05-09T01:16:30.069 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-09 01:11:13.857076162 +0000 2024-05-09T01:16:30.069 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-09 01:11:13.857076162 +0000 2024-05-09T01:16:30.069 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-09 01:11:13.857076162 +0000 2024-05-09T01:16:30.069 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T01:16:30.184 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-09T01:16:30.184 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-09T01:16:30.184 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000207589 s, 2.5 MB/s 2024-05-09T01:16:30.185 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T01:16:30.200 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-05-09T01:16:30.200 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T01:16:30.455 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:16:30.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:30 smithi019 ceph-mon[30158]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:30.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:30 smithi196 ceph-mon[35041]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:30.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:30 smithi059 ceph-mon[34351]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:30.882 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:30 smithi196 ceph-mgr[36895]: -- 172.21.15.196:0/3674553836 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae558cc5a0 con 0x55ae5a656480 2024-05-09T01:16:31.168 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:30 smithi196 bash[36876]: 9b182b9c488027d72d87e9fd933b91b59c08c038947662d84879440b5b64844c 2024-05-09T01:16:31.168 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:30 smithi196 systemd[1]: Started Ceph mgr.z for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:16:31.168 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:30 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:30.985+0000 7ff089b3a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T01:16:31.168 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:31.168+0000 7ff089b3a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:16:31.446 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:31.269+0000 7ff089b3a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T01:16:31.947 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:31.578+0000 7ff089b3a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T01:16:31.947 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:31.664+0000 7ff089b3a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T01:16:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T01:16:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T01:16:32.262 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-09T01:16:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-09T01:16:32.262 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124321 s, 843 MB/s 2024-05-09T01:16:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-09T01:16:32.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:32.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:32.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:32.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T01:16:32.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T01:16:32.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T01:16:32.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:32.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:31 smithi019 ceph-mon[30158]: Deploying daemon mgr.y on smithi059 2024-05-09T01:16:32.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:31 smithi019 ceph-mon[30158]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:32.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:32.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:32.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:32.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T01:16:32.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T01:16:32.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T01:16:32.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:32.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:31 smithi059 ceph-mon[34351]: Deploying daemon mgr.y on smithi059 2024-05-09T01:16:32.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:31 smithi059 ceph-mon[34351]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:32.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:32.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:32.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:32.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T01:16:32.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T01:16:32.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T01:16:32.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:32.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:31 smithi196 ceph-mon[35041]: Deploying daemon mgr.y on smithi059 2024-05-09T01:16:32.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:31 smithi196 ceph-mon[35041]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:32.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-05-09T01:16:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:16:33.197 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:32 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:32.857+0000 7ff089b3a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T01:16:33.197 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:33.030+0000 7ff089b3a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T01:16:33.510 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:33.204+0000 7ff089b3a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T01:16:33.510 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:33.510+0000 7ff089b3a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T01:16:33.820 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:33.607+0000 7ff089b3a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T01:16:33.849 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:33 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:33.797+0000 7fa658add200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:16:34.095 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:33.820+0000 7ff089b3a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T01:16:34.096 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:34 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:34.022+0000 7ff089b3a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:16:34.201 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:34 smithi019 ceph-mon[30158]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:34.201 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.201 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.201 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.201 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:34.245 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:34 smithi059 ceph-mon[34351]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:34.245 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.245 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.245 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.245 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.245 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:34.246 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:33 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:33.909+0000 7fa658add200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T01:16:34.246 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:34 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:34.245+0000 7fa658add200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T01:16:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:34 smithi196 ceph-mon[35041]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:34.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:34.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:34.599 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:34 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:34.343+0000 7fa658add200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T01:16:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.671+0000 7fe645d21700 1 -- 172.21.15.19:0/1863065994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640104390 msgr2=0x7fe6401047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.671+0000 7fe645d21700 1 --2- 172.21.15.19:0/1863065994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640104390 0x7fe6401047a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe634009990 tx=0x7fe634009ca0 comp rx=0 tx=0).stop 2024-05-09T01:16:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.672+0000 7fe645d21700 1 -- 172.21.15.19:0/1863065994 shutdown_connections 2024-05-09T01:16:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.672+0000 7fe645d21700 1 --2- 172.21.15.19:0/1863065994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe640104d30 0x7fe64010b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.672+0000 7fe645d21700 1 --2- 172.21.15.19:0/1863065994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe640103260 0x7fe640103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.672+0000 7fe645d21700 1 --2- 172.21.15.19:0/1863065994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640104390 0x7fe6401047a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.672+0000 7fe645d21700 1 -- 172.21.15.19:0/1863065994 >> 172.21.15.19:0/1863065994 conn(0x7fe6400fe8c0 msgr2=0x7fe640100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.672+0000 7fe645d21700 1 -- 172.21.15.19:0/1863065994 shutdown_connections 2024-05-09T01:16:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.672+0000 7fe645d21700 1 -- 172.21.15.19:0/1863065994 wait complete. 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.672+0000 7fe645d21700 1 Processor -- start 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.672+0000 7fe645d21700 1 -- start start 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe645d21700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640103260 0x7fe64019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe645d21700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe640104390 0x7fe64019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe645d21700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe640104d30 0x7fe6401a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe645d21700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe64010ecc0 con 0x7fe640103260 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe645d21700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe64010eb40 con 0x7fe640104d30 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe645d21700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe64010ee40 con 0x7fe640104390 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640103260 0x7fe64019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640103260 0x7fe64019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59442/0 (socket says 172.21.15.19:59442) 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63effd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe640104390 0x7fe64019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63f7fe700 1 -- 172.21.15.19:0/190063759 learned_addr learned my addr 172.21.15.19:0/190063759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:16:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63ffff700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe640104d30 0x7fe6401a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63f7fe700 1 -- 172.21.15.19:0/190063759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe640104390 msgr2=0x7fe64019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63f7fe700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe640104390 0x7fe64019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63f7fe700 1 -- 172.21.15.19:0/190063759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe640104d30 msgr2=0x7fe6401a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63f7fe700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe640104d30 0x7fe6401a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63f7fe700 1 -- 172.21.15.19:0/190063759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6401a2470 con 0x7fe640103260 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63effd700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe640104390 0x7fe64019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.673+0000 7fe63f7fe700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640103260 0x7fe64019a910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe63000e940 tx=0x7fe63000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.674+0000 7fe63cff9700 1 -- 172.21.15.19:0/190063759 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630004220 con 0x7fe640103260 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.674+0000 7fe63cff9700 1 -- 172.21.15.19:0/190063759 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6300043c0 con 0x7fe640103260 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.674+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6401a2760 con 0x7fe640103260 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.674+0000 7fe63cff9700 1 -- 172.21.15.19:0/190063759 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630003c60 con 0x7fe640103260 2024-05-09T01:16:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.674+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6401a2ca0 con 0x7fe640103260 2024-05-09T01:16:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.675+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6400512c0 con 0x7fe640103260 2024-05-09T01:16:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.675+0000 7fe63cff9700 1 -- 172.21.15.19:0/190063759 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe630018430 con 0x7fe640103260 2024-05-09T01:16:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.675+0000 7fe63cff9700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe62803e750 0x7fe628040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:16:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.675+0000 7fe63cff9700 1 -- 172.21.15.19:0/190063759 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe630052a90 con 0x7fe640103260 2024-05-09T01:16:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.675+0000 7fe63effd700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe62803e750 0x7fe628040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:16:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.676+0000 7fe63effd700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe62803e750 0x7fe628040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe64019b9c0 tx=0x7fe63400b720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:16:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.681+0000 7fe63cff9700 1 -- 172.21.15.19:0/190063759 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe630003e40 con 0x7fe640103260 2024-05-09T01:16:34.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:34.905+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe640100900 con 0x7fe62803e750 2024-05-09T01:16:34.906 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:34 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:34.678+0000 7ff089b3a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T01:16:34.906 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:34 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:34.906+0000 7ff089b3a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T01:16:35.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:16:35.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:16:35.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:35.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:35.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:35.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:16:35.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:16:35.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:35.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:35.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:35.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:16:35.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:16:35.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:35.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:35.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:35.697 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:35.339+0000 7ff089b3a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T01:16:35.697 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:35.436+0000 7ff089b3a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T01:16:35.985 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:35 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:35.634+0000 7fa658add200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T01:16:35.985 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:35 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:35.809+0000 7fa658add200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T01:16:35.985 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:35 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:35.985+0000 7fa658add200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T01:16:36.164 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:35.810+0000 7ff089b3a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T01:16:36.164 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:36.164+0000 7ff089b3a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T01:16:36.446 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:36.258+0000 7ff089b3a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T01:16:36.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:36.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:36.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:36.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:36.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T01:16:36.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T01:16:36.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:36.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:36.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.588 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:36.282+0000 7fa658add200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T01:16:36.588 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:36.378+0000 7fa658add200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T01:16:36.588 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:36.588+0000 7fa658add200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T01:16:36.588 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:16:36.588 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:36.588 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.588 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.589 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:36.589 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:36.589 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.589 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T01:16:36.589 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T01:16:36.589 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:36.589 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:36.849 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:36.785+0000 7fa658add200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:16:36.947 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:36.502+0000 7ff089b3a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T01:16:36.947 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:36.633+0000 7ff089b3a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T01:16:36.947 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:36.729+0000 7ff089b3a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T01:16:37.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:37 smithi019 ceph-mon[30158]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T01:16:37.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:37 smithi019 ceph-mon[30158]: Reconfiguring daemon mgr.x on smithi019 2024-05-09T01:16:37.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:37 smithi196 ceph-mon[35041]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T01:16:37.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:37 smithi196 ceph-mon[35041]: Reconfiguring daemon mgr.x on smithi019 2024-05-09T01:16:37.447 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:37.066+0000 7ff089b3a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T01:16:37.447 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:37.158+0000 7ff089b3a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T01:16:37.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:37 smithi059 ceph-mon[34351]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T01:16:37.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:37 smithi059 ceph-mon[34351]: Reconfiguring daemon mgr.x on smithi019 2024-05-09T01:16:37.599 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:37.414+0000 7fa658add200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T01:16:38.080 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:37.643+0000 7fa658add200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T01:16:38.197 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:37.763+0000 7ff089b3a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T01:16:38.197 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:16:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z[36891]: 2024-05-09T01:16:37.979+0000 7ff089b3a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T01:16:38.349 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:38.080+0000 7fa658add200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T01:16:38.349 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:38.180+0000 7fa658add200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T01:16:38.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:38.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:38.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:38.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:38.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:38.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:38.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:38.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: Standby manager daemon z started 2024-05-09T01:16:38.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T01:16:38.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T01:16:38.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T01:16:38.350 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:38 smithi059 ceph-mon[34351]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: Standby manager daemon z started 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T01:16:38.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T01:16:38.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T01:16:38.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:38 smithi019 ceph-mon[30158]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T01:16:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:38.634+0000 7fe63cff9700 1 -- 172.21.15.19:0/190063759 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7fe63001dd20 con 0x7fe640103260 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: Standby manager daemon z started 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T01:16:38.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T01:16:38.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:38 smithi196 ceph-mon[35041]: from='mgr.? 172.21.15.196:0/2292588078' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T01:16:38.849 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:38.562+0000 7fa658add200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T01:16:39.278 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:38.932+0000 7fa658add200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T01:16:39.278 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:39.027+0000 7fa658add200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T01:16:39.599 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:39.278+0000 7fa658add200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T01:16:39.599 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:39.416+0000 7fa658add200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T01:16:39.599 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:39.514+0000 7fa658add200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T01:16:39.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:39 smithi196 ceph-mon[35041]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-09T01:16:39.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:39 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T01:16:39.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:39 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/767406449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88"}]: dispatch 2024-05-09T01:16:39.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:39 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/767406449' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88"}]': finished 2024-05-09T01:16:39.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:39 smithi196 ceph-mon[35041]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T01:16:39.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:39 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:39.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:39 smithi196 ceph-mon[35041]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:39.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:39 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1730148622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T01:16:39.948 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:39.853+0000 7fa658add200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T01:16:39.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:39 smithi059 ceph-mon[34351]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-09T01:16:39.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:39 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T01:16:39.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:39 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/767406449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88"}]: dispatch 2024-05-09T01:16:39.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:39 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/767406449' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88"}]': finished 2024-05-09T01:16:39.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:39 smithi059 ceph-mon[34351]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T01:16:39.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:39 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:39.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:39 smithi059 ceph-mon[34351]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:39.949 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:39 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1730148622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T01:16:40.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:39 smithi019 ceph-mon[30158]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-09T01:16:40.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:39 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T01:16:40.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:39 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/767406449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88"}]: dispatch 2024-05-09T01:16:40.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:39 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/767406449' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88"}]': finished 2024-05-09T01:16:40.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:39 smithi019 ceph-mon[30158]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T01:16:40.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:39 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:40.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:39 smithi019 ceph-mon[30158]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:40.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:39 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1730148622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T01:16:40.349 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:39.948+0000 7fa658add200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T01:16:40.840 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:40 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:40.574+0000 7fa658add200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T01:16:40.840 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:16:40 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y[35732]: 2024-05-09T01:16:40.795+0000 7fa658add200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T01:16:40.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:40.850+0000 7fe63cff9700 1 -- 172.21.15.19:0/190063759 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fe630017cd0 con 0x7fe640103260 2024-05-09T01:16:41.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:40 smithi059 ceph-mon[34351]: Standby manager daemon y started 2024-05-09T01:16:41.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:40 smithi059 ceph-mon[34351]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T01:16:41.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:40 smithi059 ceph-mon[34351]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T01:16:41.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:40 smithi059 ceph-mon[34351]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T01:16:41.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:40 smithi059 ceph-mon[34351]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T01:16:41.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:40 smithi019 ceph-mon[30158]: Standby manager daemon y started 2024-05-09T01:16:41.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:40 smithi019 ceph-mon[30158]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T01:16:41.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:40 smithi019 ceph-mon[30158]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T01:16:41.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:40 smithi019 ceph-mon[30158]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T01:16:41.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:40 smithi019 ceph-mon[30158]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T01:16:41.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:40 smithi196 ceph-mon[35041]: Standby manager daemon y started 2024-05-09T01:16:41.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:40 smithi196 ceph-mon[35041]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T01:16:41.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:40 smithi196 ceph-mon[35041]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T01:16:41.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:40 smithi196 ceph-mon[35041]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T01:16:41.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:40 smithi196 ceph-mon[35041]: from='mgr.? 172.21.15.59:0/1163988240' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T01:16:42.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:41 smithi019 ceph-mon[30158]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-09T01:16:42.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:41 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T01:16:42.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:41 smithi019 ceph-mon[30158]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:42.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:41 smithi196 ceph-mon[35041]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-09T01:16:42.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:41 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T01:16:42.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:41 smithi196 ceph-mon[35041]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:42.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:41 smithi059 ceph-mon[34351]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-09T01:16:42.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:41 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T01:16:42.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:41 smithi059 ceph-mon[34351]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:44.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:44 smithi059 ceph-mon[34351]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:44.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:44 smithi019 ceph-mon[30158]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:44.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:44 smithi196 ceph-mon[35041]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:46.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:46 smithi019 ceph-mon[30158]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:46.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:46 smithi196 ceph-mon[35041]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:46.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:46 smithi059 ceph-mon[34351]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:47.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T01:16:47.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:47.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T01:16:47.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:47.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T01:16:47.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:48.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:48 smithi019 ceph-mon[30158]: Deploying daemon osd.0 on smithi019 2024-05-09T01:16:48.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:48 smithi019 ceph-mon[30158]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:48.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:48 smithi196 ceph-mon[35041]: Deploying daemon osd.0 on smithi019 2024-05-09T01:16:48.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:48 smithi196 ceph-mon[35041]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:48.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:48 smithi059 ceph-mon[34351]: Deploying daemon osd.0 on smithi019 2024-05-09T01:16:48.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:48 smithi059 ceph-mon[34351]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:50.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:50 smithi019 ceph-mon[30158]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:50.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:50 smithi196 ceph-mon[35041]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:50.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:50 smithi059 ceph-mon[34351]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:52.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:52 smithi019 ceph-mon[30158]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:52.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:52 smithi196 ceph-mon[35041]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:52.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:52 smithi059 ceph-mon[34351]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:53.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:53.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:53.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:53.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:53.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:53.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:54.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:54 smithi196 ceph-mon[35041]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:54.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:54 smithi059 ceph-mon[34351]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:54.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:54 smithi019 ceph-mon[30158]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:56.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:56.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:56.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:55 smithi019 ceph-mon[30158]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:56.131 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-05-09T01:16:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.131+0000 7fe63cff9700 1 -- 172.21.15.19:0/190063759 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe640100900 con 0x7fe62803e750 2024-05-09T01:16:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe62803e750 msgr2=0x7fe628040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe62803e750 0x7fe628040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe64019b9c0 tx=0x7fe63400b720 comp rx=0 tx=0).stop 2024-05-09T01:16:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640103260 msgr2=0x7fe64019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:16:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640103260 0x7fe64019a910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe63000e940 tx=0x7fe63000ec50 comp rx=0 tx=0).stop 2024-05-09T01:16:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 shutdown_connections 2024-05-09T01:16:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe640104d30 0x7fe6401a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe62803e750 0x7fe628040c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe640103260 0x7fe64019a910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 --2- 172.21.15.19:0/190063759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe640104390 0x7fe64019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:16:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.133+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 >> 172.21.15.19:0/190063759 conn(0x7fe6400fe8c0 msgr2=0x7fe6401001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:16:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.134+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 shutdown_connections 2024-05-09T01:16:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:16:56.134+0000 7fe645d21700 1 -- 172.21.15.19:0/190063759 wait complete. 2024-05-09T01:16:56.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:56.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:56.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:55 smithi196 ceph-mon[35041]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:56.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:16:56.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:16:56.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:55 smithi059 ceph-mon[34351]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:56.749 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.0.service 2024-05-09T01:16:56.751 INFO:tasks.cephadm:Deploying osd.1 on smithi059 with /dev/vg_nvme/lv_4... 2024-05-09T01:16:56.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T01:16:56.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:56 smithi019 ceph-mon[30158]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T01:16:56.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:56.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:16:57.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:56 smithi059 ceph-mon[34351]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T01:16:57.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:57.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:57.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:57.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:56 smithi196 ceph-mon[35041]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T01:16:57.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:57.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:16:57.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:16:58.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:57 smithi059 ceph-mon[34351]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T01:16:58.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:57 smithi059 ceph-mon[34351]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T01:16:58.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:57 smithi059 ceph-mon[34351]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:16:58.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:57 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:58.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:57 smithi059 ceph-mon[34351]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:58.103 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:57 smithi019 ceph-mon[30158]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T01:16:58.103 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:57 smithi019 ceph-mon[30158]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T01:16:58.103 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:57 smithi019 ceph-mon[30158]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:16:58.104 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:57 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:58.104 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:57 smithi019 ceph-mon[30158]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:58.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:57 smithi196 ceph-mon[35041]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T01:16:58.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:57 smithi196 ceph-mon[35041]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T01:16:58.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:57 smithi196 ceph-mon[35041]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:16:58.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:57 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:58.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:57 smithi196 ceph-mon[35041]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:16:58.991 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T01:16:58.991 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T01:16:58.992 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-09T01:16:58.992 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-09T01:16:58.992 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0179939 s, 583 MB/s 2024-05-09T01:16:58.992 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-09T01:16:59.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:58 smithi019 ceph-mon[30158]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:16:59.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:58 smithi019 ceph-mon[30158]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T01:16:59.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:59.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:59.148 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:16:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0[44990]: 2024-05-09T01:16:58.898+0000 7f5255d82700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T01:16:59.148 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:16:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0[44990]: 2024-05-09T01:16:58.998+0000 7f524db2f700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:16:59.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:58 smithi196 ceph-mon[35041]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:16:59.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:58 smithi196 ceph-mon[35041]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T01:16:59.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:59.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:59.284 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:58 smithi059 ceph-mon[34351]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:16:59.284 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:58 smithi059 ceph-mon[34351]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T01:16:59.284 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:59.284 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:16:59.612 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_4 2024-05-09T01:16:59.865 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:17:00.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:17:00.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:17:00.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' 2024-05-09T01:17:00.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:17:00.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:17:00.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:17:00.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:00.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:00.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:16:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:17:00.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:00.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:16:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: from='osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191]' entity='osd.0' 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:00.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:16:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:01.058 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:00 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi019 2024-05-09T01:17:01.058 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:00 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi019 to 17093M 2024-05-09T01:17:01.058 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:00 smithi059 ceph-mon[34351]: osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] boot 2024-05-09T01:17:01.058 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:00 smithi059 ceph-mon[34351]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T01:17:01.058 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:00 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:17:01.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:00 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi019 2024-05-09T01:17:01.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:00 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi019 to 17093M 2024-05-09T01:17:01.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:00 smithi196 ceph-mon[35041]: osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] boot 2024-05-09T01:17:01.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:00 smithi196 ceph-mon[35041]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T01:17:01.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:00 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:17:01.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.277+0000 7fe27677b700 1 -- 172.21.15.59:0/3810850660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270105e30 msgr2=0x7fe27010c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:01.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.277+0000 7fe27677b700 1 --2- 172.21.15.59:0/3810850660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270105e30 0x7fe27010c6e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe26800b0a0 tx=0x7fe26800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:17:01.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.277+0000 7fe27677b700 1 -- 172.21.15.59:0/3810850660 shutdown_connections 2024-05-09T01:17:01.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.277+0000 7fe27677b700 1 --2- 172.21.15.59:0/3810850660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270105e30 0x7fe27010c6e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:01.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.277+0000 7fe27677b700 1 --2- 172.21.15.59:0/3810850660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe270104360 0x7fe270104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:01.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.277+0000 7fe27677b700 1 --2- 172.21.15.59:0/3810850660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe270105490 0x7fe2701058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:01.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.277+0000 7fe27677b700 1 -- 172.21.15.59:0/3810850660 >> 172.21.15.59:0/3810850660 conn(0x7fe2700ff9c0 msgr2=0x7fe270101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:17:01.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.277+0000 7fe27677b700 1 -- 172.21.15.59:0/3810850660 shutdown_connections 2024-05-09T01:17:01.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.277+0000 7fe27677b700 1 -- 172.21.15.59:0/3810850660 wait complete. 2024-05-09T01:17:01.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe27677b700 1 Processor -- start 2024-05-09T01:17:01.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe27677b700 1 -- start start 2024-05-09T01:17:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe27677b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270104360 0x7fe27019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe27677b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe270105490 0x7fe27019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe27677b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe270105e30 0x7fe2701a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe27677b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe27010fdc0 con 0x7fe270105e30 2024-05-09T01:17:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe27677b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe27010fc40 con 0x7fe270104360 2024-05-09T01:17:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe27677b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe27010ff40 con 0x7fe270105490 2024-05-09T01:17:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe26ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270104360 0x7fe27019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe26ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270104360 0x7fe27019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33912/0 (socket says 172.21.15.59:33912) 2024-05-09T01:17:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe26ffff700 1 -- 172.21.15.59:0/3263400028 learned_addr learned my addr 172.21.15.59:0/3263400028 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-09T01:17:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe274d18700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe270105e30 0x7fe2701a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.278+0000 7fe26f7fe700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe270105490 0x7fe27019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe26ffff700 1 -- 172.21.15.59:0/3263400028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe270105490 msgr2=0x7fe27019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe26ffff700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe270105490 0x7fe27019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe26ffff700 1 -- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe270105e30 msgr2=0x7fe2701a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe26ffff700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe270105e30 0x7fe2701a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe26ffff700 1 -- 172.21.15.59:0/3263400028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2701a2430 con 0x7fe270104360 2024-05-09T01:17:01.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe26ffff700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270104360 0x7fe27019a8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe26000d950 tx=0x7fe26000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:17:01.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe26d7fa700 1 -- 172.21.15.59:0/3263400028 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260016070 con 0x7fe270104360 2024-05-09T01:17:01.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe26d7fa700 1 -- 172.21.15.59:0/3263400028 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe260005020 con 0x7fe270104360 2024-05-09T01:17:01.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe26d7fa700 1 -- 172.21.15.59:0/3263400028 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26000f9d0 con 0x7fe270104360 2024-05-09T01:17:01.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2701a2720 con 0x7fe270104360 2024-05-09T01:17:01.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.279+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2701a2c60 con 0x7fe270104360 2024-05-09T01:17:01.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.280+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2700512c0 con 0x7fe270104360 2024-05-09T01:17:01.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.282+0000 7fe26d7fa700 1 -- 172.21.15.59:0/3263400028 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fe2600051d0 con 0x7fe270104360 2024-05-09T01:17:01.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.282+0000 7fe26d7fa700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe2580b27f0 0x7fe2580b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:01.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.282+0000 7fe26d7fa700 1 -- 172.21.15.59:0/3263400028 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fe2600e0640 con 0x7fe270104360 2024-05-09T01:17:01.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.282+0000 7fe26f7fe700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe2580b27f0 0x7fe2580b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:01.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.283+0000 7fe26f7fe700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe2580b27f0 0x7fe2580b4d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe27019b980 tx=0x7fe26400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:17:01.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.286+0000 7fe26d7fa700 1 -- 172.21.15.59:0/3263400028 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe2600a9dd0 con 0x7fe270104360 2024-05-09T01:17:01.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:00 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi019 2024-05-09T01:17:01.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:00 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi019 to 17093M 2024-05-09T01:17:01.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:00 smithi019 ceph-mon[30158]: osd.0 [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] boot 2024-05-09T01:17:01.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:00 smithi019 ceph-mon[30158]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T01:17:01.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:00 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T01:17:01.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:01.510+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe270063c00 con 0x7fe2580b27f0 2024-05-09T01:17:02.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:01 smithi196 ceph-mon[35041]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T01:17:02.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:01 smithi196 ceph-mon[35041]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:02.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:17:02.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:17:02.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:02.226 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:01 smithi059 ceph-mon[34351]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T01:17:02.227 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:01 smithi059 ceph-mon[34351]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:02.227 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:17:02.227 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:17:02.227 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:02.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:01 smithi019 ceph-mon[30158]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T01:17:02.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:01 smithi019 ceph-mon[30158]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:02.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:17:02.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:17:02.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:03.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:02 smithi196 ceph-mon[35041]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:17:03.275 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:02 smithi059 ceph-mon[34351]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:17:03.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:02 smithi019 ceph-mon[30158]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:17:04.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:03 smithi019 ceph-mon[30158]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:04.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:03 smithi196 ceph-mon[35041]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:04.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:03 smithi059 ceph-mon[34351]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:05.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:04 smithi059 ceph-mon[34351]: from='client.? 172.21.15.59:0/2803903043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6305a55-e657-43de-83fd-8e9568e91363"}]: dispatch 2024-05-09T01:17:05.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:04 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6305a55-e657-43de-83fd-8e9568e91363"}]: dispatch 2024-05-09T01:17:05.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:04 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6305a55-e657-43de-83fd-8e9568e91363"}]': finished 2024-05-09T01:17:05.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:04 smithi059 ceph-mon[34351]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T01:17:05.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:05.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:04 smithi196 ceph-mon[35041]: from='client.? 172.21.15.59:0/2803903043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6305a55-e657-43de-83fd-8e9568e91363"}]: dispatch 2024-05-09T01:17:05.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:04 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6305a55-e657-43de-83fd-8e9568e91363"}]: dispatch 2024-05-09T01:17:05.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:04 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6305a55-e657-43de-83fd-8e9568e91363"}]': finished 2024-05-09T01:17:05.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:04 smithi196 ceph-mon[35041]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T01:17:05.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:05.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:04 smithi019 ceph-mon[30158]: from='client.? 172.21.15.59:0/2803903043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6305a55-e657-43de-83fd-8e9568e91363"}]: dispatch 2024-05-09T01:17:05.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:04 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6305a55-e657-43de-83fd-8e9568e91363"}]: dispatch 2024-05-09T01:17:05.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:04 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6305a55-e657-43de-83fd-8e9568e91363"}]': finished 2024-05-09T01:17:05.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:04 smithi019 ceph-mon[30158]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T01:17:05.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:06.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:05 smithi196 ceph-mon[35041]: from='client.? 172.21.15.59:0/663877327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T01:17:06.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:05 smithi196 ceph-mon[35041]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:06.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:05 smithi059 ceph-mon[34351]: from='client.? 172.21.15.59:0/663877327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T01:17:06.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:05 smithi059 ceph-mon[34351]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:06.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:05 smithi019 ceph-mon[30158]: from='client.? 172.21.15.59:0/663877327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T01:17:06.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:05 smithi019 ceph-mon[30158]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:08.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:08 smithi019 ceph-mon[30158]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:08.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:08 smithi196 ceph-mon[35041]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:08.573 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:08 smithi059 ceph-mon[34351]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:10.395 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:10 smithi059 ceph-mon[34351]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:10.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:10 smithi019 ceph-mon[30158]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:10.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:10 smithi196 ceph-mon[35041]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:12.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:12 smithi019 ceph-mon[30158]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:12.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T01:17:12.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:12.408 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:12 smithi059 ceph-mon[34351]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:12.408 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:12 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T01:17:12.408 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:12 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:12.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:12 smithi196 ceph-mon[35041]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:12.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T01:17:12.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:13.291 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:13 smithi059 ceph-mon[34351]: Deploying daemon osd.1 on smithi059 2024-05-09T01:17:13.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:13 smithi019 ceph-mon[30158]: Deploying daemon osd.1 on smithi059 2024-05-09T01:17:13.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:13 smithi196 ceph-mon[35041]: Deploying daemon osd.1 on smithi059 2024-05-09T01:17:14.396 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:14 smithi059 ceph-mon[34351]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:14.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:14 smithi019 ceph-mon[30158]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:14.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:14 smithi196 ceph-mon[35041]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:16.271 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:16 smithi059 ceph-mon[34351]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:16.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:16 smithi019 ceph-mon[30158]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:16.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:16 smithi196 ceph-mon[35041]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:18.278 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:18 smithi059 ceph-mon[34351]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:18.279 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:18 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:18.279 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:18 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:18.279 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:18 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:18 smithi019 ceph-mon[30158]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:18 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:18 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:18 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:18.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:18 smithi196 ceph-mon[35041]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:18.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:18 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:18.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:18 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:18.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:18 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:20.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:20 smithi059 ceph-mon[34351]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:20.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:20.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:20.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:20.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:20.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:20 smithi019 ceph-mon[30158]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:20.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:20 smithi196 ceph-mon[35041]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:20.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:20.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:20.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:20.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:20.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:20.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.678+0000 7fe26d7fa700 1 -- 172.21.15.59:0/3263400028 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe270063c00 con 0x7fe2580b27f0 2024-05-09T01:17:20.678 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 1 on host 'smithi059' 2024-05-09T01:17:20.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe2580b27f0 msgr2=0x7fe2580b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:20.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe2580b27f0 0x7fe2580b4d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe27019b980 tx=0x7fe26400b410 comp rx=0 tx=0).stop 2024-05-09T01:17:20.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270104360 msgr2=0x7fe27019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:20.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270104360 0x7fe27019a8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe26000d950 tx=0x7fe26000dc60 comp rx=0 tx=0).stop 2024-05-09T01:17:20.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 shutdown_connections 2024-05-09T01:17:20.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe2580b27f0 0x7fe2580b4d00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:20.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe270105e30 0x7fe2701a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:20.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe270104360 0x7fe27019a8d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:20.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 --2- 172.21.15.59:0/3263400028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe270105490 0x7fe27019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:20.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 >> 172.21.15.59:0/3263400028 conn(0x7fe2700ff9c0 msgr2=0x7fe2701086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:17:20.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 shutdown_connections 2024-05-09T01:17:20.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:17:20.681+0000 7fe27677b700 1 -- 172.21.15.59:0/3263400028 wait complete. 2024-05-09T01:17:21.255 DEBUG:teuthology.orchestra.run.smithi059:osd.1> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.1.service 2024-05-09T01:17:21.257 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-05-09T01:17:21.258 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T01:17:21.501 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:17:21.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:21.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:21.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:21.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:21 smithi059 ceph-mon[34351]: from='osd.1 [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T01:17:21.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:21 smithi059 ceph-mon[34351]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T01:17:21.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:21 smithi059 ceph-mon[34351]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:21.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:21.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:21.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:21.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:21 smithi196 ceph-mon[35041]: from='osd.1 [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T01:17:21.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:21 smithi196 ceph-mon[35041]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T01:17:21.877 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:21 smithi196 ceph-mon[35041]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:21.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:21.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:21.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:21.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:21 smithi019 ceph-mon[30158]: from='osd.1 [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T01:17:21.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:21 smithi019 ceph-mon[30158]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T01:17:21.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:21 smithi019 ceph-mon[30158]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:22.760 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:22 smithi196 ceph-mon[35041]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T01:17:22.760 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:22 smithi196 ceph-mon[35041]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T01:17:22.760 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:22.760 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:22 smithi196 ceph-mon[35041]: from='osd.1 [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:17:22.760 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:22 smithi196 ceph-mon[35041]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:17:22.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:22 smithi059 ceph-mon[34351]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T01:17:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:22 smithi059 ceph-mon[34351]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T01:17:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:22 smithi059 ceph-mon[34351]: from='osd.1 [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:17:22.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:22 smithi059 ceph-mon[34351]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:17:22.849 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:22 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1[38944]: 2024-05-09T01:17:22.658+0000 7fe61db98700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T01:17:22.849 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:22 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1[38944]: 2024-05-09T01:17:22.663+0000 7fe61994d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:17:22.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:22 smithi019 ceph-mon[30158]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T01:17:22.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:22 smithi019 ceph-mon[30158]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T01:17:22.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:22.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:22 smithi019 ceph-mon[30158]: from='osd.1 [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:17:22.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:22 smithi019 ceph-mon[30158]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:17:23.333 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T01:17:23.334 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T01:17:23.334 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-09T01:17:23.334 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-09T01:17:23.334 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112812 s, 929 MB/s 2024-05-09T01:17:23.334 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-09T01:17:23.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:23 smithi196 ceph-mon[35041]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:17:23.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:23 smithi196 ceph-mon[35041]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T01:17:23.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:23 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:23.886 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:23 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:23.887 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:23 smithi196 ceph-mon[35041]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:23.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:23 smithi019 ceph-mon[30158]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:17:23.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:23 smithi019 ceph-mon[30158]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T01:17:23.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:23 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:23.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:23 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:23.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:23 smithi019 ceph-mon[30158]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:23.914 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-05-09T01:17:23.997 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:23 smithi059 ceph-mon[34351]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:17:23.998 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:23 smithi059 ceph-mon[34351]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T01:17:23.998 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:23 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:23.998 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:23 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:23.998 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:23 smithi059 ceph-mon[34351]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T01:17:24.157 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: osd.1 [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] boot 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi059 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi059 to 17093M 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:24.948 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: osd.1 [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] boot 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi059 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:17:25.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi059 to 17093M 2024-05-09T01:17:25.100 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:25.100 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:25.100 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:25.100 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:17:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:17:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: osd.1 [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] boot 2024-05-09T01:17:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T01:17:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T01:17:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi059 2024-05-09T01:17:25.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:25.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:25.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:17:25.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi059 to 17093M 2024-05-09T01:17:25.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:25.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:25.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:25.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:25.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.737+0000 7f2df864c700 1 -- 172.21.15.196:0/2336092246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 msgr2=0x7f2df0103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:25.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.737+0000 7f2df864c700 1 --2- 172.21.15.196:0/2336092246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 0x7f2df0103670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2de800b0a0 tx=0x7f2de800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:17:25.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.737+0000 7f2df864c700 1 -- 172.21.15.196:0/2336092246 shutdown_connections 2024-05-09T01:17:25.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.737+0000 7f2df864c700 1 --2- 172.21.15.196:0/2336092246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 0x7f2df010b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:25.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.737+0000 7f2df864c700 1 --2- 172.21.15.196:0/2336092246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 0x7f2df0103670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:25.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.737+0000 7f2df864c700 1 --2- 172.21.15.196:0/2336092246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df0104390 0x7f2df01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:25.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.737+0000 7f2df864c700 1 -- 172.21.15.196:0/2336092246 >> 172.21.15.196:0/2336092246 conn(0x7f2df00fe8c0 msgr2=0x7f2df0100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:17:25.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.737+0000 7f2df864c700 1 -- 172.21.15.196:0/2336092246 shutdown_connections 2024-05-09T01:17:25.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:25 smithi196 ceph-mon[35041]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T01:17:25.737 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:25 smithi196 ceph-mon[35041]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:25.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.737+0000 7f2df864c700 1 -- 172.21.15.196:0/2336092246 wait complete. 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.738+0000 7f2df864c700 1 Processor -- start 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.738+0000 7f2df864c700 1 -- start start 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.738+0000 7f2df864c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 0x7f2df01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.738+0000 7f2df864c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df0104390 0x7f2df0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.738+0000 7f2df864c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 0x7f2df019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.738+0000 7f2df864c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010dca0 con 0x7f2df0104d30 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.738+0000 7f2df864c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010db20 con 0x7f2df0104390 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.738+0000 7f2df864c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010de20 con 0x7f2df0103260 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df63e8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 0x7f2df01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df63e8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 0x7f2df01987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38018/0 (socket says 172.21.15.196:38018) 2024-05-09T01:17:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df63e8700 1 -- 172.21.15.196:0/3516878812 learned_addr learned my addr 172.21.15.196:0/3516878812 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df5be7700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df0104390 0x7f2df0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df6be9700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 0x7f2df019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df63e8700 1 -- 172.21.15.196:0/3516878812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df0104390 msgr2=0x7f2df0198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df63e8700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df0104390 0x7f2df0198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df63e8700 1 -- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 msgr2=0x7f2df019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df63e8700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 0x7f2df019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df63e8700 1 -- 172.21.15.196:0/3516878812 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01a0310 con 0x7f2df0103260 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df6be9700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 0x7f2df019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df63e8700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 0x7f2df01987b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2de80062a0 tx=0x7f2de8004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2de77fe700 1 -- 172.21.15.196:0/3516878812 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8021070 con 0x7f2df0103260 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2de77fe700 1 -- 172.21.15.196:0/3516878812 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de8007c80 con 0x7f2df0103260 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2de77fe700 1 -- 172.21.15.196:0/3516878812 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de80047e0 con 0x7f2df0103260 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df01a05a0 con 0x7f2df0103260 2024-05-09T01:17:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.739+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a0a90 con 0x7f2df0103260 2024-05-09T01:17:25.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.740+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df00512c0 con 0x7f2df0103260 2024-05-09T01:17:25.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.742+0000 7f2de77fe700 1 -- 172.21.15.196:0/3516878812 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f2de8003a00 con 0x7f2df0103260 2024-05-09T01:17:25.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.743+0000 7f2de77fe700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ddc0b2870 0x7f2ddc0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:25.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.743+0000 7f2de77fe700 1 -- 172.21.15.196:0/3516878812 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f2de800f020 con 0x7f2df0103260 2024-05-09T01:17:25.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.743+0000 7f2df5be7700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ddc0b2870 0x7f2ddc0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:25.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.744+0000 7f2df5be7700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ddc0b2870 0x7f2ddc0b4d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2df0199860 tx=0x7f2de000b700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:17:25.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.747+0000 7f2de77fe700 1 -- 172.21.15.196:0/3516878812 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2de802a020 con 0x7f2df0103260 2024-05-09T01:17:25.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:25.954+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2df0063c00 con 0x7f2ddc0b2870 2024-05-09T01:17:26.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:25 smithi059 ceph-mon[34351]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T01:17:26.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:25 smithi059 ceph-mon[34351]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:26.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:25 smithi019 ceph-mon[30158]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T01:17:26.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:25 smithi019 ceph-mon[30158]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:26.807 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:26 smithi196 ceph-mon[35041]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:17:26.807 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:17:26.807 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:17:26.808 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:27.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:26 smithi059 ceph-mon[34351]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:17:27.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:17:27.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:17:27.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:27.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:26 smithi019 ceph-mon[30158]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:17:27.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:17:27.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:17:27.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:27.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:27 smithi196 ceph-mon[35041]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:28.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:27 smithi059 ceph-mon[34351]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:28.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:27 smithi019 ceph-mon[30158]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:28.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:28 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/636459760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "632fb397-639e-4ccf-b0a2-31e1f8c7aa65"}]: dispatch 2024-05-09T01:17:28.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:28 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "632fb397-639e-4ccf-b0a2-31e1f8c7aa65"}]: dispatch 2024-05-09T01:17:28.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:28 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "632fb397-639e-4ccf-b0a2-31e1f8c7aa65"}]': finished 2024-05-09T01:17:28.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:28 smithi196 ceph-mon[35041]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T01:17:28.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:28 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:29.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:28 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/636459760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "632fb397-639e-4ccf-b0a2-31e1f8c7aa65"}]: dispatch 2024-05-09T01:17:29.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:28 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "632fb397-639e-4ccf-b0a2-31e1f8c7aa65"}]: dispatch 2024-05-09T01:17:29.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:28 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "632fb397-639e-4ccf-b0a2-31e1f8c7aa65"}]': finished 2024-05-09T01:17:29.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:28 smithi059 ceph-mon[34351]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T01:17:29.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:28 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:29.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:28 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/636459760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "632fb397-639e-4ccf-b0a2-31e1f8c7aa65"}]: dispatch 2024-05-09T01:17:29.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:28 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "632fb397-639e-4ccf-b0a2-31e1f8c7aa65"}]: dispatch 2024-05-09T01:17:29.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:28 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "632fb397-639e-4ccf-b0a2-31e1f8c7aa65"}]': finished 2024-05-09T01:17:29.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:28 smithi019 ceph-mon[30158]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T01:17:29.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:28 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:29.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:29 smithi196 ceph-mon[35041]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:29.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:29 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/59388689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T01:17:30.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:29 smithi059 ceph-mon[34351]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:30.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:29 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/59388689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T01:17:30.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:29 smithi019 ceph-mon[30158]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:30.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:29 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/59388689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T01:17:32.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:32 smithi019 ceph-mon[30158]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:32.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:32 smithi196 ceph-mon[35041]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:32.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:32 smithi059 ceph-mon[34351]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:34.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:34 smithi196 ceph-mon[35041]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:34.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:34 smithi019 ceph-mon[30158]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:34.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:34 smithi059 ceph-mon[34351]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:36 smithi019 ceph-mon[30158]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T01:17:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:36.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:36 smithi196 ceph-mon[35041]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:36.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T01:17:36.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:36.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:36 smithi059 ceph-mon[34351]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:36.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T01:17:36.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:37.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:37 smithi019 ceph-mon[30158]: Deploying daemon osd.2 on smithi196 2024-05-09T01:17:37.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:37 smithi196 ceph-mon[35041]: Deploying daemon osd.2 on smithi196 2024-05-09T01:17:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:37 smithi059 ceph-mon[34351]: Deploying daemon osd.2 on smithi196 2024-05-09T01:17:38.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:38 smithi019 ceph-mon[30158]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:38.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:38 smithi196 ceph-mon[35041]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:38.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:38 smithi059 ceph-mon[34351]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:40.290 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:40 smithi196 ceph-mon[35041]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:40.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:40 smithi019 ceph-mon[30158]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:40.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:40 smithi059 ceph-mon[34351]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:42.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:42.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:42.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:42 smithi196 ceph-mon[35041]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:42.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:42.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:42.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:42.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:42 smithi059 ceph-mon[34351]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:42.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:42.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:42.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:42.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:42 smithi019 ceph-mon[30158]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:42.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:44.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.271+0000 7f2de77fe700 1 -- 172.21.15.196:0/3516878812 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2df0063c00 con 0x7f2ddc0b2870 2024-05-09T01:17:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-05-09T01:17:44.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:44 smithi196 ceph-mon[35041]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:44.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:44 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:44.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:44 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:44.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:44 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:44.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:44 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:44.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:44 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ddc0b2870 msgr2=0x7f2ddc0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ddc0b2870 0x7f2ddc0b4d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2df0199860 tx=0x7f2de000b700 comp rx=0 tx=0).stop 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 msgr2=0x7f2df01987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 0x7f2df01987b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2de80062a0 tx=0x7f2de8004e40 comp rx=0 tx=0).stop 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 shutdown_connections 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ddc0b2870 0x7f2ddc0b4d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df0104d30 0x7f2df019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0103260 0x7f2df01987b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 --2- 172.21.15.196:0/3516878812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df0104390 0x7f2df0198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 >> 172.21.15.196:0/3516878812 conn(0x7f2df00fe8c0 msgr2=0x7f2df01075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 shutdown_connections 2024-05-09T01:17:44.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:17:44.274+0000 7f2df864c700 1 -- 172.21.15.196:0/3516878812 wait complete. 2024-05-09T01:17:44.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:44 smithi059 ceph-mon[34351]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:44.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:44 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:44.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:44 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:44.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:44 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:44.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:44 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:44.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:44 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:44.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:44 smithi019 ceph-mon[30158]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:44.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:44.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:44.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:44.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:44.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:44.885 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.2.service 2024-05-09T01:17:44.887 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-05-09T01:17:44.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T01:17:45.133 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:44 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2[40189]: 2024-05-09T01:17:44.941+0000 7f6d4cdf37c0 -1 osd.2 0 log_to_monitors true 2024-05-09T01:17:45.148 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:17:45.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:45.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:45.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:45.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:45 smithi196 ceph-mon[35041]: from='osd.2 [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T01:17:45.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:45 smithi196 ceph-mon[35041]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T01:17:45.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:45.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:45.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:45.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:45 smithi019 ceph-mon[30158]: from='osd.2 [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T01:17:45.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:45 smithi019 ceph-mon[30158]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T01:17:45.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:45.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:45.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:17:45.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:45 smithi059 ceph-mon[34351]: from='osd.2 [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T01:17:45.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:45 smithi059 ceph-mon[34351]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T01:17:46.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:46 smithi196 ceph-mon[35041]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:46.313 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:46 smithi196 ceph-mon[35041]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T01:17:46.313 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:46 smithi196 ceph-mon[35041]: from='osd.2 [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:17:46.313 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:46 smithi196 ceph-mon[35041]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T01:17:46.313 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:46 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:46.313 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:46 smithi196 ceph-mon[35041]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:17:46.313 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2[40189]: 2024-05-09T01:17:46.284+0000 7f6d42457700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T01:17:46.313 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2[40189]: 2024-05-09T01:17:46.289+0000 7f6d3e20c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:17:46.553 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:46 smithi019 ceph-mon[30158]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:46.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:46 smithi019 ceph-mon[30158]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T01:17:46.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:46 smithi019 ceph-mon[30158]: from='osd.2 [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:17:46.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:46 smithi019 ceph-mon[30158]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T01:17:46.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:46 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:46.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:46 smithi019 ceph-mon[30158]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:17:46.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:46 smithi059 ceph-mon[34351]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:46.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:46 smithi059 ceph-mon[34351]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T01:17:46.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:46 smithi059 ceph-mon[34351]: from='osd.2 [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:17:46.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:46 smithi059 ceph-mon[34351]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T01:17:46.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:46 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:46.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:46 smithi059 ceph-mon[34351]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:17:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T01:17:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T01:17:47.092 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-09T01:17:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-09T01:17:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011237 s, 933 MB/s 2024-05-09T01:17:47.092 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 2024-05-09T01:17:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-09T01:17:47.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:47 smithi196 ceph-mon[35041]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:17:47.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:47 smithi196 ceph-mon[35041]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T01:17:47.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:47.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:47.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:47 smithi059 ceph-mon[34351]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:17:47.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:47 smithi059 ceph-mon[34351]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T01:17:47.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:47.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:47.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:47 smithi019 ceph-mon[30158]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:17:47.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:47 smithi019 ceph-mon[30158]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T01:17:47.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:47.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:47.695 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-05-09T01:17:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:17:48.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:17:48.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:17:48.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: osd.2 [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] boot 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:48.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:17:48.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:17:48.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:48.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:48.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: osd.2 [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] boot 2024-05-09T01:17:48.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T01:17:48.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:48.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:17:48.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:48.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:48.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:17:48.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:17:48.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: osd.2 [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] boot 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:17:48.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:49.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:49 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi196 2024-05-09T01:17:49.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:49 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi196 to 17088M 2024-05-09T01:17:49.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:49 smithi019 ceph-mon[30158]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T01:17:49.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:49 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:17:49.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.550+0000 7f373cc60700 1 -- 172.21.15.19:0/4223125684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738103250 msgr2=0x7f3738103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:49.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.550+0000 7f373cc60700 1 --2- 172.21.15.19:0/4223125684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738103250 0x7f3738103660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f372800b0a0 tx=0x7f372800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:17:49.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.550+0000 7f373cc60700 1 -- 172.21.15.19:0/4223125684 shutdown_connections 2024-05-09T01:17:49.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.550+0000 7f373cc60700 1 --2- 172.21.15.19:0/4223125684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3738104d20 0x7f373810b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:49.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.550+0000 7f373cc60700 1 --2- 172.21.15.19:0/4223125684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738103250 0x7f3738103660 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:49.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.550+0000 7f373cc60700 1 --2- 172.21.15.19:0/4223125684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3738104380 0x7f3738104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:49.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.550+0000 7f373cc60700 1 -- 172.21.15.19:0/4223125684 >> 172.21.15.19:0/4223125684 conn(0x7f37380fe890 msgr2=0x7f3738100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:17:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.550+0000 7f373cc60700 1 -- 172.21.15.19:0/4223125684 shutdown_connections 2024-05-09T01:17:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.550+0000 7f373cc60700 1 -- 172.21.15.19:0/4223125684 wait complete. 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.551+0000 7f373cc60700 1 Processor -- start 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.551+0000 7f373cc60700 1 -- start start 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f373cc60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3738103250 0x7f3738199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f373cc60700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3738104380 0x7f3738199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f373cc60700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738104d20 0x7f37381a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f373cc60700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f373810ec90 con 0x7f3738104d20 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f373cc60700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f373810eb10 con 0x7f3738103250 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f373cc60700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f373810ee10 con 0x7f3738104380 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3736d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738104d20 0x7f37381a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3736d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738104d20 0x7f37381a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59454/0 (socket says 172.21.15.19:59454) 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3736d9d700 1 -- 172.21.15.19:0/2376231014 learned_addr learned my addr 172.21.15.19:0/2376231014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:17:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f373659c700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3738103250 0x7f3738199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3735d9b700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3738104380 0x7f3738199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3736d9d700 1 -- 172.21.15.19:0/2376231014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3738104380 msgr2=0x7f3738199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3736d9d700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3738104380 0x7f3738199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3736d9d700 1 -- 172.21.15.19:0/2376231014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3738103250 msgr2=0x7f3738199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3736d9d700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3738103250 0x7f3738199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3736d9d700 1 -- 172.21.15.19:0/2376231014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37381a1300 con 0x7f3738104d20 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f373659c700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3738103250 0x7f3738199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.552+0000 7f3736d9d700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738104d20 0x7f37381a0d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37300098d0 tx=0x7f3730009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.553+0000 7f37277fe700 1 -- 172.21.15.19:0/2376231014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373000d020 con 0x7f3738104d20 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.553+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3738075a50 con 0x7f3738104d20 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.553+0000 7f37277fe700 1 -- 172.21.15.19:0/2376231014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373000d1c0 con 0x7f3738104d20 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.553+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3738075f90 con 0x7f3738104d20 2024-05-09T01:17:49.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.553+0000 7f37277fe700 1 -- 172.21.15.19:0/2376231014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373000c460 con 0x7f3738104d20 2024-05-09T01:17:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.554+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37380512c0 con 0x7f3738104d20 2024-05-09T01:17:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.555+0000 7f37277fe700 1 -- 172.21.15.19:0/2376231014 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f373001f020 con 0x7f3738104d20 2024-05-09T01:17:49.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.556+0000 7f37277fe700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37200b2870 0x7f37200b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:17:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.556+0000 7f373659c700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37200b2870 0x7f37200b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:17:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.556+0000 7f37277fe700 1 -- 172.21.15.19:0/2376231014 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f37300e25f0 con 0x7f3738104d20 2024-05-09T01:17:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.557+0000 7f373659c700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37200b2870 0x7f37200b4d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3728009fd0 tx=0x7f372800b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:17:49.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.560+0000 7f37277fe700 1 -- 172.21.15.19:0/2376231014 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3730026030 con 0x7f3738104d20 2024-05-09T01:17:49.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:49 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi196 2024-05-09T01:17:49.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:49 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi196 to 17088M 2024-05-09T01:17:49.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:49 smithi059 ceph-mon[34351]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T01:17:49.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:49 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:17:49.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:49 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi196 2024-05-09T01:17:49.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:49 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi196 to 17088M 2024-05-09T01:17:49.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:49 smithi196 ceph-mon[35041]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T01:17:49.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:49 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:17:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:49.791+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3738076300 con 0x7f37200b2870 2024-05-09T01:17:50.540 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 ceph-mon[30158]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:50.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T01:17:50.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 ceph-mon[30158]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T01:17:50.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:17:50.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:17:50.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:17:50.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:50.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:50 smithi059 ceph-mon[34351]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:50.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:50 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T01:17:50.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:50 smithi059 ceph-mon[34351]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T01:17:50.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:50 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:17:50.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:50 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:17:50.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:50 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:17:50.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:50 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:50.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:50 smithi196 ceph-mon[35041]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:50.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:50 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T01:17:50.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:50 smithi196 ceph-mon[35041]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T01:17:50.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:50 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:17:50.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:50 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:17:50.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:50 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:17:50.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:50 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:17:50.897 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49369]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T01:17:50.897 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49369]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:50.897 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49369]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:50.897 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49369]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:50.897 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49372]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T01:17:50.897 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49372]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:50.898 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49372]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:50.898 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49372]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 sudo[43082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T01:17:51.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 sudo[43082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 sudo[43082]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 sudo[43082]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 sudo[43085]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T01:17:51.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 sudo[43085]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 sudo[43085]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 sudo[43085]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.099 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:50 smithi059 sudo[43076]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T01:17:51.099 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:50 smithi059 sudo[43076]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.099 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:50 smithi059 sudo[43076]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.099 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:50 smithi059 sudo[43076]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.099 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:50 smithi059 sudo[43079]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T01:17:51.099 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:50 smithi059 sudo[43079]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.099 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:50 smithi059 sudo[43079]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.100 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:17:50 smithi059 sudo[43079]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 sudo[44247]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T01:17:51.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 sudo[44247]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 sudo[44247]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 sudo[44247]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 sudo[44250]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T01:17:51.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 sudo[44250]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 sudo[44250]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 sudo[44250]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.197 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:50 smithi196 sudo[44241]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T01:17:51.197 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:50 smithi196 sudo[44241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.197 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:50 smithi196 sudo[44241]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.198 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:50 smithi196 sudo[44241]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.198 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:50 smithi196 sudo[44244]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T01:17:51.198 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:50 smithi196 sudo[44244]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.198 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:50 smithi196 sudo[44244]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.198 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:17:50 smithi196 sudo[44244]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49375]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T01:17:51.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49375]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49375]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49375]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49378]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49378]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49378]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:50 smithi019 sudo[49378]: pam_unix(sudo:session): session closed for user root 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:17:51.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:17:51.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T01:17:51.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:17:51.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:17:51.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:17:51.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T01:17:51.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T01:17:51.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:17:51.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:17:51.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T01:17:51.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:17:51.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:17:51.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:17:51.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T01:17:51.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T01:17:51.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:17:51.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:17:51.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:17:51.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:51 smithi059 ceph-mon[34351]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T01:17:51.607 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:17:51.607 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:51 smithi019 ceph-mon[30158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T01:17:51.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:17:51.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T01:17:51.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T01:17:51.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T01:17:51.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T01:17:51.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:17:51.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T01:17:51.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T01:17:51.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:51 smithi196 ceph-mon[35041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T01:17:51.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:17:51.967+0000 7f37277fe700 1 -- 172.21.15.19:0/2376231014 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f37300a7430 con 0x7f3738104d20 2024-05-09T01:17:52.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:52 smithi059 ceph-mon[34351]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:52.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:52 smithi059 ceph-mon[34351]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T01:17:52.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:52 smithi059 ceph-mon[34351]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T01:17:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:52 smithi019 ceph-mon[30158]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:52.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:52 smithi019 ceph-mon[30158]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T01:17:52.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:52 smithi019 ceph-mon[30158]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T01:17:52.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:52 smithi196 ceph-mon[35041]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:52.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:52 smithi196 ceph-mon[35041]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T01:17:52.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:52 smithi196 ceph-mon[35041]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T01:17:53.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:53 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/4158185220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "067c1dbe-5d98-4603-bc43-64343344993e"}]: dispatch 2024-05-09T01:17:53.606 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:53 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/4158185220' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "067c1dbe-5d98-4603-bc43-64343344993e"}]': finished 2024-05-09T01:17:53.606 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:53 smithi059 ceph-mon[34351]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T01:17:53.606 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:17:53.606 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:53 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1621237719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T01:17:53.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:53 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/4158185220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "067c1dbe-5d98-4603-bc43-64343344993e"}]: dispatch 2024-05-09T01:17:53.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:53 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/4158185220' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "067c1dbe-5d98-4603-bc43-64343344993e"}]': finished 2024-05-09T01:17:53.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:53 smithi019 ceph-mon[30158]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T01:17:53.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:17:53.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:53 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1621237719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T01:17:53.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:53 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/4158185220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "067c1dbe-5d98-4603-bc43-64343344993e"}]: dispatch 2024-05-09T01:17:53.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:53 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/4158185220' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "067c1dbe-5d98-4603-bc43-64343344993e"}]': finished 2024-05-09T01:17:53.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:53 smithi196 ceph-mon[35041]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T01:17:53.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:17:53.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:53 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1621237719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T01:17:54.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:54 smithi059 ceph-mon[34351]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:54.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:54 smithi019 ceph-mon[30158]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:54.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:54 smithi196 ceph-mon[35041]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:56.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:56 smithi019 ceph-mon[30158]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:56.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:56 smithi059 ceph-mon[34351]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:56 smithi196 ceph-mon[35041]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:58.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:17:58 smithi019 ceph-mon[30158]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:58.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:17:58 smithi059 ceph-mon[34351]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:17:58.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:17:58 smithi196 ceph-mon[35041]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:00.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:00 smithi059 ceph-mon[34351]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:00.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:00 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T01:18:00.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:00 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:00.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:00 smithi019 ceph-mon[30158]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:00.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:00 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T01:18:00.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:00 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:00.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:00 smithi196 ceph-mon[35041]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:00.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:00 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T01:18:00.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:00 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:01.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:01 smithi059 ceph-mon[34351]: Deploying daemon osd.3 on smithi019 2024-05-09T01:18:01.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:01 smithi019 ceph-mon[30158]: Deploying daemon osd.3 on smithi019 2024-05-09T01:18:01.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:01 smithi196 ceph-mon[35041]: Deploying daemon osd.3 on smithi019 2024-05-09T01:18:02.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:02 smithi196 ceph-mon[35041]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:02.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:02 smithi059 ceph-mon[34351]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:02.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:02 smithi019 ceph-mon[30158]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:04.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:04 smithi059 ceph-mon[34351]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:04.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:04 smithi019 ceph-mon[30158]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:04.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:04 smithi196 ceph-mon[35041]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:06.471 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:06 smithi019 ceph-mon[30158]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:06.472 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:06.472 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:06.472 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:06 smithi059 ceph-mon[34351]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:06.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:06 smithi196 ceph-mon[35041]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:08.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:08 smithi059 ceph-mon[34351]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:08.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:08 smithi019 ceph-mon[30158]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:08.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:08 smithi196 ceph-mon[35041]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:09.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.457+0000 7f37277fe700 1 -- 172.21.15.19:0/2376231014 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3738076300 con 0x7f37200b2870 2024-05-09T01:18:09.456 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-05-09T01:18:09.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.459+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37200b2870 msgr2=0x7f37200b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:09.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.459+0000 7f373cc60700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37200b2870 0x7f37200b4d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3728009fd0 tx=0x7f372800b7b0 comp rx=0 tx=0).stop 2024-05-09T01:18:09.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.459+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738104d20 msgr2=0x7f37381a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:09.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.459+0000 7f373cc60700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738104d20 0x7f37381a0d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37300098d0 tx=0x7f3730009be0 comp rx=0 tx=0).stop 2024-05-09T01:18:09.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.459+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 shutdown_connections 2024-05-09T01:18:09.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.459+0000 7f373cc60700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37200b2870 0x7f37200b4d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:09.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.459+0000 7f373cc60700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3738104d20 0x7f37381a0d10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:09.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.460+0000 7f373cc60700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3738103250 0x7f3738199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:09.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.460+0000 7f373cc60700 1 --2- 172.21.15.19:0/2376231014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3738104380 0x7f3738199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:09.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.460+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 >> 172.21.15.19:0/2376231014 conn(0x7f37380fe890 msgr2=0x7f37381075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:18:09.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.460+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 shutdown_connections 2024-05-09T01:18:09.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:18:09.460+0000 7f373cc60700 1 -- 172.21.15.19:0/2376231014 wait complete. 2024-05-09T01:18:09.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:09.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:09.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:09.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:09.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:09.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:09.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:09.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:09.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:09.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:10.086 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.3.service 2024-05-09T01:18:10.088 INFO:tasks.cephadm:Deploying osd.4 on smithi059 with /dev/vg_nvme/lv_3... 2024-05-09T01:18:10.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T01:18:10.344 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:18:10.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:10 smithi019 ceph-mon[30158]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:10.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:10 smithi019 ceph-mon[30158]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T01:18:10.756 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:10 smithi059 ceph-mon[34351]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:10.756 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:10 smithi059 ceph-mon[34351]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T01:18:10.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:10 smithi196 ceph-mon[35041]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:10.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:10 smithi196 ceph-mon[35041]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T01:18:11.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:11 smithi059 ceph-mon[34351]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T01:18:11.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:11 smithi059 ceph-mon[34351]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T01:18:11.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:11 smithi059 ceph-mon[34351]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:18:11.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:11.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:11 smithi059 ceph-mon[34351]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:18:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:11 smithi019 ceph-mon[30158]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T01:18:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:11 smithi019 ceph-mon[30158]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T01:18:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:11 smithi019 ceph-mon[30158]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:18:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:11 smithi019 ceph-mon[30158]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:18:11.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:11 smithi196 ceph-mon[35041]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T01:18:11.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:11 smithi196 ceph-mon[35041]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T01:18:11.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:11 smithi196 ceph-mon[35041]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:18:11.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:11.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:11 smithi196 ceph-mon[35041]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:18:12.315 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T01:18:12.315 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T01:18:12.316 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-09T01:18:12.316 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-09T01:18:12.316 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124058 s, 845 MB/s 2024-05-09T01:18:12.316 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-09T01:18:12.657 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:12 smithi059 ceph-mon[34351]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:12.658 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:12 smithi059 ceph-mon[34351]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T01:18:12.658 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:12 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.658 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:12 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.658 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:12 smithi059 ceph-mon[34351]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' 2024-05-09T01:18:12.658 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:12 smithi059 ceph-mon[34351]: osdmap e26: 4 total, 3 up, 4 in 2024-05-09T01:18:12.658 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:12 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.658 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:12 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:12 smithi019 ceph-mon[30158]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:12.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:12 smithi019 ceph-mon[30158]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T01:18:12.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:12 smithi019 ceph-mon[30158]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' 2024-05-09T01:18:12.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:12 smithi019 ceph-mon[30158]: osdmap e26: 4 total, 3 up, 4 in 2024-05-09T01:18:12.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.800 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:18:12 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3[51497]: 2024-05-09T01:18:12.470+0000 7f8c5babc700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T01:18:12.800 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:18:12 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3[51497]: 2024-05-09T01:18:12.503+0000 7f8c53068700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:18:12.943 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_3 2024-05-09T01:18:12.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:12 smithi196 ceph-mon[35041]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:12.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:12 smithi196 ceph-mon[35041]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T01:18:12.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:12 smithi196 ceph-mon[35041]: from='osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110]' entity='osd.3' 2024-05-09T01:18:12.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:12 smithi196 ceph-mon[35041]: osdmap e26: 4 total, 3 up, 4 in 2024-05-09T01:18:12.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:12.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:13.187 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:18:13.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:18:13.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi019 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi019 to 8546M 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] boot 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T01:18:13.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:13.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:18:13.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:18:13.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:13.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi019 2024-05-09T01:18:13.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi019 to 8546M 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] boot 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T01:18:13.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:13.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:18:13.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:18:13.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T01:18:13.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi019 2024-05-09T01:18:13.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi019 to 8546M 2024-05-09T01:18:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: osd.3 [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] boot 2024-05-09T01:18:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T01:18:13.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T01:18:14.569 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:14 smithi059 ceph-mon[34351]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T01:18:14.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:14 smithi019 ceph-mon[30158]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T01:18:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.804+0000 7f1eb3393700 1 -- 172.21.15.59:0/1468535981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac103810 msgr2=0x7f1eac103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.804+0000 7f1eb3393700 1 --2- 172.21.15.59:0/1468535981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac103810 0x7f1eac103c20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80099c0 tx=0x7f1ea8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:18:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.804+0000 7f1eb3393700 1 -- 172.21.15.59:0/1468535981 shutdown_connections 2024-05-09T01:18:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.804+0000 7f1eb3393700 1 --2- 172.21.15.59:0/1468535981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac104cf0 0x7f1eac10b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.804+0000 7f1eb3393700 1 --2- 172.21.15.59:0/1468535981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac103810 0x7f1eac103c20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.804+0000 7f1eb3393700 1 --2- 172.21.15.59:0/1468535981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1eac1041f0 0x7f1eac104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.804+0000 7f1eb3393700 1 -- 172.21.15.59:0/1468535981 >> 172.21.15.59:0/1468535981 conn(0x7f1eac0fe020 msgr2=0x7f1eac1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.804+0000 7f1eb3393700 1 -- 172.21.15.59:0/1468535981 shutdown_connections 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.804+0000 7f1eb3393700 1 -- 172.21.15.59:0/1468535981 wait complete. 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb3393700 1 Processor -- start 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb3393700 1 -- start start 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb3393700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac103810 0x7f1eac19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb3393700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac1041f0 0x7f1eac19d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb3393700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1eac104cf0 0x7f1eac1a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb3393700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac10ecb0 con 0x7f1eac103810 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb3393700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac10eb30 con 0x7f1eac1041f0 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb3393700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac10ee30 con 0x7f1eac104cf0 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb092e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac1041f0 0x7f1eac19d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:18:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb092e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac1041f0 0x7f1eac19d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40962/0 (socket says 172.21.15.59:40962) 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb112f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac103810 0x7f1eac19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.805+0000 7f1eb092e700 1 -- 172.21.15.59:0/3221289226 learned_addr learned my addr 172.21.15.59:0/3221289226 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb1930700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1eac104cf0 0x7f1eac1a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb092e700 1 -- 172.21.15.59:0/3221289226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1eac104cf0 msgr2=0x7f1eac1a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb092e700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1eac104cf0 0x7f1eac1a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb092e700 1 -- 172.21.15.59:0/3221289226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac103810 msgr2=0x7f1eac19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb092e700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac103810 0x7f1eac19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb092e700 1 -- 172.21.15.59:0/3221289226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac1a4680 con 0x7f1eac1041f0 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb1930700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1eac104cf0 0x7f1eac1a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb092e700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac1041f0 0x7f1eac19d060 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00d950 tx=0x7f1e9c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1ea27fc700 1 -- 172.21.15.59:0/3221289226 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c016070 con 0x7f1eac1041f0 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac1a4970 con 0x7f1eac1041f0 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1ea27fc700 1 -- 172.21.15.59:0/3221289226 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e9c009df0 con 0x7f1eac1041f0 2024-05-09T01:18:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1ea27fc700 1 -- 172.21.15.59:0/3221289226 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c00f3c0 con 0x7f1eac1041f0 2024-05-09T01:18:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.806+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac1a4f00 con 0x7f1eac1041f0 2024-05-09T01:18:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.807+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eac0512c0 con 0x7f1eac1041f0 2024-05-09T01:18:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.809+0000 7f1ea27fc700 1 -- 172.21.15.59:0/3221289226 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1e9c019070 con 0x7f1eac1041f0 2024-05-09T01:18:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.810+0000 7f1ea27fc700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e980b2880 0x7f1e980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:18:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.810+0000 7f1ea27fc700 1 -- 172.21.15.59:0/3221289226 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f1e9c0e0c80 con 0x7f1eac1041f0 2024-05-09T01:18:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.810+0000 7f1eb112f700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e980b2880 0x7f1e980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:18:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.811+0000 7f1eb112f700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e980b2880 0x7f1e980b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009f50 tx=0x7f1ea801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:18:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:14.814+0000 7f1ea27fc700 1 -- 172.21.15.59:0/3221289226 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e9c0a9b20 con 0x7f1eac1041f0 2024-05-09T01:18:14.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:14 smithi196 ceph-mon[35041]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T01:18:15.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:15.040+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1eac1a5130 con 0x7f1e980b2880 2024-05-09T01:18:15.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:15 smithi059 ceph-mon[34351]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T01:18:15.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:15 smithi059 ceph-mon[34351]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:18:15.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:18:15.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:18:15.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:15.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:15 smithi059 ceph-mon[34351]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 161 KiB/s, 0 objects/s recovering 2024-05-09T01:18:15.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:15 smithi019 ceph-mon[30158]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T01:18:15.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:15 smithi019 ceph-mon[30158]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:18:15.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:15 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:18:15.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:15 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:18:15.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:15 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:15.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:15 smithi019 ceph-mon[30158]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 161 KiB/s, 0 objects/s recovering 2024-05-09T01:18:15.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:15 smithi196 ceph-mon[35041]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T01:18:15.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:15 smithi196 ceph-mon[35041]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:18:15.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:15 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:18:15.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:15 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:18:15.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:15 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:15.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:15 smithi196 ceph-mon[35041]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 161 KiB/s, 0 objects/s recovering 2024-05-09T01:18:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:18 smithi019 ceph-mon[30158]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-09T01:18:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:18 smithi019 ceph-mon[30158]: from='client.? 172.21.15.59:0/1045632705' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "322cb4e2-84e1-494f-93d6-bd5b09ce3a82"}]: dispatch 2024-05-09T01:18:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:18 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "322cb4e2-84e1-494f-93d6-bd5b09ce3a82"}]: dispatch 2024-05-09T01:18:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:18 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "322cb4e2-84e1-494f-93d6-bd5b09ce3a82"}]': finished 2024-05-09T01:18:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:18 smithi019 ceph-mon[30158]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T01:18:18.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:18 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:18.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:18 smithi196 ceph-mon[35041]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-09T01:18:18.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:18 smithi196 ceph-mon[35041]: from='client.? 172.21.15.59:0/1045632705' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "322cb4e2-84e1-494f-93d6-bd5b09ce3a82"}]: dispatch 2024-05-09T01:18:18.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:18 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "322cb4e2-84e1-494f-93d6-bd5b09ce3a82"}]: dispatch 2024-05-09T01:18:18.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:18 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "322cb4e2-84e1-494f-93d6-bd5b09ce3a82"}]': finished 2024-05-09T01:18:18.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:18 smithi196 ceph-mon[35041]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T01:18:18.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:18 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:18.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:18 smithi059 ceph-mon[34351]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-09T01:18:18.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:18 smithi059 ceph-mon[34351]: from='client.? 172.21.15.59:0/1045632705' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "322cb4e2-84e1-494f-93d6-bd5b09ce3a82"}]: dispatch 2024-05-09T01:18:18.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:18 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "322cb4e2-84e1-494f-93d6-bd5b09ce3a82"}]: dispatch 2024-05-09T01:18:18.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:18 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "322cb4e2-84e1-494f-93d6-bd5b09ce3a82"}]': finished 2024-05-09T01:18:18.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:18 smithi059 ceph-mon[34351]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T01:18:18.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:18 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:19.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:19 smithi019 ceph-mon[30158]: from='client.? 172.21.15.59:0/3883962425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T01:18:19.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:19 smithi196 ceph-mon[35041]: from='client.? 172.21.15.59:0/3883962425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T01:18:19.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:19 smithi059 ceph-mon[34351]: from='client.? 172.21.15.59:0/3883962425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T01:18:20.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:20 smithi019 ceph-mon[30158]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-09T01:18:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:20 smithi196 ceph-mon[35041]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-09T01:18:20.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:20 smithi059 ceph-mon[34351]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-09T01:18:22.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:22 smithi059 ceph-mon[34351]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-09T01:18:22.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:22 smithi019 ceph-mon[30158]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-09T01:18:22.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:22 smithi196 ceph-mon[35041]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-09T01:18:24.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:24 smithi059 ceph-mon[34351]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-09T01:18:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:24 smithi019 ceph-mon[30158]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-09T01:18:24.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:24 smithi196 ceph-mon[35041]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-09T01:18:26.287 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:26 smithi059 ceph-mon[34351]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:26.287 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T01:18:26.287 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:26.287 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:26 smithi059 ceph-mon[34351]: Deploying daemon osd.4 on smithi059 2024-05-09T01:18:26.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:26 smithi019 ceph-mon[30158]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:26.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T01:18:26.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:26.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:26 smithi019 ceph-mon[30158]: Deploying daemon osd.4 on smithi059 2024-05-09T01:18:26.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:26 smithi196 ceph-mon[35041]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:26.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T01:18:26.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:26.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:26 smithi196 ceph-mon[35041]: Deploying daemon osd.4 on smithi059 2024-05-09T01:18:28.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:28 smithi019 ceph-mon[30158]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:28.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:28 smithi196 ceph-mon[35041]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:28.487 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:28 smithi059 ceph-mon[34351]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:30.296 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:30 smithi059 ceph-mon[34351]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:30.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:30 smithi019 ceph-mon[30158]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:30.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:30 smithi196 ceph-mon[35041]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:31.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:31.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:31.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:31 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:31.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:31.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:31.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:31 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:31.522 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:31.522 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:31.522 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:31 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:32.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:32 smithi019 ceph-mon[30158]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:32.437 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:32 smithi059 ceph-mon[34351]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:32.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:32 smithi196 ceph-mon[35041]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:34.248 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:34 smithi059 ceph-mon[34351]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:34.248 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:34.248 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:34.248 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:34.248 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:34.248 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:34 smithi196 ceph-mon[35041]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:34.608 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 4 on host 'smithi059' 2024-05-09T01:18:34.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.608+0000 7f1ea27fc700 1 -- 172.21.15.59:0/3221289226 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1eac1a5130 con 0x7f1e980b2880 2024-05-09T01:18:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e980b2880 msgr2=0x7f1e980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e980b2880 0x7f1e980b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009f50 tx=0x7f1ea801a040 comp rx=0 tx=0).stop 2024-05-09T01:18:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac1041f0 msgr2=0x7f1eac19d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac1041f0 0x7f1eac19d060 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00d950 tx=0x7f1e9c00dc60 comp rx=0 tx=0).stop 2024-05-09T01:18:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 shutdown_connections 2024-05-09T01:18:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1eac104cf0 0x7f1eac1a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e980b2880 0x7f1e980b4d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac103810 0x7f1eac19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 --2- 172.21.15.59:0/3221289226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1eac1041f0 0x7f1eac19d060 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 >> 172.21.15.59:0/3221289226 conn(0x7f1eac0fe020 msgr2=0x7f1eac0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:18:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 shutdown_connections 2024-05-09T01:18:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:18:34.611+0000 7f1eb3393700 1 -- 172.21.15.59:0/3221289226 wait complete. 2024-05-09T01:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:34 smithi019 ceph-mon[30158]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:35.296 DEBUG:teuthology.orchestra.run.smithi059:osd.4> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.4.service 2024-05-09T01:18:35.298 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-05-09T01:18:35.299 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T01:18:35.545 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:18:35.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:35.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:35.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:35.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:35 smithi059 ceph-mon[34351]: from='osd.4 [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T01:18:35.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:35 smithi059 ceph-mon[34351]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T01:18:35.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:35 smithi059 ceph-mon[34351]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:35.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:35.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:35.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:35.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:35 smithi019 ceph-mon[30158]: from='osd.4 [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T01:18:35.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:35 smithi019 ceph-mon[30158]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T01:18:35.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:35 smithi019 ceph-mon[30158]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:35.905 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:35.905 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:35.905 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:35.905 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:35 smithi196 ceph-mon[35041]: from='osd.4 [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T01:18:35.905 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:35 smithi196 ceph-mon[35041]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T01:18:35.905 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:35 smithi196 ceph-mon[35041]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:36.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:36 smithi196 ceph-mon[35041]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T01:18:36.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:36 smithi196 ceph-mon[35041]: from='osd.4 [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:18:36.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:36 smithi196 ceph-mon[35041]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T01:18:36.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:36.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:36 smithi196 ceph-mon[35041]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:18:36.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:36 smithi059 ceph-mon[34351]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T01:18:36.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:36 smithi059 ceph-mon[34351]: from='osd.4 [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:18:36.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:36 smithi059 ceph-mon[34351]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T01:18:36.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:36.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:36 smithi059 ceph-mon[34351]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:18:36.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:36 smithi019 ceph-mon[30158]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T01:18:36.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:36 smithi019 ceph-mon[30158]: from='osd.4 [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:18:36.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:36 smithi019 ceph-mon[30158]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T01:18:36.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:36.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:36 smithi019 ceph-mon[30158]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:18:37.415 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T01:18:37.416 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T01:18:37.416 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-09T01:18:37.416 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-09T01:18:37.416 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127728 s, 821 MB/s 2024-05-09T01:18:37.416 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-09T01:18:37.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:37 smithi059 ceph-mon[34351]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:18:37.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:37 smithi059 ceph-mon[34351]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T01:18:37.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:37.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:37.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:37 smithi059 ceph-mon[34351]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:37.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:37 smithi059 ceph-mon[34351]: from='osd.4 ' entity='osd.4' 2024-05-09T01:18:37.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:37.849 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:18:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:18:37.543+0000 7f1b3a648700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T01:18:37.849 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:18:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:18:37.604+0000 7f1b323f5700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:18:37.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:37 smithi019 ceph-mon[30158]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:18:37.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:37 smithi019 ceph-mon[30158]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T01:18:37.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:37.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:37.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:37 smithi019 ceph-mon[30158]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:37.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:37 smithi019 ceph-mon[30158]: from='osd.4 ' entity='osd.4' 2024-05-09T01:18:37.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:37.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:37 smithi196 ceph-mon[35041]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:18:37.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:37 smithi196 ceph-mon[35041]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T01:18:37.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:37.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:37.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:37 smithi196 ceph-mon[35041]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-09T01:18:37.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:37 smithi196 ceph-mon[35041]: from='osd.4 ' entity='osd.4' 2024-05-09T01:18:37.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:38.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-05-09T01:18:38.248 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi059 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi059 to 8546M 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:38.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:38.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:38.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:38.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: osd.4 [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] boot 2024-05-09T01:18:38.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T01:18:38.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:38.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:18:38.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:18:38.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi059 2024-05-09T01:18:38.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:38.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:38.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:38.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:38.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi059 to 8546M 2024-05-09T01:18:38.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:38.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:38.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:38.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:38.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: osd.4 [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] boot 2024-05-09T01:18:38.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T01:18:38.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:39.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:18:39.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:18:39.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi059 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi059 to 8546M 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: osd.4 [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] boot 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T01:18:39.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T01:18:39.731 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:39 smithi196 ceph-mon[35041]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:39.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.845+0000 7fc380d99700 1 -- 172.21.15.196:0/2251071666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c104e40 msgr2=0x7fc37c107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:39.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.845+0000 7fc380d99700 1 --2- 172.21.15.196:0/2251071666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c104e40 0x7fc37c107260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009990 tx=0x7fc36c009ca0 comp rx=0 tx=0).stop 2024-05-09T01:18:39.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.845+0000 7fc380d99700 1 -- 172.21.15.196:0/2251071666 shutdown_connections 2024-05-09T01:18:39.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.845+0000 7fc380d99700 1 --2- 172.21.15.196:0/2251071666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c10a100 0x7fc37c10c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:39.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.845+0000 7fc380d99700 1 --2- 172.21.15.196:0/2251071666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c104e40 0x7fc37c107260 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:39.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.845+0000 7fc380d99700 1 --2- 172.21.15.196:0/2251071666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc37c1077a0 0x7fc37c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:39.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.845+0000 7fc380d99700 1 -- 172.21.15.196:0/2251071666 >> 172.21.15.196:0/2251071666 conn(0x7fc37c0fe880 msgr2=0x7fc37c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:18:39.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.845+0000 7fc380d99700 1 -- 172.21.15.196:0/2251071666 shutdown_connections 2024-05-09T01:18:39.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.845+0000 7fc380d99700 1 -- 172.21.15.196:0/2251071666 wait complete. 2024-05-09T01:18:39.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc380d99700 1 Processor -- start 2024-05-09T01:18:39.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc380d99700 1 -- start start 2024-05-09T01:18:39.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc380d99700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c104e40 0x7fc37c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:18:39.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc380d99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc37c1077a0 0x7fc37c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:18:39.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc380d99700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c10a100 0x7fc37c1a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc380d99700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10fd80 con 0x7fc37c1077a0 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc380d99700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10fc00 con 0x7fc37c104e40 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc380d99700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10ff00 con 0x7fc37c10a100 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc37ad9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c10a100 0x7fc37c1a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc37ad9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c10a100 0x7fc37c1a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:57580/0 (socket says 172.21.15.196:57580) 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc37ad9d700 1 -- 172.21.15.196:0/4210849471 learned_addr learned my addr 172.21.15.196:0/4210849471 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc37a59c700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c104e40 0x7fc37c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.846+0000 7fc371d9b700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc37c1077a0 0x7fc37c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc37ad9d700 1 -- 172.21.15.196:0/4210849471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c104e40 msgr2=0x7fc37c19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc37ad9d700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c104e40 0x7fc37c19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc37ad9d700 1 -- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc37c1077a0 msgr2=0x7fc37c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc37ad9d700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc37c1077a0 0x7fc37c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:39.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc37ad9d700 1 -- 172.21.15.196:0/4210849471 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c1a2430 con 0x7fc37c10a100 2024-05-09T01:18:39.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc37ad9d700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c10a100 0x7fc37c1a1db0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc374007ee0 tx=0x7fc37400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:18:39.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc373fff700 1 -- 172.21.15.196:0/4210849471 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374004bb0 con 0x7fc37c10a100 2024-05-09T01:18:39.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c1a2720 con 0x7fc37c10a100 2024-05-09T01:18:39.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc373fff700 1 -- 172.21.15.196:0/4210849471 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc374004d50 con 0x7fc37c10a100 2024-05-09T01:18:39.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc373fff700 1 -- 172.21.15.196:0/4210849471 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374005920 con 0x7fc37c10a100 2024-05-09T01:18:39.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.847+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c1a2c60 con 0x7fc37c10a100 2024-05-09T01:18:39.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.848+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc37c0512c0 con 0x7fc37c10a100 2024-05-09T01:18:39.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.850+0000 7fc373fff700 1 -- 172.21.15.196:0/4210849471 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc37401a020 con 0x7fc37c10a100 2024-05-09T01:18:39.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.851+0000 7fc373fff700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc3680b2880 0x7fc3680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:18:39.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.851+0000 7fc37a59c700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc3680b2880 0x7fc3680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:18:39.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.851+0000 7fc373fff700 1 -- 172.21.15.196:0/4210849471 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fc3740e0ee0 con 0x7fc37c10a100 2024-05-09T01:18:39.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.851+0000 7fc37a59c700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc3680b2880 0x7fc3680b4d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00b5c0 tx=0x7fc36c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:18:39.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:39.855+0000 7fc373fff700 1 -- 172.21.15.196:0/4210849471 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc374010070 con 0x7fc37c10a100 2024-05-09T01:18:40.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:40.065+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc37c063c00 con 0x7fc3680b2880 2024-05-09T01:18:40.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:39 smithi059 ceph-mon[34351]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:40.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:39 smithi019 ceph-mon[30158]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:40.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:40 smithi196 ceph-mon[35041]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T01:18:40.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:40 smithi196 ceph-mon[35041]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:18:40.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:40 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:18:40.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:40 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:18:40.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:40 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:41.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:40 smithi059 ceph-mon[34351]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T01:18:41.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:40 smithi059 ceph-mon[34351]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:18:41.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:40 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:18:41.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:40 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:18:41.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:40 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:41.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:40 smithi019 ceph-mon[30158]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T01:18:41.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:40 smithi019 ceph-mon[30158]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:18:41.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:40 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:18:41.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:40 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:18:41.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:40 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:41.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:41 smithi196 ceph-mon[35041]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:42.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:41 smithi019 ceph-mon[30158]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:42.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:41 smithi059 ceph-mon[34351]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:43.068 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:42 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/2354391684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c53b8d7f-d032-46d4-98c0-99a6b77d9fd8"}]: dispatch 2024-05-09T01:18:43.068 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:42 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c53b8d7f-d032-46d4-98c0-99a6b77d9fd8"}]: dispatch 2024-05-09T01:18:43.068 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:42 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c53b8d7f-d032-46d4-98c0-99a6b77d9fd8"}]': finished 2024-05-09T01:18:43.068 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:42 smithi196 ceph-mon[35041]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T01:18:43.068 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:18:43.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:42 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/2354391684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c53b8d7f-d032-46d4-98c0-99a6b77d9fd8"}]: dispatch 2024-05-09T01:18:43.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:42 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c53b8d7f-d032-46d4-98c0-99a6b77d9fd8"}]: dispatch 2024-05-09T01:18:43.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:42 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c53b8d7f-d032-46d4-98c0-99a6b77d9fd8"}]': finished 2024-05-09T01:18:43.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:42 smithi059 ceph-mon[34351]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T01:18:43.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:18:43.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:42 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/2354391684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c53b8d7f-d032-46d4-98c0-99a6b77d9fd8"}]: dispatch 2024-05-09T01:18:43.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:42 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c53b8d7f-d032-46d4-98c0-99a6b77d9fd8"}]: dispatch 2024-05-09T01:18:43.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:42 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c53b8d7f-d032-46d4-98c0-99a6b77d9fd8"}]': finished 2024-05-09T01:18:43.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:42 smithi019 ceph-mon[30158]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T01:18:43.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:18:44.076 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:43 smithi059 ceph-mon[34351]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:44.076 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:43 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/2808046514' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T01:18:44.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:43 smithi019 ceph-mon[30158]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:44.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:43 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/2808046514' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T01:18:44.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:43 smithi196 ceph-mon[35041]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:44.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:43 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/2808046514' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T01:18:46.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:46 smithi019 ceph-mon[30158]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:46.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:46 smithi196 ceph-mon[35041]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:46.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:46 smithi059 ceph-mon[34351]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:48.282 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:48 smithi196 ceph-mon[35041]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:48.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:48 smithi019 ceph-mon[30158]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:48.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:48 smithi059 ceph-mon[34351]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:50 smithi019 ceph-mon[30158]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:50.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:50 smithi196 ceph-mon[35041]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:50.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:50 smithi059 ceph-mon[34351]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:51.396 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T01:18:51.396 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:51.396 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:51 smithi196 ceph-mon[35041]: Deploying daemon osd.5 on smithi196 2024-05-09T01:18:51.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T01:18:51.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:51.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:51 smithi019 ceph-mon[30158]: Deploying daemon osd.5 on smithi196 2024-05-09T01:18:51.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T01:18:51.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:18:51.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:51 smithi059 ceph-mon[34351]: Deploying daemon osd.5 on smithi196 2024-05-09T01:18:52.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:52 smithi196 ceph-mon[35041]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:52.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:52 smithi019 ceph-mon[30158]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:52.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:52 smithi059 ceph-mon[34351]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:54.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:54 smithi019 ceph-mon[30158]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:54.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:54 smithi196 ceph-mon[35041]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:54.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:54 smithi059 ceph-mon[34351]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:56.667 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:56 smithi196 ceph-mon[35041]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:56.668 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:56.668 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:56.668 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:56.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:56 smithi059 ceph-mon[34351]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:56.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:56.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:56.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:56.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:56 smithi019 ceph-mon[30158]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:56.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:56.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:56.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:18:58.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:58 smithi196 ceph-mon[35041]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:58.579 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-05-09T01:18:58.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.579+0000 7fc373fff700 1 -- 172.21.15.196:0/4210849471 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc37c063c00 con 0x7fc3680b2880 2024-05-09T01:18:58.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc3680b2880 msgr2=0x7fc3680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:58.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc3680b2880 0x7fc3680b4d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00b5c0 tx=0x7fc36c01a040 comp rx=0 tx=0).stop 2024-05-09T01:18:58.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c10a100 msgr2=0x7fc37c1a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:18:58.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c10a100 0x7fc37c1a1db0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc374007ee0 tx=0x7fc37400d3b0 comp rx=0 tx=0).stop 2024-05-09T01:18:58.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 shutdown_connections 2024-05-09T01:18:58.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37c10a100 0x7fc37c1a1db0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:58.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c104e40 0x7fc37c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:58.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc3680b2880 0x7fc3680b4d90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:58.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 --2- 172.21.15.196:0/4210849471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc37c1077a0 0x7fc37c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:18:58.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.582+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 >> 172.21.15.196:0/4210849471 conn(0x7fc37c0fe880 msgr2=0x7fc37c105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:18:58.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.583+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 shutdown_connections 2024-05-09T01:18:58.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:18:58.583+0000 7fc380d99700 1 -- 172.21.15.196:0/4210849471 wait complete. 2024-05-09T01:18:58.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:58 smithi059 ceph-mon[34351]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:58.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:58 smithi019 ceph-mon[30158]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:59.182 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.5.service 2024-05-09T01:18:59.184 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-05-09T01:18:59.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T01:18:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:18:59.446 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:18:59 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:18:59.330+0000 7f7e9ed717c0 -1 osd.5 0 log_to_monitors true 2024-05-09T01:18:59.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:59 smithi019 ceph-mon[30158]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:59.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:59 smithi019 ceph-mon[30158]: from='osd.5 [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T01:18:59.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:59 smithi019 ceph-mon[30158]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T01:18:59.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:18:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:59 smithi059 ceph-mon[34351]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:59.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:59 smithi059 ceph-mon[34351]: from='osd.5 [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T01:18:59.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:59 smithi059 ceph-mon[34351]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T01:18:59.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:18:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:59 smithi196 ceph-mon[35041]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:18:59.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:59 smithi196 ceph-mon[35041]: from='osd.5 [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T01:18:59.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:59 smithi196 ceph-mon[35041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T01:18:59.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:18:59.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:18:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:00.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:00 smithi019 ceph-mon[30158]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T01:19:00.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:00 smithi019 ceph-mon[30158]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T01:19:00.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:00 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:00.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:00 smithi019 ceph-mon[30158]: from='osd.5 [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:19:00.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:00 smithi019 ceph-mon[30158]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:19:00.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:00 smithi059 ceph-mon[34351]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T01:19:00.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:00 smithi059 ceph-mon[34351]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T01:19:00.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:00 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:00.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:00 smithi059 ceph-mon[34351]: from='osd.5 [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:19:00.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:00 smithi059 ceph-mon[34351]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:19:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:00 smithi196 ceph-mon[35041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T01:19:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:00 smithi196 ceph-mon[35041]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T01:19:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:00 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:00 smithi196 ceph-mon[35041]: from='osd.5 [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:19:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:00 smithi196 ceph-mon[35041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:19:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T01:19:01.454 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T01:19:01.454 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-09T01:19:01.454 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-09T01:19:01.454 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137035 s, 765 MB/s 2024-05-09T01:19:01.454 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-09T01:19:01.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:01 smithi019 ceph-mon[30158]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:19:01.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:01 smithi019 ceph-mon[30158]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T01:19:01.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:01.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:01.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:01 smithi019 ceph-mon[30158]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:19:01.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:01 smithi019 ceph-mon[30158]: from='osd.5 ' entity='osd.5' 2024-05-09T01:19:01.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:01 smithi059 ceph-mon[34351]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:19:01.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:01 smithi059 ceph-mon[34351]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T01:19:01.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:01.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:01.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:01 smithi059 ceph-mon[34351]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:19:01.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:01 smithi059 ceph-mon[34351]: from='osd.5 ' entity='osd.5' 2024-05-09T01:19:01.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:01 smithi196 ceph-mon[35041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:19:01.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:01 smithi196 ceph-mon[35041]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T01:19:01.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:01.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:01.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:01 smithi196 ceph-mon[35041]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:19:01.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:01 smithi196 ceph-mon[35041]: from='osd.5 ' entity='osd.5' 2024-05-09T01:19:01.946 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:19:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:19:01.503+0000 7f7e943d5700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T01:19:01.947 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:19:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:19:01.668+0000 7f7e8c182700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:19:02.060 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-05-09T01:19:02.309 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:19:02.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:02 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:19:02.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:02 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:19:02.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:02.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:02 smithi019 ceph-mon[30158]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T01:19:02.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:02.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:02 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:19:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:02 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:19:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:02 smithi059 ceph-mon[34351]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T01:19:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:02.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:02 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:19:02.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:02 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:19:02.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:02.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:02 smithi196 ceph-mon[35041]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T01:19:02.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:02.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:03.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi196 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: osd.5 [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] boot 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi196 to 8544M 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:03.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:03.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:03.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:19:03.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:03 smithi019 ceph-mon[30158]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T01:19:03.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi196 2024-05-09T01:19:03.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: osd.5 [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] boot 2024-05-09T01:19:03.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T01:19:03.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:03.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi196 to 8544M 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:19:03.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:03 smithi196 ceph-mon[35041]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T01:19:04.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.030+0000 7fd7b734a700 1 -- 172.21.15.19:0/1903277476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0104d20 msgr2=0x7fd7b010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:04.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.030+0000 7fd7b734a700 1 --2- 172.21.15.19:0/1903277476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0104d20 0x7fd7b010b5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800b0a0 tx=0x7fd7a800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:19:04.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.030+0000 7fd7b734a700 1 -- 172.21.15.19:0/1903277476 shutdown_connections 2024-05-09T01:19:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.030+0000 7fd7b734a700 1 --2- 172.21.15.19:0/1903277476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0104d20 0x7fd7b010b5d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.030+0000 7fd7b734a700 1 --2- 172.21.15.19:0/1903277476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b0103250 0x7fd7b0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.030+0000 7fd7b734a700 1 --2- 172.21.15.19:0/1903277476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0104380 0x7fd7b0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.030+0000 7fd7b734a700 1 -- 172.21.15.19:0/1903277476 >> 172.21.15.19:0/1903277476 conn(0x7fd7b00fe890 msgr2=0x7fd7b0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:19:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.030+0000 7fd7b734a700 1 -- 172.21.15.19:0/1903277476 shutdown_connections 2024-05-09T01:19:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.030+0000 7fd7b734a700 1 -- 172.21.15.19:0/1903277476 wait complete. 2024-05-09T01:19:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b734a700 1 Processor -- start 2024-05-09T01:19:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b734a700 1 -- start start 2024-05-09T01:19:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b734a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0103250 0x7fd7b019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b734a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b0104380 0x7fd7b019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b734a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0104d20 0x7fd7b01a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b734a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b010ecd0 con 0x7fd7b0103250 2024-05-09T01:19:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b734a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b010eb50 con 0x7fd7b0104380 2024-05-09T01:19:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b734a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b010ee50 con 0x7fd7b0104d20 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b50e6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0103250 0x7fd7b019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b48e5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b0104380 0x7fd7b019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b48e5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b0104380 0x7fd7b019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:51406/0 (socket says 172.21.15.19:51406) 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b50e6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0103250 0x7fd7b019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42326/0 (socket says 172.21.15.19:42326) 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b48e5700 1 -- 172.21.15.19:0/169186319 learned_addr learned my addr 172.21.15.19:0/169186319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.031+0000 7fd7b58e7700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0104d20 0x7fd7b01a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7b50e6700 1 -- 172.21.15.19:0/169186319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0104d20 msgr2=0x7fd7b01a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7b50e6700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0104d20 0x7fd7b01a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7b50e6700 1 -- 172.21.15.19:0/169186319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b0104380 msgr2=0x7fd7b019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7b50e6700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b0104380 0x7fd7b019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7b50e6700 1 -- 172.21.15.19:0/169186319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b01a2440 con 0x7fd7b0103250 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7b50e6700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0103250 0x7fd7b019a8e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0102f50 tx=0x7fd7ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7a67fc700 1 -- 172.21.15.19:0/169186319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac004220 con 0x7fd7b0103250 2024-05-09T01:19:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7a67fc700 1 -- 172.21.15.19:0/169186319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7ac0043c0 con 0x7fd7b0103250 2024-05-09T01:19:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b01a2730 con 0x7fd7b0103250 2024-05-09T01:19:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7a67fc700 1 -- 172.21.15.19:0/169186319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac003be0 con 0x7fd7b0103250 2024-05-09T01:19:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.032+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b01a2c70 con 0x7fd7b0103250 2024-05-09T01:19:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.033+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b00512c0 con 0x7fd7b0103250 2024-05-09T01:19:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.036+0000 7fd7a67fc700 1 -- 172.21.15.19:0/169186319 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd7ac010700 con 0x7fd7b0103250 2024-05-09T01:19:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.037+0000 7fd7a67fc700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd79c0b2880 0x7fd79c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.037+0000 7fd7b48e5700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd79c0b2880 0x7fd79c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.037+0000 7fd7a67fc700 1 -- 172.21.15.19:0/169186319 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4533+0+0 (secure 0 0 0) 0x7fd7ac0e1820 con 0x7fd7b0103250 2024-05-09T01:19:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.037+0000 7fd7b48e5700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd79c0b2880 0x7fd79c0b4d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd7b019b990 tx=0x7fd7a000b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:19:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.041+0000 7fd7a67fc700 1 -- 172.21.15.19:0/169186319 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7ac0aa490 con 0x7fd7b0103250 2024-05-09T01:19:04.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi196 2024-05-09T01:19:04.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: osd.5 [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] boot 2024-05-09T01:19:04.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T01:19:04.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T01:19:04.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:04.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi196 to 8544M 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-09T01:19:04.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:03 smithi059 ceph-mon[34351]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T01:19:04.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:04.268+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd7b01a2f90 con 0x7fd79c0b2880 2024-05-09T01:19:05.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:04 smithi059 ceph-mon[34351]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:19:05.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:19:05.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:19:05.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:05.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:04 smithi059 ceph-mon[34351]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T01:19:05.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:04 smithi019 ceph-mon[30158]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:19:05.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:19:05.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:19:05.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:05.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:04 smithi019 ceph-mon[30158]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T01:19:05.178 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:04 smithi196 ceph-mon[35041]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:19:05.178 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:19:05.178 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:19:05.178 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:05.178 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:04 smithi196 ceph-mon[35041]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T01:19:06.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:05 smithi059 ceph-mon[34351]: pgmap v143: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T01:19:06.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:05 smithi019 ceph-mon[30158]: pgmap v143: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T01:19:06.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:05 smithi196 ceph-mon[35041]: pgmap v143: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T01:19:07.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:06 smithi059 ceph-mon[34351]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T01:19:07.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:06 smithi019 ceph-mon[30158]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T01:19:07.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:06 smithi196 ceph-mon[35041]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T01:19:08.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:07 smithi059 ceph-mon[34351]: pgmap v144: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T01:19:08.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:07 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/805738278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95d655ea-93ee-4077-96be-3bbddc722520"}]: dispatch 2024-05-09T01:19:08.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:07 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/805738278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95d655ea-93ee-4077-96be-3bbddc722520"}]': finished 2024-05-09T01:19:08.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:07 smithi059 ceph-mon[34351]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T01:19:08.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:07 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:08.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:07 smithi019 ceph-mon[30158]: pgmap v144: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T01:19:08.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:07 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/805738278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95d655ea-93ee-4077-96be-3bbddc722520"}]: dispatch 2024-05-09T01:19:08.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:07 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/805738278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95d655ea-93ee-4077-96be-3bbddc722520"}]': finished 2024-05-09T01:19:08.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:07 smithi019 ceph-mon[30158]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T01:19:08.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:07 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:08.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:07 smithi196 ceph-mon[35041]: pgmap v144: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T01:19:08.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:07 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/805738278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95d655ea-93ee-4077-96be-3bbddc722520"}]: dispatch 2024-05-09T01:19:08.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:07 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/805738278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95d655ea-93ee-4077-96be-3bbddc722520"}]': finished 2024-05-09T01:19:08.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:07 smithi196 ceph-mon[35041]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T01:19:08.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:07 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:09.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:08 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/857783108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T01:19:09.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:08 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/857783108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T01:19:09.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:08 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/857783108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T01:19:10.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:09 smithi059 ceph-mon[34351]: pgmap v146: 1 pgs: 1 activating+degraded; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T01:19:10.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:09 smithi019 ceph-mon[30158]: pgmap v146: 1 pgs: 1 activating+degraded; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T01:19:10.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:09 smithi196 ceph-mon[35041]: pgmap v146: 1 pgs: 1 activating+degraded; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T01:19:11.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:11 smithi196 ceph-mon[35041]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T01:19:11.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:11 smithi196 ceph-mon[35041]: Cluster is now healthy 2024-05-09T01:19:11.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:11 smithi019 ceph-mon[30158]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T01:19:11.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:11 smithi019 ceph-mon[30158]: Cluster is now healthy 2024-05-09T01:19:11.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:11 smithi059 ceph-mon[34351]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T01:19:11.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:11 smithi059 ceph-mon[34351]: Cluster is now healthy 2024-05-09T01:19:12.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:12 smithi196 ceph-mon[35041]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-09T01:19:12.517 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:12 smithi019 ceph-mon[30158]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-09T01:19:12.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:12 smithi059 ceph-mon[34351]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-09T01:19:14.336 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:14 smithi019 ceph-mon[30158]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-09T01:19:14.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:14 smithi196 ceph-mon[35041]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-09T01:19:14.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:14 smithi059 ceph-mon[34351]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-09T01:19:16.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:16 smithi019 ceph-mon[30158]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:19:16.347 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:16 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T01:19:16.347 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:16 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:16.347 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:16 smithi019 ceph-mon[30158]: Deploying daemon osd.6 on smithi019 2024-05-09T01:19:16.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:16 smithi196 ceph-mon[35041]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:19:16.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:16 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T01:19:16.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:16 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:16.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:16 smithi196 ceph-mon[35041]: Deploying daemon osd.6 on smithi019 2024-05-09T01:19:16.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:16 smithi059 ceph-mon[34351]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:19:16.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:16 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T01:19:16.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:16 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:16.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:16 smithi059 ceph-mon[34351]: Deploying daemon osd.6 on smithi019 2024-05-09T01:19:18.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:18 smithi196 ceph-mon[35041]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:19:18.455 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:18 smithi019 ceph-mon[30158]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:19:18.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:18 smithi059 ceph-mon[34351]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:19:20.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:20 smithi019 ceph-mon[30158]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T01:19:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:20 smithi196 ceph-mon[35041]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T01:19:20.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:20 smithi059 ceph-mon[34351]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T01:19:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:22 smithi059 ceph-mon[34351]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T01:19:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:22.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:22 smithi019 ceph-mon[30158]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T01:19:22.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:22.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:22.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:22.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:22 smithi196 ceph-mon[35041]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T01:19:22.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:22.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:22.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:24 smithi019 ceph-mon[30158]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:24.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:24.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:24 smithi059 ceph-mon[34351]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:24.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:24 smithi196 ceph-mon[35041]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:24.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:24.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:24.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:24.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:24.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.379+0000 7fd7a67fc700 1 -- 172.21.15.19:0/169186319 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd7b01a2f90 con 0x7fd79c0b2880 2024-05-09T01:19:25.379 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd79c0b2880 msgr2=0x7fd79c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd79c0b2880 0x7fd79c0b4d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd7b019b990 tx=0x7fd7a000b480 comp rx=0 tx=0).stop 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0103250 msgr2=0x7fd7b019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0103250 0x7fd7b019a8e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0102f50 tx=0x7fd7ac00eca0 comp rx=0 tx=0).stop 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 shutdown_connections 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0104d20 0x7fd7b01a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd79c0b2880 0x7fd79c0b4d90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b0103250 0x7fd7b019a8e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 --2- 172.21.15.19:0/169186319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b0104380 0x7fd7b019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 >> 172.21.15.19:0/169186319 conn(0x7fd7b00fe890 msgr2=0x7fd7b0100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:19:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 shutdown_connections 2024-05-09T01:19:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:19:25.382+0000 7fd7b734a700 1 -- 172.21.15.19:0/169186319 wait complete. 2024-05-09T01:19:25.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:25 smithi059 ceph-mon[34351]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T01:19:25.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:25 smithi019 ceph-mon[30158]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T01:19:25.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:25 smithi196 ceph-mon[35041]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T01:19:26.022 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.6.service 2024-05-09T01:19:26.025 INFO:tasks.cephadm:Deploying osd.7 on smithi059 with /dev/vg_nvme/lv_2... 2024-05-09T01:19:26.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T01:19:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:19:26.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:26 smithi059 ceph-mon[34351]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:26.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:26 smithi059 ceph-mon[34351]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T01:19:26.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:26 smithi059 ceph-mon[34351]: osdmap e43: 7 total, 6 up, 7 in 2024-05-09T01:19:26.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:26 smithi059 ceph-mon[34351]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:19:26.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:26.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:26.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:26.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:26 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:26 smithi019 ceph-mon[30158]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:26 smithi019 ceph-mon[30158]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T01:19:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:26 smithi019 ceph-mon[30158]: osdmap e43: 7 total, 6 up, 7 in 2024-05-09T01:19:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:26 smithi019 ceph-mon[30158]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:19:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:26.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:26.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:26 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:26.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:26 smithi196 ceph-mon[35041]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:26.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:26 smithi196 ceph-mon[35041]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T01:19:26.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:26 smithi196 ceph-mon[35041]: osdmap e43: 7 total, 6 up, 7 in 2024-05-09T01:19:26.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:26 smithi196 ceph-mon[35041]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:19:26.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:26.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:26.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:26.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:26 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:27.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:27 smithi019 ceph-mon[30158]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:19:27.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:27 smithi019 ceph-mon[30158]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T01:19:27.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:27.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:27.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:27 smithi019 ceph-mon[30158]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' 2024-05-09T01:19:27.397 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:19:27 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:19:27.242+0000 7f63c469d700 -1 osd.6 0 waiting for initial osdmap 2024-05-09T01:19:27.397 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:19:27 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:19:27.353+0000 7f63bc44a700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:19:27.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:27 smithi059 ceph-mon[34351]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:19:27.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:27 smithi059 ceph-mon[34351]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T01:19:27.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:27.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:27.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:27 smithi059 ceph-mon[34351]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' 2024-05-09T01:19:27.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:27 smithi196 ceph-mon[35041]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:19:27.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:27 smithi196 ceph-mon[35041]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T01:19:27.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:27.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:27.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:27 smithi196 ceph-mon[35041]: from='osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508]' entity='osd.6' 2024-05-09T01:19:28.324 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:28 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:19:28.325 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:28 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:19:28.408 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T01:19:28.408 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T01:19:28.409 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-09T01:19:28.409 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-09T01:19:28.409 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121565 s, 863 MB/s 2024-05-09T01:19:28.409 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-09T01:19:28.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:28 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:19:28.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:28 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:19:28.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:28 smithi019 ceph-mon[30158]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:28.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:28 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:28.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:28 smithi059 ceph-mon[34351]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:28.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:28 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:28.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:28 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:19:28.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:28 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:19:28.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:28 smithi196 ceph-mon[35041]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:28.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:28 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:29.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_2 2024-05-09T01:19:29.496 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:19:29.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:29.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] boot 2024-05-09T01:19:29.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: osdmap e45: 7 total, 7 up, 7 in 2024-05-09T01:19:29.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:29.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:29.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:29.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:29.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] boot 2024-05-09T01:19:29.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: osdmap e45: 7 total, 7 up, 7 in 2024-05-09T01:19:29.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:29.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:29.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: osd.6 [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] boot 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: osdmap e45: 7 total, 7 up, 7 in 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:29.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:30.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:30 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi019 2024-05-09T01:19:30.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:30 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi019 to 5697M 2024-05-09T01:19:30.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:30 smithi059 ceph-mon[34351]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:30.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:30 smithi059 ceph-mon[34351]: osdmap e46: 7 total, 7 up, 7 in 2024-05-09T01:19:30.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:30 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi019 2024-05-09T01:19:30.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:30 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi019 to 5697M 2024-05-09T01:19:30.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:30 smithi019 ceph-mon[30158]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:30.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:30 smithi019 ceph-mon[30158]: osdmap e46: 7 total, 7 up, 7 in 2024-05-09T01:19:30.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:30 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi019 2024-05-09T01:19:30.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:30 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi019 to 5697M 2024-05-09T01:19:30.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:30 smithi196 ceph-mon[35041]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-09T01:19:30.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:30 smithi196 ceph-mon[35041]: osdmap e46: 7 total, 7 up, 7 in 2024-05-09T01:19:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 -- 172.21.15.59:0/4093233738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff94104cf0 msgr2=0x7eff9410b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 --2- 172.21.15.59:0/4093233738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff94104cf0 0x7eff9410b780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b0a0 tx=0x7eff8c00b3b0 comp rx=0 tx=0).stop 2024-05-09T01:19:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 -- 172.21.15.59:0/4093233738 shutdown_connections 2024-05-09T01:19:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 --2- 172.21.15.59:0/4093233738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff94104cf0 0x7eff9410b780 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 --2- 172.21.15.59:0/4093233738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff941037b0 0x7eff94103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 --2- 172.21.15.59:0/4093233738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff94104190 0x7eff94104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 -- 172.21.15.59:0/4093233738 >> 172.21.15.59:0/4093233738 conn(0x7eff940fe020 msgr2=0x7eff941004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 -- 172.21.15.59:0/4093233738 shutdown_connections 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 -- 172.21.15.59:0/4093233738 wait complete. 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.140+0000 7eff99156700 1 Processor -- start 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff99156700 1 -- start start 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff99156700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff941037b0 0x7eff9419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff99156700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff94104190 0x7eff9419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff99156700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff94104cf0 0x7eff941a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff99156700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410ed40 con 0x7eff94104190 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff99156700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410ebc0 con 0x7eff941037b0 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff99156700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410eec0 con 0x7eff94104cf0 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff92d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff941037b0 0x7eff9419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff92d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff941037b0 0x7eff9419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59434/0 (socket says 172.21.15.59:59434) 2024-05-09T01:19:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff92d9d700 1 -- 172.21.15.59:0/2474483288 learned_addr learned my addr 172.21.15.59:0/2474483288 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff9359e700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff94104cf0 0x7eff941a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff9259c700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff94104190 0x7eff9419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff92d9d700 1 -- 172.21.15.59:0/2474483288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff94104cf0 msgr2=0x7eff941a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff92d9d700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff94104cf0 0x7eff941a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff92d9d700 1 -- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff94104190 msgr2=0x7eff9419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff92d9d700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff94104190 0x7eff9419d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff92d9d700 1 -- 172.21.15.59:0/2474483288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff941a4650 con 0x7eff941037b0 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.141+0000 7eff9259c700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff94104190 0x7eff9419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.142+0000 7eff92d9d700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff941037b0 0x7eff9419caf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7eff8400d950 tx=0x7eff8400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.142+0000 7eff7bfff700 1 -- 172.21.15.59:0/2474483288 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84017070 con 0x7eff941037b0 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.142+0000 7eff7bfff700 1 -- 172.21.15.59:0/2474483288 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff84009de0 con 0x7eff941037b0 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.142+0000 7eff7bfff700 1 -- 172.21.15.59:0/2474483288 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8400f3c0 con 0x7eff941037b0 2024-05-09T01:19:31.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.142+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff941a4940 con 0x7eff941037b0 2024-05-09T01:19:31.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.142+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff941a4e80 con 0x7eff941037b0 2024-05-09T01:19:31.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.143+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff940512c0 con 0x7eff941037b0 2024-05-09T01:19:31.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.145+0000 7eff7bfff700 1 -- 172.21.15.59:0/2474483288 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7eff8401a070 con 0x7eff941037b0 2024-05-09T01:19:31.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.145+0000 7eff7bfff700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff7c0b2880 0x7eff7c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:31.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.145+0000 7eff7bfff700 1 -- 172.21.15.59:0/2474483288 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4965+0+0 (secure 0 0 0) 0x7eff840e0d60 con 0x7eff941037b0 2024-05-09T01:19:31.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.145+0000 7eff9259c700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff7c0b2880 0x7eff7c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:31.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.146+0000 7eff9259c700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff7c0b2880 0x7eff7c0b4d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff9419dba0 tx=0x7eff88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:19:31.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.149+0000 7eff7bfff700 1 -- 172.21.15.59:0/2474483288 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff840a9790 con 0x7eff941037b0 2024-05-09T01:19:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:31.374+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7eff941a5110 con 0x7eff7c0b2880 2024-05-09T01:19:32.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:32 smithi059 ceph-mon[34351]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:32.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:32 smithi059 ceph-mon[34351]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:19:32.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:19:32.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:19:32.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:32 smithi019 ceph-mon[30158]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:32 smithi019 ceph-mon[30158]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:19:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:19:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:19:32.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:32.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:32 smithi196 ceph-mon[35041]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:32.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:32 smithi196 ceph-mon[35041]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:19:32.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:19:32.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:19:32.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:34.474 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:34 smithi059 ceph-mon[34351]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:34.475 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:34 smithi059 ceph-mon[34351]: from='client.? 172.21.15.59:0/129921604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792a321f-5e7a-4ec4-b6bd-020b1fb8bf07"}]: dispatch 2024-05-09T01:19:34.475 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:34 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792a321f-5e7a-4ec4-b6bd-020b1fb8bf07"}]: dispatch 2024-05-09T01:19:34.475 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:34 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "792a321f-5e7a-4ec4-b6bd-020b1fb8bf07"}]': finished 2024-05-09T01:19:34.475 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:34 smithi059 ceph-mon[34351]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T01:19:34.475 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:34 smithi019 ceph-mon[30158]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:34 smithi019 ceph-mon[30158]: from='client.? 172.21.15.59:0/129921604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792a321f-5e7a-4ec4-b6bd-020b1fb8bf07"}]: dispatch 2024-05-09T01:19:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:34 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792a321f-5e7a-4ec4-b6bd-020b1fb8bf07"}]: dispatch 2024-05-09T01:19:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:34 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "792a321f-5e7a-4ec4-b6bd-020b1fb8bf07"}]': finished 2024-05-09T01:19:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:34 smithi019 ceph-mon[30158]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T01:19:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:34.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:34 smithi196 ceph-mon[35041]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:34.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:34 smithi196 ceph-mon[35041]: from='client.? 172.21.15.59:0/129921604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792a321f-5e7a-4ec4-b6bd-020b1fb8bf07"}]: dispatch 2024-05-09T01:19:34.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:34 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792a321f-5e7a-4ec4-b6bd-020b1fb8bf07"}]: dispatch 2024-05-09T01:19:34.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:34 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "792a321f-5e7a-4ec4-b6bd-020b1fb8bf07"}]': finished 2024-05-09T01:19:34.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:34 smithi196 ceph-mon[35041]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T01:19:34.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:35.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:35 smithi059 ceph-mon[34351]: from='client.? 172.21.15.59:0/3978051612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T01:19:35.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:35 smithi019 ceph-mon[30158]: from='client.? 172.21.15.59:0/3978051612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T01:19:35.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:35 smithi196 ceph-mon[35041]: from='client.? 172.21.15.59:0/3978051612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T01:19:36.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:36 smithi059 ceph-mon[34351]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:36.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:36 smithi019 ceph-mon[30158]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:36.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:36 smithi196 ceph-mon[35041]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:38.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:38 smithi059 ceph-mon[34351]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:38.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:38 smithi019 ceph-mon[30158]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:38.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:38 smithi196 ceph-mon[35041]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:40.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:40 smithi059 ceph-mon[34351]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:40.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:40 smithi019 ceph-mon[30158]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:40.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:40 smithi196 ceph-mon[35041]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:42.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:42 smithi059 ceph-mon[34351]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:42.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T01:19:42.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:42.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:42 smithi019 ceph-mon[30158]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:42.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T01:19:42.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:42.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:42 smithi196 ceph-mon[35041]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:42.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T01:19:42.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:43.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:43 smithi059 ceph-mon[34351]: Deploying daemon osd.7 on smithi059 2024-05-09T01:19:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:43 smithi019 ceph-mon[30158]: Deploying daemon osd.7 on smithi059 2024-05-09T01:19:43.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:43 smithi196 ceph-mon[35041]: Deploying daemon osd.7 on smithi059 2024-05-09T01:19:44.573 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:44 smithi059 ceph-mon[34351]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:44.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:44 smithi019 ceph-mon[30158]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:44.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:44 smithi196 ceph-mon[35041]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:46.590 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:46 smithi059 ceph-mon[34351]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:46.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:46 smithi019 ceph-mon[30158]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:46.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:46 smithi196 ceph-mon[35041]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:48.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:48 smithi059 ceph-mon[34351]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:48.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:48.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:48.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:48.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:48 smithi019 ceph-mon[30158]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:48.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:48.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:48.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:48.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:48 smithi196 ceph-mon[35041]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:48.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:48.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:48.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:49.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:49 smithi059 ceph-mon[34351]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:49.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:49 smithi019 ceph-mon[30158]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:49.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:49 smithi196 ceph-mon[35041]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:51.094 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 7 on host 'smithi059' 2024-05-09T01:19:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.094+0000 7eff7bfff700 1 -- 172.21.15.59:0/2474483288 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff941a5110 con 0x7eff7c0b2880 2024-05-09T01:19:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff7c0b2880 msgr2=0x7eff7c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff7c0b2880 0x7eff7c0b4d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff9419dba0 tx=0x7eff88009260 comp rx=0 tx=0).stop 2024-05-09T01:19:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff941037b0 msgr2=0x7eff9419caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff941037b0 0x7eff9419caf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7eff8400d950 tx=0x7eff8400dc60 comp rx=0 tx=0).stop 2024-05-09T01:19:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 shutdown_connections 2024-05-09T01:19:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff94104cf0 0x7eff941a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff941037b0 0x7eff9419caf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff7c0b2880 0x7eff7c0b4d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 --2- 172.21.15.59:0/2474483288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff94104190 0x7eff9419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 >> 172.21.15.59:0/2474483288 conn(0x7eff940fe020 msgr2=0x7eff940ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:19:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 shutdown_connections 2024-05-09T01:19:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:19:51.097+0000 7eff99156700 1 -- 172.21.15.59:0/2474483288 wait complete. 2024-05-09T01:19:51.743 DEBUG:teuthology.orchestra.run.smithi059:osd.7> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.7.service 2024-05-09T01:19:51.746 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-05-09T01:19:51.746 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T01:19:51.993 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:19:52.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:52 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:52 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:52 smithi059 ceph-mon[34351]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:52.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:52 smithi059 ceph-mon[34351]: from='osd.7 [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T01:19:52.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:52 smithi059 ceph-mon[34351]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T01:19:52.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:52 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:52 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:52 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:52 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:52 smithi196 ceph-mon[35041]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:52.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:52 smithi196 ceph-mon[35041]: from='osd.7 [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T01:19:52.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:52 smithi196 ceph-mon[35041]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T01:19:52.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:52 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:52 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:52 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:52 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:52 smithi019 ceph-mon[30158]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:52.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:52 smithi019 ceph-mon[30158]: from='osd.7 [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T01:19:52.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:52 smithi019 ceph-mon[30158]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T01:19:52.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:52 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:52.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:52 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:53.337 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:53 smithi196 ceph-mon[35041]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T01:19:53.338 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:53 smithi196 ceph-mon[35041]: osdmap e48: 8 total, 7 up, 8 in 2024-05-09T01:19:53.338 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:53.338 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:53 smithi196 ceph-mon[35041]: from='osd.7 [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:19:53.338 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:53 smithi196 ceph-mon[35041]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:19:53.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:53 smithi059 ceph-mon[34351]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T01:19:53.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:53 smithi059 ceph-mon[34351]: osdmap e48: 8 total, 7 up, 8 in 2024-05-09T01:19:53.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:53.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:53 smithi059 ceph-mon[34351]: from='osd.7 [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:19:53.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:53 smithi059 ceph-mon[34351]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:19:53.348 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:19:53 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:19:53.098+0000 7f1535178700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T01:19:53.349 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:19:53 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:19:53.110+0000 7f1530f2d700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:19:53.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:53 smithi019 ceph-mon[30158]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T01:19:53.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:53 smithi019 ceph-mon[30158]: osdmap e48: 8 total, 7 up, 8 in 2024-05-09T01:19:53.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:53.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:53 smithi019 ceph-mon[30158]: from='osd.7 [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:19:53.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:53 smithi019 ceph-mon[30158]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:19:53.916 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T01:19:53.916 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T01:19:53.916 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-09T01:19:53.916 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-09T01:19:53.916 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124037 s, 845 MB/s 2024-05-09T01:19:53.916 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-09T01:19:54.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:54 smithi059 ceph-mon[34351]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:54.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:54 smithi059 ceph-mon[34351]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:19:54.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:54 smithi059 ceph-mon[34351]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T01:19:54.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:54.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:54.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:54 smithi059 ceph-mon[34351]: osd.7 [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] boot 2024-05-09T01:19:54.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:54 smithi059 ceph-mon[34351]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T01:19:54.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:54.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:54 smithi019 ceph-mon[30158]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:54.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:54 smithi019 ceph-mon[30158]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:19:54.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:54 smithi019 ceph-mon[30158]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T01:19:54.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:54.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:54.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:54 smithi019 ceph-mon[30158]: osd.7 [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] boot 2024-05-09T01:19:54.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:54 smithi019 ceph-mon[30158]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T01:19:54.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:54.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:54 smithi196 ceph-mon[35041]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-05-09T01:19:54.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:54 smithi196 ceph-mon[35041]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:19:54.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:54 smithi196 ceph-mon[35041]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T01:19:54.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:54.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:54.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:54 smithi196 ceph-mon[35041]: osd.7 [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] boot 2024-05-09T01:19:54.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:54 smithi196 ceph-mon[35041]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T01:19:54.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T01:19:54.461 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-05-09T01:19:54.707 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:19:55.276 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:55 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:19:55.276 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:55 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:19:55.277 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:55 smithi059 ceph-mon[34351]: osdmap e51: 8 total, 8 up, 8 in 2024-05-09T01:19:55.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:55 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:19:55.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:55 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:19:55.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:55 smithi019 ceph-mon[30158]: osdmap e51: 8 total, 8 up, 8 in 2024-05-09T01:19:55.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:55 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:19:55.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:55 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:19:55.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:55 smithi196 ceph-mon[35041]: osdmap e51: 8 total, 8 up, 8 in 2024-05-09T01:19:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.259+0000 7f1d154c4700 1 -- 172.21.15.196:0/2654836487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10103240 msgr2=0x7f1d10103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.259+0000 7f1d154c4700 1 --2- 172.21.15.196:0/2654836487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10103240 0x7f1d10103650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009990 tx=0x7f1d00009ca0 comp rx=0 tx=0).stop 2024-05-09T01:19:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.259+0000 7f1d154c4700 1 -- 172.21.15.196:0/2654836487 shutdown_connections 2024-05-09T01:19:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.259+0000 7f1d154c4700 1 --2- 172.21.15.196:0/2654836487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d10104d10 0x7f1d1010b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.259+0000 7f1d154c4700 1 --2- 172.21.15.196:0/2654836487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10103240 0x7f1d10103650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.259+0000 7f1d154c4700 1 --2- 172.21.15.196:0/2654836487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10104370 0x7f1d10104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.259+0000 7f1d154c4700 1 -- 172.21.15.196:0/2654836487 >> 172.21.15.196:0/2654836487 conn(0x7f1d100fe880 msgr2=0x7f1d10100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:19:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.259+0000 7f1d154c4700 1 -- 172.21.15.196:0/2654836487 shutdown_connections 2024-05-09T01:19:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.259+0000 7f1d154c4700 1 -- 172.21.15.196:0/2654836487 wait complete. 2024-05-09T01:19:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d154c4700 1 Processor -- start 2024-05-09T01:19:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d154c4700 1 -- start start 2024-05-09T01:19:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d154c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d10103240 0x7f1d10199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d154c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10104370 0x7f1d10199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d154c4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10104d10 0x7f1d101a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d154c4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1010ecd0 con 0x7f1d10104370 2024-05-09T01:19:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d154c4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1010eb50 con 0x7f1d10103240 2024-05-09T01:19:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d154c4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1010ee50 con 0x7f1d10104d10 2024-05-09T01:19:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10104d10 0x7f1d101a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10104d10 0x7f1d101a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:42074/0 (socket says 172.21.15.196:42074) 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0f7fe700 1 -- 172.21.15.196:0/3342108691 learned_addr learned my addr 172.21.15.196:0/3342108691 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0e7fc700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10104370 0x7f1d10199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0effd700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d10103240 0x7f1d10199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0f7fe700 1 -- 172.21.15.196:0/3342108691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d10103240 msgr2=0x7f1d10199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0f7fe700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d10103240 0x7f1d10199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0f7fe700 1 -- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10104370 msgr2=0x7f1d10199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0f7fe700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10104370 0x7f1d10199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0f7fe700 1 -- 172.21.15.196:0/3342108691 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d101a1350 con 0x7f1d10104d10 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.260+0000 7f1d0effd700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d10103240 0x7f1d10199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.261+0000 7f1d0f7fe700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10104d10 0x7f1d101a0d60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1d08007d70 tx=0x7f1d0800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.261+0000 7f1cf7fff700 1 -- 172.21.15.196:0/3342108691 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08016020 con 0x7f1d10104d10 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.261+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d101a1640 con 0x7f1d10104d10 2024-05-09T01:19:56.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.261+0000 7f1cf7fff700 1 -- 172.21.15.196:0/3342108691 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0800ad40 con 0x7f1d10104d10 2024-05-09T01:19:56.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.261+0000 7f1cf7fff700 1 -- 172.21.15.196:0/3342108691 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d080115e0 con 0x7f1d10104d10 2024-05-09T01:19:56.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.261+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d101a1b30 con 0x7f1d10104d10 2024-05-09T01:19:56.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.262+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d100512c0 con 0x7f1d10104d10 2024-05-09T01:19:56.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.263+0000 7f1cf7fff700 1 -- 172.21.15.196:0/3342108691 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1d08019020 con 0x7f1d10104d10 2024-05-09T01:19:56.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.264+0000 7f1cf7fff700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1cf80b2880 0x7f1cf80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:19:56.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.264+0000 7f1cf7fff700 1 -- 172.21.15.196:0/3342108691 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5397+0+0 (secure 0 0 0) 0x7f1d080e1b50 con 0x7f1d10104d10 2024-05-09T01:19:56.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.264+0000 7f1d0effd700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1cf80b2880 0x7f1cf80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:19:56.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.265+0000 7f1d0effd700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1cf80b2880 0x7f1cf80b4d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d000038a0 tx=0x7f1d0001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:19:56.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.268+0000 7f1cf7fff700 1 -- 172.21.15.196:0/3342108691 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d08020020 con 0x7f1d10104d10 2024-05-09T01:19:56.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:19:56.484+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1d10063c00 con 0x7f1cf80b2880 2024-05-09T01:19:56.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:19:56.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi059 2024-05-09T01:19:56.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:56.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:56.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi059 to 5697M 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:56 smithi059 ceph-mon[34351]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T01:19:56.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:19:56.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi059 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi059 to 5697M 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:56 smithi019 ceph-mon[30158]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi059 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi059 to 5697M 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:56.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:19:56.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:56.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:19:56.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:19:56.697 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:56 smithi196 ceph-mon[35041]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T01:19:57.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:57 smithi059 ceph-mon[34351]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:19:57.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:57 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:19:57.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:57 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:19:57.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:57 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:57.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:57 smithi019 ceph-mon[30158]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:19:57.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:57 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:19:57.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:57 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:19:57.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:57 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:57.681 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:57 smithi196 ceph-mon[35041]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:19:57.681 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:57 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:19:57.681 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:57 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:19:57.682 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:57 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:19:58.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:58 smithi196 ceph-mon[35041]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:19:58.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:58 smithi059 ceph-mon[34351]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:19:58.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:58 smithi019 ceph-mon[30158]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:19:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:59 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/2924118533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9e69b9-50a5-4379-be85-ef221d1b6cf9"}]: dispatch 2024-05-09T01:19:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:59 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9e69b9-50a5-4379-be85-ef221d1b6cf9"}]: dispatch 2024-05-09T01:19:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:59 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a9e69b9-50a5-4379-be85-ef221d1b6cf9"}]': finished 2024-05-09T01:19:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:59 smithi059 ceph-mon[34351]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T01:19:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:19:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:19:59.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:59 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/2924118533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9e69b9-50a5-4379-be85-ef221d1b6cf9"}]: dispatch 2024-05-09T01:19:59.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:59 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9e69b9-50a5-4379-be85-ef221d1b6cf9"}]: dispatch 2024-05-09T01:19:59.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:59 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a9e69b9-50a5-4379-be85-ef221d1b6cf9"}]': finished 2024-05-09T01:19:59.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:59 smithi019 ceph-mon[30158]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T01:19:59.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:19:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:19:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:59 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/2924118533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9e69b9-50a5-4379-be85-ef221d1b6cf9"}]: dispatch 2024-05-09T01:19:59.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:59 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9e69b9-50a5-4379-be85-ef221d1b6cf9"}]: dispatch 2024-05-09T01:19:59.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:59 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a9e69b9-50a5-4379-be85-ef221d1b6cf9"}]': finished 2024-05-09T01:19:59.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:59 smithi196 ceph-mon[35041]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T01:19:59.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:19:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:00.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:00 smithi196 ceph-mon[35041]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:00.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:00 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/793201406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T01:20:00.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:00 smithi196 ceph-mon[35041]: overall HEALTH_OK 2024-05-09T01:20:00.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:00 smithi059 ceph-mon[34351]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:00.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:00 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/793201406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T01:20:00.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:00 smithi059 ceph-mon[34351]: overall HEALTH_OK 2024-05-09T01:20:00.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:00 smithi019 ceph-mon[30158]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:00.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:00 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/793201406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T01:20:00.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:00 smithi019 ceph-mon[30158]: overall HEALTH_OK 2024-05-09T01:20:02.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:02 smithi059 ceph-mon[34351]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 177 KiB/s, 0 objects/s recovering 2024-05-09T01:20:02.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:02 smithi019 ceph-mon[30158]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 177 KiB/s, 0 objects/s recovering 2024-05-09T01:20:02.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:02 smithi196 ceph-mon[35041]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 177 KiB/s, 0 objects/s recovering 2024-05-09T01:20:04.425 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:04 smithi196 ceph-mon[35041]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T01:20:04.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:04 smithi059 ceph-mon[34351]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T01:20:04.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:04 smithi019 ceph-mon[30158]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T01:20:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:06 smithi059 ceph-mon[34351]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-09T01:20:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T01:20:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:06.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:06 smithi019 ceph-mon[30158]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-09T01:20:06.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T01:20:06.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:06 smithi196 ceph-mon[35041]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-09T01:20:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T01:20:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:07.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:07 smithi196 ceph-mon[35041]: Deploying daemon osd.8 on smithi196 2024-05-09T01:20:07.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:07 smithi019 ceph-mon[30158]: Deploying daemon osd.8 on smithi196 2024-05-09T01:20:07.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:07 smithi059 ceph-mon[34351]: Deploying daemon osd.8 on smithi196 2024-05-09T01:20:08.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:08 smithi019 ceph-mon[30158]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T01:20:08.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:08 smithi196 ceph-mon[35041]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T01:20:08.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:08 smithi059 ceph-mon[34351]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T01:20:10.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:10 smithi019 ceph-mon[30158]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T01:20:10.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:10 smithi196 ceph-mon[35041]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T01:20:10.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:10 smithi059 ceph-mon[34351]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T01:20:12.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:12 smithi019 ceph-mon[30158]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T01:20:12.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:12.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:12.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:12 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:12.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:12 smithi196 ceph-mon[35041]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T01:20:12.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:12.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:12.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:12 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:12.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:12 smithi059 ceph-mon[34351]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T01:20:12.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:12 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:12.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:12 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:12.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:12 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:14.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:14 smithi196 ceph-mon[35041]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:14.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:14 smithi019 ceph-mon[30158]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:14.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:14 smithi059 ceph-mon[34351]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:14.898 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-05-09T01:20:14.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.899+0000 7f1cf7fff700 1 -- 172.21.15.196:0/3342108691 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d10063c00 con 0x7f1cf80b2880 2024-05-09T01:20:14.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1cf80b2880 msgr2=0x7f1cf80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:14.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1cf80b2880 0x7f1cf80b4d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d000038a0 tx=0x7f1d0001a040 comp rx=0 tx=0).stop 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10104d10 msgr2=0x7f1d101a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10104d10 0x7f1d101a0d60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1d08007d70 tx=0x7f1d0800a340 comp rx=0 tx=0).stop 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 shutdown_connections 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d10104d10 0x7f1d101a0d60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d10103240 0x7f1d10199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1cf80b2880 0x7f1cf80b4d90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 --2- 172.21.15.196:0/3342108691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10104370 0x7f1d10199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 >> 172.21.15.196:0/3342108691 conn(0x7f1d100fe880 msgr2=0x7f1d101075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 shutdown_connections 2024-05-09T01:20:14.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:20:14.901+0000 7f1d154c4700 1 -- 172.21.15.196:0/3342108691 wait complete. 2024-05-09T01:20:15.553 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.8.service 2024-05-09T01:20:15.555 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-05-09T01:20:15.555 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T01:20:15.823 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:20:16.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:15 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:15 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:15 smithi019 ceph-mon[30158]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:16.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:15 smithi019 ceph-mon[30158]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T01:20:16.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:15 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:15 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:15 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:15 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:15 smithi196 ceph-mon[35041]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:16.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:15 smithi196 ceph-mon[35041]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T01:20:16.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:15 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:15 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:15 smithi059 ceph-mon[34351]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:16.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:15 smithi059 ceph-mon[34351]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T01:20:16.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:16.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:15 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:17.136 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:16 smithi019 ceph-mon[30158]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T01:20:17.136 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:16 smithi019 ceph-mon[30158]: osdmap e54: 9 total, 8 up, 9 in 2024-05-09T01:20:17.136 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:16 smithi019 ceph-mon[30158]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:20:17.136 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:16 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:17.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:16 smithi196 ceph-mon[35041]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T01:20:17.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:16 smithi196 ceph-mon[35041]: osdmap e54: 9 total, 8 up, 9 in 2024-05-09T01:20:17.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:16 smithi196 ceph-mon[35041]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:20:17.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:16 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:17.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:16 smithi059 ceph-mon[34351]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T01:20:17.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:16 smithi059 ceph-mon[34351]: osdmap e54: 9 total, 8 up, 9 in 2024-05-09T01:20:17.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:16 smithi059 ceph-mon[34351]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:20:17.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:16 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:17.870 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:20:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:20:17.768+0000 7f012e032700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T01:20:17.870 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:20:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:20:17.868+0000 7f0125ddf700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:20:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T01:20:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T01:20:17.883 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-09T01:20:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-09T01:20:17.883 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129906 s, 807 MB/s 2024-05-09T01:20:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-09T01:20:18.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:17 smithi019 ceph-mon[30158]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:20:18.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:17 smithi019 ceph-mon[30158]: osdmap e55: 9 total, 8 up, 9 in 2024-05-09T01:20:18.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:17 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:18.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:17 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:18.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:17 smithi019 ceph-mon[30158]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:18.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:17 smithi019 ceph-mon[30158]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' 2024-05-09T01:20:18.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:17 smithi196 ceph-mon[35041]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:20:18.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:17 smithi196 ceph-mon[35041]: osdmap e55: 9 total, 8 up, 9 in 2024-05-09T01:20:18.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:17 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:18.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:17 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:18.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:17 smithi196 ceph-mon[35041]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:18.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:17 smithi196 ceph-mon[35041]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' 2024-05-09T01:20:18.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:17 smithi059 ceph-mon[34351]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:20:18.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:17 smithi059 ceph-mon[34351]: osdmap e55: 9 total, 8 up, 9 in 2024-05-09T01:20:18.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:17 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:18.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:17 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:18.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:17 smithi059 ceph-mon[34351]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-09T01:20:18.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:17 smithi059 ceph-mon[34351]: from='osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882]' entity='osd.8' 2024-05-09T01:20:18.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-05-09T01:20:18.963 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:20:18.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:18 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:20:18.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:18 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:20:18.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:18 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:18.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:18 smithi019 ceph-mon[30158]: osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] boot 2024-05-09T01:20:18.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:18 smithi019 ceph-mon[30158]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T01:20:18.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:18 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:18.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:18 smithi019 ceph-mon[30158]: osdmap e57: 9 total, 9 up, 9 in 2024-05-09T01:20:19.083 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:18 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:20:19.083 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:18 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:20:19.083 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:18 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:19.083 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:18 smithi196 ceph-mon[35041]: osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] boot 2024-05-09T01:20:19.083 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:18 smithi196 ceph-mon[35041]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T01:20:19.083 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:18 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:19.084 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:18 smithi196 ceph-mon[35041]: osdmap e57: 9 total, 9 up, 9 in 2024-05-09T01:20:19.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:18 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:20:19.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:18 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:20:19.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:18 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:19.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:18 smithi059 ceph-mon[34351]: osd.8 [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] boot 2024-05-09T01:20:19.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:18 smithi059 ceph-mon[34351]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T01:20:19.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:18 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T01:20:19.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:18 smithi059 ceph-mon[34351]: osdmap e57: 9 total, 9 up, 9 in 2024-05-09T01:20:20.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:20.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi196 2024-05-09T01:20:20.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi196 to 5696M 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:20 smithi019 ceph-mon[30158]: osdmap e58: 9 total, 9 up, 9 in 2024-05-09T01:20:20.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi196 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi196 to 5696M 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:20.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:20.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:20.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:20 smithi196 ceph-mon[35041]: osdmap e58: 9 total, 9 up, 9 in 2024-05-09T01:20:20.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:20.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi196 2024-05-09T01:20:20.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:20.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:20.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi196 to 5696M 2024-05-09T01:20:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:20.600 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:20.601 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:20 smithi059 ceph-mon[34351]: osdmap e58: 9 total, 9 up, 9 in 2024-05-09T01:20:20.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 -- 172.21.15.19:0/557419161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 msgr2=0x7f9afc104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:20.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 --2- 172.21.15.19:0/557419161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 0x7f9afc104750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b0a0 tx=0x7f9aec00b3b0 comp rx=0 tx=0).stop 2024-05-09T01:20:20.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 -- 172.21.15.19:0/557419161 shutdown_connections 2024-05-09T01:20:20.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 --2- 172.21.15.19:0/557419161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9afc105e10 0x7f9afc10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 --2- 172.21.15.19:0/557419161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 0x7f9afc104750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 --2- 172.21.15.19:0/557419161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9afc105470 0x7f9afc105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 -- 172.21.15.19:0/557419161 >> 172.21.15.19:0/557419161 conn(0x7f9afc0ff9c0 msgr2=0x7f9afc101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 -- 172.21.15.19:0/557419161 shutdown_connections 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 -- 172.21.15.19:0/557419161 wait complete. 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 Processor -- start 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.770+0000 7f9b043af700 1 -- start start 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b043af700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 0x7f9afc077460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b043af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9afc105470 0x7f9afc075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b043af700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9afc105e10 0x7f9afc076050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b043af700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc10fd90 con 0x7f9afc104340 2024-05-09T01:20:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b043af700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc10fc10 con 0x7f9afc105470 2024-05-09T01:20:20.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b043af700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc10ff10 con 0x7f9afc105e10 2024-05-09T01:20:20.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0214b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 0x7f9afc077460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:20:20.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0214b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 0x7f9afc077460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48022/0 (socket says 172.21.15.19:48022) 2024-05-09T01:20:20.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0214b700 1 -- 172.21.15.19:0/904186620 learned_addr learned my addr 172.21.15.19:0/904186620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:20:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0194a700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9afc105470 0x7f9afc075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:20:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0294c700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9afc105e10 0x7f9afc076050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:20:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0214b700 1 -- 172.21.15.19:0/904186620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9afc105e10 msgr2=0x7f9afc076050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0214b700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9afc105e10 0x7f9afc076050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0214b700 1 -- 172.21.15.19:0/904186620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9afc105470 msgr2=0x7f9afc075b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0214b700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9afc105470 0x7f9afc075b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0214b700 1 -- 172.21.15.19:0/904186620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9afc0767b0 con 0x7f9afc104340 2024-05-09T01:20:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0294c700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9afc105e10 0x7f9afc076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:20:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0194a700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9afc105470 0x7f9afc075b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:20:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9b0214b700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 0x7f9afc077460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b780 tx=0x7f9aec004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:20:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.771+0000 7f9afb7fe700 1 -- 172.21.15.19:0/904186620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec021070 con 0x7f9afc104340 2024-05-09T01:20:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.772+0000 7f9afb7fe700 1 -- 172.21.15.19:0/904186620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aec007c80 con 0x7f9afc104340 2024-05-09T01:20:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.772+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9afc1a4a60 con 0x7f9afc104340 2024-05-09T01:20:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.772+0000 7f9afb7fe700 1 -- 172.21.15.19:0/904186620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec005350 con 0x7f9afc104340 2024-05-09T01:20:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.772+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9afc1a4ec0 con 0x7f9afc104340 2024-05-09T01:20:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.773+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9afc0512c0 con 0x7f9afc104340 2024-05-09T01:20:20.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.774+0000 7f9afb7fe700 1 -- 172.21.15.19:0/904186620 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9aec024070 con 0x7f9afc104340 2024-05-09T01:20:20.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.775+0000 7f9afb7fe700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ae40b2880 0x7f9ae40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:20:20.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.775+0000 7f9b0194a700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ae40b2880 0x7f9ae40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:20:20.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.775+0000 7f9afb7fe700 1 -- 172.21.15.19:0/904186620 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5829+0+0 (secure 0 0 0) 0x7f9aec00f020 con 0x7f9afc104340 2024-05-09T01:20:20.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.776+0000 7f9b0194a700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ae40b2880 0x7f9ae40b4d90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9afc076e10 tx=0x7f9af0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:20:20.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:20.779+0000 7f9afb7fe700 1 -- 172.21.15.19:0/904186620 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9aec02b020 con 0x7f9afc104340 2024-05-09T01:20:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:21.003+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9afc063c00 con 0x7f9ae40b2880 2024-05-09T01:20:21.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:20:21.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:20:21.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:21.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:20:21.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:20:21.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:20:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:20:21.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:22.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:22 smithi019 ceph-mon[30158]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:20:22.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:22 smithi019 ceph-mon[30158]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:22.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:22 smithi196 ceph-mon[35041]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:20:22.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:22 smithi196 ceph-mon[35041]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:22 smithi059 ceph-mon[34351]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:20:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:22 smithi059 ceph-mon[34351]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:24 smithi019 ceph-mon[30158]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:24 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1583230082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7437a16-b125-4a63-80f3-987d87d3365d"}]: dispatch 2024-05-09T01:20:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:24 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1583230082' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7437a16-b125-4a63-80f3-987d87d3365d"}]': finished 2024-05-09T01:20:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:24 smithi019 ceph-mon[30158]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T01:20:24.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:24.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:24 smithi196 ceph-mon[35041]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:24.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:24 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1583230082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7437a16-b125-4a63-80f3-987d87d3365d"}]: dispatch 2024-05-09T01:20:24.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:24 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1583230082' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7437a16-b125-4a63-80f3-987d87d3365d"}]': finished 2024-05-09T01:20:24.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:24 smithi196 ceph-mon[35041]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T01:20:24.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:24 smithi059 ceph-mon[34351]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:24 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1583230082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7437a16-b125-4a63-80f3-987d87d3365d"}]: dispatch 2024-05-09T01:20:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:24 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1583230082' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7437a16-b125-4a63-80f3-987d87d3365d"}]': finished 2024-05-09T01:20:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:24 smithi059 ceph-mon[34351]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T01:20:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:25.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:25 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1509857505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T01:20:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:25 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1509857505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T01:20:25.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:25 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1509857505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T01:20:26.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:26 smithi196 ceph-mon[35041]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-09T01:20:26.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:26 smithi059 ceph-mon[34351]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-09T01:20:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:26 smithi019 ceph-mon[30158]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-09T01:20:28.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:28 smithi196 ceph-mon[35041]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T01:20:28.566 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:28 smithi019 ceph-mon[30158]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T01:20:28.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:28 smithi059 ceph-mon[34351]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T01:20:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:30 smithi019 ceph-mon[30158]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-09T01:20:30.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:30 smithi196 ceph-mon[35041]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-09T01:20:30.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:30 smithi059 ceph-mon[34351]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-09T01:20:32.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:32 smithi196 ceph-mon[35041]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:20:32.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T01:20:32.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:32.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:32 smithi019 ceph-mon[30158]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:20:32.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T01:20:32.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:32.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:32 smithi059 ceph-mon[34351]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:20:32.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T01:20:32.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:33.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:33 smithi196 ceph-mon[35041]: Deploying daemon osd.9 on smithi019 2024-05-09T01:20:33.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:33 smithi019 ceph-mon[30158]: Deploying daemon osd.9 on smithi019 2024-05-09T01:20:33.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:33 smithi059 ceph-mon[34351]: Deploying daemon osd.9 on smithi019 2024-05-09T01:20:34.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:34 smithi019 ceph-mon[30158]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:20:34.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:34 smithi196 ceph-mon[35041]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:20:34.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:34 smithi059 ceph-mon[34351]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:20:36.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:36 smithi019 ceph-mon[30158]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-09T01:20:36.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:36 smithi196 ceph-mon[35041]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-09T01:20:36.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:36 smithi059 ceph-mon[34351]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-09T01:20:38.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:38 smithi196 ceph-mon[35041]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:38.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:38.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:38.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:38.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:38 smithi019 ceph-mon[30158]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:38.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:38.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:38.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:38.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:38 smithi059 ceph-mon[34351]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:38.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:38.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:38.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:40.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:40 smithi196 ceph-mon[35041]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:40.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:40 smithi059 ceph-mon[34351]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:40.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:40 smithi019 ceph-mon[30158]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:41.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.479+0000 7f9afb7fe700 1 -- 172.21.15.19:0/904186620 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9afc063c00 con 0x7f9ae40b2880 2024-05-09T01:20:41.479 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-05-09T01:20:41.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.481+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ae40b2880 msgr2=0x7f9ae40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:41.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.481+0000 7f9b043af700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ae40b2880 0x7f9ae40b4d90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9afc076e10 tx=0x7f9af0009260 comp rx=0 tx=0).stop 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.481+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 msgr2=0x7f9afc077460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.481+0000 7f9b043af700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 0x7f9afc077460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b780 tx=0x7f9aec004e20 comp rx=0 tx=0).stop 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.482+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 shutdown_connections 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.482+0000 7f9b043af700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9afc105e10 0x7f9afc076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.482+0000 7f9b043af700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ae40b2880 0x7f9ae40b4d90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.482+0000 7f9b043af700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9afc104340 0x7f9afc077460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.482+0000 7f9b043af700 1 --2- 172.21.15.19:0/904186620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9afc105470 0x7f9afc075b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.482+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 >> 172.21.15.19:0/904186620 conn(0x7f9afc0ff9c0 msgr2=0x7f9afc1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.482+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 shutdown_connections 2024-05-09T01:20:41.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:20:41.482+0000 7f9b043af700 1 -- 172.21.15.19:0/904186620 wait complete. 2024-05-09T01:20:42.138 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.9.service 2024-05-09T01:20:42.140 INFO:tasks.cephadm:Deploying osd.10 on smithi059 with /dev/vg_nvme/lv_1... 2024-05-09T01:20:42.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T01:20:42.407 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:20:42.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:42 smithi019 ceph-mon[30158]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:42.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:42 smithi019 ceph-mon[30158]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T01:20:42.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:42 smithi059 ceph-mon[34351]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:42.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.676 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:42 smithi059 ceph-mon[34351]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T01:20:42.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:42 smithi196 ceph-mon[35041]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:42.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:42.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:42 smithi196 ceph-mon[35041]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T01:20:43.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:43 smithi019 ceph-mon[30158]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T01:20:43.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:43 smithi019 ceph-mon[30158]: osdmap e60: 10 total, 9 up, 10 in 2024-05-09T01:20:43.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:43 smithi019 ceph-mon[30158]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:20:43.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:43 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:43.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:43 smithi019 ceph-mon[30158]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:43.609 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:20:43 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:20:43.575+0000 7fef2aae1700 -1 osd.9 0 waiting for initial osdmap 2024-05-09T01:20:43.609 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:20:43 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:20:43.589+0000 7fef26896700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:20:43.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:43 smithi059 ceph-mon[34351]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T01:20:43.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:43 smithi059 ceph-mon[34351]: osdmap e60: 10 total, 9 up, 10 in 2024-05-09T01:20:43.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:43 smithi059 ceph-mon[34351]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:20:43.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:43 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:43.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:43 smithi059 ceph-mon[34351]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:43.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:43 smithi196 ceph-mon[35041]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T01:20:43.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:43 smithi196 ceph-mon[35041]: osdmap e60: 10 total, 9 up, 10 in 2024-05-09T01:20:43.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:43 smithi196 ceph-mon[35041]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T01:20:43.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:43 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:43.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:43 smithi196 ceph-mon[35041]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T01:20:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T01:20:44.436 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-09T01:20:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-09T01:20:44.436 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118508 s, 885 MB/s 2024-05-09T01:20:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-09T01:20:44.753 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:44 smithi059 ceph-mon[34351]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:20:44.753 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:44 smithi059 ceph-mon[34351]: osdmap e61: 10 total, 9 up, 10 in 2024-05-09T01:20:44.753 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:44 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:44.754 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:44 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:44.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:44 smithi019 ceph-mon[30158]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:20:44.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:44 smithi019 ceph-mon[30158]: osdmap e61: 10 total, 9 up, 10 in 2024-05-09T01:20:44.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:44.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:44.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:44 smithi196 ceph-mon[35041]: from='osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T01:20:44.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:44 smithi196 ceph-mon[35041]: osdmap e61: 10 total, 9 up, 10 in 2024-05-09T01:20:44.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:44 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:44.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:44 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:45.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_1 2024-05-09T01:20:45.392 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:20:45.771 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] boot 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: osdmap e62: 10 total, 10 up, 10 in 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi019 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.772 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-09T01:20:45.773 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.773 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:45.773 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:45.773 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:20:45.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:20:45.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] boot 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: osdmap e62: 10 total, 10 up, 10 in 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi019 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:45.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:45.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: osd.9 [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] boot 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: osdmap e62: 10 total, 10 up, 10 in 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi019 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:20:45.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-09T01:20:45.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:45.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:45.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:45.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:46.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:46 smithi059 ceph-mon[34351]: osdmap e63: 10 total, 10 up, 10 in 2024-05-09T01:20:46.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:46 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:46.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:46 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:46.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:46 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:46.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:46 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:46.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:46 smithi019 ceph-mon[30158]: osdmap e63: 10 total, 10 up, 10 in 2024-05-09T01:20:46.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:46 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:46.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:46 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:46.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:46 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:46.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:46 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:46.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:46 smithi196 ceph-mon[35041]: osdmap e63: 10 total, 10 up, 10 in 2024-05-09T01:20:46.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:46 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:20:46.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:46 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:46.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:46 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:20:46.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:46 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:20:47.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.130+0000 7f16081da700 1 -- 172.21.15.59:0/3745799574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104d20 msgr2=0x7f160010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:47.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.130+0000 7f16081da700 1 --2- 172.21.15.59:0/3745799574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104d20 0x7f160010b5d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b0a0 tx=0x7f15f800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:20:47.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.131+0000 7f16081da700 1 -- 172.21.15.59:0/3745799574 shutdown_connections 2024-05-09T01:20:47.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.131+0000 7f16081da700 1 --2- 172.21.15.59:0/3745799574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104d20 0x7f160010b5d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:47.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.131+0000 7f16081da700 1 --2- 172.21.15.59:0/3745799574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600103250 0x7f1600103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:47.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.131+0000 7f16081da700 1 --2- 172.21.15.59:0/3745799574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1600104380 0x7f1600104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:47.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.131+0000 7f16081da700 1 -- 172.21.15.59:0/3745799574 >> 172.21.15.59:0/3745799574 conn(0x7f16000fe890 msgr2=0x7f1600100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:20:47.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.131+0000 7f16081da700 1 -- 172.21.15.59:0/3745799574 shutdown_connections 2024-05-09T01:20:47.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.131+0000 7f16081da700 1 -- 172.21.15.59:0/3745799574 wait complete. 2024-05-09T01:20:47.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.131+0000 7f16081da700 1 Processor -- start 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.131+0000 7f16081da700 1 -- start start 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f16081da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600103250 0x7f160019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f16081da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104380 0x7f160019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f16081da700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1600104d20 0x7f16001a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f16081da700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f160010ed10 con 0x7f1600103250 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f16081da700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f160010eb90 con 0x7f1600104380 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f16081da700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f160010ee90 con 0x7f1600104d20 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605775700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104380 0x7f160019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605775700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104380 0x7f160019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40718/0 (socket says 172.21.15.59:40718) 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605775700 1 -- 172.21.15.59:0/2036140419 learned_addr learned my addr 172.21.15.59:0/2036140419 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605f76700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600103250 0x7f160019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1606777700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1600104d20 0x7f16001a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605775700 1 -- 172.21.15.59:0/2036140419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1600104d20 msgr2=0x7f16001a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:47.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605775700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1600104d20 0x7f16001a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605775700 1 -- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600103250 msgr2=0x7f160019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605775700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600103250 0x7f160019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605775700 1 -- 172.21.15.59:0/2036140419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16001055a0 con 0x7f1600104380 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f1605775700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104380 0x7f160019ae70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f15f000d950 tx=0x7f15f000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f15f6ffd700 1 -- 172.21.15.59:0/2036140419 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0016070 con 0x7f1600104380 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f15f6ffd700 1 -- 172.21.15.59:0/2036140419 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f000fd40 con 0x7f1600104380 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.132+0000 7f15f6ffd700 1 -- 172.21.15.59:0/2036140419 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f000e3b0 con 0x7f1600104380 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.133+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1600105890 con 0x7f1600104380 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.133+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1600105d80 con 0x7f1600104380 2024-05-09T01:20:47.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.134+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16000512c0 con 0x7f1600104380 2024-05-09T01:20:47.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.135+0000 7f15f6ffd700 1 -- 172.21.15.59:0/2036140419 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f15f0019070 con 0x7f1600104380 2024-05-09T01:20:47.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.136+0000 7f15f6ffd700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f15ec0b2900 0x7f15ec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:20:47.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.136+0000 7f15f6ffd700 1 -- 172.21.15.59:0/2036140419 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6261+0+0 (secure 0 0 0) 0x7f15f00e13e0 con 0x7f1600104380 2024-05-09T01:20:47.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.136+0000 7f1605f76700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f15ec0b2900 0x7f15ec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:20:47.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.137+0000 7f1605f76700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f15ec0b2900 0x7f15ec0b4e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1600104280 tx=0x7f15fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:20:47.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.140+0000 7f15f6ffd700 1 -- 172.21.15.59:0/2036140419 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15f00a9900 con 0x7f1600104380 2024-05-09T01:20:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:20:47.364+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1600100870 con 0x7f15ec0b2900 2024-05-09T01:20:47.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:47 smithi059 ceph-mon[34351]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:47.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:47 smithi059 ceph-mon[34351]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:20:47.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:20:47.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:20:47.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:47.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:47 smithi019 ceph-mon[30158]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:47.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:47 smithi019 ceph-mon[30158]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:20:47.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:20:47.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:20:47.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:47.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:47 smithi196 ceph-mon[35041]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:47.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:47 smithi196 ceph-mon[35041]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:20:47.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:20:47.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:20:47.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:50.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:50 smithi196 ceph-mon[35041]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:50.585 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:50 smithi059 ceph-mon[34351]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:50.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:50 smithi019 ceph-mon[30158]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:51.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:51 smithi196 ceph-mon[35041]: from='client.? 172.21.15.59:0/1735238182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56aa036a-6771-45e1-80a9-eb0fbee10f5e"}]: dispatch 2024-05-09T01:20:51.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:51 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56aa036a-6771-45e1-80a9-eb0fbee10f5e"}]: dispatch 2024-05-09T01:20:51.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:51 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56aa036a-6771-45e1-80a9-eb0fbee10f5e"}]': finished 2024-05-09T01:20:51.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:51 smithi196 ceph-mon[35041]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T01:20:51.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:20:51.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:51 smithi196 ceph-mon[35041]: from='client.? 172.21.15.59:0/2406331074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T01:20:51.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:51 smithi059 ceph-mon[34351]: from='client.? 172.21.15.59:0/1735238182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56aa036a-6771-45e1-80a9-eb0fbee10f5e"}]: dispatch 2024-05-09T01:20:51.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:51 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56aa036a-6771-45e1-80a9-eb0fbee10f5e"}]: dispatch 2024-05-09T01:20:51.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:51 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56aa036a-6771-45e1-80a9-eb0fbee10f5e"}]': finished 2024-05-09T01:20:51.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:51 smithi059 ceph-mon[34351]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T01:20:51.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:20:51.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:51 smithi059 ceph-mon[34351]: from='client.? 172.21.15.59:0/2406331074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T01:20:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:51 smithi019 ceph-mon[30158]: from='client.? 172.21.15.59:0/1735238182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56aa036a-6771-45e1-80a9-eb0fbee10f5e"}]: dispatch 2024-05-09T01:20:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:51 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56aa036a-6771-45e1-80a9-eb0fbee10f5e"}]: dispatch 2024-05-09T01:20:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:51 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56aa036a-6771-45e1-80a9-eb0fbee10f5e"}]': finished 2024-05-09T01:20:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:51 smithi019 ceph-mon[30158]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T01:20:51.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:20:51.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:51 smithi019 ceph-mon[30158]: from='client.? 172.21.15.59:0/2406331074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T01:20:52.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:52 smithi059 ceph-mon[34351]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:52 smithi019 ceph-mon[30158]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:52.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:52 smithi196 ceph-mon[35041]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:54.591 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:54 smithi059 ceph-mon[34351]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:54.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:54 smithi019 ceph-mon[30158]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:54.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:54 smithi196 ceph-mon[35041]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:56.582 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:56 smithi059 ceph-mon[34351]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:56.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:56 smithi019 ceph-mon[30158]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:56.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:56 smithi196 ceph-mon[35041]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:58.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:58 smithi059 ceph-mon[34351]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:58.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T01:20:58.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:58.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:58 smithi019 ceph-mon[30158]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:58.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T01:20:58.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:58 smithi196 ceph-mon[35041]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:20:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T01:20:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:20:59.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:20:59 smithi059 ceph-mon[34351]: Deploying daemon osd.10 on smithi059 2024-05-09T01:20:59.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:20:59 smithi019 ceph-mon[30158]: Deploying daemon osd.10 on smithi059 2024-05-09T01:20:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:20:59 smithi196 ceph-mon[35041]: Deploying daemon osd.10 on smithi059 2024-05-09T01:21:00.339 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:00 smithi059 ceph-mon[34351]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:00.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:00 smithi019 ceph-mon[30158]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:00.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:00 smithi196 ceph-mon[35041]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:02.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:02 smithi059 ceph-mon[34351]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:02.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:02 smithi019 ceph-mon[30158]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:02.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:02 smithi196 ceph-mon[35041]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:04.552 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:04 smithi059 ceph-mon[34351]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:04.552 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:04.552 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:04.552 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:04.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:04 smithi019 ceph-mon[30158]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:04.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:04.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:04.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:04.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:04 smithi196 ceph-mon[35041]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:04.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:04.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:04.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:06.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:06 smithi059 ceph-mon[34351]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:06.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:06 smithi019 ceph-mon[30158]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:06.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:06 smithi196 ceph-mon[35041]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.437+0000 7f15f6ffd700 1 -- 172.21.15.59:0/2036140419 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1600100870 con 0x7f15ec0b2900 2024-05-09T01:21:07.437 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 10 on host 'smithi059' 2024-05-09T01:21:07.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f15ec0b2900 msgr2=0x7f15ec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:07.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f15ec0b2900 0x7f15ec0b4e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1600104280 tx=0x7f15fc009260 comp rx=0 tx=0).stop 2024-05-09T01:21:07.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104380 msgr2=0x7f160019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:07.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104380 0x7f160019ae70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f15f000d950 tx=0x7f15f000dc60 comp rx=0 tx=0).stop 2024-05-09T01:21:07.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 shutdown_connections 2024-05-09T01:21:07.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1600104d20 0x7f16001a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:07.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f15ec0b2900 0x7f15ec0b4e10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:07.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600103250 0x7f160019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:07.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 --2- 172.21.15.59:0/2036140419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1600104380 0x7f160019ae70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:07.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 >> 172.21.15.59:0/2036140419 conn(0x7f16000fe890 msgr2=0x7f1600100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:07.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 shutdown_connections 2024-05-09T01:21:07.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T01:21:07.441+0000 7f16081da700 1 -- 172.21.15.59:0/2036140419 wait complete. 2024-05-09T01:21:08.118 DEBUG:teuthology.orchestra.run.smithi059:osd.10> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.10.service 2024-05-09T01:21:08.121 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-05-09T01:21:08.121 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T01:21:08.365 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:21:08.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:08 smithi196 ceph-mon[35041]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:08.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:08 smithi196 ceph-mon[35041]: from='osd.10 [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T01:21:08.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:08 smithi196 ceph-mon[35041]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T01:21:08.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:08 smithi059 ceph-mon[34351]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:08.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:08 smithi059 ceph-mon[34351]: from='osd.10 [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T01:21:08.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:08 smithi059 ceph-mon[34351]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T01:21:08.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:08 smithi019 ceph-mon[30158]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:08.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:08.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:08 smithi019 ceph-mon[30158]: from='osd.10 [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T01:21:08.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:08 smithi019 ceph-mon[30158]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T01:21:09.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:09 smithi196 ceph-mon[35041]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T01:21:09.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:09 smithi196 ceph-mon[35041]: osdmap e65: 11 total, 10 up, 11 in 2024-05-09T01:21:09.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:09 smithi196 ceph-mon[35041]: from='osd.10 [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:21:09.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:09.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:09 smithi196 ceph-mon[35041]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:21:09.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:09 smithi059 ceph-mon[34351]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T01:21:09.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:09 smithi059 ceph-mon[34351]: osdmap e65: 11 total, 10 up, 11 in 2024-05-09T01:21:09.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:09 smithi059 ceph-mon[34351]: from='osd.10 [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:21:09.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:09.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:09 smithi059 ceph-mon[34351]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:21:09.848 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:21:09 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:21:09.520+0000 7f94c62e8700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T01:21:09.848 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:21:09 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:21:09.537+0000 7f94c209d700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:21:09.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:09 smithi019 ceph-mon[30158]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T01:21:09.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:09 smithi019 ceph-mon[30158]: osdmap e65: 11 total, 10 up, 11 in 2024-05-09T01:21:09.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:09 smithi019 ceph-mon[30158]: from='osd.10 [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:21:09.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:09.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:09 smithi019 ceph-mon[30158]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T01:21:10.296 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T01:21:10.296 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T01:21:10.296 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-09T01:21:10.296 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-09T01:21:10.296 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125284 s, 837 MB/s 2024-05-09T01:21:10.296 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-09T01:21:10.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:10 smithi059 ceph-mon[34351]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:10.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:10 smithi059 ceph-mon[34351]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:21:10.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:10 smithi059 ceph-mon[34351]: osdmap e66: 11 total, 10 up, 11 in 2024-05-09T01:21:10.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:10 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:10.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:10 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:10.869 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:10 smithi196 ceph-mon[35041]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:10.869 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:10 smithi196 ceph-mon[35041]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:21:10.869 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:10 smithi196 ceph-mon[35041]: osdmap e66: 11 total, 10 up, 11 in 2024-05-09T01:21:10.869 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:10 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:10.869 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:10 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:10.898 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-05-09T01:21:10.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:10 smithi019 ceph-mon[30158]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-09T01:21:10.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:10 smithi019 ceph-mon[30158]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T01:21:10.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:10 smithi019 ceph-mon[30158]: osdmap e66: 11 total, 10 up, 11 in 2024-05-09T01:21:10.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:10 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:10.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:10 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:11.138 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:21:11.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:21:11.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:21:11.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: osd.10 [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] boot 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:11.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:11.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:11.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: osd.10 [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] boot 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:11.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:11.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:11.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:11.849 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:21:11.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:21:11.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: osd.10 [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] boot 2024-05-09T01:21:11.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T01:21:11.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T01:21:11.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:11.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:12.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.678+0000 7f6055215700 1 -- 172.21.15.196:0/3599896110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 msgr2=0x7f605010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:12.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.678+0000 7f6055215700 1 --2- 172.21.15.196:0/3599896110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 0x7f605010b5d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f604000b0a0 tx=0x7f604000b3b0 comp rx=0 tx=0).stop 2024-05-09T01:21:12.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.678+0000 7f6055215700 1 -- 172.21.15.196:0/3599896110 shutdown_connections 2024-05-09T01:21:12.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.678+0000 7f6055215700 1 --2- 172.21.15.196:0/3599896110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 0x7f605010b5d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:12.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.678+0000 7f6055215700 1 --2- 172.21.15.196:0/3599896110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6050103250 0x7f6050103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:12.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.678+0000 7f6055215700 1 --2- 172.21.15.196:0/3599896110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6050104380 0x7f6050104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:12.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.678+0000 7f6055215700 1 -- 172.21.15.196:0/3599896110 >> 172.21.15.196:0/3599896110 conn(0x7f60500fe890 msgr2=0x7f6050100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:12.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.678+0000 7f6055215700 1 -- 172.21.15.196:0/3599896110 shutdown_connections 2024-05-09T01:21:12.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.678+0000 7f6055215700 1 -- 172.21.15.196:0/3599896110 wait complete. 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f6055215700 1 Processor -- start 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f6055215700 1 -- start start 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f6055215700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6050103250 0x7f605019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f6055215700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6050104380 0x7f605019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f6055215700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 0x7f60501a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f6055215700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f605010ecb0 con 0x7f6050104380 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f6055215700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f605010eb30 con 0x7f6050103250 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f6055215700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f605010ee30 con 0x7f6050104d20 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f604f59e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 0x7f60501a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f604f59e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 0x7f60501a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59370/0 (socket says 172.21.15.196:59370) 2024-05-09T01:21:12.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f604f59e700 1 -- 172.21.15.196:0/3337159821 learned_addr learned my addr 172.21.15.196:0/3337159821 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.679+0000 7f604e59c700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6050104380 0x7f605019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f604ed9d700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6050103250 0x7f605019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f604f59e700 1 -- 172.21.15.196:0/3337159821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6050103250 msgr2=0x7f605019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f604f59e700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6050103250 0x7f605019a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f604f59e700 1 -- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6050104380 msgr2=0x7f605019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f604f59e700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6050104380 0x7f605019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f604f59e700 1 -- 172.21.15.196:0/3337159821 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60501a23e0 con 0x7f6050104d20 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f604f59e700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 0x7f60501a1d60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60400062a0 tx=0x7f6040004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f6037fff700 1 -- 172.21.15.196:0/3337159821 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040021070 con 0x7f6050104d20 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f6037fff700 1 -- 172.21.15.196:0/3337159821 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6040007c80 con 0x7f6050104d20 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60501a2670 con 0x7f6050104d20 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f6037fff700 1 -- 172.21.15.196:0/3337159821 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040004850 con 0x7f6050104d20 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.680+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60501a2bb0 con 0x7f6050104d20 2024-05-09T01:21:12.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.681+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60500512c0 con 0x7f6050104d20 2024-05-09T01:21:12.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.683+0000 7f6037fff700 1 -- 172.21.15.196:0/3337159821 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6040003a00 con 0x7f6050104d20 2024-05-09T01:21:12.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.684+0000 7f6037fff700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60380b2880 0x7f60380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:12.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.684+0000 7f604ed9d700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60380b2880 0x7f60380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:12.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.684+0000 7f6037fff700 1 -- 172.21.15.196:0/3337159821 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6694+0+0 (secure 0 0 0) 0x7f604000f020 con 0x7f6050104d20 2024-05-09T01:21:12.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.684+0000 7f604ed9d700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60380b2880 0x7f60380b4d90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6048005950 tx=0x7f60480058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:12.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.688+0000 7f6037fff700 1 -- 172.21.15.196:0/3337159821 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f604002a020 con 0x7f6050104d20 2024-05-09T01:21:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:12 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi059 2024-05-09T01:21:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:12 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi059 to 4273M 2024-05-09T01:21:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:12 smithi019 ceph-mon[30158]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:12 smithi019 ceph-mon[30158]: osdmap e68: 11 total, 11 up, 11 in 2024-05-09T01:21:12.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:12 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi059 2024-05-09T01:21:12.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:12 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi059 to 4273M 2024-05-09T01:21:12.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:12 smithi059 ceph-mon[34351]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:12.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:12 smithi059 ceph-mon[34351]: osdmap e68: 11 total, 11 up, 11 in 2024-05-09T01:21:12.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:12 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi059 2024-05-09T01:21:12.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:12 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi059 to 4273M 2024-05-09T01:21:12.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:12 smithi196 ceph-mon[35041]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:12.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:12 smithi196 ceph-mon[35041]: osdmap e68: 11 total, 11 up, 11 in 2024-05-09T01:21:12.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:12.900+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6050100870 con 0x7f60380b2880 2024-05-09T01:21:13.756 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:13 smithi196 ceph-mon[35041]: osdmap e69: 11 total, 11 up, 11 in 2024-05-09T01:21:13.756 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:13 smithi196 ceph-mon[35041]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:21:13.756 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:21:13.756 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:21:13.756 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:13.756 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:13 smithi196 ceph-mon[35041]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:13.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:13 smithi059 ceph-mon[34351]: osdmap e69: 11 total, 11 up, 11 in 2024-05-09T01:21:13.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:13 smithi059 ceph-mon[34351]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:21:13.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:21:13.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:21:13.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:13.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:13 smithi059 ceph-mon[34351]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:13.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:13 smithi019 ceph-mon[30158]: osdmap e69: 11 total, 11 up, 11 in 2024-05-09T01:21:13.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:13 smithi019 ceph-mon[30158]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:21:13.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T01:21:13.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T01:21:13.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:13.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:13 smithi019 ceph-mon[30158]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:16.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:16 smithi196 ceph-mon[35041]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-09T01:21:16.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:16 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/2568763552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b4010a5-eb86-4f92-93de-4d7733ff5ec2"}]: dispatch 2024-05-09T01:21:16.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:16 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b4010a5-eb86-4f92-93de-4d7733ff5ec2"}]: dispatch 2024-05-09T01:21:16.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:16 smithi196 ceph-mon[35041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b4010a5-eb86-4f92-93de-4d7733ff5ec2"}]': finished 2024-05-09T01:21:16.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:16 smithi196 ceph-mon[35041]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T01:21:16.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:16 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:16.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:16 smithi059 ceph-mon[34351]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-09T01:21:16.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:16 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/2568763552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b4010a5-eb86-4f92-93de-4d7733ff5ec2"}]: dispatch 2024-05-09T01:21:16.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:16 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b4010a5-eb86-4f92-93de-4d7733ff5ec2"}]: dispatch 2024-05-09T01:21:16.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:16 smithi059 ceph-mon[34351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b4010a5-eb86-4f92-93de-4d7733ff5ec2"}]': finished 2024-05-09T01:21:16.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:16 smithi059 ceph-mon[34351]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T01:21:16.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:16 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:16.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:16 smithi019 ceph-mon[30158]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-09T01:21:16.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:16 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/2568763552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b4010a5-eb86-4f92-93de-4d7733ff5ec2"}]: dispatch 2024-05-09T01:21:16.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:16 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b4010a5-eb86-4f92-93de-4d7733ff5ec2"}]: dispatch 2024-05-09T01:21:16.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:16 smithi019 ceph-mon[30158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b4010a5-eb86-4f92-93de-4d7733ff5ec2"}]': finished 2024-05-09T01:21:16.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:16 smithi019 ceph-mon[30158]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T01:21:16.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:16 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:17.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:17 smithi196 ceph-mon[35041]: from='client.? 172.21.15.196:0/661335899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T01:21:17.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:17 smithi059 ceph-mon[34351]: from='client.? 172.21.15.196:0/661335899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T01:21:17.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:17 smithi019 ceph-mon[30158]: from='client.? 172.21.15.196:0/661335899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T01:21:18.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:18 smithi196 ceph-mon[35041]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T01:21:18.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:18 smithi059 ceph-mon[34351]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T01:21:18.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:18 smithi019 ceph-mon[30158]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T01:21:20.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:20 smithi196 ceph-mon[35041]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-09T01:21:20.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:20 smithi059 ceph-mon[34351]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-09T01:21:20.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:20 smithi019 ceph-mon[30158]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-09T01:21:22.427 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:22 smithi196 ceph-mon[35041]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-09T01:21:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:22 smithi059 ceph-mon[34351]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-09T01:21:22.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:22 smithi019 ceph-mon[30158]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-09T01:21:23.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:23 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T01:21:23.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:23 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:23.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:23 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T01:21:23.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:23 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:23.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:23 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T01:21:23.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:23 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:24.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:24 smithi196 ceph-mon[35041]: Deploying daemon osd.11 on smithi196 2024-05-09T01:21:24.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:24 smithi196 ceph-mon[35041]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:21:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:24 smithi019 ceph-mon[30158]: Deploying daemon osd.11 on smithi196 2024-05-09T01:21:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:24 smithi019 ceph-mon[30158]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:21:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:24 smithi059 ceph-mon[34351]: Deploying daemon osd.11 on smithi196 2024-05-09T01:21:24.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:24 smithi059 ceph-mon[34351]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T01:21:26.319 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:26 smithi196 ceph-mon[35041]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:26.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:26 smithi059 ceph-mon[34351]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:26 smithi019 ceph-mon[30158]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:28.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:28 smithi196 ceph-mon[35041]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:28.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:28 smithi059 ceph-mon[34351]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:28.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:28 smithi019 ceph-mon[30158]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:29.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:29.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:29.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:29.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:29 smithi196 ceph-mon[35041]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:30.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:30.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:30.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:30.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:29 smithi059 ceph-mon[34351]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:30.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:30.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:30.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:30.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:29 smithi019 ceph-mon[30158]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:32.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.066+0000 7f6037fff700 1 -- 172.21.15.196:0/3337159821 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6050100870 con 0x7f60380b2880 2024-05-09T01:21:32.065 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-05-09T01:21:32.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60380b2880 msgr2=0x7f60380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60380b2880 0x7f60380b4d90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6048005950 tx=0x7f60480058e0 comp rx=0 tx=0).stop 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 msgr2=0x7f60501a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 0x7f60501a1d60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60400062a0 tx=0x7f6040004e40 comp rx=0 tx=0).stop 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 shutdown_connections 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050104d20 0x7f60501a1d60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6050103250 0x7f605019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60380b2880 0x7f60380b4d90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 --2- 172.21.15.196:0/3337159821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6050104380 0x7f605019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.068+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 >> 172.21.15.196:0/3337159821 conn(0x7f60500fe890 msgr2=0x7f6050100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.069+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 shutdown_connections 2024-05-09T01:21:32.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:32.069+0000 7f6055215700 1 -- 172.21.15.196:0/3337159821 wait complete. 2024-05-09T01:21:32.375 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:32 smithi196 ceph-mon[35041]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:32.375 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:32.375 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:32.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:32 smithi059 ceph-mon[34351]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:32.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:32.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:32 smithi019 ceph-mon[30158]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:32.686 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.11.service 2024-05-09T01:21:32.688 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T01:21:32.689 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T01:21:32.951 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:21:33.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:33 smithi019 ceph-mon[30158]: from='osd.11 [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T01:21:33.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:33 smithi019 ceph-mon[30158]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T01:21:33.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:33 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:33.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:33 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:33.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:33 smithi196 ceph-mon[35041]: from='osd.11 [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T01:21:33.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:33 smithi196 ceph-mon[35041]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T01:21:33.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:33 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:33.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:33 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:33.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:33 smithi059 ceph-mon[34351]: from='osd.11 [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T01:21:33.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:33 smithi059 ceph-mon[34351]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T01:21:33.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:33 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:33.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:33 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:34.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:34 smithi019 ceph-mon[30158]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:34.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:34 smithi019 ceph-mon[30158]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T01:21:34.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:34 smithi019 ceph-mon[30158]: from='osd.11 [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:21:34.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:34 smithi019 ceph-mon[30158]: osdmap e71: 12 total, 11 up, 12 in 2024-05-09T01:21:34.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:34 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:34.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:34 smithi019 ceph-mon[30158]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:21:34.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:34 smithi196 ceph-mon[35041]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:34 smithi196 ceph-mon[35041]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T01:21:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:34 smithi196 ceph-mon[35041]: from='osd.11 [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:21:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:34 smithi196 ceph-mon[35041]: osdmap e71: 12 total, 11 up, 12 in 2024-05-09T01:21:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:34 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:34.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:34 smithi196 ceph-mon[35041]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:21:34.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:34 smithi059 ceph-mon[34351]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:34.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:34 smithi059 ceph-mon[34351]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T01:21:34.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:34 smithi059 ceph-mon[34351]: from='osd.11 [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:21:34.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:34 smithi059 ceph-mon[34351]: osdmap e71: 12 total, 11 up, 12 in 2024-05-09T01:21:34.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:34 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:34.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:34 smithi059 ceph-mon[34351]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T01:21:34.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.619+0000 7f228de22700 1 -- 172.21.15.19:0/3227974610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 msgr2=0x7f2288104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:34.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.619+0000 7f228de22700 1 --2- 172.21.15.19:0/3227974610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 0x7f2288104660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f227c009990 tx=0x7f227c009ca0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.619+0000 7f228de22700 1 -- 172.21.15.19:0/3227974610 shutdown_connections 2024-05-09T01:21:34.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.619+0000 7f228de22700 1 --2- 172.21.15.19:0/3227974610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2288104cf0 0x7f228810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.619+0000 7f228de22700 1 --2- 172.21.15.19:0/3227974610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288103810 0x7f2288103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.619+0000 7f228de22700 1 --2- 172.21.15.19:0/3227974610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 0x7f2288104660 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.619+0000 7f228de22700 1 -- 172.21.15.19:0/3227974610 >> 172.21.15.19:0/3227974610 conn(0x7f22880fe020 msgr2=0x7f22881004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:34.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.619+0000 7f228de22700 1 -- 172.21.15.19:0/3227974610 shutdown_connections 2024-05-09T01:21:34.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.619+0000 7f228de22700 1 -- 172.21.15.19:0/3227974610 wait complete. 2024-05-09T01:21:34.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.620+0000 7f228de22700 1 Processor -- start 2024-05-09T01:21:34.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.620+0000 7f228de22700 1 -- start start 2024-05-09T01:21:34.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f228de22700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288103810 0x7f2288102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:34.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f228de22700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 0x7f228819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:34.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f228de22700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2288104cf0 0x7f22881a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:34.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f228de22700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f228810ecd0 con 0x7f22881041f0 2024-05-09T01:21:34.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f228de22700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f228810eb50 con 0x7f2288104cf0 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f228de22700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f228810ee50 con 0x7f2288103810 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2286ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 0x7f228819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f22877fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288103810 0x7f2288102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2286ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 0x7f228819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46532/0 (socket says 172.21.15.19:46532) 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2286ffd700 1 -- 172.21.15.19:0/2060769901 learned_addr learned my addr 172.21.15.19:0/2060769901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2287fff700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2288104cf0 0x7f22881a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2286ffd700 1 -- 172.21.15.19:0/2060769901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288103810 msgr2=0x7f2288102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2286ffd700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288103810 0x7f2288102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2286ffd700 1 -- 172.21.15.19:0/2060769901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2288104cf0 msgr2=0x7f22881a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2286ffd700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2288104cf0 0x7f22881a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2286ffd700 1 -- 172.21.15.19:0/2060769901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22881a2450 con 0x7f22881041f0 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2286ffd700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 0x7f228819f6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f227c0098e0 tx=0x7f227c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.621+0000 7f2284ff9700 1 -- 172.21.15.19:0/2060769901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c026070 con 0x7f22881041f0 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.622+0000 7f2284ff9700 1 -- 172.21.15.19:0/2060769901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f227c004800 con 0x7f22881041f0 2024-05-09T01:21:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.622+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22881a26e0 con 0x7f22881041f0 2024-05-09T01:21:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.622+0000 7f2284ff9700 1 -- 172.21.15.19:0/2060769901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c005430 con 0x7f22881041f0 2024-05-09T01:21:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.622+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a2b30 con 0x7f22881041f0 2024-05-09T01:21:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.623+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22880512c0 con 0x7f22881041f0 2024-05-09T01:21:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.624+0000 7f2284ff9700 1 -- 172.21.15.19:0/2060769901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f227c029070 con 0x7f22881041f0 2024-05-09T01:21:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.625+0000 7f2284ff9700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22700b2880 0x7f22700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.625+0000 7f22877fe700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22700b2880 0x7f22700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.625+0000 7f2284ff9700 1 -- 172.21.15.19:0/2060769901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6847+0+0 (secure 0 0 0) 0x7f227c0e2c50 con 0x7f22881041f0 2024-05-09T01:21:34.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.625+0000 7f22877fe700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22700b2880 0x7f22700b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f228806c000 tx=0x7f2278009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:34.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.629+0000 7f2284ff9700 1 -- 172.21.15.19:0/2060769901 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f227c030080 con 0x7f22881041f0 2024-05-09T01:21:34.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.847+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f22880643d0 con 0x7f22881041f0 2024-05-09T01:21:34.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.847+0000 7f2284ff9700 1 -- 172.21.15.19:0/2060769901 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f227c013070 con 0x7f22881041f0 2024-05-09T01:21:34.847 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:21:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.849+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22700b2880 msgr2=0x7f22700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.849+0000 7f228de22700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22700b2880 0x7f22700b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f228806c000 tx=0x7f2278009260 comp rx=0 tx=0).stop 2024-05-09T01:21:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.849+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 msgr2=0x7f228819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.849+0000 7f228de22700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 0x7f228819f6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f227c0098e0 tx=0x7f227c004790 comp rx=0 tx=0).stop 2024-05-09T01:21:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.850+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 shutdown_connections 2024-05-09T01:21:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.850+0000 7f228de22700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2288104cf0 0x7f22881a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.850+0000 7f228de22700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288103810 0x7f2288102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.850+0000 7f228de22700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22700b2880 0x7f22700b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.850+0000 7f228de22700 1 --2- 172.21.15.19:0/2060769901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22881041f0 0x7f228819f6b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:34.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.850+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 >> 172.21.15.19:0/2060769901 conn(0x7f22880fe020 msgr2=0x7f22881077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:34.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.850+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 shutdown_connections 2024-05-09T01:21:34.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:34.850+0000 7f228de22700 1 -- 172.21.15.19:0/2060769901 wait complete. 2024-05-09T01:21:35.154 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:21:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:21:35.092+0000 7f045edd1700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T01:21:35.154 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:21:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:21:35.154+0000 7f0456b7e700 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T01:21:35.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:35 smithi196 ceph-mon[35041]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:21:35.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:35 smithi196 ceph-mon[35041]: osdmap e72: 12 total, 11 up, 12 in 2024-05-09T01:21:35.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:35.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:35.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:35 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2060769901' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T01:21:35.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:35 smithi196 ceph-mon[35041]: from='osd.11 ' entity='osd.11' 2024-05-09T01:21:35.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:35 smithi059 ceph-mon[34351]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:21:35.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:35 smithi059 ceph-mon[34351]: osdmap e72: 12 total, 11 up, 12 in 2024-05-09T01:21:35.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:35.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:35.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:35 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2060769901' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T01:21:35.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:35 smithi059 ceph-mon[34351]: from='osd.11 ' entity='osd.11' 2024-05-09T01:21:35.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:35 smithi019 ceph-mon[30158]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T01:21:35.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:35 smithi019 ceph-mon[30158]: osdmap e72: 12 total, 11 up, 12 in 2024-05-09T01:21:35.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:35.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:35.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:35 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2060769901' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T01:21:35.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:35 smithi019 ceph-mon[30158]: from='osd.11 ' entity='osd.11' 2024-05-09T01:21:35.646 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1715217670,"num_in_osds":12,"osd_in_since":1715217675,"num_remapped_pgs":0} 2024-05-09T01:21:36.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:36 smithi196 ceph-mon[35041]: purged_snaps scrub starts 2024-05-09T01:21:36.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:36 smithi196 ceph-mon[35041]: purged_snaps scrub ok 2024-05-09T01:21:36.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:36 smithi196 ceph-mon[35041]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:36.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:36.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:36 smithi196 ceph-mon[35041]: osd.11 [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] boot 2024-05-09T01:21:36.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:36 smithi196 ceph-mon[35041]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T01:21:36.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:36.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:36 smithi059 ceph-mon[34351]: purged_snaps scrub starts 2024-05-09T01:21:36.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:36 smithi059 ceph-mon[34351]: purged_snaps scrub ok 2024-05-09T01:21:36.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:36 smithi059 ceph-mon[34351]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:36.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:36.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:36 smithi059 ceph-mon[34351]: osd.11 [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] boot 2024-05-09T01:21:36.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:36 smithi059 ceph-mon[34351]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T01:21:36.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:36.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:36 smithi019 ceph-mon[30158]: purged_snaps scrub starts 2024-05-09T01:21:36.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:36 smithi019 ceph-mon[30158]: purged_snaps scrub ok 2024-05-09T01:21:36.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:36 smithi019 ceph-mon[30158]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-05-09T01:21:36.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:36.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:36 smithi019 ceph-mon[30158]: osd.11 [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] boot 2024-05-09T01:21:36.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:36 smithi019 ceph-mon[30158]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T01:21:36.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T01:21:36.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T01:21:36.900 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:21:37.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: Detected new or changed devices on smithi196 2024-05-09T01:21:37.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi196 to 4272M 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:37.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:37 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: Detected new or changed devices on smithi196 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi196 to 4272M 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:37.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:37.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:37.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:37.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:37 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: Detected new or changed devices on smithi196 2024-05-09T01:21:37.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi196 to 4272M 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:37.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:37 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.634+0000 7f7062e5a700 1 -- 172.21.15.19:0/253485279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c1054a0 msgr2=0x7f705c1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.634+0000 7f7062e5a700 1 --2- 172.21.15.19:0/253485279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c1054a0 0x7f705c1058b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7048009990 tx=0x7f7048009ca0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.635+0000 7f7062e5a700 1 -- 172.21.15.19:0/253485279 shutdown_connections 2024-05-09T01:21:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.635+0000 7f7062e5a700 1 --2- 172.21.15.19:0/253485279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f705c105e40 0x7f705c10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.635+0000 7f7062e5a700 1 --2- 172.21.15.19:0/253485279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c104370 0x7f705c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.635+0000 7f7062e5a700 1 --2- 172.21.15.19:0/253485279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c1054a0 0x7f705c1058b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.635+0000 7f7062e5a700 1 -- 172.21.15.19:0/253485279 >> 172.21.15.19:0/253485279 conn(0x7f705c0ff9d0 msgr2=0x7f705c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.635+0000 7f7062e5a700 1 -- 172.21.15.19:0/253485279 shutdown_connections 2024-05-09T01:21:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.635+0000 7f7062e5a700 1 -- 172.21.15.19:0/253485279 wait complete. 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.635+0000 7f7062e5a700 1 Processor -- start 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.635+0000 7f7062e5a700 1 -- start start 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7062e5a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c104370 0x7f705c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7062e5a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c1054a0 0x7f705c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7062e5a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f705c105e40 0x7f705c1a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7062e5a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f705c10fde0 con 0x7f705c104370 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7062e5a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f705c10fc60 con 0x7f705c105e40 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7062e5a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f705c10ff60 con 0x7f705c1054a0 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7060bf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c104370 0x7f705c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7060bf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c104370 0x7f705c19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46558/0 (socket says 172.21.15.19:46558) 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7060bf6700 1 -- 172.21.15.19:0/611422345 learned_addr learned my addr 172.21.15.19:0/611422345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f705bfff700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c1054a0 0x7f705c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f70613f7700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f705c105e40 0x7f705c1a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7060bf6700 1 -- 172.21.15.19:0/611422345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c1054a0 msgr2=0x7f705c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7060bf6700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c1054a0 0x7f705c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7060bf6700 1 -- 172.21.15.19:0/611422345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f705c105e40 msgr2=0x7f705c1a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7060bf6700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f705c105e40 0x7f705c1a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7060bf6700 1 -- 172.21.15.19:0/611422345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c1a2360 con 0x7f705c104370 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f70613f7700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f705c105e40 0x7f705c1a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7060bf6700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c104370 0x7f705c19a890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f705000c970 tx=0x7f705000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.636+0000 7f7059ffb700 1 -- 172.21.15.19:0/611422345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050007b00 con 0x7f705c104370 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.637+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c1a2650 con 0x7f705c104370 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.637+0000 7f7059ffb700 1 -- 172.21.15.19:0/611422345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7050007ca0 con 0x7f705c104370 2024-05-09T01:21:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.637+0000 7f7059ffb700 1 -- 172.21.15.19:0/611422345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050018660 con 0x7f705c104370 2024-05-09T01:21:38.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.637+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1a2be0 con 0x7f705c104370 2024-05-09T01:21:38.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.638+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c0512c0 con 0x7f705c104370 2024-05-09T01:21:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.640+0000 7f7059ffb700 1 -- 172.21.15.19:0/611422345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f705000f450 con 0x7f705c104370 2024-05-09T01:21:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.641+0000 7f7059ffb700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f704c0b2880 0x7f704c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.641+0000 7f705bfff700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f704c0b2880 0x7f704c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:38.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.641+0000 7f7059ffb700 1 -- 172.21.15.19:0/611422345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f70500e1ef0 con 0x7f705c104370 2024-05-09T01:21:38.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.642+0000 7f705bfff700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f704c0b2880 0x7f704c0b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f704800b5c0 tx=0x7f704801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.645+0000 7f7059ffb700 1 -- 172.21.15.19:0/611422345 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7050053df0 con 0x7f705c104370 2024-05-09T01:21:38.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:38 smithi059 ceph-mon[34351]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:38.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:38 smithi059 ceph-mon[34351]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T01:21:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.858+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f705c02e5f0 con 0x7f705c104370 2024-05-09T01:21:38.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:38 smithi019 ceph-mon[30158]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:38.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:38 smithi019 ceph-mon[30158]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T01:21:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.859+0000 7f7059ffb700 1 -- 172.21.15.19:0/611422345 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f7050014030 con 0x7f705c104370 2024-05-09T01:21:38.858 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f704c0b2880 msgr2=0x7f704c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f704c0b2880 0x7f704c0b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f704800b5c0 tx=0x7f704801a040 comp rx=0 tx=0).stop 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c104370 msgr2=0x7f705c19a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c104370 0x7f705c19a890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f705000c970 tx=0x7f705000cc80 comp rx=0 tx=0).stop 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 shutdown_connections 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f705c105e40 0x7f705c1a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f704c0b2880 0x7f704c0b4d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c104370 0x7f705c19a890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 --2- 172.21.15.19:0/611422345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c1054a0 0x7f705c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 >> 172.21.15.19:0/611422345 conn(0x7f705c0ff9d0 msgr2=0x7f705c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 shutdown_connections 2024-05-09T01:21:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:38.861+0000 7f7062e5a700 1 -- 172.21.15.19:0/611422345 wait complete. 2024-05-09T01:21:38.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:38 smithi196 ceph-mon[35041]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:38.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:38 smithi196 ceph-mon[35041]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T01:21:39.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:39 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/611422345' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T01:21:39.577 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1715217696,"num_in_osds":12,"osd_in_since":1715217675,"num_remapped_pgs":0} 2024-05-09T01:21:39.577 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:21:39.826 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:21:39.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:39 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/611422345' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T01:21:39.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:39 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/611422345' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T01:21:40.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:40 smithi019 ceph-mon[30158]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:40.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:40 smithi059 ceph-mon[34351]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:40.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:40 smithi196 ceph-mon[35041]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.442+0000 7f1db38e0700 1 -- 172.21.15.19:0/3638092964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac107e10 msgr2=0x7f1dac108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.442+0000 7f1db38e0700 1 --2- 172.21.15.19:0/3638092964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac107e10 0x7f1dac108220 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0099c0 tx=0x7f1d9c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.442+0000 7f1db38e0700 1 -- 172.21.15.19:0/3638092964 shutdown_connections 2024-05-09T01:21:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.442+0000 7f1db38e0700 1 --2- 172.21.15.19:0/3638092964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dac108760 0x7f1dac1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.442+0000 7f1db38e0700 1 --2- 172.21.15.19:0/3638092964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dac1034e0 0x7f1dac1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.442+0000 7f1db38e0700 1 --2- 172.21.15.19:0/3638092964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac107e10 0x7f1dac108220 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.442+0000 7f1db38e0700 1 -- 172.21.15.19:0/3638092964 >> 172.21.15.19:0/3638092964 conn(0x7f1dac0fef30 msgr2=0x7f1dac101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.442+0000 7f1db38e0700 1 -- 172.21.15.19:0/3638092964 shutdown_connections 2024-05-09T01:21:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.442+0000 7f1db38e0700 1 -- 172.21.15.19:0/3638092964 wait complete. 2024-05-09T01:21:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db38e0700 1 Processor -- start 2024-05-09T01:21:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db38e0700 1 -- start start 2024-05-09T01:21:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db38e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac1034e0 0x7f1dac19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db38e0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dac107e10 0x7f1dac19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db38e0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dac108760 0x7f1dac19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db38e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1133c0 con 0x7f1dac1034e0 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db38e0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac113240 con 0x7f1dac108760 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db38e0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac113540 con 0x7f1dac107e10 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db167c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac1034e0 0x7f1dac19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db167c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac1034e0 0x7f1dac19ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53138/0 (socket says 172.21.15.19:53138) 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.443+0000 7f1db167c700 1 -- 172.21.15.19:0/4234596067 learned_addr learned my addr 172.21.15.19:0/4234596067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db0e7b700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dac107e10 0x7f1dac19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db1e7d700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dac108760 0x7f1dac19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db167c700 1 -- 172.21.15.19:0/4234596067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dac107e10 msgr2=0x7f1dac19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db167c700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dac107e10 0x7f1dac19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db167c700 1 -- 172.21.15.19:0/4234596067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dac108760 msgr2=0x7f1dac19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db167c700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dac108760 0x7f1dac19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db167c700 1 -- 172.21.15.19:0/4234596067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac19fc80 con 0x7f1dac1034e0 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db0e7b700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dac107e10 0x7f1dac19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:21:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db1e7d700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dac108760 0x7f1dac19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:21:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db167c700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac1034e0 0x7f1dac19ae60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1da800d950 tx=0x7f1da800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1da27fc700 1 -- 172.21.15.19:0/4234596067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8016070 con 0x7f1dac1034e0 2024-05-09T01:21:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac19ff70 con 0x7f1dac1034e0 2024-05-09T01:21:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1da27fc700 1 -- 172.21.15.19:0/4234596067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da8010460 con 0x7f1dac1034e0 2024-05-09T01:21:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1da27fc700 1 -- 172.21.15.19:0/4234596067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da800f620 con 0x7f1dac1034e0 2024-05-09T01:21:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.444+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac1a7940 con 0x7f1dac1034e0 2024-05-09T01:21:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.445+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac0512c0 con 0x7f1dac1034e0 2024-05-09T01:21:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.447+0000 7f1da27fc700 1 -- 172.21.15.19:0/4234596067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1da8019070 con 0x7f1dac1034e0 2024-05-09T01:21:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.448+0000 7f1da27fc700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d980b2880 0x7f1d980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.448+0000 7f1db0e7b700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d980b2880 0x7f1d980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.448+0000 7f1da27fc700 1 -- 172.21.15.19:0/4234596067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f1da80e1a00 con 0x7f1dac1034e0 2024-05-09T01:21:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.448+0000 7f1db0e7b700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d980b2880 0x7f1d980b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c006010 tx=0x7f1d9c003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.453+0000 7f1da27fc700 1 -- 172.21.15.19:0/4234596067 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1da80a9bc0 con 0x7f1dac1034e0 2024-05-09T01:21:41.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.670+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dac19c1c0 con 0x7f1dac1034e0 2024-05-09T01:21:41.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.671+0000 7f1da27fc700 1 -- 172.21.15.19:0/4234596067 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18428 (secure 0 0 0) 0x7f1da80a94c0 con 0x7f1dac1034e0 2024-05-09T01:21:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:21:41.670 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:21:37.436642+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:21:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d980b2880 msgr2=0x7f1d980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d980b2880 0x7f1d980b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c006010 tx=0x7f1d9c003680 comp rx=0 tx=0).stop 2024-05-09T01:21:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac1034e0 msgr2=0x7f1dac19ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac1034e0 0x7f1dac19ae60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1da800d950 tx=0x7f1da800dd10 comp rx=0 tx=0).stop 2024-05-09T01:21:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 shutdown_connections 2024-05-09T01:21:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dac108760 0x7f1dac19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d980b2880 0x7f1d980b4d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac1034e0 0x7f1dac19ae60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 --2- 172.21.15.19:0/4234596067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dac107e10 0x7f1dac19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:41.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.673+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 >> 172.21.15.19:0/4234596067 conn(0x7f1dac0fef30 msgr2=0x7f1dac06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:41.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.674+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 shutdown_connections 2024-05-09T01:21:41.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:41.674+0000 7f1db38e0700 1 -- 172.21.15.19:0/4234596067 wait complete. 2024-05-09T01:21:42.498 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T01:17:49.120859+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T01:21:42.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T01:21:42.752 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:21:42.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:42 smithi059 ceph-mon[34351]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:42.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:42 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/4234596067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:21:42.870 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:42 smithi019 ceph-mon[30158]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:42 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/4234596067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:21:42.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:42 smithi196 ceph-mon[35041]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:42.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:42 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/4234596067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:21:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.409+0000 7f0e51629700 1 -- 172.21.15.19:0/3819211055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 msgr2=0x7f0e4c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.409+0000 7f0e51629700 1 --2- 172.21.15.19:0/3819211055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 0x7f0e4c109310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0e400099c0 tx=0x7f0e40009cd0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.409+0000 7f0e51629700 1 -- 172.21.15.19:0/3819211055 shutdown_connections 2024-05-09T01:21:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.409+0000 7f0e51629700 1 --2- 172.21.15.19:0/3819211055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4c109850 0x7f0e4c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.409+0000 7f0e51629700 1 --2- 172.21.15.19:0/3819211055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c102ae0 0x7f0e4c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.409+0000 7f0e51629700 1 --2- 172.21.15.19:0/3819211055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 0x7f0e4c109310 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.409+0000 7f0e51629700 1 -- 172.21.15.19:0/3819211055 >> 172.21.15.19:0/3819211055 conn(0x7f0e4c0fe540 msgr2=0x7f0e4c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.409+0000 7f0e51629700 1 -- 172.21.15.19:0/3819211055 shutdown_connections 2024-05-09T01:21:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.409+0000 7f0e51629700 1 -- 172.21.15.19:0/3819211055 wait complete. 2024-05-09T01:21:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.410+0000 7f0e51629700 1 Processor -- start 2024-05-09T01:21:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.410+0000 7f0e51629700 1 -- start start 2024-05-09T01:21:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.410+0000 7f0e51629700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c102ae0 0x7f0e4c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.410+0000 7f0e51629700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 0x7f0e4c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.410+0000 7f0e51629700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4c109850 0x7f0e4c1044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.410+0000 7f0e51629700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c114460 con 0x7f0e4c108f00 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.410+0000 7f0e51629700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1142e0 con 0x7f0e4c102ae0 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.410+0000 7f0e51629700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1145e0 con 0x7f0e4c109850 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 0x7f0e4c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 0x7f0e4c103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53150/0 (socket says 172.21.15.19:53150) 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4a7fc700 1 -- 172.21.15.19:0/880528289 learned_addr learned my addr 172.21.15.19:0/880528289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4b7fe700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4c109850 0x7f0e4c1044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4affd700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c102ae0 0x7f0e4c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4a7fc700 1 -- 172.21.15.19:0/880528289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4c109850 msgr2=0x7f0e4c1044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4a7fc700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4c109850 0x7f0e4c1044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4a7fc700 1 -- 172.21.15.19:0/880528289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c102ae0 msgr2=0x7f0e4c103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4a7fc700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c102ae0 0x7f0e4c103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4a7fc700 1 -- 172.21.15.19:0/880528289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4c1a9140 con 0x7f0e4c108f00 2024-05-09T01:21:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e4a7fc700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 0x7f0e4c103f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e40006010 tx=0x7f0e400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e33fff700 1 -- 172.21.15.19:0/880528289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40027070 con 0x7f0e4c108f00 2024-05-09T01:21:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e33fff700 1 -- 172.21.15.19:0/880528289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e40003f20 con 0x7f0e4c108f00 2024-05-09T01:21:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4c1a93d0 con 0x7f0e4c108f00 2024-05-09T01:21:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e33fff700 1 -- 172.21.15.19:0/880528289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40022cd0 con 0x7f0e4c108f00 2024-05-09T01:21:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.411+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4c1a9820 con 0x7f0e4c108f00 2024-05-09T01:21:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.413+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e4c0512c0 con 0x7f0e4c108f00 2024-05-09T01:21:44.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.414+0000 7f0e33fff700 1 -- 172.21.15.19:0/880528289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0e4002a070 con 0x7f0e4c108f00 2024-05-09T01:21:44.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.415+0000 7f0e33fff700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e340b2880 0x7f0e340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:44.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.415+0000 7f0e4affd700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e340b2880 0x7f0e340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:44.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.415+0000 7f0e33fff700 1 -- 172.21.15.19:0/880528289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f0e400e2c30 con 0x7f0e4c108f00 2024-05-09T01:21:44.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.416+0000 7f0e4affd700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e340b2880 0x7f0e340b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c005ea0 tx=0x7f0e3c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:44.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.420+0000 7f0e33fff700 1 -- 172.21.15.19:0/880528289 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e40013070 con 0x7f0e4c108f00 2024-05-09T01:21:44.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:44 smithi059 ceph-mon[34351]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:44.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.633+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e4c02e7b0 con 0x7f0e4c108f00 2024-05-09T01:21:44.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.633+0000 7f0e33fff700 1 -- 172.21.15.19:0/880528289 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f0e400b1e20 con 0x7f0e4c108f00 2024-05-09T01:21:44.632 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-09T01:21:44.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e340b2880 msgr2=0x7f0e340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e340b2880 0x7f0e340b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c005ea0 tx=0x7f0e3c005e30 comp rx=0 tx=0).stop 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 msgr2=0x7f0e4c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 0x7f0e4c103f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e40006010 tx=0x7f0e400047b0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 shutdown_connections 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e4c109850 0x7f0e4c1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c102ae0 0x7f0e4c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e340b2880 0x7f0e340b4d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 --2- 172.21.15.19:0/880528289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c108f00 0x7f0e4c103f70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 >> 172.21.15.19:0/880528289 conn(0x7f0e4c0fe540 msgr2=0x7f0e4c107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 shutdown_connections 2024-05-09T01:21:44.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:44.636+0000 7f0e51629700 1 -- 172.21.15.19:0/880528289 wait complete. 2024-05-09T01:21:44.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:44 smithi196 ceph-mon[35041]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:44.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:44 smithi019 ceph-mon[30158]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:45.333 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.b on smithi059 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.e on smithi059 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.h on smithi059 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.k on smithi059 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-05-09T01:21:45.334 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-05-09T01:21:45.334 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-05-09T01:21:45.582 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:21:45.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:45 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/880528289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:21:45.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:45 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/880528289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:21:45.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:45 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/880528289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:21:46.819 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:46 smithi196 ceph-mon[35041]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:46.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:46 smithi059 ceph-mon[34351]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:46.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:46 smithi019 ceph-mon[30158]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.154+0000 7f5deaa80700 1 -- 172.21.15.196:0/1139812797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 msgr2=0x7f5de4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.154+0000 7f5deaa80700 1 --2- 172.21.15.196:0/1139812797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 0x7f5de4103be0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009990 tx=0x7f5dd8009ca0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.155+0000 7f5deaa80700 1 -- 172.21.15.196:0/1139812797 shutdown_connections 2024-05-09T01:21:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.155+0000 7f5deaa80700 1 --2- 172.21.15.196:0/1139812797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de4104d10 0x7f5de410b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.155+0000 7f5deaa80700 1 --2- 172.21.15.196:0/1139812797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 0x7f5de4103be0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.155+0000 7f5deaa80700 1 --2- 172.21.15.196:0/1139812797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de41041b0 0x7f5de4104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.155+0000 7f5deaa80700 1 -- 172.21.15.196:0/1139812797 >> 172.21.15.196:0/1139812797 conn(0x7f5de40fe020 msgr2=0x7f5de41004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.155+0000 7f5deaa80700 1 -- 172.21.15.196:0/1139812797 shutdown_connections 2024-05-09T01:21:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.155+0000 7f5deaa80700 1 -- 172.21.15.196:0/1139812797 wait complete. 2024-05-09T01:21:47.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.155+0000 7f5deaa80700 1 Processor -- start 2024-05-09T01:21:47.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5deaa80700 1 -- start start 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5deaa80700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 0x7f5de4102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5deaa80700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de41041b0 0x7f5de419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5deaa80700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de4104d10 0x7f5de41a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5deaa80700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5de410eca0 con 0x7f5de4104d10 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5deaa80700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5de410eb20 con 0x7f5de41041b0 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5deaa80700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5de410ee20 con 0x7f5de41037d0 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de881c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 0x7f5de4102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de881c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 0x7f5de4102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34258/0 (socket says 172.21.15.196:34258) 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de881c700 1 -- 172.21.15.196:0/4072354455 learned_addr learned my addr 172.21.15.196:0/4072354455 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de3fff700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de41041b0 0x7f5de419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de901d700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de4104d10 0x7f5de41a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de881c700 1 -- 172.21.15.196:0/4072354455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de41041b0 msgr2=0x7f5de419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de881c700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de41041b0 0x7f5de419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de881c700 1 -- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de4104d10 msgr2=0x7f5de41a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:47.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de881c700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de4104d10 0x7f5de41a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de881c700 1 -- 172.21.15.196:0/4072354455 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de41a2420 con 0x7f5de41037d0 2024-05-09T01:21:47.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.156+0000 7f5de3fff700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de41041b0 0x7f5de419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:21:47.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.157+0000 7f5de881c700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 0x7f5de4102eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800b760 tx=0x7f5dd8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:47.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.157+0000 7f5de1ffb700 1 -- 172.21.15.196:0/4072354455 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8026070 con 0x7f5de41037d0 2024-05-09T01:21:47.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.157+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de41a26b0 con 0x7f5de41037d0 2024-05-09T01:21:47.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.157+0000 7f5de1ffb700 1 -- 172.21.15.196:0/4072354455 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd8021410 con 0x7f5de41037d0 2024-05-09T01:21:47.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.157+0000 7f5de1ffb700 1 -- 172.21.15.196:0/4072354455 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8017650 con 0x7f5de41037d0 2024-05-09T01:21:47.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.157+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41a2b00 con 0x7f5de41037d0 2024-05-09T01:21:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.158+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de40512c0 con 0x7f5de41037d0 2024-05-09T01:21:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.159+0000 7f5de1ffb700 1 -- 172.21.15.196:0/4072354455 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5dd8029070 con 0x7f5de41037d0 2024-05-09T01:21:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.160+0000 7f5de1ffb700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5dd40b2880 0x7f5dd40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.160+0000 7f5de1ffb700 1 -- 172.21.15.196:0/4072354455 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f5dd80e2c90 con 0x7f5de41037d0 2024-05-09T01:21:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.160+0000 7f5de3fff700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5dd40b2880 0x7f5dd40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:47.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.161+0000 7f5de3fff700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5dd40b2880 0x7f5dd40b4d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5de4102860 tx=0x7f5dd0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:47.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.165+0000 7f5de1ffb700 1 -- 172.21.15.196:0/4072354455 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5dd8013070 con 0x7f5de41037d0 2024-05-09T01:21:47.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.401+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7f5de4063dc0 con 0x7f5dd40b2880 2024-05-09T01:21:47.409 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-05-09T01:21:47.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.410+0000 7f5de1ffb700 1 -- 172.21.15.196:0/4072354455 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5de4063dc0 con 0x7f5dd40b2880 2024-05-09T01:21:47.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5dd40b2880 msgr2=0x7f5dd40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:47.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5dd40b2880 0x7f5dd40b4d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5de4102860 tx=0x7f5dd0009260 comp rx=0 tx=0).stop 2024-05-09T01:21:47.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 msgr2=0x7f5de4102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:47.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 0x7f5de4102eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800b760 tx=0x7f5dd8009580 comp rx=0 tx=0).stop 2024-05-09T01:21:47.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 shutdown_connections 2024-05-09T01:21:47.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5dd40b2880 0x7f5dd40b4d90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de4104d10 0x7f5de41a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de41037d0 0x7f5de4102eb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 --2- 172.21.15.196:0/4072354455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de41041b0 0x7f5de419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:47.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.412+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 >> 172.21.15.196:0/4072354455 conn(0x7f5de40fe020 msgr2=0x7f5de4107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:47.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.413+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 shutdown_connections 2024-05-09T01:21:47.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T01:21:47.413+0000 7f5deaa80700 1 -- 172.21.15.196:0/4072354455 wait complete. 2024-05-09T01:21:48.013 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.a.service 2024-05-09T01:21:48.016 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.d.service 2024-05-09T01:21:48.059 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.g.service 2024-05-09T01:21:48.102 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.j.service 2024-05-09T01:21:48.145 DEBUG:teuthology.orchestra.run.smithi059:mds.b> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.b.service 2024-05-09T01:21:48.148 DEBUG:teuthology.orchestra.run.smithi059:mds.e> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.e.service 2024-05-09T01:21:48.191 DEBUG:teuthology.orchestra.run.smithi059:mds.h> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.h.service 2024-05-09T01:21:48.234 DEBUG:teuthology.orchestra.run.smithi059:mds.k> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.k.service 2024-05-09T01:21:48.276 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.c.service 2024-05-09T01:21:48.279 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.f.service 2024-05-09T01:21:48.321 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.i.service 2024-05-09T01:21:48.364 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.l.service 2024-05-09T01:21:48.407 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T01:21:48.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:21:48.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:48.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:21:48.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-05-09T01:21:48.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:48.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:48.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:48.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:48.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:48.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:48.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:48.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:21:48.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:48.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:21:48.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-05-09T01:21:48.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:48.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:48.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:48.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:48.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:48.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:48.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:48.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:49.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:49 smithi196 ceph-mon[35041]: Deploying daemon mds.l on smithi196 2024-05-09T01:21:49.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:49 smithi019 ceph-mon[30158]: Deploying daemon mds.l on smithi196 2024-05-09T01:21:49.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:49 smithi059 ceph-mon[34351]: Deploying daemon mds.l on smithi196 2024-05-09T01:21:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.354+0000 7f3a557b8700 1 -- 172.21.15.19:0/2884388647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 msgr2=0x7f3a5010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.354+0000 7f3a557b8700 1 --2- 172.21.15.19:0/2884388647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 0x7f3a5010e3a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009990 tx=0x7f3a38009ca0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.355+0000 7f3a557b8700 1 -- 172.21.15.19:0/2884388647 shutdown_connections 2024-05-09T01:21:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.355+0000 7f3a557b8700 1 --2- 172.21.15.19:0/2884388647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5010e8e0 0x7f3a50110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.355+0000 7f3a557b8700 1 --2- 172.21.15.19:0/2884388647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a50105500 0x7f3a50105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.355+0000 7f3a557b8700 1 --2- 172.21.15.19:0/2884388647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 0x7f3a5010e3a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.355+0000 7f3a557b8700 1 -- 172.21.15.19:0/2884388647 >> 172.21.15.19:0/2884388647 conn(0x7f3a500fedc0 msgr2=0x7f3a501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.355+0000 7f3a557b8700 1 -- 172.21.15.19:0/2884388647 shutdown_connections 2024-05-09T01:21:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.355+0000 7f3a557b8700 1 -- 172.21.15.19:0/2884388647 wait complete. 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.355+0000 7f3a557b8700 1 Processor -- start 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a557b8700 1 -- start start 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a557b8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a50105500 0x7f3a501031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a557b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 0x7f3a50103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a557b8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5010e8e0 0x7f3a50103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a557b8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a50114440 con 0x7f3a50105e50 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a557b8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a501142c0 con 0x7f3a5010e8e0 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a557b8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a501145c0 con 0x7f3a50105500 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 0x7f3a50103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 0x7f3a50103730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43658/0 (socket says 172.21.15.19:43658) 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4e7fc700 1 -- 172.21.15.19:0/3571037050 learned_addr learned my addr 172.21.15.19:0/3571037050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4effd700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a50105500 0x7f3a501031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4f7fe700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5010e8e0 0x7f3a50103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4e7fc700 1 -- 172.21.15.19:0/3571037050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a50105500 msgr2=0x7f3a501031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4e7fc700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a50105500 0x7f3a501031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4e7fc700 1 -- 172.21.15.19:0/3571037050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5010e8e0 msgr2=0x7f3a50103c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4e7fc700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5010e8e0 0x7f3a50103c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.356+0000 7f3a4e7fc700 1 -- 172.21.15.19:0/3571037050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a501a91d0 con 0x7f3a50105e50 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.357+0000 7f3a4e7fc700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 0x7f3a50103730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3a38000c00 tx=0x7f3a38009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.357+0000 7f3a47fff700 1 -- 172.21.15.19:0/3571037050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38027070 con 0x7f3a50105e50 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.357+0000 7f3a47fff700 1 -- 172.21.15.19:0/3571037050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a38022470 con 0x7f3a50105e50 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.357+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a501a9460 con 0x7f3a50105e50 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.357+0000 7f3a47fff700 1 -- 172.21.15.19:0/3571037050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38017650 con 0x7f3a50105e50 2024-05-09T01:21:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.357+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a501a9940 con 0x7f3a50105e50 2024-05-09T01:21:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.358+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a500512c0 con 0x7f3a50105e50 2024-05-09T01:21:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.359+0000 7f3a47fff700 1 -- 172.21.15.19:0/3571037050 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3a3802a070 con 0x7f3a50105e50 2024-05-09T01:21:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.360+0000 7f3a47fff700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a3c0b2880 0x7f3a3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.360+0000 7f3a4effd700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a3c0b2880 0x7f3a3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.360+0000 7f3a47fff700 1 -- 172.21.15.19:0/3571037050 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f3a380e3290 con 0x7f3a50105e50 2024-05-09T01:21:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.361+0000 7f3a4effd700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a3c0b2880 0x7f3a3c0b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000f540 tx=0x7f3a4000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.364+0000 7f3a47fff700 1 -- 172.21.15.19:0/3571037050 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a38031020 con 0x7f3a50105e50 2024-05-09T01:21:50.393 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:21:50 smithi196 systemd[1]: Starting Ceph mds.l for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:21:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.630+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a5002e5f0 con 0x7f3a50105e50 2024-05-09T01:21:50.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:50 smithi019 ceph-mon[30158]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:50.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.630+0000 7f3a47fff700 1 -- 172.21.15.19:0/3571037050 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f3a38013070 con 0x7f3a50105e50 2024-05-09T01:21:50.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:21:50.629 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-09T01:21:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a3c0b2880 msgr2=0x7f3a3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a3c0b2880 0x7f3a3c0b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000f540 tx=0x7f3a4000b040 comp rx=0 tx=0).stop 2024-05-09T01:21:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 msgr2=0x7f3a50103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 0x7f3a50103730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3a38000c00 tx=0x7f3a38009580 comp rx=0 tx=0).stop 2024-05-09T01:21:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 shutdown_connections 2024-05-09T01:21:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5010e8e0 0x7f3a50103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a50105500 0x7f3a501031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a3c0b2880 0x7f3a3c0b4d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 --2- 172.21.15.19:0/3571037050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a50105e50 0x7f3a50103730 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 >> 172.21.15.19:0/3571037050 conn(0x7f3a500fedc0 msgr2=0x7f3a500ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 shutdown_connections 2024-05-09T01:21:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:50.633+0000 7f3a557b8700 1 -- 172.21.15.19:0/3571037050 wait complete. 2024-05-09T01:21:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 1 2024-05-09T01:21:50.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:50 smithi196 ceph-mon[35041]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:50.692 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:21:50 smithi196 podman[65060]: 2024-05-09 01:21:50.298491971 +0000 UTC m=+0.018080883 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:21:50.692 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:21:50 smithi196 podman[65060]: 2024-05-09 01:21:50.400793113 +0000 UTC m=+0.120382015 container create e2168d199905b8017f860093dfec4ddfb5333e134354136383166d78c8a022cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T01:21:50.693 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:21:50 smithi196 podman[65060]: 2024-05-09 01:21:50.695263572 +0000 UTC m=+0.414852473 container init e2168d199905b8017f860093dfec4ddfb5333e134354136383166d78c8a022cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T01:21:50.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:50 smithi059 ceph-mon[34351]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:50.945 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:21:50 smithi196 podman[65060]: 2024-05-09 01:21:50.697971221 +0000 UTC m=+0.417560126 container start e2168d199905b8017f860093dfec4ddfb5333e134354136383166d78c8a022cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-05-09T01:21:50.945 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:21:50 smithi196 ceph-mds[65079]: -- 172.21.15.196:0/2523347828 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc723c1a40 con 0x55dc72409a80 2024-05-09T01:21:50.945 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:21:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l[65075]: starting mds.l at 2024-05-09T01:21:50.945 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:21:50 smithi196 bash[65060]: e2168d199905b8017f860093dfec4ddfb5333e134354136383166d78c8a022cb 2024-05-09T01:21:50.945 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:21:50 smithi196 systemd[1]: Started Ceph mds.l for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3571037050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6832/2736793414,v1:172.21.15.196:6833/2736793414] up:boot 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: fsmap 1 up:standby 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3571037050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6832/2736793414,v1:172.21.15.196:6833/2736793414] up:boot 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: fsmap 1 up:standby 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:51.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:51.739 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3571037050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:21:51.739 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6832/2736793414,v1:172.21.15.196:6833/2736793414] up:boot 2024-05-09T01:21:51.739 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: fsmap 1 up:standby 2024-05-09T01:21:51.739 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T01:21:51.739 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:51.740 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:51.740 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:51.740 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:51.740 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:51.740 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:52.676 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:52 smithi059 ceph-mon[34351]: Deploying daemon mds.k on smithi059 2024-05-09T01:21:52.676 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:52 smithi059 ceph-mon[34351]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:52.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:52 smithi196 ceph-mon[35041]: Deploying daemon mds.k on smithi059 2024-05-09T01:21:52.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:52 smithi196 ceph-mon[35041]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:52.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:52 smithi019 ceph-mon[30158]: Deploying daemon mds.k on smithi059 2024-05-09T01:21:52.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:52 smithi019 ceph-mon[30158]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:53.347 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:21:53 smithi059 systemd[1]: Starting Ceph mds.k for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:21:53.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:21:53.639 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:21:53 smithi059 podman[64186]: 2024-05-09 01:21:53.363565385 +0000 UTC m=+0.101185894 container create 5cd8671a4aa277a3e30026dec4de02f1b04f716baaf0247dd3aa9f6447cf0206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-09T01:21:53.639 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:21:53 smithi059 podman[64186]: 2024-05-09 01:21:53.28031011 +0000 UTC m=+0.017930616 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:21:53.639 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:21:53 smithi059 podman[64186]: 2024-05-09 01:21:53.55829695 +0000 UTC m=+0.295917455 container init 5cd8671a4aa277a3e30026dec4de02f1b04f716baaf0247dd3aa9f6447cf0206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T01:21:53.639 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:21:53 smithi059 podman[64186]: 2024-05-09 01:21:53.561024211 +0000 UTC m=+0.298644724 container start 5cd8671a4aa277a3e30026dec4de02f1b04f716baaf0247dd3aa9f6447cf0206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T01:21:53.639 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:21:53 smithi059 ceph-mds[64205]: -- 172.21.15.59:0/1152020671 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7ba903a40 con 0x55c7ba94b600 2024-05-09T01:21:53.639 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:21:53 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k[64201]: starting mds.k at 2024-05-09T01:21:53.640 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:21:53 smithi059 bash[64186]: 5cd8671a4aa277a3e30026dec4de02f1b04f716baaf0247dd3aa9f6447cf0206 2024-05-09T01:21:53.658 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:21:54.097 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:21:53 smithi059 systemd[1]: Started Ceph mds.k for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:21:54.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:54 smithi196 ceph-mon[35041]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:54.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:54.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:54.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:54.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:54.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:54.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:54.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:54 smithi019 ceph-mon[30158]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:54.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:54.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:54.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:54.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:54.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:54.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:54 smithi059 ceph-mon[34351]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:55.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:55 smithi196 ceph-mon[35041]: Deploying daemon mds.j on smithi019 2024-05-09T01:21:55.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:55 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6832/3715662430,v1:172.21.15.59:6833/3715662430] up:boot 2024-05-09T01:21:55.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:55 smithi196 ceph-mon[35041]: fsmap 2 up:standby 2024-05-09T01:21:55.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T01:21:55.700 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:55 smithi059 ceph-mon[34351]: Deploying daemon mds.j on smithi019 2024-05-09T01:21:55.700 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:55 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6832/3715662430,v1:172.21.15.59:6833/3715662430] up:boot 2024-05-09T01:21:55.700 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:55 smithi059 ceph-mon[34351]: fsmap 2 up:standby 2024-05-09T01:21:55.700 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T01:21:55.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:55 smithi019 ceph-mon[30158]: Deploying daemon mds.j on smithi019 2024-05-09T01:21:55.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:55 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6832/3715662430,v1:172.21.15.59:6833/3715662430] up:boot 2024-05-09T01:21:55.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:55 smithi019 ceph-mon[30158]: fsmap 2 up:standby 2024-05-09T01:21:55.752 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T01:21:56.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 -- 172.21.15.19:0/2533331954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a80108200 msgr2=0x7f4a80108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:56.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 --2- 172.21.15.19:0/2533331954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a80108200 0x7f4a80108610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009990 tx=0x7f4a7c009ca0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 -- 172.21.15.19:0/2533331954 shutdown_connections 2024-05-09T01:21:56.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 --2- 172.21.15.19:0/2533331954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a801042b0 0x7f4a80110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 --2- 172.21.15.19:0/2533331954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a80108200 0x7f4a80108610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 --2- 172.21.15.19:0/2533331954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80103960 0x7f4a80103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 -- 172.21.15.19:0/2533331954 >> 172.21.15.19:0/2533331954 conn(0x7f4a800fe4e0 msgr2=0x7f4a80100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 -- 172.21.15.19:0/2533331954 shutdown_connections 2024-05-09T01:21:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 -- 172.21.15.19:0/2533331954 wait complete. 2024-05-09T01:21:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.119+0000 7f4a87f20700 1 Processor -- start 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a87f20700 1 -- start start 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a87f20700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a80103960 0x7f4a8019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a87f20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a801042b0 0x7f4a8019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a87f20700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80108200 0x7f4a8019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a87f20700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80113370 con 0x7f4a801042b0 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a87f20700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4a801131f0 con 0x7f4a80103960 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a87f20700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a801134f0 con 0x7f4a80108200 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a854bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a801042b0 0x7f4a8019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a85cbc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a80103960 0x7f4a8019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a85cbc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a80103960 0x7f4a8019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43388/0 (socket says 172.21.15.19:43388) 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a85cbc700 1 -- 172.21.15.19:0/3903980220 learned_addr learned my addr 172.21.15.19:0/3903980220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a854bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a801042b0 0x7f4a8019c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43668/0 (socket says 172.21.15.19:43668) 2024-05-09T01:21:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a864bd700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80108200 0x7f4a8019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a85cbc700 1 -- 172.21.15.19:0/3903980220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80108200 msgr2=0x7f4a8019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a85cbc700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80108200 0x7f4a8019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a85cbc700 1 -- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a801042b0 msgr2=0x7f4a8019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a85cbc700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a801042b0 0x7f4a8019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a85cbc700 1 -- 172.21.15.19:0/3903980220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a801a6fa0 con 0x7f4a80103960 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.120+0000 7f4a854bb700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a801042b0 0x7f4a8019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.121+0000 7f4a85cbc700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a80103960 0x7f4a8019bc50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c006010 tx=0x7f4a7c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.121+0000 7f4a72ffd700 1 -- 172.21.15.19:0/3903980220 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c027070 con 0x7f4a80103960 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.121+0000 7f4a72ffd700 1 -- 172.21.15.19:0/3903980220 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a7c022470 con 0x7f4a80103960 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.121+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a801a7230 con 0x7f4a80103960 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.121+0000 7f4a72ffd700 1 -- 172.21.15.19:0/3903980220 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c017650 con 0x7f4a80103960 2024-05-09T01:21:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.121+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a801a76e0 con 0x7f4a80103960 2024-05-09T01:21:56.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.122+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8010b7a0 con 0x7f4a80103960 2024-05-09T01:21:56.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.124+0000 7f4a72ffd700 1 -- 172.21.15.19:0/3903980220 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4a7c02a070 con 0x7f4a80103960 2024-05-09T01:21:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.124+0000 7f4a72ffd700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a6c0b2900 0x7f4a6c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:21:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.124+0000 7f4a854bb700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a6c0b2900 0x7f4a6c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:21:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.124+0000 7f4a72ffd700 1 -- 172.21.15.19:0/3903980220 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f4a7c0e32e0 con 0x7f4a80103960 2024-05-09T01:21:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.125+0000 7f4a854bb700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a6c0b2900 0x7f4a6c0b4e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4a80107930 tx=0x7f4a7400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:21:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.128+0000 7f4a72ffd700 1 -- 172.21.15.19:0/3903980220 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a7c013070 con 0x7f4a80103960 2024-05-09T01:21:56.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.394+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a8010b180 con 0x7f4a80103960 2024-05-09T01:21:56.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.395+0000 7f4a72ffd700 1 -- 172.21.15.19:0/3903980220 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f4a7c0ab4d0 con 0x7f4a80103960 2024-05-09T01:21:56.393 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:21:56.393 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T01:21:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a6c0b2900 msgr2=0x7f4a6c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a6c0b2900 0x7f4a6c0b4e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4a80107930 tx=0x7f4a7400a300 comp rx=0 tx=0).stop 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a80103960 msgr2=0x7f4a8019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a80103960 0x7f4a8019bc50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c006010 tx=0x7f4a7c0047b0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 shutdown_connections 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80108200 0x7f4a8019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a80103960 0x7f4a8019bc50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a6c0b2900 0x7f4a6c0b4e10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 --2- 172.21.15.19:0/3903980220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a801042b0 0x7f4a8019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 >> 172.21.15.19:0/3903980220 conn(0x7f4a800fe4e0 msgr2=0x7f4a80105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 shutdown_connections 2024-05-09T01:21:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:21:56.397+0000 7f4a87f20700 1 -- 172.21.15.19:0/3903980220 wait complete. 2024-05-09T01:21:56.397 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-05-09T01:21:56.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:56 smithi019 ceph-mon[30158]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:56.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:56 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3903980220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:21:56.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:56 smithi196 ceph-mon[35041]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:56.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:56 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3903980220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:21:56.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:56 smithi059 ceph-mon[34351]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:56.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:56 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3903980220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:21:57.896 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:21:57 smithi019 podman[70934]: 2024-05-09 01:21:57.809012195 +0000 UTC m=+0.128318615 container create ed0d63ac177663e9b002a49825edeef8d663704a04bb84785014186cdf629a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T01:21:58.396 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:21:58 smithi019 podman[70934]: 2024-05-09 01:21:58.044850667 +0000 UTC m=+0.364157099 container init ed0d63ac177663e9b002a49825edeef8d663704a04bb84785014186cdf629a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:21:58.396 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:21:58 smithi019 podman[70934]: 2024-05-09 01:21:58.047883608 +0000 UTC m=+0.367190032 container start ed0d63ac177663e9b002a49825edeef8d663704a04bb84785014186cdf629a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:21:58.396 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:21:58 smithi019 ceph-mds[70953]: -- 172.21.15.19:0/372620868 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f0cef3a40 con 0x562f0cf3b600 2024-05-09T01:21:58.396 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:21:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-j[70949]: starting mds.j at 2024-05-09T01:21:58.396 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:21:58 smithi019 bash[70934]: ed0d63ac177663e9b002a49825edeef8d663704a04bb84785014186cdf629a2c 2024-05-09T01:21:58.396 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:21:58 smithi019 systemd[1]: Started Ceph mds.j for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:21:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:58 smithi196 ceph-mon[35041]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:58.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:58.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:58.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:58 smithi059 ceph-mon[34351]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:58.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:58.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:58.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:58.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:58.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:58.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:58.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:58 smithi019 ceph-mon[30158]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:21:58.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:58.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:58.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:21:58.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:21:58.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:21:58.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:21:59.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:59 smithi019 ceph-mon[30158]: Deploying daemon mds.d on smithi019 2024-05-09T01:21:59.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:59 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.19:6834/523355371,v1:172.21.15.19:6835/523355371] up:boot 2024-05-09T01:21:59.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:59 smithi019 ceph-mon[30158]: fsmap 3 up:standby 2024-05-09T01:21:59.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:21:59 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T01:21:59.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:21:59.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:59 smithi059 ceph-mon[34351]: Deploying daemon mds.d on smithi019 2024-05-09T01:21:59.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:59 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.19:6834/523355371,v1:172.21.15.19:6835/523355371] up:boot 2024-05-09T01:21:59.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:59 smithi059 ceph-mon[34351]: fsmap 3 up:standby 2024-05-09T01:21:59.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:21:59 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T01:21:59.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:59 smithi196 ceph-mon[35041]: Deploying daemon mds.d on smithi019 2024-05-09T01:21:59.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:59 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.19:6834/523355371,v1:172.21.15.19:6835/523355371] up:boot 2024-05-09T01:21:59.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:59 smithi196 ceph-mon[35041]: fsmap 3 up:standby 2024-05-09T01:21:59.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:21:59 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T01:22:00.218 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:00.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:00 smithi019 ceph-mon[30158]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:00.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:00 smithi059 ceph-mon[34351]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:00.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:00 smithi196 ceph-mon[35041]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.543+0000 7fc431f8e700 1 -- 172.21.15.19:0/2671269665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c108f10 msgr2=0x7fc42c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.543+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2671269665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c108f10 0x7fc42c109320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc4200099c0 tx=0x7fc420009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.543+0000 7fc431f8e700 1 -- 172.21.15.19:0/2671269665 shutdown_connections 2024-05-09T01:22:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.543+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2671269665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc42c109860 0x7fc42c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.543+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2671269665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc42c1045e0 0x7fc42c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.543+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2671269665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c108f10 0x7fc42c109320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.543+0000 7fc431f8e700 1 -- 172.21.15.19:0/2671269665 >> 172.21.15.19:0/2671269665 conn(0x7fc42c0ffed0 msgr2=0x7fc42c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.543+0000 7fc431f8e700 1 -- 172.21.15.19:0/2671269665 shutdown_connections 2024-05-09T01:22:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.543+0000 7fc431f8e700 1 -- 172.21.15.19:0/2671269665 wait complete. 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.544+0000 7fc431f8e700 1 Processor -- start 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.544+0000 7fc431f8e700 1 -- start start 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc431f8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c1045e0 0x7fc42c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc431f8e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc42c108f10 0x7fc42c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc431f8e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc42c109860 0x7fc42c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc431f8e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c114470 con 0x7fc42c1045e0 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc431f8e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1142f0 con 0x7fc42c108f10 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc431f8e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1145f0 con 0x7fc42c109860 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c1045e0 0x7fc42c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c1045e0 0x7fc42c19ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47528/0 (socket says 172.21.15.19:47528) 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42b7fe700 1 -- 172.21.15.19:0/2591323495 learned_addr learned my addr 172.21.15.19:0/2591323495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42affd700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc42c108f10 0x7fc42c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42bfff700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc42c109860 0x7fc42c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42b7fe700 1 -- 172.21.15.19:0/2591323495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc42c109860 msgr2=0x7fc42c19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42b7fe700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc42c109860 0x7fc42c19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42b7fe700 1 -- 172.21.15.19:0/2591323495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc42c108f10 msgr2=0x7fc42c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42b7fe700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc42c108f10 0x7fc42c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42b7fe700 1 -- 172.21.15.19:0/2591323495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c19fc80 con 0x7fc42c1045e0 2024-05-09T01:22:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc42b7fe700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c1045e0 0x7fc42c19ae60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c9b0 tx=0x7fc41c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc428ff9700 1 -- 172.21.15.19:0/2591323495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c007b00 con 0x7fc42c1045e0 2024-05-09T01:22:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.545+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c19ff70 con 0x7fc42c1045e0 2024-05-09T01:22:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.546+0000 7fc428ff9700 1 -- 172.21.15.19:0/2591323495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c007ca0 con 0x7fc42c1045e0 2024-05-09T01:22:02.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.546+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1a04d0 con 0x7fc42c1045e0 2024-05-09T01:22:02.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.546+0000 7fc428ff9700 1 -- 172.21.15.19:0/2591323495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c018660 con 0x7fc42c1045e0 2024-05-09T01:22:02.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.548+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c105a90 con 0x7fc42c1045e0 2024-05-09T01:22:02.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.549+0000 7fc428ff9700 1 -- 172.21.15.19:0/2591323495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc41c00f450 con 0x7fc42c1045e0 2024-05-09T01:22:02.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.549+0000 7fc428ff9700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4140b2880 0x7fc4140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:02.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.550+0000 7fc42affd700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4140b2880 0x7fc4140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:02.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.550+0000 7fc428ff9700 1 -- 172.21.15.19:0/2591323495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fc41c0e1d50 con 0x7fc42c1045e0 2024-05-09T01:22:02.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.550+0000 7fc42affd700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4140b2880 0x7fc4140b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc420006010 tx=0x7fc420000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.555+0000 7fc428ff9700 1 -- 172.21.15.19:0/2591323495 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc41c0e5030 con 0x7fc42c1045e0 2024-05-09T01:22:02.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:02 smithi196 ceph-mon[35041]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:02.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:02.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:02.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:02.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:02.749 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:02.749 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.819+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc42c19c1c0 con 0x7fc42c1045e0 2024-05-09T01:22:02.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:02 smithi019 ceph-mon[30158]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:02.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:02.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:02.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:02.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:02.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:02.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:02.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.819+0000 7fc428ff9700 1 -- 172.21.15.19:0/2591323495 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fc41c014030 con 0x7fc42c1045e0 2024-05-09T01:22:02.819 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:22:02.819 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T01:22:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4140b2880 msgr2=0x7fc4140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4140b2880 0x7fc4140b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc420006010 tx=0x7fc420000bc0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c1045e0 msgr2=0x7fc42c19ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c1045e0 0x7fc42c19ae60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c9b0 tx=0x7fc41c00ccc0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 shutdown_connections 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc42c109860 0x7fc42c19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4140b2880 0x7fc4140b4d90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42c1045e0 0x7fc42c19ae60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 --2- 172.21.15.19:0/2591323495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc42c108f10 0x7fc42c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 >> 172.21.15.19:0/2591323495 conn(0x7fc42c0ffed0 msgr2=0x7fc42c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 shutdown_connections 2024-05-09T01:22:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:02.822+0000 7fc431f8e700 1 -- 172.21.15.19:0/2591323495 wait complete. 2024-05-09T01:22:02.822 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 5 2024-05-09T01:22:02.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:02 smithi059 ceph-mon[34351]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:02.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:02.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:03.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:03 smithi059 ceph-mon[34351]: Deploying daemon mds.e on smithi059 2024-05-09T01:22:03.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:03 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.19:6836/3245904831,v1:172.21.15.19:6837/3245904831] up:boot 2024-05-09T01:22:03.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:03 smithi059 ceph-mon[34351]: fsmap 4 up:standby 2024-05-09T01:22:03.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T01:22:03.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:03 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2591323495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:03.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:03 smithi019 ceph-mon[30158]: Deploying daemon mds.e on smithi059 2024-05-09T01:22:03.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:03 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.19:6836/3245904831,v1:172.21.15.19:6837/3245904831] up:boot 2024-05-09T01:22:03.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:03 smithi019 ceph-mon[30158]: fsmap 4 up:standby 2024-05-09T01:22:03.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T01:22:03.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:03 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2591323495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:03.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:03 smithi196 ceph-mon[35041]: Deploying daemon mds.e on smithi059 2024-05-09T01:22:03.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:03 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.19:6836/3245904831,v1:172.21.15.19:6837/3245904831] up:boot 2024-05-09T01:22:03.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:03 smithi196 ceph-mon[35041]: fsmap 4 up:standby 2024-05-09T01:22:03.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T01:22:03.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:03 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2591323495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:04.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:04 smithi059 ceph-mon[34351]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:04.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:04 smithi019 ceph-mon[30158]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:04.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:04 smithi196 ceph-mon[35041]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:05.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:22:05.597 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:22:05 smithi059 systemd[1]: Started Ceph mds.e for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:22:05.723 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:06.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:06.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:06.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:06.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:06.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:06.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:06.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:06.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: Deploying daemon mds.f on smithi196 2024-05-09T01:22:06.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] up:boot 2024-05-09T01:22:06.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: fsmap 5 up:standby 2024-05-09T01:22:06.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:06 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T01:22:06.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:06.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:06.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:06.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:06.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: Deploying daemon mds.f on smithi196 2024-05-09T01:22:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] up:boot 2024-05-09T01:22:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: fsmap 5 up:standby 2024-05-09T01:22:06.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:06 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T01:22:06.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:06.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:06.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:06.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:06.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:06.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: Deploying daemon mds.f on smithi196 2024-05-09T01:22:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] up:boot 2024-05-09T01:22:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: fsmap 5 up:standby 2024-05-09T01:22:06.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:06 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T01:22:07.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.397+0000 7f06aa712700 1 -- 172.21.15.19:0/1141261799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4108740 msgr2=0x7f06a41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.397+0000 7f06aa712700 1 --2- 172.21.15.19:0/1141261799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4108740 0x7f06a41100a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f069c00b0d0 tx=0x7f069c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.397+0000 7f06aa712700 1 -- 172.21.15.19:0/1141261799 shutdown_connections 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.397+0000 7f06aa712700 1 --2- 172.21.15.19:0/1141261799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4108740 0x7f06a41100a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.397+0000 7f06aa712700 1 --2- 172.21.15.19:0/1141261799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a41034c0 0x7f06a41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.397+0000 7f06aa712700 1 --2- 172.21.15.19:0/1141261799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a4107df0 0x7f06a4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.397+0000 7f06aa712700 1 -- 172.21.15.19:0/1141261799 >> 172.21.15.19:0/1141261799 conn(0x7f06a40fedf0 msgr2=0x7f06a4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.397+0000 7f06aa712700 1 -- 172.21.15.19:0/1141261799 shutdown_connections 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.397+0000 7f06aa712700 1 -- 172.21.15.19:0/1141261799 wait complete. 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.398+0000 7f06aa712700 1 Processor -- start 2024-05-09T01:22:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.398+0000 7f06aa712700 1 -- start start 2024-05-09T01:22:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f06aa712700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a41034c0 0x7f06a419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f06aa712700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4107df0 0x7f06a419c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f06aa712700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a4108740 0x7f06a419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f06aa712700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4113370 con 0x7f06a4107df0 2024-05-09T01:22:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f06aa712700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41131f0 con 0x7f06a4108740 2024-05-09T01:22:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f06aa712700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41134f0 con 0x7f06a41034c0 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f069bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4107df0 0x7f06a419c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f069bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4107df0 0x7f06a419c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47536/0 (socket says 172.21.15.19:47536) 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f06a3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a41034c0 0x7f06a419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f069bfff700 1 -- 172.21.15.19:0/1115422479 learned_addr learned my addr 172.21.15.19:0/1115422479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f06a8caf700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a4108740 0x7f06a419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f069bfff700 1 -- 172.21.15.19:0/1115422479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a41034c0 msgr2=0x7f06a419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f069bfff700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a41034c0 0x7f06a419bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f069bfff700 1 -- 172.21.15.19:0/1115422479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a4108740 msgr2=0x7f06a419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f069bfff700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a4108740 0x7f06a419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.399+0000 7f069bfff700 1 -- 172.21.15.19:0/1115422479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a41a6fa0 con 0x7f06a4107df0 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.400+0000 7f069bfff700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4107df0 0x7f06a419c190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f068c00d950 tx=0x7f068c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.400+0000 7f06a1ffb700 1 -- 172.21.15.19:0/1115422479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c016070 con 0x7f06a4107df0 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.400+0000 7f06a1ffb700 1 -- 172.21.15.19:0/1115422479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f068c004830 con 0x7f06a4107df0 2024-05-09T01:22:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.400+0000 7f06a1ffb700 1 -- 172.21.15.19:0/1115422479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c005520 con 0x7f06a4107df0 2024-05-09T01:22:07.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.400+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a41a7290 con 0x7f06a4107df0 2024-05-09T01:22:07.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.400+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a41a7820 con 0x7f06a4107df0 2024-05-09T01:22:07.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.401+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a40512c0 con 0x7f06a4107df0 2024-05-09T01:22:07.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.403+0000 7f06a1ffb700 1 -- 172.21.15.19:0/1115422479 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f068c004220 con 0x7f06a4107df0 2024-05-09T01:22:07.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.404+0000 7f06a1ffb700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06900b2900 0x7f06900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:07.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.404+0000 7f06a3fff700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06900b2900 0x7f06900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:07.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.404+0000 7f06a1ffb700 1 -- 172.21.15.19:0/1115422479 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f068c0e1ef0 con 0x7f06a4107df0 2024-05-09T01:22:07.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.405+0000 7f06a3fff700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06900b2900 0x7f06900b4e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0694005950 tx=0x7f069400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:07.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.408+0000 7f06a1ffb700 1 -- 172.21.15.19:0/1115422479 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f068c01f020 con 0x7f06a4107df0 2024-05-09T01:22:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.671+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06a4075fe0 con 0x7f06a4107df0 2024-05-09T01:22:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.672+0000 7f06a1ffb700 1 -- 172.21.15.19:0/1115422479 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f068c0aa010 con 0x7f06a4107df0 2024-05-09T01:22:07.670 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:22:07.671 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06900b2900 msgr2=0x7f06900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06900b2900 0x7f06900b4e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0694005950 tx=0x7f069400b410 comp rx=0 tx=0).stop 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4107df0 msgr2=0x7f06a419c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4107df0 0x7f06a419c190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f068c00d950 tx=0x7f068c00dc60 comp rx=0 tx=0).stop 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 shutdown_connections 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a4108740 0x7f06a419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a41034c0 0x7f06a419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06900b2900 0x7f06900b4e10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 --2- 172.21.15.19:0/1115422479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a4107df0 0x7f06a419c190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 >> 172.21.15.19:0/1115422479 conn(0x7f06a40fedf0 msgr2=0x7f06a4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 shutdown_connections 2024-05-09T01:22:07.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:07.674+0000 7f06aa712700 1 -- 172.21.15.19:0/1115422479 wait complete. 2024-05-09T01:22:07.674 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-05-09T01:22:07.675 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:22:07 smithi196 systemd[1]: Starting Ceph mds.f for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:22:07.929 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:22:07 smithi196 podman[65383]: 2024-05-09 01:22:07.677351331 +0000 UTC m=+0.105678448 container create 7a5bab63e85e8ac0bb25bde20f769efe95cca5f1927fcd6d49670bd6ae9ea2e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0) 2024-05-09T01:22:07.929 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:22:07 smithi196 podman[65383]: 2024-05-09 01:22:07.589774443 +0000 UTC m=+0.018101562 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:22:07.929 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:22:07 smithi196 podman[65383]: 2024-05-09 01:22:07.896661985 +0000 UTC m=+0.324989106 container init 7a5bab63e85e8ac0bb25bde20f769efe95cca5f1927fcd6d49670bd6ae9ea2e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507) 2024-05-09T01:22:07.929 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:22:07 smithi196 podman[65383]: 2024-05-09 01:22:07.899160384 +0000 UTC m=+0.327487502 container start 7a5bab63e85e8ac0bb25bde20f769efe95cca5f1927fcd6d49670bd6ae9ea2e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-09T01:22:08.195 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:22:07 smithi196 ceph-mds[65403]: -- 172.21.15.196:0/1516403534 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55baaed3fa40 con 0x55baaed87a80 2024-05-09T01:22:08.195 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:22:07 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f[65399]: starting mds.f at 2024-05-09T01:22:08.195 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:22:07 smithi196 bash[65383]: 7a5bab63e85e8ac0bb25bde20f769efe95cca5f1927fcd6d49670bd6ae9ea2e1 2024-05-09T01:22:08.195 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:22:07 smithi196 systemd[1]: Started Ceph mds.f for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:22:08.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:08 smithi059 ceph-mon[34351]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:08.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:08 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1115422479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:08.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:08.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:08.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:08.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:08.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:08.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:08 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:08.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:08 smithi019 ceph-mon[30158]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:08.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:08 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1115422479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:08.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:08.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:08.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:08.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:08.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:08.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:08 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:08 smithi196 ceph-mon[35041]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:08 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1115422479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:08 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:09.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:09 smithi059 ceph-mon[34351]: Deploying daemon mds.c on smithi196 2024-05-09T01:22:09.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:09 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6834/1184717757,v1:172.21.15.196:6835/1184717757] up:boot 2024-05-09T01:22:09.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:09 smithi059 ceph-mon[34351]: fsmap 6 up:standby 2024-05-09T01:22:09.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:09 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T01:22:09.600 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:09 smithi019 ceph-mon[30158]: Deploying daemon mds.c on smithi196 2024-05-09T01:22:09.600 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:09 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6834/1184717757,v1:172.21.15.196:6835/1184717757] up:boot 2024-05-09T01:22:09.600 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:09 smithi019 ceph-mon[30158]: fsmap 6 up:standby 2024-05-09T01:22:09.600 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:09 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T01:22:09.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:09 smithi196 ceph-mon[35041]: Deploying daemon mds.c on smithi196 2024-05-09T01:22:09.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:09 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6834/1184717757,v1:172.21.15.196:6835/1184717757] up:boot 2024-05-09T01:22:09.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:09 smithi196 ceph-mon[35041]: fsmap 6 up:standby 2024-05-09T01:22:09.660 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:09 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T01:22:10.195 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:22:10 smithi196 systemd[1]: Starting Ceph mds.c for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:22:10.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:22:10.567 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:10 smithi196 ceph-mon[35041]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:10.567 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:22:10 smithi196 podman[65694]: 2024-05-09 01:22:10.210119587 +0000 UTC m=+0.017629927 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:22:10.567 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:22:10 smithi196 podman[65694]: 2024-05-09 01:22:10.323439555 +0000 UTC m=+0.130949937 container create e5a570786a3e44fbc9d79bf6f227bd25d2d39fbfdb2518023e477e5ad42bdd6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-09T01:22:10.567 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:22:10 smithi196 podman[65694]: 2024-05-09 01:22:10.534095541 +0000 UTC m=+0.341605882 container init e5a570786a3e44fbc9d79bf6f227bd25d2d39fbfdb2518023e477e5ad42bdd6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-09T01:22:10.567 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:22:10 smithi196 podman[65694]: 2024-05-09 01:22:10.536717621 +0000 UTC m=+0.344227970 container start e5a570786a3e44fbc9d79bf6f227bd25d2d39fbfdb2518023e477e5ad42bdd6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T01:22:10.567 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:22:10 smithi196 ceph-mds[65714]: -- 172.21.15.196:0/2299402683 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f62a365a40 con 0x55f62a3ada80 2024-05-09T01:22:10.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:10 smithi059 ceph-mon[34351]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:10.603 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:10.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:10 smithi019 ceph-mon[30158]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:10.945 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:22:10 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c[65710]: starting mds.c at 2024-05-09T01:22:10.945 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:22:10 smithi196 bash[65694]: e5a570786a3e44fbc9d79bf6f227bd25d2d39fbfdb2518023e477e5ad42bdd6a 2024-05-09T01:22:10.945 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:22:10 smithi196 systemd[1]: Started Ceph mds.c for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: Deploying daemon mds.h on smithi059 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] up:boot 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: fsmap 7 up:standby 2024-05-09T01:22:11.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:11 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: Deploying daemon mds.h on smithi059 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] up:boot 2024-05-09T01:22:11.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: fsmap 7 up:standby 2024-05-09T01:22:11.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:11 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T01:22:12.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:12.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:12.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:12.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:12.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:12.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:12.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: Deploying daemon mds.h on smithi059 2024-05-09T01:22:12.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:12.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] up:boot 2024-05-09T01:22:12.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: fsmap 7 up:standby 2024-05-09T01:22:12.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:11 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T01:22:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.324+0000 7f7d61dcc700 1 -- 172.21.15.19:0/4012877903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 msgr2=0x7f7d5c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.324+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/4012877903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 0x7f7d5c1038f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0099c0 tx=0x7f7d4c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.324+0000 7f7d61dcc700 1 -- 172.21.15.19:0/4012877903 shutdown_connections 2024-05-09T01:22:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.324+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/4012877903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c108760 0x7f7d5c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.324+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/4012877903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 0x7f7d5c1038f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.324+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/4012877903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c107e10 0x7f7d5c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.324+0000 7f7d61dcc700 1 -- 172.21.15.19:0/4012877903 >> 172.21.15.19:0/4012877903 conn(0x7f7d5c0fef30 msgr2=0x7f7d5c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.324+0000 7f7d61dcc700 1 -- 172.21.15.19:0/4012877903 shutdown_connections 2024-05-09T01:22:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.324+0000 7f7d61dcc700 1 -- 172.21.15.19:0/4012877903 wait complete. 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d61dcc700 1 Processor -- start 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d61dcc700 1 -- start start 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d61dcc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 0x7f7d5c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d61dcc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c107e10 0x7f7d5c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d61dcc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c108760 0x7f7d5c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d61dcc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c113390 con 0x7f7d5c1034e0 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d61dcc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c113210 con 0x7f7d5c107e10 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d61dcc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c113510 con 0x7f7d5c108760 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d5b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 0x7f7d5c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d5b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 0x7f7d5c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37644/0 (socket says 172.21.15.19:37644) 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d5b7fe700 1 -- 172.21.15.19:0/411266463 learned_addr learned my addr 172.21.15.19:0/411266463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.325+0000 7f7d5affd700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c107e10 0x7f7d5c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d5bfff700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c108760 0x7f7d5c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d5b7fe700 1 -- 172.21.15.19:0/411266463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c108760 msgr2=0x7f7d5c19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d5b7fe700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c108760 0x7f7d5c19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d5b7fe700 1 -- 172.21.15.19:0/411266463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c107e10 msgr2=0x7f7d5c19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d5b7fe700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c107e10 0x7f7d5c19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d5b7fe700 1 -- 172.21.15.19:0/411266463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c1a7000 con 0x7f7d5c1034e0 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d5b7fe700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 0x7f7d5c19bcb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c006010 tx=0x7f7d4c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d58ff9700 1 -- 172.21.15.19:0/411266463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c026070 con 0x7f7d5c1034e0 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c1a7290 con 0x7f7d5c1034e0 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d58ff9700 1 -- 172.21.15.19:0/411266463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d4c00f460 con 0x7f7d5c1034e0 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c1a7740 con 0x7f7d5c1034e0 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.326+0000 7f7d58ff9700 1 -- 172.21.15.19:0/411266463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c009e60 con 0x7f7d5c1034e0 2024-05-09T01:22:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.327+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5c0512c0 con 0x7f7d5c1034e0 2024-05-09T01:22:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.329+0000 7f7d58ff9700 1 -- 172.21.15.19:0/411266463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7d4c029070 con 0x7f7d5c1034e0 2024-05-09T01:22:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.329+0000 7f7d58ff9700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7d440b2880 0x7f7d440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.330+0000 7f7d5affd700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7d440b2880 0x7f7d440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.330+0000 7f7d58ff9700 1 -- 172.21.15.19:0/411266463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f7d4c0e2b50 con 0x7f7d5c1034e0 2024-05-09T01:22:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.330+0000 7f7d5affd700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7d440b2880 0x7f7d440b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c1068f0 tx=0x7f7d500073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.333+0000 7f7d58ff9700 1 -- 172.21.15.19:0/411266463 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d4c013070 con 0x7f7d5c1034e0 2024-05-09T01:22:12.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.594+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d5c02e5f0 con 0x7f7d5c1034e0 2024-05-09T01:22:12.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.595+0000 7f7d58ff9700 1 -- 172.21.15.19:0/411266463 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f7d4c0aada0 con 0x7f7d5c1034e0 2024-05-09T01:22:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:22:12.593 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.597+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7d440b2880 msgr2=0x7f7d440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.597+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7d440b2880 0x7f7d440b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c1068f0 tx=0x7f7d500073d0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.597+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 msgr2=0x7f7d5c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.597+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 0x7f7d5c19bcb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c006010 tx=0x7f7d4c0045e0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.598+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 shutdown_connections 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.598+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c108760 0x7f7d5c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.598+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7d440b2880 0x7f7d440b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.598+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d5c1034e0 0x7f7d5c19bcb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.598+0000 7f7d61dcc700 1 --2- 172.21.15.19:0/411266463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c107e10 0x7f7d5c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.598+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 >> 172.21.15.19:0/411266463 conn(0x7f7d5c0fef30 msgr2=0x7f7d5c1096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.598+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 shutdown_connections 2024-05-09T01:22:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:12.598+0000 7f7d61dcc700 1 -- 172.21.15.19:0/411266463 wait complete. 2024-05-09T01:22:12.597 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-05-09T01:22:12.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:12 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/411266463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:12.977 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:12 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/411266463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:13.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:12 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/411266463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:13.597 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:22:13 smithi059 systemd[1]: Started Ceph mds.h for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:22:14.010 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:13 smithi196 ceph-mon[35041]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:14.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:14.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:14.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:14.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:14.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:14.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:13 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:14.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:13 smithi059 ceph-mon[34351]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:14.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:14.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:14.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:14.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:14.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:14.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:13 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:14.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:13 smithi019 ceph-mon[30158]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:14.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:14.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:14.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:14.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:14.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:14.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:13 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:15.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:14 smithi196 ceph-mon[35041]: Deploying daemon mds.i on smithi196 2024-05-09T01:22:15.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:14 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] up:boot 2024-05-09T01:22:15.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:14 smithi196 ceph-mon[35041]: fsmap 8 up:standby 2024-05-09T01:22:15.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:14 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T01:22:15.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:14 smithi059 ceph-mon[34351]: Deploying daemon mds.i on smithi196 2024-05-09T01:22:15.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:14 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] up:boot 2024-05-09T01:22:15.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:14 smithi059 ceph-mon[34351]: fsmap 8 up:standby 2024-05-09T01:22:15.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:14 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T01:22:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:14 smithi019 ceph-mon[30158]: Deploying daemon mds.i on smithi196 2024-05-09T01:22:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:14 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] up:boot 2024-05-09T01:22:15.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:14 smithi019 ceph-mon[30158]: fsmap 8 up:standby 2024-05-09T01:22:15.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:14 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T01:22:15.284 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:22:15.549 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:15.695 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:22:15 smithi196 systemd[1]: Starting Ceph mds.i for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:22:15.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:15 smithi019 ceph-mon[30158]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:15.991 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:15 smithi196 ceph-mon[35041]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:15.991 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:22:15 smithi196 podman[66006]: 2024-05-09 01:22:15.639029431 +0000 UTC m=+0.017801118 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:22:15.991 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:22:15 smithi196 podman[66006]: 2024-05-09 01:22:15.770640507 +0000 UTC m=+0.149412227 container create ae57a6613523a7075718f55cbfe72b9a1cfaf55fa3fecf750f48beb97f515a1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux ) 2024-05-09T01:22:15.991 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:22:15 smithi196 podman[66006]: 2024-05-09 01:22:15.958267064 +0000 UTC m=+0.337038739 container init ae57a6613523a7075718f55cbfe72b9a1cfaf55fa3fecf750f48beb97f515a1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:22:15.991 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:22:15 smithi196 podman[66006]: 2024-05-09 01:22:15.960756519 +0000 UTC m=+0.339528205 container start ae57a6613523a7075718f55cbfe72b9a1cfaf55fa3fecf750f48beb97f515a1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True) 2024-05-09T01:22:16.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:15 smithi059 ceph-mon[34351]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:16.445 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:22:15 smithi196 ceph-mds[66025]: -- 172.21.15.196:0/1535286089 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561dd48bfa40 con 0x561dd4907a80 2024-05-09T01:22:16.445 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:22:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i[66021]: starting mds.i at 2024-05-09T01:22:16.445 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:22:16 smithi196 bash[66006]: ae57a6613523a7075718f55cbfe72b9a1cfaf55fa3fecf750f48beb97f515a1f 2024-05-09T01:22:16.445 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:22:16 smithi196 systemd[1]: Started Ceph mds.i for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:22:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.327+0000 7f7ab36a9700 1 -- 172.21.15.19:0/2046922217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 msgr2=0x7f7aac1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.327+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/2046922217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 0x7f7aac1038f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400b0d0 tx=0x7f7aa400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.327+0000 7f7ab36a9700 1 -- 172.21.15.19:0/2046922217 shutdown_connections 2024-05-09T01:22:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.327+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/2046922217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac108760 0x7f7aac1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.327+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/2046922217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 0x7f7aac1038f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.327+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/2046922217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7aac107e10 0x7f7aac108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.327+0000 7f7ab36a9700 1 -- 172.21.15.19:0/2046922217 >> 172.21.15.19:0/2046922217 conn(0x7f7aac0fef30 msgr2=0x7f7aac101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:17.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.327+0000 7f7ab36a9700 1 -- 172.21.15.19:0/2046922217 shutdown_connections 2024-05-09T01:22:17.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.327+0000 7f7ab36a9700 1 -- 172.21.15.19:0/2046922217 wait complete. 2024-05-09T01:22:17.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:17.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:17.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:17.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:17.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:17.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:17.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: Deploying daemon mds.g on smithi019 2024-05-09T01:22:17.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] up:boot 2024-05-09T01:22:17.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: fsmap 9 up:standby 2024-05-09T01:22:17.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:17 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T01:22:17.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.328+0000 7f7ab36a9700 1 Processor -- start 2024-05-09T01:22:17.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.328+0000 7f7ab36a9700 1 -- start start 2024-05-09T01:22:17.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.328+0000 7f7ab36a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 0x7f7aac19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:17.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.328+0000 7f7ab36a9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac107e10 0x7f7aac19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab36a9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7aac108760 0x7f7aac19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab36a9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1144d0 con 0x7f7aac1034e0 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab36a9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac114350 con 0x7f7aac107e10 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab36a9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac114650 con 0x7f7aac108760 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab1445700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 0x7f7aac19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab1445700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 0x7f7aac19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37662/0 (socket says 172.21.15.19:37662) 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab1445700 1 -- 172.21.15.19:0/622644506 learned_addr learned my addr 172.21.15.19:0/622644506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab0c44700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac107e10 0x7f7aac19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab1c46700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7aac108760 0x7f7aac19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab1445700 1 -- 172.21.15.19:0/622644506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7aac108760 msgr2=0x7f7aac19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab1445700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7aac108760 0x7f7aac19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab1445700 1 -- 172.21.15.19:0/622644506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac107e10 msgr2=0x7f7aac19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab1445700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac107e10 0x7f7aac19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab1445700 1 -- 172.21.15.19:0/622644506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac19fd10 con 0x7f7aac1034e0 2024-05-09T01:22:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.329+0000 7f7ab0c44700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac107e10 0x7f7aac19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:22:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.330+0000 7f7ab1445700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 0x7f7aac19ae40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400b0a0 tx=0x7f7aa4004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.330+0000 7f7aaa7fc700 1 -- 172.21.15.19:0/622644506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4021070 con 0x7f7aac1034e0 2024-05-09T01:22:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.330+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac19ffa0 con 0x7f7aac1034e0 2024-05-09T01:22:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.330+0000 7f7aaa7fc700 1 -- 172.21.15.19:0/622644506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa4007c40 con 0x7f7aac1034e0 2024-05-09T01:22:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.330+0000 7f7aaa7fc700 1 -- 172.21.15.19:0/622644506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40052a0 con 0x7f7aac1034e0 2024-05-09T01:22:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.330+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac111b60 con 0x7f7aac1034e0 2024-05-09T01:22:17.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.331+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac104990 con 0x7f7aac1034e0 2024-05-09T01:22:17.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.334+0000 7f7aaa7fc700 1 -- 172.21.15.19:0/622644506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7aa4024070 con 0x7f7aac1034e0 2024-05-09T01:22:17.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.335+0000 7f7aaa7fc700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a940b2880 0x7f7a940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:17.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.335+0000 7f7aaa7fc700 1 -- 172.21.15.19:0/622644506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f7aa400f020 con 0x7f7aac1034e0 2024-05-09T01:22:17.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.335+0000 7f7ab0c44700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a940b2880 0x7f7a940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:17.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.336+0000 7f7ab0c44700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a940b2880 0x7f7a940b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7aac107750 tx=0x7f7aa00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.338+0000 7f7aaa7fc700 1 -- 172.21.15.19:0/622644506 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7aa402b020 con 0x7f7aac1034e0 2024-05-09T01:22:17.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:17.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:17.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:17.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:17.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:17.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:17.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: Deploying daemon mds.g on smithi019 2024-05-09T01:22:17.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] up:boot 2024-05-09T01:22:17.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: fsmap 9 up:standby 2024-05-09T01:22:17.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:17 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T01:22:17.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:17.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:17.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:17.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:17.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:17.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:17.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: Deploying daemon mds.g on smithi019 2024-05-09T01:22:17.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] up:boot 2024-05-09T01:22:17.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: fsmap 9 up:standby 2024-05-09T01:22:17.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:17 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T01:22:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.601+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aac19c1a0 con 0x7f7aac1034e0 2024-05-09T01:22:17.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.602+0000 7f7aaa7fc700 1 -- 172.21.15.19:0/622644506 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f7aa40abae0 con 0x7f7aac1034e0 2024-05-09T01:22:17.600 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:22:17.600 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-09T01:22:17.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a940b2880 msgr2=0x7f7a940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:17.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a940b2880 0x7f7a940b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7aac107750 tx=0x7f7aa00073d0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 msgr2=0x7f7aac19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:17.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 0x7f7aac19ae40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400b0a0 tx=0x7f7aa4004cc0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 shutdown_connections 2024-05-09T01:22:17.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7aac108760 0x7f7aac19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a940b2880 0x7f7a940b4d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aac1034e0 0x7f7aac19ae40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 --2- 172.21.15.19:0/622644506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac107e10 0x7f7aac19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:17.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.604+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 >> 172.21.15.19:0/622644506 conn(0x7f7aac0fef30 msgr2=0x7f7aac1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:17.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.605+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 shutdown_connections 2024-05-09T01:22:17.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:17.605+0000 7f7ab36a9700 1 -- 172.21.15.19:0/622644506 wait complete. 2024-05-09T01:22:17.604 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-05-09T01:22:18.348 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:18 smithi019 ceph-mon[30158]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:18.348 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:18 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/622644506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:18.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:18 smithi196 ceph-mon[35041]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:18.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:18 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/622644506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:18.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:18 smithi059 ceph-mon[34351]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:18.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:18 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/622644506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:19.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:19 smithi019 ceph-mon[30158]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:19.896 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:22:19 smithi019 systemd[1]: Started Ceph mds.g for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:22:19.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:19 smithi196 ceph-mon[35041]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:20.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:19 smithi059 ceph-mon[34351]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:21.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:21.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:21.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:21.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:21.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:21.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:21.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: Deploying daemon mds.a on smithi019 2024-05-09T01:22:21.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.19:6838/909874272,v1:172.21.15.19:6839/909874272] up:boot 2024-05-09T01:22:21.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: fsmap 10 up:standby 2024-05-09T01:22:21.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:20 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T01:22:21.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:22:21.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:21.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:21.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:21.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:21.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:21.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:21.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: Deploying daemon mds.a on smithi019 2024-05-09T01:22:21.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.19:6838/909874272,v1:172.21.15.19:6839/909874272] up:boot 2024-05-09T01:22:21.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: fsmap 10 up:standby 2024-05-09T01:22:21.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:20 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T01:22:21.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:21.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:21.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:21.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:21.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:21.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:21.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: Deploying daemon mds.a on smithi019 2024-05-09T01:22:21.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.19:6838/909874272,v1:172.21.15.19:6839/909874272] up:boot 2024-05-09T01:22:21.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: fsmap 10 up:standby 2024-05-09T01:22:21.348 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:20 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T01:22:21.675 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:22.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:21 smithi019 ceph-mon[30158]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:22.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:21 smithi196 ceph-mon[35041]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:22.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:21 smithi059 ceph-mon[34351]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.488+0000 7f92566f3700 1 -- 172.21.15.19:0/1397851420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 msgr2=0x7f925010f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.488+0000 7f92566f3700 1 --2- 172.21.15.19:0/1397851420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 0x7f925010f890 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f924800c210 tx=0x7f924800c520 comp rx=0 tx=0).stop 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.488+0000 7f92566f3700 1 -- 172.21.15.19:0/1397851420 shutdown_connections 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.488+0000 7f92566f3700 1 --2- 172.21.15.19:0/1397851420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 0x7f925010f890 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.488+0000 7f92566f3700 1 --2- 172.21.15.19:0/1397851420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92501089f0 0x7f9250108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.488+0000 7f92566f3700 1 --2- 172.21.15.19:0/1397851420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92501029c0 0x7f9250102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.488+0000 7f92566f3700 1 -- 172.21.15.19:0/1397851420 >> 172.21.15.19:0/1397851420 conn(0x7f92500fe520 msgr2=0x7f9250100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.488+0000 7f92566f3700 1 -- 172.21.15.19:0/1397851420 shutdown_connections 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.488+0000 7f92566f3700 1 -- 172.21.15.19:0/1397851420 wait complete. 2024-05-09T01:22:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.489+0000 7f92566f3700 1 Processor -- start 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.489+0000 7f92566f3700 1 -- start start 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.489+0000 7f92566f3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92501029c0 0x7f925019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.489+0000 7f92566f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 0x7f925019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.489+0000 7f92566f3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92501089f0 0x7f925019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f92566f3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9250112310 con 0x7f9250103310 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f92566f3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9250112190 con 0x7f92501029c0 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f92566f3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9250112490 con 0x7f92501089f0 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 0x7f925019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 0x7f925019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39960/0 (socket says 172.21.15.19:39960) 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924f7fe700 1 -- 172.21.15.19:0/2908915443 learned_addr learned my addr 172.21.15.19:0/2908915443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924ffff700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92501029c0 0x7f925019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f9254c90700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92501089f0 0x7f925019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:23.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924f7fe700 1 -- 172.21.15.19:0/2908915443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92501089f0 msgr2=0x7f925019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:23.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924f7fe700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92501089f0 0x7f925019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:23.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924f7fe700 1 -- 172.21.15.19:0/2908915443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92501029c0 msgr2=0x7f925019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:23.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924f7fe700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92501029c0 0x7f925019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:23.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924f7fe700 1 -- 172.21.15.19:0/2908915443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925019fd00 con 0x7f9250103310 2024-05-09T01:22:23.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924f7fe700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 0x7f925019b3d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f924400c8f0 tx=0x7f924400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:23.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.490+0000 7f924d7fa700 1 -- 172.21.15.19:0/2908915443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92440078d0 con 0x7f9250103310 2024-05-09T01:22:23.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.491+0000 7f924d7fa700 1 -- 172.21.15.19:0/2908915443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f924400f450 con 0x7f9250103310 2024-05-09T01:22:23.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.491+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925019fff0 con 0x7f9250103310 2024-05-09T01:22:23.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.491+0000 7f924d7fa700 1 -- 172.21.15.19:0/2908915443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244018660 con 0x7f9250103310 2024-05-09T01:22:23.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.491+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92501a7910 con 0x7f9250103310 2024-05-09T01:22:23.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.492+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92500512c0 con 0x7f9250103310 2024-05-09T01:22:23.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.493+0000 7f924d7fa700 1 -- 172.21.15.19:0/2908915443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f924400f900 con 0x7f9250103310 2024-05-09T01:22:23.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.494+0000 7f924d7fa700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f92380b2900 0x7f92380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:23.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.494+0000 7f924ffff700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f92380b2900 0x7f92380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:23.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.494+0000 7f924d7fa700 1 -- 172.21.15.19:0/2908915443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f92440e1b20 con 0x7f9250103310 2024-05-09T01:22:23.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.495+0000 7f924ffff700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f92380b2900 0x7f92380b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9250103b00 tx=0x7f9240009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.499+0000 7f924d7fa700 1 -- 172.21.15.19:0/2908915443 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9244014030 con 0x7f9250103310 2024-05-09T01:22:23.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.762+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f925019c190 con 0x7f9250103310 2024-05-09T01:22:23.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.762+0000 7f924d7fa700 1 -- 172.21.15.19:0/2908915443 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f92440a9ce0 con 0x7f9250103310 2024-05-09T01:22:23.761 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:22:23.761 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-09T01:22:23.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f92380b2900 msgr2=0x7f92380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:23.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f92380b2900 0x7f92380b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9250103b00 tx=0x7f9240009190 comp rx=0 tx=0).stop 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 msgr2=0x7f925019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 0x7f925019b3d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f924400c8f0 tx=0x7f924400cc00 comp rx=0 tx=0).stop 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 shutdown_connections 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92501089f0 0x7f925019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92501029c0 0x7f925019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f92380b2900 0x7f92380b4e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 --2- 172.21.15.19:0/2908915443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9250103310 0x7f925019b3d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 >> 172.21.15.19:0/2908915443 conn(0x7f92500fe520 msgr2=0x7f9250106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 shutdown_connections 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:23.765+0000 7f92566f3700 1 -- 172.21.15.19:0/2908915443 wait complete. 2024-05-09T01:22:23.764 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-05-09T01:22:24.018 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:22:23 smithi019 systemd[1]: Started Ceph mds.a for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:22:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:24 smithi019 ceph-mon[30158]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:24 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2908915443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:24 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:24.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:24 smithi196 ceph-mon[35041]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:24.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:24 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2908915443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:24.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:24.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:24.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:24.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:24.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:24.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:24 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:24 smithi059 ceph-mon[34351]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:24 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2908915443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T01:22:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T01:22:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:24 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:25.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:25 smithi059 ceph-mon[34351]: Deploying daemon mds.b on smithi059 2024-05-09T01:22:25.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:25 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.19:6840/555235417,v1:172.21.15.19:6841/555235417] up:boot 2024-05-09T01:22:25.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:25 smithi059 ceph-mon[34351]: fsmap 11 up:standby 2024-05-09T01:22:25.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:25 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T01:22:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:25 smithi196 ceph-mon[35041]: Deploying daemon mds.b on smithi059 2024-05-09T01:22:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:25 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.19:6840/555235417,v1:172.21.15.19:6841/555235417] up:boot 2024-05-09T01:22:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:25 smithi196 ceph-mon[35041]: fsmap 11 up:standby 2024-05-09T01:22:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:25 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T01:22:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:25 smithi019 ceph-mon[30158]: Deploying daemon mds.b on smithi059 2024-05-09T01:22:25.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:25 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.19:6840/555235417,v1:172.21.15.19:6841/555235417] up:boot 2024-05-09T01:22:25.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:25 smithi019 ceph-mon[30158]: fsmap 11 up:standby 2024-05-09T01:22:25.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:25 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T01:22:26.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:26 smithi196 ceph-mon[35041]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:26.525 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:26 smithi059 ceph-mon[34351]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:26.525 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:26 smithi059 systemd[1]: Starting Ceph mds.b for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:22:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:26 smithi019 ceph-mon[30158]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:26.749 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:22:26.776 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:26 smithi059 podman[65124]: 2024-05-09 01:22:26.430516092 +0000 UTC m=+0.018536561 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:22:26.776 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:26 smithi059 podman[65124]: 2024-05-09 01:22:26.531652032 +0000 UTC m=+0.119672491 container create b3d7d32a0ba77efdb56d691f69dc014d325a1ca2b4e55a3ccceae7009d586ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:22:26.776 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:26 smithi059 podman[65124]: 2024-05-09 01:22:26.742300579 +0000 UTC m=+0.330321040 container init b3d7d32a0ba77efdb56d691f69dc014d325a1ca2b4e55a3ccceae7009d586ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:22:26.776 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:26 smithi059 podman[65124]: 2024-05-09 01:22:26.744891694 +0000 UTC m=+0.332912156 container start b3d7d32a0ba77efdb56d691f69dc014d325a1ca2b4e55a3ccceae7009d586ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:22:26.777 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:26 smithi059 ceph-mds[65143]: -- 172.21.15.59:0/3861794461 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576541afa40 con 0x5576541f7a80 2024-05-09T01:22:26.777 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:26 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: starting mds.b at 2024-05-09T01:22:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:27.097 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:26 smithi059 bash[65124]: b3d7d32a0ba77efdb56d691f69dc014d325a1ca2b4e55a3ccceae7009d586ad7 2024-05-09T01:22:27.097 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:26 smithi059 systemd[1]: Started Ceph mds.b for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:22:28.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:28.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:22:28.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] up:boot 2024-05-09T01:22:28.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: fsmap 12 up:standby 2024-05-09T01:22:28.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:27 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:22:28.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:28.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:22:28.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] up:boot 2024-05-09T01:22:28.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: fsmap 12 up:standby 2024-05-09T01:22:28.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:27 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] up:boot 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: fsmap 12 up:standby 2024-05-09T01:22:28.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:27 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:22:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.661+0000 7f5c50b6d700 1 -- 172.21.15.19:0/241792654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c1034e0 msgr2=0x7f5c4c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.661+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/241792654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c1034e0 0x7f5c4c1038f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0099c0 tx=0x7f5c3c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.661+0000 7f5c50b6d700 1 -- 172.21.15.19:0/241792654 shutdown_connections 2024-05-09T01:22:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.661+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/241792654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c108760 0x7f5c4c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.661+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/241792654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c1034e0 0x7f5c4c1038f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.661+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/241792654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c107e10 0x7f5c4c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.661+0000 7f5c50b6d700 1 -- 172.21.15.19:0/241792654 >> 172.21.15.19:0/241792654 conn(0x7f5c4c0fef30 msgr2=0x7f5c4c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.661+0000 7f5c50b6d700 1 -- 172.21.15.19:0/241792654 shutdown_connections 2024-05-09T01:22:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.661+0000 7f5c50b6d700 1 -- 172.21.15.19:0/241792654 wait complete. 2024-05-09T01:22:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.662+0000 7f5c50b6d700 1 Processor -- start 2024-05-09T01:22:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.662+0000 7f5c50b6d700 1 -- start start 2024-05-09T01:22:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c50b6d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c1034e0 0x7f5c4c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c50b6d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c107e10 0x7f5c4c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c50b6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108760 0x7f5c4c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c50b6d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c113390 con 0x7f5c4c107e10 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c50b6d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c113210 con 0x7f5c4c108760 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c50b6d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c113510 con 0x7f5c4c1034e0 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c49d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c107e10 0x7f5c4c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c49d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c107e10 0x7f5c4c19c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40006/0 (socket says 172.21.15.19:40006) 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c49d9b700 1 -- 172.21.15.19:0/1447423246 learned_addr learned my addr 172.21.15.19:0/1447423246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c4a59c700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c1034e0 0x7f5c4c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c4ad9d700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108760 0x7f5c4c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c49d9b700 1 -- 172.21.15.19:0/1447423246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c1034e0 msgr2=0x7f5c4c19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c49d9b700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c1034e0 0x7f5c4c19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c49d9b700 1 -- 172.21.15.19:0/1447423246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108760 msgr2=0x7f5c4c19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c49d9b700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108760 0x7f5c4c19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c49d9b700 1 -- 172.21.15.19:0/1447423246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c1a7000 con 0x7f5c4c107e10 2024-05-09T01:22:28.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c4a59c700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c1034e0 0x7f5c4c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:22:28.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.663+0000 7f5c4ad9d700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108760 0x7f5c4c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:22:28.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.664+0000 7f5c49d9b700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c107e10 0x7f5c4c19c1f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000e970 tx=0x7f5c4000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:28.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.664+0000 7f5c3b7fe700 1 -- 172.21.15.19:0/1447423246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40004a30 con 0x7f5c4c107e10 2024-05-09T01:22:28.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.664+0000 7f5c3b7fe700 1 -- 172.21.15.19:0/1447423246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c4000cc90 con 0x7f5c4c107e10 2024-05-09T01:22:28.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.664+0000 7f5c3b7fe700 1 -- 172.21.15.19:0/1447423246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40005610 con 0x7f5c4c107e10 2024-05-09T01:22:28.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.664+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4c1a72f0 con 0x7f5c4c107e10 2024-05-09T01:22:28.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.664+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4c1a7820 con 0x7f5c4c107e10 2024-05-09T01:22:28.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.665+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4c0512c0 con 0x7f5c4c107e10 2024-05-09T01:22:28.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.667+0000 7f5c3b7fe700 1 -- 172.21.15.19:0/1447423246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5c4000c7e0 con 0x7f5c4c107e10 2024-05-09T01:22:28.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.668+0000 7f5c3b7fe700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2900 0x7f5c340b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:28.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.668+0000 7f5c4a59c700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2900 0x7f5c340b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:28.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.668+0000 7f5c3b7fe700 1 -- 172.21.15.19:0/1447423246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f5c400e1ed0 con 0x7f5c4c107e10 2024-05-09T01:22:28.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.668+0000 7f5c4a59c700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2900 0x7f5c340b4e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c000c00 tx=0x7f5c3c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:28.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.673+0000 7f5c3b7fe700 1 -- 172.21.15.19:0/1447423246 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c400a9ff0 con 0x7f5c4c107e10 2024-05-09T01:22:28.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.947+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c4c02e5f0 con 0x7f5c4c107e10 2024-05-09T01:22:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.948+0000 7f5c3b7fe700 1 -- 172.21.15.19:0/1447423246 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7f5c400a98f0 con 0x7f5c4c107e10 2024-05-09T01:22:28.946 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:22:28.946 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-09T01:22:28.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2900 msgr2=0x7f5c340b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:28.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2900 0x7f5c340b4e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c000c00 tx=0x7f5c3c01a040 comp rx=0 tx=0).stop 2024-05-09T01:22:28.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c107e10 msgr2=0x7f5c4c19c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:28.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c107e10 0x7f5c4c19c1f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000e970 tx=0x7f5c4000ec80 comp rx=0 tx=0).stop 2024-05-09T01:22:28.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 shutdown_connections 2024-05-09T01:22:28.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108760 0x7f5c4c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c1034e0 0x7f5c4c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2900 0x7f5c340b4e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 --2- 172.21.15.19:0/1447423246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c107e10 0x7f5c4c19c1f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.951+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 >> 172.21.15.19:0/1447423246 conn(0x7f5c4c0fef30 msgr2=0x7f5c4c1096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.952+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 shutdown_connections 2024-05-09T01:22:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:28.952+0000 7f5c50b6d700 1 -- 172.21.15.19:0/1447423246 wait complete. 2024-05-09T01:22:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-05-09T01:22:29.674 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-09T01:22:29.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:29.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:29.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:29.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:29 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:29.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:29 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1447423246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:29.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:29 smithi059 ceph-mon[34351]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:29.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:29 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:29 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1447423246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:29 smithi019 ceph-mon[30158]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:30.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:30.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:30.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:30.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:29 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:30.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:29 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1447423246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:22:30.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:29 smithi196 ceph-mon[35041]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.663+0000 7f673d2a6700 1 -- 172.21.15.19:0/2199946115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 msgr2=0x7f67381100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.663+0000 7f673d2a6700 1 --2- 172.21.15.19:0/2199946115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 0x7f67381100a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f67300099c0 tx=0x7f6730009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.663+0000 7f673d2a6700 1 -- 172.21.15.19:0/2199946115 shutdown_connections 2024-05-09T01:22:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.663+0000 7f673d2a6700 1 --2- 172.21.15.19:0/2199946115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 0x7f67381100a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.663+0000 7f673d2a6700 1 --2- 172.21.15.19:0/2199946115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67381034c0 0x7f67381038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.663+0000 7f673d2a6700 1 --2- 172.21.15.19:0/2199946115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6738107df0 0x7f6738108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.663+0000 7f673d2a6700 1 -- 172.21.15.19:0/2199946115 >> 172.21.15.19:0/2199946115 conn(0x7f67380fedc0 msgr2=0x7f67381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.664+0000 7f673d2a6700 1 -- 172.21.15.19:0/2199946115 shutdown_connections 2024-05-09T01:22:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.664+0000 7f673d2a6700 1 -- 172.21.15.19:0/2199946115 wait complete. 2024-05-09T01:22:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.664+0000 7f673d2a6700 1 Processor -- start 2024-05-09T01:22:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.664+0000 7f673d2a6700 1 -- start start 2024-05-09T01:22:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f673d2a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67381034c0 0x7f6738199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f673d2a6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6738107df0 0x7f673819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f673d2a6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 0x7f673819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f673d2a6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6738113440 con 0x7f6738108740 2024-05-09T01:22:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f673d2a6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67381132c0 con 0x7f67381034c0 2024-05-09T01:22:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f673d2a6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67381135c0 con 0x7f6738107df0 2024-05-09T01:22:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 0x7f673819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 0x7f673819e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54846/0 (socket says 172.21.15.19:54846) 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67377fe700 1 -- 172.21.15.19:0/3178777029 learned_addr learned my addr 172.21.15.19:0/3178777029 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f6736ffd700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67381034c0 0x7f6738199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67367fc700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6738107df0 0x7f673819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67377fe700 1 -- 172.21.15.19:0/3178777029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6738107df0 msgr2=0x7f673819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67377fe700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6738107df0 0x7f673819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67377fe700 1 -- 172.21.15.19:0/3178777029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67381034c0 msgr2=0x7f6738199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67377fe700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67381034c0 0x7f6738199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67377fe700 1 -- 172.21.15.19:0/3178777029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673819ec00 con 0x7f6738108740 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.665+0000 7f67367fc700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6738107df0 0x7f673819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.666+0000 7f67377fe700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 0x7f673819e560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6730009990 tx=0x7f67300057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.666+0000 7f671ffff700 1 -- 172.21.15.19:0/3178777029 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730027070 con 0x7f6738108740 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.666+0000 7f671ffff700 1 -- 172.21.15.19:0/3178777029 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6730003f20 con 0x7f6738108740 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.666+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673819ee90 con 0x7f6738108740 2024-05-09T01:22:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.666+0000 7f671ffff700 1 -- 172.21.15.19:0/3178777029 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730022be0 con 0x7f6738108740 2024-05-09T01:22:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.666+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673806b610 con 0x7f6738108740 2024-05-09T01:22:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.667+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6738104970 con 0x7f6738108740 2024-05-09T01:22:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.669+0000 7f671ffff700 1 -- 172.21.15.19:0/3178777029 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f673002a070 con 0x7f6738108740 2024-05-09T01:22:31.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.670+0000 7f671ffff700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f67200b2900 0x7f67200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:31.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.670+0000 7f6736ffd700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f67200b2900 0x7f67200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:31.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.670+0000 7f671ffff700 1 -- 172.21.15.19:0/3178777029 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f67300e3090 con 0x7f6738108740 2024-05-09T01:22:31.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.670+0000 7f6736ffd700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f67200b2900 0x7f67200b4e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f672800a400 tx=0x7f672800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.675+0000 7f671ffff700 1 -- 172.21.15.19:0/3178777029 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6730013070 con 0x7f6738108740 2024-05-09T01:22:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.983+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f673819b090 con 0x7f6738108740 2024-05-09T01:22:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.983+0000 7f671ffff700 1 -- 172.21.15.19:0/3178777029 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7f67300ab2c0 con 0x7f6738108740 2024-05-09T01:22:31.982 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:22:31.982 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-05-09T01:22:31.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:22:31.982 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-05-09T01:22:31.982 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-05-09T01:22:31.982 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:22:31.982 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f67200b2900 msgr2=0x7f67200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f67200b2900 0x7f67200b4e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f672800a400 tx=0x7f672800d040 comp rx=0 tx=0).stop 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 msgr2=0x7f673819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 0x7f673819e560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6730009990 tx=0x7f67300057c0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 shutdown_connections 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f67200b2900 0x7f67200b4e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6738108740 0x7f673819e560 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67381034c0 0x7f6738199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 --2- 172.21.15.19:0/3178777029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6738107df0 0x7f673819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.985+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 >> 172.21.15.19:0/3178777029 conn(0x7f67380fedc0 msgr2=0x7f6738107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.986+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 shutdown_connections 2024-05-09T01:22:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:31.986+0000 7f673d2a6700 1 -- 172.21.15.19:0/3178777029 wait complete. 2024-05-09T01:22:32.341 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:32.341 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:32 smithi019 ceph-mon[30158]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:32.341 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:32 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3178777029' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T01:22:32.341 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:32.341 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:32 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:32.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:32.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:32 smithi196 ceph-mon[35041]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:32.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:32 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3178777029' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T01:22:32.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:32.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:32 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:32.550 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:32.550 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:32 smithi059 ceph-mon[34351]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:32.550 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:32 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3178777029' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T01:22:32.550 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:32.550 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:32 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:32.703 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T01:22:32.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T01:22:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:33.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:33 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:33.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:33 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:22:33.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:33 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:33.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:33 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:22:33.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:33 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:33.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:33 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:22:33.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:33 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:33.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:33 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:22:33.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:33 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:33.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:33 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:22:33.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:33 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:33.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:33 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:22:34.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:34 smithi196 ceph-mon[35041]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:34.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:34 smithi059 ceph-mon[34351]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:34.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:34 smithi019 ceph-mon[30158]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:34.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.693+0000 7f50d8747700 1 -- 172.21.15.19:0/1411268241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d0108f10 msgr2=0x7f50d0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:34.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.693+0000 7f50d8747700 1 --2- 172.21.15.19:0/1411268241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d0108f10 0x7f50d0109320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f50c40099c0 tx=0x7f50c4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:34.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.694+0000 7f50d8747700 1 -- 172.21.15.19:0/1411268241 shutdown_connections 2024-05-09T01:22:34.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.694+0000 7f50d8747700 1 --2- 172.21.15.19:0/1411268241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50d0109860 0x7f50d01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:34.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.694+0000 7f50d8747700 1 --2- 172.21.15.19:0/1411268241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d01045e0 0x7f50d01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:34.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.694+0000 7f50d8747700 1 --2- 172.21.15.19:0/1411268241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d0108f10 0x7f50d0109320 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:34.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.694+0000 7f50d8747700 1 -- 172.21.15.19:0/1411268241 >> 172.21.15.19:0/1411268241 conn(0x7f50d00ffe70 msgr2=0x7f50d01022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:34.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.694+0000 7f50d8747700 1 -- 172.21.15.19:0/1411268241 shutdown_connections 2024-05-09T01:22:34.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.694+0000 7f50d8747700 1 -- 172.21.15.19:0/1411268241 wait complete. 2024-05-09T01:22:34.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d8747700 1 Processor -- start 2024-05-09T01:22:34.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d8747700 1 -- start start 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d8747700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d01045e0 0x7f50d019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d8747700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d0108f10 0x7f50d019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d8747700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50d0109860 0x7f50d019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d8747700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50d0114470 con 0x7f50d0108f10 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d8747700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50d01142f0 con 0x7f50d01045e0 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d8747700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50d01145f0 con 0x7f50d0109860 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d64e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d01045e0 0x7f50d019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d64e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d01045e0 0x7f50d019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60472/0 (socket says 172.21.15.19:60472) 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d64e3700 1 -- 172.21.15.19:0/2504816760 learned_addr learned my addr 172.21.15.19:0/2504816760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d6ce4700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50d0109860 0x7f50d019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d5ce2700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d0108f10 0x7f50d019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d64e3700 1 -- 172.21.15.19:0/2504816760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50d0109860 msgr2=0x7f50d019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d64e3700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50d0109860 0x7f50d019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d64e3700 1 -- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d0108f10 msgr2=0x7f50d019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.695+0000 7f50d64e3700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d0108f10 0x7f50d019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:34.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.696+0000 7f50d64e3700 1 -- 172.21.15.19:0/2504816760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d019fcf0 con 0x7f50d01045e0 2024-05-09T01:22:34.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.696+0000 7f50d64e3700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d01045e0 0x7f50d019ae20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ef10 tx=0x7f50c000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:34.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.696+0000 7f50cf7fe700 1 -- 172.21.15.19:0/2504816760 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c000c9c0 con 0x7f50d01045e0 2024-05-09T01:22:34.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.696+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d019ff80 con 0x7f50d01045e0 2024-05-09T01:22:34.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.696+0000 7f50cf7fe700 1 -- 172.21.15.19:0/2504816760 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c0019070 con 0x7f50d01045e0 2024-05-09T01:22:34.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.696+0000 7f50cf7fe700 1 -- 172.21.15.19:0/2504816760 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0014480 con 0x7f50d01045e0 2024-05-09T01:22:34.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.696+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d01a0490 con 0x7f50d01045e0 2024-05-09T01:22:34.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.697+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d0105a90 con 0x7f50d01045e0 2024-05-09T01:22:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.700+0000 7f50cf7fe700 1 -- 172.21.15.19:0/2504816760 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f50c0012050 con 0x7f50d01045e0 2024-05-09T01:22:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.701+0000 7f50cf7fe700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50b80b2880 0x7f50b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.701+0000 7f50d5ce2700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50b80b2880 0x7f50b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.701+0000 7f50cf7fe700 1 -- 172.21.15.19:0/2504816760 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f50c00e2470 con 0x7f50d01045e0 2024-05-09T01:22:34.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.701+0000 7f50d5ce2700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50b80b2880 0x7f50b80b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000c00 tx=0x7f50c401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:34.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.703+0000 7f50cf7fe700 1 -- 172.21.15.19:0/2504816760 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50c0010040 con 0x7f50d01045e0 2024-05-09T01:22:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:34.925+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f50d002e5f0 con 0x7f50d01045e0 2024-05-09T01:22:35.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.787+0000 7f50cf7fe700 1 -- 172.21.15.19:0/2504816760 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7f50c00aa590 con 0x7f50d01045e0 2024-05-09T01:22:35.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50b80b2880 msgr2=0x7f50b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:35.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50b80b2880 0x7f50b80b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000c00 tx=0x7f50c401a040 comp rx=0 tx=0).stop 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d01045e0 msgr2=0x7f50d019ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d01045e0 0x7f50d019ae20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ef10 tx=0x7f50c000c3a0 comp rx=0 tx=0).stop 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 shutdown_connections 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50d0109860 0x7f50d019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d01045e0 0x7f50d019ae20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50b80b2880 0x7f50b80b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 --2- 172.21.15.19:0/2504816760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d0108f10 0x7f50d019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 >> 172.21.15.19:0/2504816760 conn(0x7f50d00ffe70 msgr2=0x7f50d01086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 shutdown_connections 2024-05-09T01:22:35.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:35.791+0000 7f50d8747700 1 -- 172.21.15.19:0/2504816760 wait complete. 2024-05-09T01:22:35.791 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-05-09T01:22:36.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:35 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:35 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2504816760' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T01:22:36.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:35 smithi059 ceph-mon[34351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T01:22:36.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:35 smithi059 ceph-mon[34351]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:36.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:35 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:35 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2504816760' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T01:22:36.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:35 smithi019 ceph-mon[30158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T01:22:36.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:35 smithi019 ceph-mon[30158]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:36.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:35 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:36.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:35 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2504816760' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T01:22:36.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:35 smithi196 ceph-mon[35041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T01:22:36.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:35 smithi196 ceph-mon[35041]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:36.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T01:22:36.715 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:37.038 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:37.038 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:37.038 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:37.038 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:22:37.038 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:37.038 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:36 smithi019 ceph-mon[30158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T01:22:37.038 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:36 smithi019 ceph-mon[30158]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T01:22:37.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:37.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:37.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:37.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:22:37.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:37.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:36 smithi059 ceph-mon[34351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T01:22:37.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:36 smithi059 ceph-mon[34351]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T01:22:37.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:37.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:37.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:22:37.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:22:37.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:22:37.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:36 smithi196 ceph-mon[35041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T01:22:37.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:36 smithi196 ceph-mon[35041]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T01:22:38.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:37 smithi019 ceph-mon[30158]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T01:22:38.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:37 smithi019 ceph-mon[30158]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:38.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:37 smithi059 ceph-mon[34351]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T01:22:38.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:37 smithi059 ceph-mon[34351]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:38.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:37 smithi196 ceph-mon[35041]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T01:22:38.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:37 smithi196 ceph-mon[35041]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:38.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.313+0000 7efdd6ace700 1 -- 172.21.15.19:0/2766395930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd01034e0 msgr2=0x7efdd01038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.313+0000 7efdd6ace700 1 --2- 172.21.15.19:0/2766395930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd01034e0 0x7efdd01038f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efdc00099c0 tx=0x7efdc0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.313+0000 7efdd6ace700 1 -- 172.21.15.19:0/2766395930 shutdown_connections 2024-05-09T01:22:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.313+0000 7efdd6ace700 1 --2- 172.21.15.19:0/2766395930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdd0108760 0x7efdd01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.313+0000 7efdd6ace700 1 --2- 172.21.15.19:0/2766395930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd01034e0 0x7efdd01038f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.313+0000 7efdd6ace700 1 --2- 172.21.15.19:0/2766395930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd0107e10 0x7efdd0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.313+0000 7efdd6ace700 1 -- 172.21.15.19:0/2766395930 >> 172.21.15.19:0/2766395930 conn(0x7efdd00fef50 msgr2=0x7efdd0101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.313+0000 7efdd6ace700 1 -- 172.21.15.19:0/2766395930 shutdown_connections 2024-05-09T01:22:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.313+0000 7efdd6ace700 1 -- 172.21.15.19:0/2766395930 wait complete. 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.314+0000 7efdd6ace700 1 Processor -- start 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.314+0000 7efdd6ace700 1 -- start start 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd6ace700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdd01034e0 0x7efdd00ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd6ace700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd0107e10 0x7efdd00ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd6ace700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd0108760 0x7efdd00ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd6ace700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efdd0113390 con 0x7efdd0108760 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd6ace700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efdd0113210 con 0x7efdd01034e0 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd6ace700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efdd0113510 con 0x7efdd0107e10 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd506b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd0108760 0x7efdd00ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd506b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd0108760 0x7efdd00ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54876/0 (socket says 172.21.15.19:54876) 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd506b700 1 -- 172.21.15.19:0/3211492965 learned_addr learned my addr 172.21.15.19:0/3211492965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd486a700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdd01034e0 0x7efdd00ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdcffff700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd0107e10 0x7efdd00ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd506b700 1 -- 172.21.15.19:0/3211492965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd0107e10 msgr2=0x7efdd00ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd506b700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd0107e10 0x7efdd00ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd506b700 1 -- 172.21.15.19:0/3211492965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdd01034e0 msgr2=0x7efdd00ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd506b700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdd01034e0 0x7efdd00ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd506b700 1 -- 172.21.15.19:0/3211492965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd01a9250 con 0x7efdd0108760 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdcffff700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd0107e10 0x7efdd00ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.315+0000 7efdd506b700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd0108760 0x7efdd00ffeb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efdc8007ee0 tx=0x7efdc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.316+0000 7efdcdffb700 1 -- 172.21.15.19:0/3211492965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc800dbd0 con 0x7efdd0108760 2024-05-09T01:22:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.316+0000 7efdcdffb700 1 -- 172.21.15.19:0/3211492965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc800dd70 con 0x7efdd0108760 2024-05-09T01:22:38.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.316+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd01a9540 con 0x7efdd0108760 2024-05-09T01:22:38.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.316+0000 7efdcdffb700 1 -- 172.21.15.19:0/3211492965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8005990 con 0x7efdd0108760 2024-05-09T01:22:38.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.316+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd01a9a80 con 0x7efdd0108760 2024-05-09T01:22:38.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.317+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd00512c0 con 0x7efdd0108760 2024-05-09T01:22:38.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.319+0000 7efdcdffb700 1 -- 172.21.15.19:0/3211492965 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efdc801a020 con 0x7efdd0108760 2024-05-09T01:22:38.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.320+0000 7efdcdffb700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efdb80b2900 0x7efdb80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:38.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.320+0000 7efdd486a700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efdb80b2900 0x7efdb80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:38.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.320+0000 7efdcdffb700 1 -- 172.21.15.19:0/3211492965 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7efdc80e1c00 con 0x7efdd0108760 2024-05-09T01:22:38.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.320+0000 7efdd486a700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efdb80b2900 0x7efdb80b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009990 tx=0x7efdc0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:38.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.324+0000 7efdcdffb700 1 -- 172.21.15.19:0/3211492965 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efdc8010070 con 0x7efdd0108760 2024-05-09T01:22:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.550+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efdd002e5f0 con 0x7efdd0108760 2024-05-09T01:22:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.802+0000 7efdcdffb700 1 -- 172.21.15.19:0/3211492965 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7efdc80a9c50 con 0x7efdd0108760 2024-05-09T01:22:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.804+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efdb80b2900 msgr2=0x7efdb80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.804+0000 7efdd6ace700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efdb80b2900 0x7efdb80b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009990 tx=0x7efdc0000bc0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd0108760 msgr2=0x7efdd00ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd0108760 0x7efdd00ffeb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efdc8007ee0 tx=0x7efdc800d3b0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 shutdown_connections 2024-05-09T01:22:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efdb80b2900 0x7efdb80b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd0108760 0x7efdd00ffeb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdd01034e0 0x7efdd00ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 --2- 172.21.15.19:0/3211492965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd0107e10 0x7efdd00ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 >> 172.21.15.19:0/3211492965 conn(0x7efdd00fef50 msgr2=0x7efdd0075e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 shutdown_connections 2024-05-09T01:22:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:38.805+0000 7efdd6ace700 1 -- 172.21.15.19:0/3211492965 wait complete. 2024-05-09T01:22:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-05-09T01:22:39.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:38 smithi059 ceph-mon[34351]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T01:22:39.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:38 smithi059 ceph-mon[34351]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T01:22:39.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:38 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3211492965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T01:22:39.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:38 smithi019 ceph-mon[30158]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T01:22:39.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:38 smithi019 ceph-mon[30158]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T01:22:39.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:38 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3211492965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T01:22:39.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:38 smithi196 ceph-mon[35041]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T01:22:39.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:38 smithi196 ceph-mon[35041]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T01:22:39.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:38 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3211492965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T01:22:39.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T01:22:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:40.029 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:39 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3211492965' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T01:22:40.030 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:39 smithi019 ceph-mon[30158]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T01:22:40.030 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:39 smithi019 ceph-mon[30158]: pgmap v287: 97 pgs: 12 creating+peering, 23 active+clean, 62 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:40.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:39 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3211492965' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T01:22:40.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:39 smithi059 ceph-mon[34351]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T01:22:40.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:39 smithi059 ceph-mon[34351]: pgmap v287: 97 pgs: 12 creating+peering, 23 active+clean, 62 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:40.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:39 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3211492965' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T01:22:40.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:39 smithi196 ceph-mon[35041]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T01:22:40.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:39 smithi196 ceph-mon[35041]: pgmap v287: 97 pgs: 12 creating+peering, 23 active+clean, 62 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:41.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:40 smithi059 ceph-mon[34351]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T01:22:41.104 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:40 smithi019 ceph-mon[30158]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T01:22:41.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:40 smithi196 ceph-mon[35041]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T01:22:41.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.437+0000 7f26aa8b2700 1 -- 172.21.15.19:0/929923865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4107df0 msgr2=0x7f26a4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:41.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.437+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/929923865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4107df0 0x7f26a4108200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f268c0099c0 tx=0x7f268c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.438+0000 7f26aa8b2700 1 -- 172.21.15.19:0/929923865 shutdown_connections 2024-05-09T01:22:41.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.438+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/929923865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26a4108740 0x7f26a41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.438+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/929923865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26a41034c0 0x7f26a41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.438+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/929923865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4107df0 0x7f26a4108200 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.438+0000 7f26aa8b2700 1 -- 172.21.15.19:0/929923865 >> 172.21.15.19:0/929923865 conn(0x7f26a40fedf0 msgr2=0x7f26a4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.438+0000 7f26aa8b2700 1 -- 172.21.15.19:0/929923865 shutdown_connections 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.438+0000 7f26aa8b2700 1 -- 172.21.15.19:0/929923865 wait complete. 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.438+0000 7f26aa8b2700 1 Processor -- start 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26aa8b2700 1 -- start start 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26aa8b2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26a41034c0 0x7f26a419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26aa8b2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26a4107df0 0x7f26a419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26aa8b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4108740 0x7f26a419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26aa8b2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4113350 con 0x7f26a4108740 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26aa8b2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41131d0 con 0x7f26a4107df0 2024-05-09T01:22:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26aa8b2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41134d0 con 0x7f26a41034c0 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26a8e4f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4108740 0x7f26a419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f269bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26a4107df0 0x7f26a419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26a8e4f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4108740 0x7f26a419c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42720/0 (socket says 172.21.15.19:42720) 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26a8e4f700 1 -- 172.21.15.19:0/732159496 learned_addr learned my addr 172.21.15.19:0/732159496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26a3fff700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26a41034c0 0x7f26a419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26a8e4f700 1 -- 172.21.15.19:0/732159496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26a41034c0 msgr2=0x7f26a419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26a8e4f700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26a41034c0 0x7f26a419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26a8e4f700 1 -- 172.21.15.19:0/732159496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26a4107df0 msgr2=0x7f26a419c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26a8e4f700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26a4107df0 0x7f26a419c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.439+0000 7f26a8e4f700 1 -- 172.21.15.19:0/732159496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a41a7010 con 0x7f26a4108740 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.440+0000 7f26a8e4f700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4108740 0x7f26a419c790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f269c00ada0 tx=0x7f269c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.440+0000 7f26a1ffb700 1 -- 172.21.15.19:0/732159496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c017020 con 0x7f26a4108740 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.440+0000 7f26a1ffb700 1 -- 172.21.15.19:0/732159496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f269c00cd80 con 0x7f26a4108740 2024-05-09T01:22:41.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.440+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a41a7300 con 0x7f26a4108740 2024-05-09T01:22:41.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.440+0000 7f26a1ffb700 1 -- 172.21.15.19:0/732159496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c005260 con 0x7f26a4108740 2024-05-09T01:22:41.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.440+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a41a77c0 con 0x7f26a4108740 2024-05-09T01:22:41.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.441+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a40512c0 con 0x7f26a4108740 2024-05-09T01:22:41.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.443+0000 7f26a1ffb700 1 -- 172.21.15.19:0/732159496 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f269c00a090 con 0x7f26a4108740 2024-05-09T01:22:41.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.443+0000 7f26a1ffb700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26900b2880 0x7f26900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:41.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.443+0000 7f26a3fff700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26900b2880 0x7f26900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:41.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.444+0000 7f26a1ffb700 1 -- 172.21.15.19:0/732159496 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f269c0e1ce0 con 0x7f26a4108740 2024-05-09T01:22:41.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.444+0000 7f26a3fff700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26900b2880 0x7f26900b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2694005950 tx=0x7f269400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:41.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.447+0000 7f26a1ffb700 1 -- 172.21.15.19:0/732159496 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f269c021020 con 0x7f26a4108740 2024-05-09T01:22:41.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.709+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f26a402e5f0 con 0x7f26a4108740 2024-05-09T01:22:41.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.822+0000 7f26a1ffb700 1 -- 172.21.15.19:0/732159496 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f269c0a9c40 con 0x7f26a4108740 2024-05-09T01:22:41.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26900b2880 msgr2=0x7f26900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:41.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26900b2880 0x7f26900b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2694005950 tx=0x7f269400b410 comp rx=0 tx=0).stop 2024-05-09T01:22:41.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4108740 msgr2=0x7f26a419c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:41.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4108740 0x7f26a419c790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f269c00ada0 tx=0x7f269c00c330 comp rx=0 tx=0).stop 2024-05-09T01:22:41.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 shutdown_connections 2024-05-09T01:22:41.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26900b2880 0x7f26900b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4108740 0x7f26a419c790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26a41034c0 0x7f26a419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 --2- 172.21.15.19:0/732159496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26a4107df0 0x7f26a419c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:41.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 >> 172.21.15.19:0/732159496 conn(0x7f26a40fedf0 msgr2=0x7f26a41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:41.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 shutdown_connections 2024-05-09T01:22:41.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:41.825+0000 7f26aa8b2700 1 -- 172.21.15.19:0/732159496 wait complete. 2024-05-09T01:22:41.824 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T01:22:42.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:41 smithi059 ceph-mon[34351]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T01:22:42.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:41 smithi059 ceph-mon[34351]: pgmap v290: 97 pgs: 24 creating+peering, 47 active+clean, 26 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:42.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:41 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/732159496' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T01:22:42.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:41 smithi059 ceph-mon[34351]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T01:22:42.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:41 smithi019 ceph-mon[30158]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T01:22:42.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:41 smithi019 ceph-mon[30158]: pgmap v290: 97 pgs: 24 creating+peering, 47 active+clean, 26 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:42.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:41 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/732159496' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T01:22:42.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:41 smithi019 ceph-mon[30158]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T01:22:42.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:41 smithi196 ceph-mon[35041]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T01:22:42.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:41 smithi196 ceph-mon[35041]: pgmap v290: 97 pgs: 24 creating+peering, 47 active+clean, 26 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:42.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:41 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/732159496' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T01:22:42.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:41 smithi196 ceph-mon[35041]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T01:22:42.515 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:22:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:43.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:42 smithi059 ceph-mon[34351]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T01:22:43.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:42 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/732159496' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T01:22:43.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:42 smithi059 ceph-mon[34351]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T01:22:43.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:42 smithi059 ceph-mon[34351]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T01:22:43.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:42 smithi019 ceph-mon[30158]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T01:22:43.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:42 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/732159496' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T01:22:43.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:42 smithi019 ceph-mon[30158]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T01:22:43.133 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:42 smithi019 ceph-mon[30158]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T01:22:43.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:42 smithi196 ceph-mon[35041]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T01:22:43.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:42 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/732159496' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T01:22:43.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:42 smithi196 ceph-mon[35041]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T01:22:43.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:42 smithi196 ceph-mon[35041]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T01:22:43.854 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:43 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] up:active 2024-05-09T01:22:43.855 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:43 smithi019 ceph-mon[30158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:43.855 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:43 smithi019 ceph-mon[30158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:43.855 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:43 smithi019 ceph-mon[30158]: pgmap v292: 97 pgs: 24 creating+peering, 54 active+clean, 19 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:43.855 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:43 smithi019 ceph-mon[30158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T01:22:44.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:43 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] up:active 2024-05-09T01:22:44.098 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:43 smithi059 ceph-mon[34351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:44.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:43 smithi059 ceph-mon[34351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:44.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:43 smithi059 ceph-mon[34351]: pgmap v292: 97 pgs: 24 creating+peering, 54 active+clean, 19 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:44.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:43 smithi059 ceph-mon[34351]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T01:22:44.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:43 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] up:active 2024-05-09T01:22:44.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:43 smithi196 ceph-mon[35041]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:44.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:43 smithi196 ceph-mon[35041]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:44.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:43 smithi196 ceph-mon[35041]: pgmap v292: 97 pgs: 24 creating+peering, 54 active+clean, 19 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:22:44.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:43 smithi196 ceph-mon[35041]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T01:22:44.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.400+0000 7f6f6f5af700 1 -- 172.21.15.19:0/1537477067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f681034c0 msgr2=0x7f6f681038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:44.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.400+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/1537477067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f681034c0 0x7f6f681038d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6f640099c0 tx=0x7f6f64009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.400+0000 7f6f6f5af700 1 -- 172.21.15.19:0/1537477067 shutdown_connections 2024-05-09T01:22:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.400+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/1537477067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68108740 0x7f6f681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.400+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/1537477067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f681034c0 0x7f6f681038d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.400+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/1537477067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f68107df0 0x7f6f68108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.400+0000 7f6f6f5af700 1 -- 172.21.15.19:0/1537477067 >> 172.21.15.19:0/1537477067 conn(0x7f6f680fedf0 msgr2=0x7f6f68101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.400+0000 7f6f6f5af700 1 -- 172.21.15.19:0/1537477067 shutdown_connections 2024-05-09T01:22:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.400+0000 7f6f6f5af700 1 -- 172.21.15.19:0/1537477067 wait complete. 2024-05-09T01:22:44.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6f5af700 1 Processor -- start 2024-05-09T01:22:44.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6f5af700 1 -- start start 2024-05-09T01:22:44.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6f5af700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f681034c0 0x7f6f6819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:44.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6f5af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68107df0 0x7f6f6819c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:44.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6f5af700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f68108740 0x7f6f6819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:44.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6f5af700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68113370 con 0x7f6f68108740 2024-05-09T01:22:44.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6f5af700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681131f0 con 0x7f6f68107df0 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6f5af700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681134f0 con 0x7f6f681034c0 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6db4c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f68108740 0x7f6f6819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6db4c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f68108740 0x7f6f6819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42728/0 (socket says 172.21.15.19:42728) 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6db4c700 1 -- 172.21.15.19:0/2290933504 learned_addr learned my addr 172.21.15.19:0/2290933504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.401+0000 7f6f6cb4a700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68107df0 0x7f6f6819c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f6db4c700 1 -- 172.21.15.19:0/2290933504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f681034c0 msgr2=0x7f6f6819bc70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f6db4c700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f681034c0 0x7f6f6819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f6db4c700 1 -- 172.21.15.19:0/2290933504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68107df0 msgr2=0x7f6f6819c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f6db4c700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68107df0 0x7f6f6819c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f6db4c700 1 -- 172.21.15.19:0/2290933504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f681a6fc0 con 0x7f6f68108740 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f6cb4a700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68107df0 0x7f6f6819c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:22:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f6db4c700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f68108740 0x7f6f6819c710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000ef30 tx=0x7f6f6000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f5e7fc700 1 -- 172.21.15.19:0/2290933504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f600092e0 con 0x7f6f68108740 2024-05-09T01:22:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f5e7fc700 1 -- 172.21.15.19:0/2290933504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f60004bb0 con 0x7f6f68108740 2024-05-09T01:22:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f5e7fc700 1 -- 172.21.15.19:0/2290933504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60004280 con 0x7f6f68108740 2024-05-09T01:22:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f681a7250 con 0x7f6f68108740 2024-05-09T01:22:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.402+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f681a7740 con 0x7f6f68108740 2024-05-09T01:22:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.403+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f680512c0 con 0x7f6f68108740 2024-05-09T01:22:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.405+0000 7f6f5e7fc700 1 -- 172.21.15.19:0/2290933504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6f6001b030 con 0x7f6f68108740 2024-05-09T01:22:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.406+0000 7f6f5e7fc700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f540b2880 0x7f6f540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.406+0000 7f6f6d34b700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f540b2880 0x7f6f540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.406+0000 7f6f5e7fc700 1 -- 172.21.15.19:0/2290933504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f6f600e26a0 con 0x7f6f68108740 2024-05-09T01:22:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.407+0000 7f6f6d34b700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f540b2880 0x7f6f540b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f640098e0 tx=0x7f6f64000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.410+0000 7f6f5e7fc700 1 -- 172.21.15.19:0/2290933504 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f60022020 con 0x7f6f68108740 2024-05-09T01:22:44.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.628+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f6802e5f0 con 0x7f6f68108740 2024-05-09T01:22:44.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.629+0000 7f6f5e7fc700 1 -- 172.21.15.19:0/2290933504 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22191 (secure 0 0 0) 0x7f6f60010040 con 0x7f6f68108740 2024-05-09T01:22:44.628 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:22:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:22:41.711322+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:22:44.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f540b2880 msgr2=0x7f6f540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:44.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f540b2880 0x7f6f540b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f640098e0 tx=0x7f6f64000bc0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f68108740 msgr2=0x7f6f6819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:44.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f68108740 0x7f6f6819c710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000ef30 tx=0x7f6f6000c450 comp rx=0 tx=0).stop 2024-05-09T01:22:44.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 shutdown_connections 2024-05-09T01:22:44.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f540b2880 0x7f6f540b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f68108740 0x7f6f6819c710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f681034c0 0x7f6f6819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 --2- 172.21.15.19:0/2290933504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68107df0 0x7f6f6819c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:44.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 >> 172.21.15.19:0/2290933504 conn(0x7f6f680fedf0 msgr2=0x7f6f68107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:44.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 shutdown_connections 2024-05-09T01:22:44.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:44.632+0000 7f6f6f5af700 1 -- 172.21.15.19:0/2290933504 wait complete. 2024-05-09T01:22:45.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:44 smithi059 ceph-mon[34351]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T01:22:45.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:44 smithi059 ceph-mon[34351]: Cluster is now healthy 2024-05-09T01:22:45.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:44 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2290933504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:22:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:44 smithi019 ceph-mon[30158]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T01:22:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:44 smithi019 ceph-mon[30158]: Cluster is now healthy 2024-05-09T01:22:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:44 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2290933504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:22:45.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:44 smithi196 ceph-mon[35041]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T01:22:45.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:44 smithi196 ceph-mon[35041]: Cluster is now healthy 2024-05-09T01:22:45.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:44 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2290933504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:22:45.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:22:45.685 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:46.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:45 smithi019 ceph-mon[30158]: pgmap v293: 97 pgs: 97 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T01:22:46.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:45 smithi059 ceph-mon[34351]: pgmap v293: 97 pgs: 97 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T01:22:46.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:45 smithi196 ceph-mon[35041]: pgmap v293: 97 pgs: 97 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T01:22:47.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.285+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2114889993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c107df0 msgr2=0x7f8b2c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:47.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.285+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2114889993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c107df0 0x7f8b2c108200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8b200099c0 tx=0x7f8b20009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.285+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2114889993 shutdown_connections 2024-05-09T01:22:47.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.285+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2114889993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c108740 0x7f8b2c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.285+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2114889993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b2c1034c0 0x7f8b2c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.285+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2114889993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c107df0 0x7f8b2c108200 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.285+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2114889993 >> 172.21.15.19:0/2114889993 conn(0x7f8b2c0fedf0 msgr2=0x7f8b2c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:47.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.285+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2114889993 shutdown_connections 2024-05-09T01:22:47.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.285+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2114889993 wait complete. 2024-05-09T01:22:47.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.286+0000 7f8b31bc8700 1 Processor -- start 2024-05-09T01:22:47.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.286+0000 7f8b31bc8700 1 -- start start 2024-05-09T01:22:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.286+0000 7f8b31bc8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c1034c0 0x7f8b2c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.286+0000 7f8b31bc8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b2c107df0 0x7f8b2c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.286+0000 7f8b31bc8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c108740 0x7f8b2c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.286+0000 7f8b31bc8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c113340 con 0x7f8b2c1034c0 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.286+0000 7f8b31bc8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1131c0 con 0x7f8b2c107df0 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.286+0000 7f8b31bc8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1134c0 con 0x7f8b2c108740 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c1034c0 0x7f8b2c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c1034c0 0x7f8b2c0ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42734/0 (socket says 172.21.15.19:42734) 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2b7fe700 1 -- 172.21.15.19:0/2269043950 learned_addr learned my addr 172.21.15.19:0/2269043950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2affd700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b2c107df0 0x7f8b2c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2bfff700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c108740 0x7f8b2c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2b7fe700 1 -- 172.21.15.19:0/2269043950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c108740 msgr2=0x7f8b2c0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2b7fe700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c108740 0x7f8b2c0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2b7fe700 1 -- 172.21.15.19:0/2269043950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b2c107df0 msgr2=0x7f8b2c0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:47.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2b7fe700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b2c107df0 0x7f8b2c0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2b7fe700 1 -- 172.21.15.19:0/2269043950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c1a91c0 con 0x7f8b2c1034c0 2024-05-09T01:22:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b2b7fe700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c1034c0 0x7f8b2c0ff2d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00d980 tx=0x7f8b1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b28ff9700 1 -- 172.21.15.19:0/2269043950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c016070 con 0x7f8b2c1034c0 2024-05-09T01:22:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.287+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c1a94b0 con 0x7f8b2c1034c0 2024-05-09T01:22:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.288+0000 7f8b28ff9700 1 -- 172.21.15.19:0/2269043950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b1c004830 con 0x7f8b2c1034c0 2024-05-09T01:22:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.288+0000 7f8b28ff9700 1 -- 172.21.15.19:0/2269043950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c005540 con 0x7f8b2c1034c0 2024-05-09T01:22:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.288+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c1a9a10 con 0x7f8b2c1034c0 2024-05-09T01:22:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.288+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c0512c0 con 0x7f8b2c1034c0 2024-05-09T01:22:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.290+0000 7f8b28ff9700 1 -- 172.21.15.19:0/2269043950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8b1c004220 con 0x7f8b2c1034c0 2024-05-09T01:22:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.291+0000 7f8b28ff9700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b140b2880 0x7f8b140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.291+0000 7f8b2affd700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b140b2880 0x7f8b140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.291+0000 7f8b28ff9700 1 -- 172.21.15.19:0/2269043950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f8b1c0e2480 con 0x7f8b2c1034c0 2024-05-09T01:22:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.291+0000 7f8b2affd700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b140b2880 0x7f8b140b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b20006010 tx=0x7f8b20000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:47.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.295+0000 7f8b28ff9700 1 -- 172.21.15.19:0/2269043950 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b1c01f020 con 0x7f8b2c1034c0 2024-05-09T01:22:47.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.508+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b2c02e5f0 con 0x7f8b2c1034c0 2024-05-09T01:22:47.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.510+0000 7f8b28ff9700 1 -- 172.21.15.19:0/2269043950 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22191 (secure 0 0 0) 0x7f8b1c0aa250 con 0x7f8b2c1034c0 2024-05-09T01:22:47.508 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:22:47.509 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:22:41.711322+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:22:47.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.512+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b140b2880 msgr2=0x7f8b140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:47.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.512+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b140b2880 0x7f8b140b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b20006010 tx=0x7f8b20000bc0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.512+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c1034c0 msgr2=0x7f8b2c0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.512+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c1034c0 0x7f8b2c0ff2d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00d980 tx=0x7f8b1c00dc90 comp rx=0 tx=0).stop 2024-05-09T01:22:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.513+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 shutdown_connections 2024-05-09T01:22:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.513+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c108740 0x7f8b2c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.513+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8b140b2880 0x7f8b140b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.513+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b2c1034c0 0x7f8b2c0ff2d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.513+0000 7f8b31bc8700 1 --2- 172.21.15.19:0/2269043950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b2c107df0 0x7f8b2c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.513+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 >> 172.21.15.19:0/2269043950 conn(0x7f8b2c0fedf0 msgr2=0x7f8b2c075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.513+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 shutdown_connections 2024-05-09T01:22:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:47.513+0000 7f8b31bc8700 1 -- 172.21.15.19:0/2269043950 wait complete. 2024-05-09T01:22:47.597 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: 2024-05-09T01:22:47.193+0000 7f49a9dbf700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T01:22:48.248 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-09T01:22:48.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:48 smithi196 ceph-mon[35041]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-09T01:22:48.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:48 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2269043950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:22:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:48.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:48 smithi059 ceph-mon[34351]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-09T01:22:48.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:48 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2269043950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:22:48.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:48 smithi019 ceph-mon[30158]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-09T01:22:48.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:48 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2269043950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:22:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.200+0000 7f7582d29700 1 -- 172.21.15.19:0/616859030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c103c10 msgr2=0x7f757c104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.200+0000 7f7582d29700 1 --2- 172.21.15.19:0/616859030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c103c10 0x7f757c104020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f75780099c0 tx=0x7f7578009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.201+0000 7f7582d29700 1 -- 172.21.15.19:0/616859030 shutdown_connections 2024-05-09T01:22:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.201+0000 7f7582d29700 1 --2- 172.21.15.19:0/616859030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f757c108970 0x7f757c1102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.201+0000 7f7582d29700 1 --2- 172.21.15.19:0/616859030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c103c10 0x7f757c104020 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.201+0000 7f7582d29700 1 --2- 172.21.15.19:0/616859030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f757c108020 0x7f757c108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.201+0000 7f7582d29700 1 -- 172.21.15.19:0/616859030 >> 172.21.15.19:0/616859030 conn(0x7f757c0fe4e0 msgr2=0x7f757c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.201+0000 7f7582d29700 1 -- 172.21.15.19:0/616859030 shutdown_connections 2024-05-09T01:22:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.201+0000 7f7582d29700 1 -- 172.21.15.19:0/616859030 wait complete. 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.201+0000 7f7582d29700 1 Processor -- start 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.201+0000 7f7582d29700 1 -- start start 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7582d29700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f757c103c10 0x7f757c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7582d29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c108020 0x7f757c19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7582d29700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f757c108970 0x7f757c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7582d29700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f757c1135a0 con 0x7f757c108020 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7582d29700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f757c113420 con 0x7f757c108970 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7582d29700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f757c113720 con 0x7f757c103c10 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7573fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c108020 0x7f757c19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7573fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c108020 0x7f757c19b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35172/0 (socket says 172.21.15.19:35172) 2024-05-09T01:22:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f75812c6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f757c108970 0x7f757c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7573fff700 1 -- 172.21.15.19:0/3389660644 learned_addr learned my addr 172.21.15.19:0/3389660644 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7580ac5700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f757c103c10 0x7f757c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7573fff700 1 -- 172.21.15.19:0/3389660644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f757c103c10 msgr2=0x7f757c19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7573fff700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f757c103c10 0x7f757c19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7573fff700 1 -- 172.21.15.19:0/3389660644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f757c108970 msgr2=0x7f757c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7573fff700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f757c108970 0x7f757c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7573fff700 1 -- 172.21.15.19:0/3389660644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c19fde0 con 0x7f757c108020 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.202+0000 7f7580ac5700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f757c103c10 0x7f757c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.203+0000 7f7573fff700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c108020 0x7f757c19b560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f756c00d980 tx=0x7f756c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.203+0000 7f7571ffb700 1 -- 172.21.15.19:0/3389660644 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c017070 con 0x7f757c108020 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.203+0000 7f7571ffb700 1 -- 172.21.15.19:0/3389660644 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f756c009de0 con 0x7f757c108020 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.203+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757c1a00d0 con 0x7f757c108020 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.203+0000 7f7571ffb700 1 -- 172.21.15.19:0/3389660644 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c00f3c0 con 0x7f757c108020 2024-05-09T01:22:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.203+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757c1a0610 con 0x7f757c108020 2024-05-09T01:22:50.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.204+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757c0512c0 con 0x7f757c108020 2024-05-09T01:22:50.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.206+0000 7f7571ffb700 1 -- 172.21.15.19:0/3389660644 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f756c009930 con 0x7f757c108020 2024-05-09T01:22:50.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.206+0000 7f7571ffb700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75640b2880 0x7f75640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:50.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.207+0000 7f7571ffb700 1 -- 172.21.15.19:0/3389660644 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f756c0e1c10 con 0x7f757c108020 2024-05-09T01:22:50.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.207+0000 7f7580ac5700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75640b2880 0x7f75640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:50.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.207+0000 7f7580ac5700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75640b2880 0x7f75640b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7578000c00 tx=0x7f757801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:50.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.211+0000 7f7571ffb700 1 -- 172.21.15.19:0/3389660644 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f756c0a9a80 con 0x7f757c108020 2024-05-09T01:22:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:50 smithi019 ceph-mon[30158]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T01:22:50.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:50 smithi196 ceph-mon[35041]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T01:22:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:50.472+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f757c06c120 con 0x7f757c108020 2024-05-09T01:22:50.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:50 smithi059 ceph-mon[34351]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T01:22:51.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.187+0000 7f7571ffb700 1 -- 172.21.15.19:0/3389660644 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f756c0a9380 con 0x7f757c108020 2024-05-09T01:22:51.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75640b2880 msgr2=0x7f75640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:51.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75640b2880 0x7f75640b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7578000c00 tx=0x7f757801a040 comp rx=0 tx=0).stop 2024-05-09T01:22:51.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c108020 msgr2=0x7f757c19b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c108020 0x7f757c19b560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f756c00d980 tx=0x7f756c00dc90 comp rx=0 tx=0).stop 2024-05-09T01:22:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 shutdown_connections 2024-05-09T01:22:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f757c108970 0x7f757c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f757c103c10 0x7f757c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75640b2880 0x7f75640b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 --2- 172.21.15.19:0/3389660644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757c108020 0x7f757c19b560 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 >> 172.21.15.19:0/3389660644 conn(0x7f757c0fe4e0 msgr2=0x7f757c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 shutdown_connections 2024-05-09T01:22:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:51.191+0000 7f7582d29700 1 -- 172.21.15.19:0/3389660644 wait complete. 2024-05-09T01:22:51.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:51 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3389660644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T01:22:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:51 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3389660644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T01:22:51.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:51 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3389660644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T01:22:51.983 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-09T01:22:51.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-09T01:22:52.245 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:52.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:52 smithi019 ceph-mon[30158]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T01:22:52.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:52 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3389660644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T01:22:52.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:52 smithi019 ceph-mon[30158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:52.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:52 smithi196 ceph-mon[35041]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T01:22:52.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:52 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3389660644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T01:22:52.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:52 smithi196 ceph-mon[35041]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:52.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:52 smithi059 ceph-mon[34351]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T01:22:52.692 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:52 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3389660644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T01:22:52.692 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:52 smithi059 ceph-mon[34351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.940+0000 7f61ab59e700 1 -- 172.21.15.19:0/2434358270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac10e720 msgr2=0x7f61ac110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.940+0000 7f61ab59e700 1 --2- 172.21.15.19:0/2434358270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac10e720 0x7f61ac110c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f61a400b0a0 tx=0x7f61a400b3b0 comp rx=0 tx=0).stop 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.940+0000 7f61ab59e700 1 -- 172.21.15.19:0/2434358270 shutdown_connections 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.940+0000 7f61ab59e700 1 --2- 172.21.15.19:0/2434358270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac10e720 0x7f61ac110c00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.940+0000 7f61ab59e700 1 --2- 172.21.15.19:0/2434358270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61ac1052b0 0x7f61ac1056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.940+0000 7f61ab59e700 1 --2- 172.21.15.19:0/2434358270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ac105c00 0x7f61ac10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.940+0000 7f61ab59e700 1 -- 172.21.15.19:0/2434358270 >> 172.21.15.19:0/2434358270 conn(0x7f61ac0fed00 msgr2=0x7f61ac101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.940+0000 7f61ab59e700 1 -- 172.21.15.19:0/2434358270 shutdown_connections 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.940+0000 7f61ab59e700 1 -- 172.21.15.19:0/2434358270 wait complete. 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.941+0000 7f61ab59e700 1 Processor -- start 2024-05-09T01:22:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.941+0000 7f61ab59e700 1 -- start start 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.941+0000 7f61ab59e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ac1052b0 0x7f61ac1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.941+0000 7f61ab59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac105c00 0x7f61ac1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.941+0000 7f61ab59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61ac10e720 0x7f61ac19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.941+0000 7f61ab59e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac114300 con 0x7f61ac105c00 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.941+0000 7f61ab59e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac114180 con 0x7f61ac10e720 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.941+0000 7f61ab59e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac114480 con 0x7f61ac1052b0 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac105c00 0x7f61ac1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac105c00 0x7f61ac1a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35186/0 (socket says 172.21.15.19:35186) 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61a9d9b700 1 -- 172.21.15.19:0/827727115 learned_addr learned my addr 172.21.15.19:0/827727115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61aa59c700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ac1052b0 0x7f61ac1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61aad9d700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61ac10e720 0x7f61ac19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61a9d9b700 1 -- 172.21.15.19:0/827727115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ac1052b0 msgr2=0x7f61ac1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61a9d9b700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ac1052b0 0x7f61ac1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61a9d9b700 1 -- 172.21.15.19:0/827727115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61ac10e720 msgr2=0x7f61ac19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61a9d9b700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61ac10e720 0x7f61ac19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61a9d9b700 1 -- 172.21.15.19:0/827727115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac19b040 con 0x7f61ac105c00 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61aa59c700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ac1052b0 0x7f61ac1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61a9d9b700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac105c00 0x7f61ac1a0d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f61a000b530 tx=0x7f61a000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f619b7fe700 1 -- 172.21.15.19:0/827727115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0015020 con 0x7f61ac105c00 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac19b330 con 0x7f61ac105c00 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f619b7fe700 1 -- 172.21.15.19:0/827727115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61a0011ce0 con 0x7f61ac105c00 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f619b7fe700 1 -- 172.21.15.19:0/827727115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0010d20 con 0x7f61ac105c00 2024-05-09T01:22:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.942+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac102ed0 con 0x7f61ac105c00 2024-05-09T01:22:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.944+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ac06c930 con 0x7f61ac105c00 2024-05-09T01:22:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.945+0000 7f619b7fe700 1 -- 172.21.15.19:0/827727115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f61a0018020 con 0x7f61ac105c00 2024-05-09T01:22:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.946+0000 7f619b7fe700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f61940b2880 0x7f61940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.946+0000 7f61aa59c700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f61940b2880 0x7f61940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.946+0000 7f619b7fe700 1 -- 172.21.15.19:0/827727115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f61a00e18e0 con 0x7f61ac105c00 2024-05-09T01:22:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.946+0000 7f61aa59c700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f61940b2880 0x7f61940b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f619c005fd0 tx=0x7f619c005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:53.953+0000 7f619b7fe700 1 -- 172.21.15.19:0/827727115 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61a001f020 con 0x7f61ac105c00 2024-05-09T01:22:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:54.217+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f61ac19bf30 con 0x7f61ac105c00 2024-05-09T01:22:54.216 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:54 smithi019 ceph-mon[30158]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T01:22:54.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:54 smithi196 ceph-mon[35041]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T01:22:54.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:54 smithi059 ceph-mon[34351]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T01:22:55.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.200+0000 7f619b7fe700 1 -- 172.21.15.19:0/827727115 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f61a00a9750 con 0x7f61ac105c00 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f61940b2880 msgr2=0x7f61940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f61940b2880 0x7f61940b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f619c005fd0 tx=0x7f619c005e40 comp rx=0 tx=0).stop 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac105c00 msgr2=0x7f61ac1a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac105c00 0x7f61ac1a0d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f61a000b530 tx=0x7f61a000b8f0 comp rx=0 tx=0).stop 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 shutdown_connections 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61ac10e720 0x7f61ac19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ac1052b0 0x7f61ac1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f61940b2880 0x7f61940b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 --2- 172.21.15.19:0/827727115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac105c00 0x7f61ac1a0d20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.204+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 >> 172.21.15.19:0/827727115 conn(0x7f61ac0fed00 msgr2=0x7f61ac0ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.205+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 shutdown_connections 2024-05-09T01:22:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:55.205+0000 7f61ab59e700 1 -- 172.21.15.19:0/827727115 wait complete. 2024-05-09T01:22:55.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:55 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/827727115' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T01:22:55.539 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:55 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/827727115' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T01:22:55.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:55 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/827727115' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T01:22:55.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-09T01:22:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:56.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:56 smithi196 ceph-mon[35041]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T01:22:56.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:56 smithi196 ceph-mon[35041]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T01:22:56.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:56 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/827727115' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T01:22:56.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:56 smithi196 ceph-mon[35041]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T01:22:56.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:56 smithi196 ceph-mon[35041]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:56.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:56 smithi196 ceph-mon[35041]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-09T01:22:56.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:56 smithi196 ceph-mon[35041]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-09T01:22:56.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:56 smithi019 ceph-mon[30158]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T01:22:56.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:56 smithi019 ceph-mon[30158]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T01:22:56.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:56 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/827727115' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T01:22:56.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:56 smithi019 ceph-mon[30158]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T01:22:56.499 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:56 smithi019 ceph-mon[30158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:56.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:56 smithi019 ceph-mon[30158]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-09T01:22:56.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:56 smithi019 ceph-mon[30158]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-09T01:22:56.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:56 smithi059 ceph-mon[34351]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T01:22:56.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:56 smithi059 ceph-mon[34351]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T01:22:56.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:56 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/827727115' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T01:22:56.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:56 smithi059 ceph-mon[34351]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T01:22:56.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:56 smithi059 ceph-mon[34351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T01:22:56.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:56 smithi059 ceph-mon[34351]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-09T01:22:56.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:56 smithi059 ceph-mon[34351]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-09T01:22:57.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:57 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] up:active 2024-05-09T01:22:57.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:57 smithi019 ceph-mon[30158]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T01:22:57.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:57 smithi019 ceph-mon[30158]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-09T01:22:57.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:57 smithi019 ceph-mon[30158]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-09T01:22:57.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:57 smithi019 ceph-mon[30158]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-09T01:22:57.597 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:22:57 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: 2024-05-09T01:22:57.193+0000 7f49a9dbf700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-09T01:22:57.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:57 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] up:active 2024-05-09T01:22:57.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:57 smithi059 ceph-mon[34351]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T01:22:57.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:57 smithi059 ceph-mon[34351]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-09T01:22:57.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:57 smithi059 ceph-mon[34351]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-09T01:22:57.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:57 smithi059 ceph-mon[34351]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-09T01:22:57.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:57 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] up:active 2024-05-09T01:22:57.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:57 smithi196 ceph-mon[35041]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T01:22:57.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:57 smithi196 ceph-mon[35041]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-09T01:22:57.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:57 smithi196 ceph-mon[35041]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-09T01:22:57.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:57 smithi196 ceph-mon[35041]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-09T01:22:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 -- 172.21.15.19:0/1639464565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae58107e10 msgr2=0x7fae58108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 --2- 172.21.15.19:0/1639464565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae58107e10 0x7fae58108220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0099c0 tx=0x7fae4c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:22:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 -- 172.21.15.19:0/1639464565 shutdown_connections 2024-05-09T01:22:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 --2- 172.21.15.19:0/1639464565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae58108760 0x7fae581100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 --2- 172.21.15.19:0/1639464565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae581034e0 0x7fae581038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 --2- 172.21.15.19:0/1639464565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae58107e10 0x7fae58108220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 -- 172.21.15.19:0/1639464565 >> 172.21.15.19:0/1639464565 conn(0x7fae580fef30 msgr2=0x7fae58101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 -- 172.21.15.19:0/1639464565 shutdown_connections 2024-05-09T01:22:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 -- 172.21.15.19:0/1639464565 wait complete. 2024-05-09T01:22:57.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.926+0000 7fae5da9b700 1 Processor -- start 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae5da9b700 1 -- start start 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae5da9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae581034e0 0x7fae5819d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae5da9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae58107e10 0x7fae5819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae5da9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae58108760 0x7fae581a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae5da9b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae58113430 con 0x7fae581034e0 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae5da9b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae581132b0 con 0x7fae58108760 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae5da9b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae581135b0 con 0x7fae58107e10 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae577fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae581034e0 0x7fae5819d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae57fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae58108760 0x7fae581a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae577fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae581034e0 0x7fae5819d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35196/0 (socket says 172.21.15.19:35196) 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae577fe700 1 -- 172.21.15.19:0/613515649 learned_addr learned my addr 172.21.15.19:0/613515649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae56ffd700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae58107e10 0x7fae5819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae57fff700 1 -- 172.21.15.19:0/613515649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae58107e10 msgr2=0x7fae5819d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae57fff700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae58107e10 0x7fae5819d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae57fff700 1 -- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae581034e0 msgr2=0x7fae5819d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.927+0000 7fae57fff700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae581034e0 0x7fae5819d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.928+0000 7fae57fff700 1 -- 172.21.15.19:0/613515649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae581a2090 con 0x7fae58108760 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.928+0000 7fae577fe700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae581034e0 0x7fae5819d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.928+0000 7fae57fff700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae58108760 0x7fae581a19f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fae5000efb0 tx=0x7fae5000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.928+0000 7fae54ff9700 1 -- 172.21.15.19:0/613515649 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50003a00 con 0x7fae58108760 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.928+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae581a2380 con 0x7fae58108760 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.928+0000 7fae54ff9700 1 -- 172.21.15.19:0/613515649 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5000ce50 con 0x7fae58108760 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.928+0000 7fae54ff9700 1 -- 172.21.15.19:0/613515649 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae500047c0 con 0x7fae58108760 2024-05-09T01:22:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.928+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae581a9ca0 con 0x7fae58108760 2024-05-09T01:22:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.929+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae58104990 con 0x7fae58108760 2024-05-09T01:22:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.931+0000 7fae54ff9700 1 -- 172.21.15.19:0/613515649 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fae50012050 con 0x7fae58108760 2024-05-09T01:22:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.931+0000 7fae54ff9700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fae400b2880 0x7fae400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:22:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.931+0000 7fae577fe700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fae400b2880 0x7fae400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:22:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.932+0000 7fae54ff9700 1 -- 172.21.15.19:0/613515649 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fae500e1c40 con 0x7fae58108760 2024-05-09T01:22:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.932+0000 7fae577fe700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fae400b2880 0x7fae400b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fae48009ea0 tx=0x7fae48009290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:22:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:57.935+0000 7fae54ff9700 1 -- 172.21.15.19:0/613515649 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae50022060 con 0x7fae58108760 2024-05-09T01:22:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.198+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fae5819e520 con 0x7fae58108760 2024-05-09T01:22:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.231+0000 7fae54ff9700 1 -- 172.21.15.19:0/613515649 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7fae50010040 con 0x7fae58108760 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fae400b2880 msgr2=0x7fae400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fae400b2880 0x7fae400b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fae48009ea0 tx=0x7fae48009290 comp rx=0 tx=0).stop 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae58108760 msgr2=0x7fae581a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae58108760 0x7fae581a19f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fae5000efb0 tx=0x7fae5000c3a0 comp rx=0 tx=0).stop 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 shutdown_connections 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae58108760 0x7fae581a19f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fae400b2880 0x7fae400b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae581034e0 0x7fae5819d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 --2- 172.21.15.19:0/613515649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae58107e10 0x7fae5819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.233+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 >> 172.21.15.19:0/613515649 conn(0x7fae580fef30 msgr2=0x7fae58109800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.234+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 shutdown_connections 2024-05-09T01:22:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:22:58.234+0000 7fae5da9b700 1 -- 172.21.15.19:0/613515649 wait complete. 2024-05-09T01:22:58.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:58 smithi019 ceph-mon[30158]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-09T01:22:58.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:58 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] up:active 2024-05-09T01:22:58.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:58 smithi019 ceph-mon[30158]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T01:22:58.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:58 smithi019 ceph-mon[30158]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-09T01:22:58.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:58 smithi019 ceph-mon[30158]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-05-09T01:22:58.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:58 smithi019 ceph-mon[30158]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-05-09T01:22:58.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:58 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/613515649' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T01:22:58.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:58 smithi019 ceph-mon[30158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T01:22:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:58 smithi059 ceph-mon[34351]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-09T01:22:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:58 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] up:active 2024-05-09T01:22:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:58 smithi059 ceph-mon[34351]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T01:22:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:58 smithi059 ceph-mon[34351]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-09T01:22:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:58 smithi059 ceph-mon[34351]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-05-09T01:22:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:58 smithi059 ceph-mon[34351]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-05-09T01:22:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:58 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/613515649' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T01:22:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:58 smithi059 ceph-mon[34351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T01:22:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:58 smithi196 ceph-mon[35041]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-09T01:22:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:58 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] up:active 2024-05-09T01:22:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:58 smithi196 ceph-mon[35041]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T01:22:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:58 smithi196 ceph-mon[35041]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-09T01:22:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:58 smithi196 ceph-mon[35041]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-05-09T01:22:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:58 smithi196 ceph-mon[35041]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-05-09T01:22:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:58 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/613515649' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T01:22:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:58 smithi196 ceph-mon[35041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T01:22:59.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-09T01:22:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] up:active 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: Cluster is now healthy 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] up:active 2024-05-09T01:22:59.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:22:59 smithi019 ceph-mon[30158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:22:59.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T01:22:59.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] up:active 2024-05-09T01:22:59.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T01:22:59.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T01:22:59.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: Cluster is now healthy 2024-05-09T01:22:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-05-09T01:22:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T01:22:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-09T01:22:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] up:active 2024-05-09T01:22:59.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:22:59 smithi059 ceph-mon[34351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:22:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T01:22:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] up:active 2024-05-09T01:22:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T01:22:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T01:22:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: Cluster is now healthy 2024-05-09T01:22:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-05-09T01:22:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T01:22:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-09T01:22:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] up:active 2024-05-09T01:22:59.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:22:59 smithi196 ceph-mon[35041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:23:00.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:00 smithi019 ceph-mon[30158]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-09T01:23:00.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:00 smithi059 ceph-mon[34351]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-09T01:23:00.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:00 smithi196 ceph-mon[35041]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-09T01:23:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.034+0000 7fda40960700 1 -- 172.21.15.19:0/3150146557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 msgr2=0x7fda3c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.034+0000 7fda40960700 1 --2- 172.21.15.19:0/3150146557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 0x7fda3c108200 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fda300099c0 tx=0x7fda30009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.034+0000 7fda40960700 1 -- 172.21.15.19:0/3150146557 shutdown_connections 2024-05-09T01:23:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.034+0000 7fda40960700 1 --2- 172.21.15.19:0/3150146557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda3c108740 0x7fda3c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.034+0000 7fda40960700 1 --2- 172.21.15.19:0/3150146557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3c1034c0 0x7fda3c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.034+0000 7fda40960700 1 --2- 172.21.15.19:0/3150146557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 0x7fda3c108200 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.034+0000 7fda40960700 1 -- 172.21.15.19:0/3150146557 >> 172.21.15.19:0/3150146557 conn(0x7fda3c0fedf0 msgr2=0x7fda3c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.034+0000 7fda40960700 1 -- 172.21.15.19:0/3150146557 shutdown_connections 2024-05-09T01:23:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.034+0000 7fda40960700 1 -- 172.21.15.19:0/3150146557 wait complete. 2024-05-09T01:23:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.035+0000 7fda40960700 1 Processor -- start 2024-05-09T01:23:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.035+0000 7fda40960700 1 -- start start 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda40960700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3c1034c0 0x7fda3c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda40960700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 0x7fda3c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda40960700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda3c108740 0x7fda3c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda40960700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c113350 con 0x7fda3c107df0 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda40960700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1131d0 con 0x7fda3c108740 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda40960700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1134d0 con 0x7fda3c1034c0 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda39d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 0x7fda3c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda39d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 0x7fda3c19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58678/0 (socket says 172.21.15.19:58678) 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda39d9b700 1 -- 172.21.15.19:0/2402426520 learned_addr learned my addr 172.21.15.19:0/2402426520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda3a59c700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3c1034c0 0x7fda3c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda3ad9d700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda3c108740 0x7fda3c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda39d9b700 1 -- 172.21.15.19:0/2402426520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3c1034c0 msgr2=0x7fda3c19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda39d9b700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3c1034c0 0x7fda3c19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda39d9b700 1 -- 172.21.15.19:0/2402426520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda3c108740 msgr2=0x7fda3c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda39d9b700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda3c108740 0x7fda3c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.036+0000 7fda39d9b700 1 -- 172.21.15.19:0/2402426520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c19fcb0 con 0x7fda3c107df0 2024-05-09T01:23:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.037+0000 7fda39d9b700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 0x7fda3c19b430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda30006010 tx=0x7fda300047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.037+0000 7fda2b7fe700 1 -- 172.21.15.19:0/2402426520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30027070 con 0x7fda3c107df0 2024-05-09T01:23:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.037+0000 7fda2b7fe700 1 -- 172.21.15.19:0/2402426520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda30003f20 con 0x7fda3c107df0 2024-05-09T01:23:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.037+0000 7fda2b7fe700 1 -- 172.21.15.19:0/2402426520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30022c60 con 0x7fda3c107df0 2024-05-09T01:23:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.037+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c19ff40 con 0x7fda3c107df0 2024-05-09T01:23:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.037+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1a0420 con 0x7fda3c107df0 2024-05-09T01:23:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.038+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c0512c0 con 0x7fda3c107df0 2024-05-09T01:23:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.040+0000 7fda2b7fe700 1 -- 172.21.15.19:0/2402426520 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fda3002a070 con 0x7fda3c107df0 2024-05-09T01:23:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.040+0000 7fda2b7fe700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda240b2880 0x7fda240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.041+0000 7fda3a59c700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda240b2880 0x7fda240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.041+0000 7fda2b7fe700 1 -- 172.21.15.19:0/2402426520 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fda30022e00 con 0x7fda3c107df0 2024-05-09T01:23:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.041+0000 7fda3a59c700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda240b2880 0x7fda240b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda2c005950 tx=0x7fda2c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:01.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.046+0000 7fda2b7fe700 1 -- 172.21.15.19:0/2402426520 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda30013070 con 0x7fda3c107df0 2024-05-09T01:23:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:01.309+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fda3c0643d0 con 0x7fda3c107df0 2024-05-09T01:23:02.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.241+0000 7fda2b7fe700 1 -- 172.21.15.19:0/2402426520 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fda300aad80 con 0x7fda3c107df0 2024-05-09T01:23:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.244+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda240b2880 msgr2=0x7fda240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.244+0000 7fda40960700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda240b2880 0x7fda240b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda2c005950 tx=0x7fda2c00b410 comp rx=0 tx=0).stop 2024-05-09T01:23:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.244+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 msgr2=0x7fda3c19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.244+0000 7fda40960700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 0x7fda3c19b430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda30006010 tx=0x7fda300047b0 comp rx=0 tx=0).stop 2024-05-09T01:23:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.244+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 shutdown_connections 2024-05-09T01:23:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.245+0000 7fda40960700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda3c108740 0x7fda3c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.245+0000 7fda40960700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3c1034c0 0x7fda3c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.245+0000 7fda40960700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda240b2880 0x7fda240b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.245+0000 7fda40960700 1 --2- 172.21.15.19:0/2402426520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c107df0 0x7fda3c19b430 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.245+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 >> 172.21.15.19:0/2402426520 conn(0x7fda3c0fedf0 msgr2=0x7fda3c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.245+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 shutdown_connections 2024-05-09T01:23:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:02.245+0000 7fda40960700 1 -- 172.21.15.19:0/2402426520 wait complete. 2024-05-09T01:23:02.498 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:02 smithi196 ceph-mon[35041]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-09T01:23:02.498 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:02 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2402426520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T01:23:02.553 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:02 smithi019 ceph-mon[30158]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-09T01:23:02.553 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:02 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2402426520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T01:23:02.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:02 smithi059 ceph-mon[34351]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-09T01:23:02.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:02 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2402426520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T01:23:02.597 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:23:02 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: 2024-05-09T01:23:02.194+0000 7f49a9dbf700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-09T01:23:02.597 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:23:02 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: 2024-05-09T01:23:02.194+0000 7f49a9dbf700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-09T01:23:02.971 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-09T01:23:02.972 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-09T01:23:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:03.513 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:03 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2402426520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T01:23:03.513 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:03 smithi196 ceph-mon[35041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:23:03.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:03 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2402426520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T01:23:03.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:03 smithi019 ceph-mon[30158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:23:03.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:03 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2402426520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T01:23:03.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:03 smithi059 ceph-mon[34351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:23:04.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:04 smithi019 ceph-mon[30158]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-09T01:23:04.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:04 smithi059 ceph-mon[34351]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-09T01:23:04.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:04 smithi196 ceph-mon[35041]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-09T01:23:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.957+0000 7eff07f3f700 1 -- 172.21.15.19:0/2780996571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff001034c0 msgr2=0x7eff001038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.957+0000 7eff07f3f700 1 --2- 172.21.15.19:0/2780996571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff001034c0 0x7eff001038d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efefc0099c0 tx=0x7efefc009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.957+0000 7eff07f3f700 1 -- 172.21.15.19:0/2780996571 shutdown_connections 2024-05-09T01:23:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.957+0000 7eff07f3f700 1 --2- 172.21.15.19:0/2780996571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff00108740 0x7eff001100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.957+0000 7eff07f3f700 1 --2- 172.21.15.19:0/2780996571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff001034c0 0x7eff001038d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.957+0000 7eff07f3f700 1 --2- 172.21.15.19:0/2780996571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff00107df0 0x7eff00108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.957+0000 7eff07f3f700 1 -- 172.21.15.19:0/2780996571 >> 172.21.15.19:0/2780996571 conn(0x7eff000fedc0 msgr2=0x7eff001011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.957+0000 7eff07f3f700 1 -- 172.21.15.19:0/2780996571 shutdown_connections 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.957+0000 7eff07f3f700 1 -- 172.21.15.19:0/2780996571 wait complete. 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff07f3f700 1 Processor -- start 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff07f3f700 1 -- start start 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff07f3f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff001034c0 0x7eff0019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff07f3f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff00107df0 0x7eff0019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff07f3f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff00108740 0x7eff0019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff07f3f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff00113370 con 0x7eff00107df0 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff07f3f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7eff001131f0 con 0x7eff00108740 2024-05-09T01:23:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff07f3f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff001134f0 con 0x7eff001034c0 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff054da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff00107df0 0x7eff0019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff054da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff00107df0 0x7eff0019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58688/0 (socket says 172.21.15.19:58688) 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff054da700 1 -- 172.21.15.19:0/67251268 learned_addr learned my addr 172.21.15.19:0/67251268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff05cdb700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff001034c0 0x7eff0019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff064dc700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff00108740 0x7eff0019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff054da700 1 -- 172.21.15.19:0/67251268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff001034c0 msgr2=0x7eff0019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff054da700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff001034c0 0x7eff0019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff054da700 1 -- 172.21.15.19:0/67251268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff00108740 msgr2=0x7eff0019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.958+0000 7eff054da700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff00108740 0x7eff0019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.959+0000 7eff054da700 1 -- 172.21.15.19:0/67251268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0019fc80 con 0x7eff00107df0 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.959+0000 7eff05cdb700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff001034c0 0x7eff0019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.959+0000 7eff054da700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff00107df0 0x7eff0019b350 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efef000d950 tx=0x7efef000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.959+0000 7efef6ffd700 1 -- 172.21.15.19:0/67251268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0016070 con 0x7eff00107df0 2024-05-09T01:23:04.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.959+0000 7efef6ffd700 1 -- 172.21.15.19:0/67251268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efef0004830 con 0x7eff00107df0 2024-05-09T01:23:04.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.959+0000 7efef6ffd700 1 -- 172.21.15.19:0/67251268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0005440 con 0x7eff00107df0 2024-05-09T01:23:04.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.959+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0019ff70 con 0x7eff00107df0 2024-05-09T01:23:04.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.959+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff001a7890 con 0x7eff00107df0 2024-05-09T01:23:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.960+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff000512c0 con 0x7eff00107df0 2024-05-09T01:23:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.962+0000 7efef6ffd700 1 -- 172.21.15.19:0/67251268 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efef0004220 con 0x7eff00107df0 2024-05-09T01:23:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.962+0000 7efef6ffd700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efeec0b2880 0x7efeec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.963+0000 7eff05cdb700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efeec0b2880 0x7efeec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.963+0000 7efef6ffd700 1 -- 172.21.15.19:0/67251268 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7efef00e1e90 con 0x7eff00107df0 2024-05-09T01:23:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.963+0000 7eff05cdb700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efeec0b2880 0x7efeec0b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efefc000c00 tx=0x7efefc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:04.968+0000 7efef6ffd700 1 -- 172.21.15.19:0/67251268 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efef001f020 con 0x7eff00107df0 2024-05-09T01:23:05.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.214+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7eff0019c110 con 0x7efeec0b2880 2024-05-09T01:23:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.246+0000 7efef6ffd700 1 -- 172.21.15.19:0/67251268 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7eff0019c110 con 0x7efeec0b2880 2024-05-09T01:23:05.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efeec0b2880 msgr2=0x7efeec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efeec0b2880 0x7efeec0b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efefc000c00 tx=0x7efefc01a040 comp rx=0 tx=0).stop 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff00107df0 msgr2=0x7eff0019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff00107df0 0x7eff0019b350 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efef000d950 tx=0x7efef000dc60 comp rx=0 tx=0).stop 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 shutdown_connections 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff00108740 0x7eff0019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff001034c0 0x7eff0019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efeec0b2880 0x7efeec0b4d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 --2- 172.21.15.19:0/67251268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff00107df0 0x7eff0019b350 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 >> 172.21.15.19:0/67251268 conn(0x7eff000fedc0 msgr2=0x7eff0006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 shutdown_connections 2024-05-09T01:23:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:05.248+0000 7eff07f3f700 1 -- 172.21.15.19:0/67251268 wait complete. 2024-05-09T01:23:05.568 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:23:05 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:23:05.219+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:23:05.568 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:23:05 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:23:05.219+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:23:05.568 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:23:05 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:23:05.219+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:23:05.568 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:23:05 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:23:05.219+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:23:05.568 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:23:05 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:23:05.219+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:23:05.900 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T01:23:06.156 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:06.563 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:06 smithi019 ceph-mon[30158]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-09T01:23:06.563 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:06 smithi019 ceph-mon[30158]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:06.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:06 smithi059 ceph-mon[34351]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-09T01:23:06.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:06 smithi059 ceph-mon[34351]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:06.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:06 smithi196 ceph-mon[35041]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-09T01:23:06.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:06 smithi196 ceph-mon[35041]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:07.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.885+0000 7f76fd250700 1 -- 172.21.15.19:0/3127490928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 msgr2=0x7f76f81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:07.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.885+0000 7f76fd250700 1 --2- 172.21.15.19:0/3127490928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 0x7f76f81100a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76e80099c0 tx=0x7f76e8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:07.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.886+0000 7f76fd250700 1 -- 172.21.15.19:0/3127490928 shutdown_connections 2024-05-09T01:23:07.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.886+0000 7f76fd250700 1 --2- 172.21.15.19:0/3127490928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 0x7f76f81100a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:07.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.886+0000 7f76fd250700 1 --2- 172.21.15.19:0/3127490928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f81034c0 0x7f76f81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:07.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.886+0000 7f76fd250700 1 --2- 172.21.15.19:0/3127490928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f8107df0 0x7f76f8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:07.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.886+0000 7f76fd250700 1 -- 172.21.15.19:0/3127490928 >> 172.21.15.19:0/3127490928 conn(0x7f76f80fedf0 msgr2=0x7f76f8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:07.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.886+0000 7f76fd250700 1 -- 172.21.15.19:0/3127490928 shutdown_connections 2024-05-09T01:23:07.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.886+0000 7f76fd250700 1 -- 172.21.15.19:0/3127490928 wait complete. 2024-05-09T01:23:07.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.886+0000 7f76fd250700 1 Processor -- start 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76fd250700 1 -- start start 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76fd250700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f81034c0 0x7f76f819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76fd250700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f8107df0 0x7f76f819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76fd250700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 0x7f76f819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76fd250700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76f81133e0 con 0x7f76f8108740 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76fd250700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76f8113260 con 0x7f76f81034c0 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76fd250700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76f8113560 con 0x7f76f8107df0 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76f759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 0x7f76f819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76f759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 0x7f76f819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58718/0 (socket says 172.21.15.19:58718) 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76f759e700 1 -- 172.21.15.19:0/3870351412 learned_addr learned my addr 172.21.15.19:0/3870351412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76f659c700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f8107df0 0x7f76f819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.887+0000 7f76f6d9d700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f81034c0 0x7f76f819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:07.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76f759e700 1 -- 172.21.15.19:0/3870351412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f8107df0 msgr2=0x7f76f819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76f759e700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f8107df0 0x7f76f819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76f759e700 1 -- 172.21.15.19:0/3870351412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f81034c0 msgr2=0x7f76f819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76f759e700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f81034c0 0x7f76f819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76f759e700 1 -- 172.21.15.19:0/3870351412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f819fcb0 con 0x7f76f8108740 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76f759e700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 0x7f76f819f610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009990 tx=0x7f76e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76effff700 1 -- 172.21.15.19:0/3870351412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8027070 con 0x7f76f8108740 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f819ff40 con 0x7f76f8108740 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76effff700 1 -- 172.21.15.19:0/3870351412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e8003fa0 con 0x7f76f8108740 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76effff700 1 -- 172.21.15.19:0/3870351412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8022d10 con 0x7f76f8108740 2024-05-09T01:23:07.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.888+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f81a0420 con 0x7f76f8108740 2024-05-09T01:23:07.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.889+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f80512c0 con 0x7f76f8108740 2024-05-09T01:23:07.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.891+0000 7f76effff700 1 -- 172.21.15.19:0/3870351412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f76e802a070 con 0x7f76f8108740 2024-05-09T01:23:07.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.892+0000 7f76effff700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f76e40b2910 0x7f76e40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:07.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.892+0000 7f76f6d9d700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f76e40b2910 0x7f76e40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:07.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.892+0000 7f76effff700 1 -- 172.21.15.19:0/3870351412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f76e80e35d0 con 0x7f76f8108740 2024-05-09T01:23:07.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.892+0000 7f76f6d9d700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f76e40b2910 0x7f76e40b4e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f76f00060b0 tx=0x7f76f0010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:07.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:07.895+0000 7f76effff700 1 -- 172.21.15.19:0/3870351412 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f76e8031080 con 0x7f76f8108740 2024-05-09T01:23:08.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.145+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f76f8063c00 con 0x7f76e40b2910 2024-05-09T01:23:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.174+0000 7f76effff700 1 -- 172.21.15.19:0/3870351412 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f76f8063c00 con 0x7f76e40b2910 2024-05-09T01:23:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f76e40b2910 msgr2=0x7f76e40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f76e40b2910 0x7f76e40b4e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f76f00060b0 tx=0x7f76f0010040 comp rx=0 tx=0).stop 2024-05-09T01:23:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 msgr2=0x7f76f819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 0x7f76f819f610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009990 tx=0x7f76e80047b0 comp rx=0 tx=0).stop 2024-05-09T01:23:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 shutdown_connections 2024-05-09T01:23:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f76e40b2910 0x7f76e40b4e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8108740 0x7f76f819f610 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f81034c0 0x7f76f819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 --2- 172.21.15.19:0/3870351412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f8107df0 0x7f76f819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 >> 172.21.15.19:0/3870351412 conn(0x7f76f80fedf0 msgr2=0x7f76f8075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 shutdown_connections 2024-05-09T01:23:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:08.176+0000 7f76fd250700 1 -- 172.21.15.19:0/3870351412 wait complete. 2024-05-09T01:23:08.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:08 smithi019 ceph-mon[30158]: pgmap v304: 97 pgs: 97 active+clean; 591 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 21 op/s 2024-05-09T01:23:08.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:08 smithi019 ceph-mon[30158]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T01:23:08.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:08 smithi059 ceph-mon[34351]: pgmap v304: 97 pgs: 97 active+clean; 591 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 21 op/s 2024-05-09T01:23:08.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:08 smithi059 ceph-mon[34351]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T01:23:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:08 smithi196 ceph-mon[35041]: pgmap v304: 97 pgs: 97 active+clean; 591 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 21 op/s 2024-05-09T01:23:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:08 smithi196 ceph-mon[35041]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T01:23:08.939 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T01:23:09.208 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:09.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:09 smithi059 ceph-mon[34351]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:09.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:09 smithi019 ceph-mon[30158]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:09.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:09 smithi196 ceph-mon[35041]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:10.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:10 smithi019 ceph-mon[30158]: pgmap v305: 97 pgs: 97 active+clean; 592 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.9 KiB/s wr, 23 op/s 2024-05-09T01:23:10.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:10 smithi059 ceph-mon[34351]: pgmap v305: 97 pgs: 97 active+clean; 592 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.9 KiB/s wr, 23 op/s 2024-05-09T01:23:10.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:10 smithi196 ceph-mon[35041]: pgmap v305: 97 pgs: 97 active+clean; 592 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.9 KiB/s wr, 23 op/s 2024-05-09T01:23:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.046+0000 7f0199547700 1 -- 172.21.15.19:0/1428221767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01941034c0 msgr2=0x7f01941038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.046+0000 7f0199547700 1 --2- 172.21.15.19:0/1428221767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01941034c0 0x7f01941038d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f018400b0d0 tx=0x7f018400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.046+0000 7f0199547700 1 -- 172.21.15.19:0/1428221767 shutdown_connections 2024-05-09T01:23:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.046+0000 7f0199547700 1 --2- 172.21.15.19:0/1428221767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0194108740 0x7f01941100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.046+0000 7f0199547700 1 --2- 172.21.15.19:0/1428221767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01941034c0 0x7f01941038d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.046+0000 7f0199547700 1 --2- 172.21.15.19:0/1428221767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0194107df0 0x7f0194108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.046+0000 7f0199547700 1 -- 172.21.15.19:0/1428221767 >> 172.21.15.19:0/1428221767 conn(0x7f01940fedf0 msgr2=0x7f0194101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.046+0000 7f0199547700 1 -- 172.21.15.19:0/1428221767 shutdown_connections 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.046+0000 7f0199547700 1 -- 172.21.15.19:0/1428221767 wait complete. 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.047+0000 7f0199547700 1 Processor -- start 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.047+0000 7f0199547700 1 -- start start 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.047+0000 7f0199547700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01941034c0 0x7f0194110880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.047+0000 7f0199547700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0194107df0 0x7f0194110dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.047+0000 7f0199547700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0194108740 0x7f0194111300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.047+0000 7f0199547700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01941144c0 con 0x7f0194108740 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.047+0000 7f0199547700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0194114340 con 0x7f01941034c0 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.047+0000 7f0199547700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0194114640 con 0x7f0194107df0 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0194108740 0x7f0194111300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0194108740 0x7f0194111300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59104/0 (socket says 172.21.15.19:59104) 2024-05-09T01:23:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01937fe700 1 -- 172.21.15.19:0/1624692458 learned_addr learned my addr 172.21.15.19:0/1624692458 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f0192ffd700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01941034c0 0x7f0194110880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01927fc700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0194107df0 0x7f0194110dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01937fe700 1 -- 172.21.15.19:0/1624692458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0194107df0 msgr2=0x7f0194110dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01937fe700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0194107df0 0x7f0194110dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01937fe700 1 -- 172.21.15.19:0/1624692458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01941034c0 msgr2=0x7f0194110880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01937fe700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01941034c0 0x7f0194110880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01937fe700 1 -- 172.21.15.19:0/1624692458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01941a9100 con 0x7f0194108740 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f01937fe700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0194108740 0x7f0194111300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f018c00d980 tx=0x7f018c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f017bfff700 1 -- 172.21.15.19:0/1624692458 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c017070 con 0x7f0194108740 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01941a93f0 con 0x7f0194108740 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f017bfff700 1 -- 172.21.15.19:0/1624692458 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f018c005020 con 0x7f0194108740 2024-05-09T01:23:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f017bfff700 1 -- 172.21.15.19:0/1624692458 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0099b0 con 0x7f0194108740 2024-05-09T01:23:11.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.048+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01941a9930 con 0x7f0194108740 2024-05-09T01:23:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.050+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01940512c0 con 0x7f0194108740 2024-05-09T01:23:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.051+0000 7f017bfff700 1 -- 172.21.15.19:0/1624692458 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f018c009c10 con 0x7f0194108740 2024-05-09T01:23:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.052+0000 7f017bfff700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f017c0b2910 0x7f017c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.052+0000 7f0192ffd700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f017c0b2910 0x7f017c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.052+0000 7f017bfff700 1 -- 172.21.15.19:0/1624692458 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f018c0e2190 con 0x7f0194108740 2024-05-09T01:23:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.052+0000 7f0192ffd700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f017c0b2910 0x7f017c0b4e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0184009900 tx=0x7f018400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:11.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.057+0000 7f017bfff700 1 -- 172.21.15.19:0/1624692458 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f018c020020 con 0x7f0194108740 2024-05-09T01:23:11.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.308+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0194063c00 con 0x7f017c0b2910 2024-05-09T01:23:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.316+0000 7f017bfff700 1 -- 172.21.15.19:0/1624692458 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0194063c00 con 0x7f017c0b2910 2024-05-09T01:23:11.313 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/2042f2fe-93b5-4b65-8c3c-a2d7f3c27255 2024-05-09T01:23:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f017c0b2910 msgr2=0x7f017c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f017c0b2910 0x7f017c0b4e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0184009900 tx=0x7f018400b040 comp rx=0 tx=0).stop 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0194108740 msgr2=0x7f0194111300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0194108740 0x7f0194111300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f018c00d980 tx=0x7f018c00dc90 comp rx=0 tx=0).stop 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 shutdown_connections 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f017c0b2910 0x7f017c0b4e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0194108740 0x7f0194111300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01941034c0 0x7f0194110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 --2- 172.21.15.19:0/1624692458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0194107df0 0x7f0194110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 >> 172.21.15.19:0/1624692458 conn(0x7f01940fedf0 msgr2=0x7f0194101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.317+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 shutdown_connections 2024-05-09T01:23:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:11.318+0000 7f0199547700 1 -- 172.21.15.19:0/1624692458 wait complete. 2024-05-09T01:23:12.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T01:23:12.295 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:12.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:12 smithi019 ceph-mon[30158]: pgmap v306: 97 pgs: 97 active+clean; 606 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-05-09T01:23:12.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:12 smithi019 ceph-mon[30158]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:12.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:12 smithi059 ceph-mon[34351]: pgmap v306: 97 pgs: 97 active+clean; 606 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-05-09T01:23:12.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:12 smithi059 ceph-mon[34351]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:12.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:12 smithi196 ceph-mon[35041]: pgmap v306: 97 pgs: 97 active+clean; 606 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-05-09T01:23:12.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:12 smithi196 ceph-mon[35041]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:14.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.010+0000 7fb89fa9c700 1 -- 172.21.15.19:0/2047470812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8981034e0 msgr2=0x7fb8981038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:14.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.010+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/2047470812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8981034e0 0x7fb8981038f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb88800b0d0 tx=0x7fb88800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.010+0000 7fb89fa9c700 1 -- 172.21.15.19:0/2047470812 shutdown_connections 2024-05-09T01:23:14.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.010+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/2047470812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb898108760 0x7fb8981100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.010+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/2047470812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8981034e0 0x7fb8981038f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.010+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/2047470812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb898107e10 0x7fb898108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.010+0000 7fb89fa9c700 1 -- 172.21.15.19:0/2047470812 >> 172.21.15.19:0/2047470812 conn(0x7fb8980fef30 msgr2=0x7fb898101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.010+0000 7fb89fa9c700 1 -- 172.21.15.19:0/2047470812 shutdown_connections 2024-05-09T01:23:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.010+0000 7fb89fa9c700 1 -- 172.21.15.19:0/2047470812 wait complete. 2024-05-09T01:23:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89fa9c700 1 Processor -- start 2024-05-09T01:23:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89fa9c700 1 -- start start 2024-05-09T01:23:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89fa9c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8981034e0 0x7fb89819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89fa9c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898107e10 0x7fb89819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89fa9c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb898108760 0x7fb89819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89fa9c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb898113390 con 0x7fb898107e10 2024-05-09T01:23:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89fa9c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb898113210 con 0x7fb898108760 2024-05-09T01:23:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89fa9c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb898113510 con 0x7fb8981034e0 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89d037700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898107e10 0x7fb89819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89d037700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898107e10 0x7fb89819c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59124/0 (socket says 172.21.15.19:59124) 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89d037700 1 -- 172.21.15.19:0/16560956 learned_addr learned my addr 172.21.15.19:0/16560956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89d838700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8981034e0 0x7fb89819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.011+0000 7fb89e039700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb898108760 0x7fb89819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb89d037700 1 -- 172.21.15.19:0/16560956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8981034e0 msgr2=0x7fb89819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb89d037700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8981034e0 0x7fb89819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb89d037700 1 -- 172.21.15.19:0/16560956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb898108760 msgr2=0x7fb89819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb89d037700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb898108760 0x7fb89819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb89d037700 1 -- 172.21.15.19:0/16560956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8981a7000 con 0x7fb898107e10 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb89e039700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb898108760 0x7fb89819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb89d037700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898107e10 0x7fb89819c1d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00d980 tx=0x7fb88c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb896ffd700 1 -- 172.21.15.19:0/16560956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c017070 con 0x7fb898107e10 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8981a72f0 con 0x7fb898107e10 2024-05-09T01:23:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb896ffd700 1 -- 172.21.15.19:0/16560956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb88c009de0 con 0x7fb898107e10 2024-05-09T01:23:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb896ffd700 1 -- 172.21.15.19:0/16560956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00f3c0 con 0x7fb898107e10 2024-05-09T01:23:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.012+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8981a7830 con 0x7fb898107e10 2024-05-09T01:23:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.013+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8980512c0 con 0x7fb898107e10 2024-05-09T01:23:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.015+0000 7fb896ffd700 1 -- 172.21.15.19:0/16560956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb88c009930 con 0x7fb898107e10 2024-05-09T01:23:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.015+0000 7fb896ffd700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb8800b2910 0x7fb8800b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.016+0000 7fb89d838700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb8800b2910 0x7fb8800b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.016+0000 7fb896ffd700 1 -- 172.21.15.19:0/16560956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb88c0e1dc0 con 0x7fb898107e10 2024-05-09T01:23:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.016+0000 7fb89d838700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb8800b2910 0x7fb8800b4e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb888009900 tx=0x7fb88800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:14.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.019+0000 7fb896ffd700 1 -- 172.21.15.19:0/16560956 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb88c0a9a30 con 0x7fb898107e10 2024-05-09T01:23:14.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.269+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb898063c00 con 0x7fb8800b2910 2024-05-09T01:23:14.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.297+0000 7fb896ffd700 1 -- 172.21.15.19:0/16560956 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb898063c00 con 0x7fb8800b2910 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb8800b2910 msgr2=0x7fb8800b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb8800b2910 0x7fb8800b4e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb888009900 tx=0x7fb88800b040 comp rx=0 tx=0).stop 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898107e10 msgr2=0x7fb89819c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898107e10 0x7fb89819c1d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00d980 tx=0x7fb88c00dc90 comp rx=0 tx=0).stop 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 shutdown_connections 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb898108760 0x7fb89819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8981034e0 0x7fb89819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb8800b2910 0x7fb8800b4e20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 --2- 172.21.15.19:0/16560956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898107e10 0x7fb89819c1d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 >> 172.21.15.19:0/16560956 conn(0x7fb8980fef30 msgr2=0x7fb8981063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 shutdown_connections 2024-05-09T01:23:14.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:14.300+0000 7fb89fa9c700 1 -- 172.21.15.19:0/16560956 wait complete. 2024-05-09T01:23:14.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:14 smithi059 ceph-mon[34351]: pgmap v307: 97 pgs: 97 active+clean; 606 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-09T01:23:14.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:14 smithi019 ceph-mon[30158]: pgmap v307: 97 pgs: 97 active+clean; 606 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-09T01:23:14.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:14 smithi196 ceph-mon[35041]: pgmap v307: 97 pgs: 97 active+clean; 606 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-09T01:23:14.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T01:23:15.223 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:15.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:15 smithi059 ceph-mon[34351]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:15.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:15 smithi019 ceph-mon[30158]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:15.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:15 smithi196 ceph-mon[35041]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:16.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:16 smithi019 ceph-mon[30158]: pgmap v308: 97 pgs: 97 active+clean; 606 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-05-09T01:23:16.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:16 smithi059 ceph-mon[34351]: pgmap v308: 97 pgs: 97 active+clean; 606 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-05-09T01:23:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:16 smithi196 ceph-mon[35041]: pgmap v308: 97 pgs: 97 active+clean; 606 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-05-09T01:23:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.968+0000 7fd393893700 1 -- 172.21.15.19:0/3435416004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 msgr2=0x7fd38c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.968+0000 7fd393893700 1 --2- 172.21.15.19:0/3435416004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 0x7fd38c1038d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b0d0 tx=0x7fd38400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:23:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.968+0000 7fd393893700 1 -- 172.21.15.19:0/3435416004 shutdown_connections 2024-05-09T01:23:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.968+0000 7fd393893700 1 --2- 172.21.15.19:0/3435416004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38c108740 0x7fd38c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.968+0000 7fd393893700 1 --2- 172.21.15.19:0/3435416004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 0x7fd38c1038d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.968+0000 7fd393893700 1 --2- 172.21.15.19:0/3435416004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38c107df0 0x7fd38c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.968+0000 7fd393893700 1 -- 172.21.15.19:0/3435416004 >> 172.21.15.19:0/3435416004 conn(0x7fd38c0fedf0 msgr2=0x7fd38c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.968+0000 7fd393893700 1 -- 172.21.15.19:0/3435416004 shutdown_connections 2024-05-09T01:23:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.968+0000 7fd393893700 1 -- 172.21.15.19:0/3435416004 wait complete. 2024-05-09T01:23:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.969+0000 7fd393893700 1 Processor -- start 2024-05-09T01:23:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.969+0000 7fd393893700 1 -- start start 2024-05-09T01:23:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.969+0000 7fd393893700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 0x7fd38c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.969+0000 7fd393893700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38c107df0 0x7fd38c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.969+0000 7fd393893700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38c108740 0x7fd38c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.969+0000 7fd393893700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c113370 con 0x7fd38c1034c0 2024-05-09T01:23:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.969+0000 7fd393893700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1131f0 con 0x7fd38c108740 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.969+0000 7fd393893700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1134f0 con 0x7fd38c107df0 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd39162f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 0x7fd38c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd39162f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 0x7fd38c19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59140/0 (socket says 172.21.15.19:59140) 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd39162f700 1 -- 172.21.15.19:0/616663466 learned_addr learned my addr 172.21.15.19:0/616663466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd390e2e700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38c107df0 0x7fd38c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd391e30700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38c108740 0x7fd38c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd39162f700 1 -- 172.21.15.19:0/616663466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38c107df0 msgr2=0x7fd38c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd39162f700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38c107df0 0x7fd38c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd39162f700 1 -- 172.21.15.19:0/616663466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38c108740 msgr2=0x7fd38c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd39162f700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38c108740 0x7fd38c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd39162f700 1 -- 172.21.15.19:0/616663466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c19fbf0 con 0x7fd38c1034c0 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd391e30700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38c108740 0x7fd38c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd39162f700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 0x7fd38c19add0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd384007a20 tx=0x7fd384007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd38a7fc700 1 -- 172.21.15.19:0/616663466 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38401f070 con 0x7fd38c1034c0 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd38a7fc700 1 -- 172.21.15.19:0/616663466 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd384003c00 con 0x7fd38c1034c0 2024-05-09T01:23:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c19fe80 con 0x7fd38c1034c0 2024-05-09T01:23:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd38a7fc700 1 -- 172.21.15.19:0/616663466 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38401b730 con 0x7fd38c1034c0 2024-05-09T01:23:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.970+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c1a03b0 con 0x7fd38c1034c0 2024-05-09T01:23:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.972+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c0512c0 con 0x7fd38c1034c0 2024-05-09T01:23:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.973+0000 7fd38a7fc700 1 -- 172.21.15.19:0/616663466 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd384022070 con 0x7fd38c1034c0 2024-05-09T01:23:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.974+0000 7fd38a7fc700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3740b2990 0x7fd3740b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.974+0000 7fd390e2e700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3740b2990 0x7fd3740b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.974+0000 7fd38a7fc700 1 -- 172.21.15.19:0/616663466 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fd38400f020 con 0x7fd38c1034c0 2024-05-09T01:23:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.974+0000 7fd390e2e700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3740b2990 0x7fd3740b4ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd380005ea0 tx=0x7fd380005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:16.978+0000 7fd38a7fc700 1 -- 172.21.15.19:0/616663466 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd384029020 con 0x7fd38c1034c0 2024-05-09T01:23:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.229+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd38c063c00 con 0x7fd3740b2990 2024-05-09T01:23:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.234+0000 7fd38a7fc700 1 -- 172.21.15.19:0/616663466 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd38c063c00 con 0x7fd3740b2990 2024-05-09T01:23:17.232 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/3c21c65b-0a8c-4ecf-8aec-2b4de05b1a10 2024-05-09T01:23:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3740b2990 msgr2=0x7fd3740b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3740b2990 0x7fd3740b4ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd380005ea0 tx=0x7fd380005e30 comp rx=0 tx=0).stop 2024-05-09T01:23:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 msgr2=0x7fd38c19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 0x7fd38c19add0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd384007a20 tx=0x7fd384007b00 comp rx=0 tx=0).stop 2024-05-09T01:23:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 shutdown_connections 2024-05-09T01:23:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38c108740 0x7fd38c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3740b2990 0x7fd3740b4ea0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:17.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38c1034c0 0x7fd38c19add0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:17.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 --2- 172.21.15.19:0/616663466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38c107df0 0x7fd38c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:17.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 >> 172.21.15.19:0/616663466 conn(0x7fd38c0fedf0 msgr2=0x7fd38c075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:17.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 shutdown_connections 2024-05-09T01:23:17.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:17.237+0000 7fd393893700 1 -- 172.21.15.19:0/616663466 wait complete. 2024-05-09T01:23:17.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:23:18.209 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:18.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:18 smithi019 ceph-mon[30158]: pgmap v309: 97 pgs: 97 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T01:23:18.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:18 smithi019 ceph-mon[30158]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:18.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:18 smithi059 ceph-mon[34351]: pgmap v309: 97 pgs: 97 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T01:23:18.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:18 smithi059 ceph-mon[34351]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:18.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:18 smithi196 ceph-mon[35041]: pgmap v309: 97 pgs: 97 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T01:23:18.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:18 smithi196 ceph-mon[35041]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.905+0000 7f5587665700 1 -- 172.21.15.19:0/2752602917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55801034c0 msgr2=0x7f55801038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.905+0000 7f5587665700 1 --2- 172.21.15.19:0/2752602917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55801034c0 0x7f55801038d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f55740099c0 tx=0x7f5574009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.905+0000 7f5587665700 1 -- 172.21.15.19:0/2752602917 shutdown_connections 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.905+0000 7f5587665700 1 --2- 172.21.15.19:0/2752602917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5580108740 0x7f55801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.905+0000 7f5587665700 1 --2- 172.21.15.19:0/2752602917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55801034c0 0x7f55801038d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.905+0000 7f5587665700 1 --2- 172.21.15.19:0/2752602917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5580107df0 0x7f5580108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.905+0000 7f5587665700 1 -- 172.21.15.19:0/2752602917 >> 172.21.15.19:0/2752602917 conn(0x7f55800fedc0 msgr2=0x7f55801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.905+0000 7f5587665700 1 -- 172.21.15.19:0/2752602917 shutdown_connections 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.905+0000 7f5587665700 1 -- 172.21.15.19:0/2752602917 wait complete. 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5587665700 1 Processor -- start 2024-05-09T01:23:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5587665700 1 -- start start 2024-05-09T01:23:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5587665700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55801034c0 0x7f558019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5587665700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5580107df0 0x7f558019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5587665700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5580108740 0x7f558019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5587665700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5580113370 con 0x7f5580108740 2024-05-09T01:23:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5587665700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55801131f0 con 0x7f5580107df0 2024-05-09T01:23:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5587665700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55801134f0 con 0x7f55801034c0 2024-05-09T01:23:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5585c02700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5580108740 0x7f558019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5585c02700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5580108740 0x7f558019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56818/0 (socket says 172.21.15.19:56818) 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5585c02700 1 -- 172.21.15.19:0/1997011035 learned_addr learned my addr 172.21.15.19:0/1997011035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5585401700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55801034c0 0x7f558019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.906+0000 7f5584c00700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5580107df0 0x7f558019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f5585c02700 1 -- 172.21.15.19:0/1997011035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55801034c0 msgr2=0x7f558019bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f5585c02700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55801034c0 0x7f558019bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f5585c02700 1 -- 172.21.15.19:0/1997011035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5580107df0 msgr2=0x7f558019c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f5585c02700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5580107df0 0x7f558019c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f5585c02700 1 -- 172.21.15.19:0/1997011035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55801a6fc0 con 0x7f5580108740 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f5585c02700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5580108740 0x7f558019c710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5578007ee0 tx=0x7f557800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f55727fc700 1 -- 172.21.15.19:0/1997011035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578004bb0 con 0x7f5580108740 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55801a72b0 con 0x7f5580108740 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f55727fc700 1 -- 172.21.15.19:0/1997011035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5578004d50 con 0x7f5580108740 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f55727fc700 1 -- 172.21.15.19:0/1997011035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578005920 con 0x7f5580108740 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.907+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55801a77a0 con 0x7f5580108740 2024-05-09T01:23:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.908+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55800512c0 con 0x7f5580108740 2024-05-09T01:23:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.910+0000 7f55727fc700 1 -- 172.21.15.19:0/1997011035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f557801a020 con 0x7f5580108740 2024-05-09T01:23:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.911+0000 7f55727fc700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f556c0b2910 0x7f556c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.911+0000 7f5585401700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f556c0b2910 0x7f556c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.911+0000 7f55727fc700 1 -- 172.21.15.19:0/1997011035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f55780e1e50 con 0x7f5580108740 2024-05-09T01:23:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.911+0000 7f5585401700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f556c0b2910 0x7f556c0b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5574000c00 tx=0x7f557401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:19.914+0000 7f55727fc700 1 -- 172.21.15.19:0/1997011035 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5578010070 con 0x7f5580108740 2024-05-09T01:23:20.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.174+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f558002e7b0 con 0x7f5580108740 2024-05-09T01:23:20.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.175+0000 7f55727fc700 1 -- 172.21.15.19:0/1997011035 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11924 (secure 0 0 0) 0x7f55780a9bc0 con 0x7f5580108740 2024-05-09T01:23:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:23:20.173 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:23:01.311002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24451,24497,24473,24452,24466]},"id":1}]} 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f556c0b2910 msgr2=0x7f556c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f556c0b2910 0x7f556c0b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5574000c00 tx=0x7f557401a040 comp rx=0 tx=0).stop 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5580108740 msgr2=0x7f558019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5580108740 0x7f558019c710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5578007ee0 tx=0x7f557800d3b0 comp rx=0 tx=0).stop 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 shutdown_connections 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f556c0b2910 0x7f556c0b4e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5580108740 0x7f558019c710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55801034c0 0x7f558019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 --2- 172.21.15.19:0/1997011035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5580107df0 0x7f558019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 >> 172.21.15.19:0/1997011035 conn(0x7f55800fedc0 msgr2=0x7f55801096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.177+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 shutdown_connections 2024-05-09T01:23:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:20.178+0000 7f5587665700 1 -- 172.21.15.19:0/1997011035 wait complete. 2024-05-09T01:23:20.177 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-09T01:23:20.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:20 smithi019 ceph-mon[30158]: pgmap v310: 97 pgs: 97 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-05-09T01:23:20.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:20 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1997011035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:23:20.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:20 smithi059 ceph-mon[34351]: pgmap v310: 97 pgs: 97 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-05-09T01:23:20.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:20 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1997011035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:23:20.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:20 smithi196 ceph-mon[35041]: pgmap v310: 97 pgs: 97 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-05-09T01:23:20.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:20 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1997011035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:23:20.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:23:21.088 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:21.463 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T01:23:21.463 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-09T01:23:21.463 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-09T01:23:21.463 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-09T01:23:21.463 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-09T01:23:21.463 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T01:23:21.463 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-05-09T01:23:21.463 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-09T01:23:21.464 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-09T01:23:21.464 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:21 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-09T01:23:21.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:21 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-09T01:23:21.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:21 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-05-09T01:23:22.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-05-09T01:23:22.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T01:23:22.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-09T01:23:22.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-09T01:23:22.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-09T01:23:22.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-09T01:23:22.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T01:23:22.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-05-09T01:23:22.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-09T01:23:22.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-09T01:23:22.538 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-05-09T01:23:22.538 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:22 smithi019 ceph-mon[30158]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T01:23:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-05-09T01:23:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T01:23:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-09T01:23:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-09T01:23:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-09T01:23:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-09T01:23:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T01:23:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-05-09T01:23:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-09T01:23:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-09T01:23:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-05-09T01:23:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:22 smithi059 ceph-mon[34351]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T01:23:22.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-05-09T01:23:22.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T01:23:22.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-09T01:23:22.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-09T01:23:22.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-09T01:23:22.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-09T01:23:22.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T01:23:22.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-05-09T01:23:22.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-09T01:23:22.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-09T01:23:22.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-05-09T01:23:22.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:22 smithi196 ceph-mon[35041]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T01:23:22.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.776+0000 7f9a86034700 1 -- 172.21.15.19:0/526124348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80109860 msgr2=0x7f9a801111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:22.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.776+0000 7f9a86034700 1 --2- 172.21.15.19:0/526124348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80109860 0x7f9a801111c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800b0d0 tx=0x7f9a7800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:23:22.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.776+0000 7f9a86034700 1 -- 172.21.15.19:0/526124348 shutdown_connections 2024-05-09T01:23:22.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.776+0000 7f9a86034700 1 --2- 172.21.15.19:0/526124348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80109860 0x7f9a801111c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.776+0000 7f9a86034700 1 --2- 172.21.15.19:0/526124348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a801045e0 0x7f9a801049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.776+0000 7f9a86034700 1 --2- 172.21.15.19:0/526124348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80108f10 0x7f9a80109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.776+0000 7f9a86034700 1 -- 172.21.15.19:0/526124348 >> 172.21.15.19:0/526124348 conn(0x7f9a800ffef0 msgr2=0x7f9a80102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.776+0000 7f9a86034700 1 -- 172.21.15.19:0/526124348 shutdown_connections 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.776+0000 7f9a86034700 1 -- 172.21.15.19:0/526124348 wait complete. 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.777+0000 7f9a86034700 1 Processor -- start 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.777+0000 7f9a86034700 1 -- start start 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a86034700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a801045e0 0x7f9a8019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a86034700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108f10 0x7f9a8019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a86034700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80109860 0x7f9a8019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a86034700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80114490 con 0x7f9a80108f10 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a86034700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80114310 con 0x7f9a801045e0 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a86034700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80114610 con 0x7f9a80109860 2024-05-09T01:23:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108f10 0x7f9a8019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108f10 0x7f9a8019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56834/0 (socket says 172.21.15.19:56834) 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7effd700 1 -- 172.21.15.19:0/1470476308 learned_addr learned my addr 172.21.15.19:0/1470476308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7ffff700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80109860 0x7f9a8019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7f7fe700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a801045e0 0x7f9a8019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7effd700 1 -- 172.21.15.19:0/1470476308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80109860 msgr2=0x7f9a8019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7effd700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80109860 0x7f9a8019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7effd700 1 -- 172.21.15.19:0/1470476308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a801045e0 msgr2=0x7f9a8019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7effd700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a801045e0 0x7f9a8019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7effd700 1 -- 172.21.15.19:0/1470476308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8019fd30 con 0x7f9a80108f10 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7f7fe700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a801045e0 0x7f9a8019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7effd700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108f10 0x7f9a8019b400 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800d980 tx=0x7f9a6800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7cff9700 1 -- 172.21.15.19:0/1470476308 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68017070 con 0x7f9a80108f10 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7cff9700 1 -- 172.21.15.19:0/1470476308 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a68010460 con 0x7f9a80108f10 2024-05-09T01:23:22.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a7cff9700 1 -- 172.21.15.19:0/1470476308 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800de60 con 0x7f9a80108f10 2024-05-09T01:23:22.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.778+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a801a0020 con 0x7f9a80108f10 2024-05-09T01:23:22.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.779+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a801a7940 con 0x7f9a80108f10 2024-05-09T01:23:22.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.780+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a800512c0 con 0x7f9a80108f10 2024-05-09T01:23:22.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.781+0000 7f9a7cff9700 1 -- 172.21.15.19:0/1470476308 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9a68010610 con 0x7f9a80108f10 2024-05-09T01:23:22.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.782+0000 7f9a7cff9700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9a6c0b2990 0x7f9a6c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:22.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.782+0000 7f9a7f7fe700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9a6c0b2990 0x7f9a6c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:22.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.782+0000 7f9a7cff9700 1 -- 172.21.15.19:0/1470476308 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9a680e1ef0 con 0x7f9a80108f10 2024-05-09T01:23:22.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.783+0000 7f9a7f7fe700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9a6c0b2990 0x7f9a6c0b4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9a70005fd0 tx=0x7f9a70005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:22.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:22.786+0000 7f9a7cff9700 1 -- 172.21.15.19:0/1470476308 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a680a9ab0 con 0x7f9a80108f10 2024-05-09T01:23:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.001+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a8002e5f0 con 0x7f9a80108f10 2024-05-09T01:23:23.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.002+0000 7f9a7cff9700 1 -- 172.21.15.19:0/1470476308 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22716 (secure 0 0 0) 0x7f9a680a93b0 con 0x7f9a80108f10 2024-05-09T01:23:23.001 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:23:23.001 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:23:22.298486+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9a6c0b2990 msgr2=0x7f9a6c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9a6c0b2990 0x7f9a6c0b4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9a70005fd0 tx=0x7f9a70005ee0 comp rx=0 tx=0).stop 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108f10 msgr2=0x7f9a8019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108f10 0x7f9a8019b400 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800d980 tx=0x7f9a6800dd40 comp rx=0 tx=0).stop 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 shutdown_connections 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80109860 0x7f9a8019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a801045e0 0x7f9a8019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9a6c0b2990 0x7f9a6c0b4ea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 --2- 172.21.15.19:0/1470476308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108f10 0x7f9a8019b400 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 >> 172.21.15.19:0/1470476308 conn(0x7f9a800ffef0 msgr2=0x7f9a80108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:23.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 shutdown_connections 2024-05-09T01:23:23.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:23.005+0000 7f9a86034700 1 -- 172.21.15.19:0/1470476308 wait complete. 2024-05-09T01:23:23.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:23 smithi059 ceph-mon[34351]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T01:23:23.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:23 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1470476308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:23:23.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:23 smithi019 ceph-mon[30158]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T01:23:23.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:23 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1470476308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:23:23.684 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T01:23:23.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T01:23:23.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:23 smithi196 ceph-mon[35041]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T01:23:23.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:23 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1470476308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:23:23.947 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:24.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:24 smithi019 ceph-mon[30158]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-05-09T01:23:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:24 smithi059 ceph-mon[34351]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-05-09T01:23:24.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:24 smithi196 ceph-mon[35041]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-05-09T01:23:25.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.598+0000 7ff5eebb6700 1 -- 172.21.15.19:0/3786680403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 msgr2=0x7ff5e8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:25.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.598+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/3786680403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 0x7ff5e8102ef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00099c0 tx=0x7ff5d0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.598+0000 7ff5eebb6700 1 -- 172.21.15.19:0/3786680403 shutdown_connections 2024-05-09T01:23:25.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.598+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/3786680403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8109850 0x7ff5e81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.598+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/3786680403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 0x7ff5e8102ef0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.598+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/3786680403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5e8108f00 0x7ff5e8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.598+0000 7ff5eebb6700 1 -- 172.21.15.19:0/3786680403 >> 172.21.15.19:0/3786680403 conn(0x7ff5e80fe540 msgr2=0x7ff5e8100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:25.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.598+0000 7ff5eebb6700 1 -- 172.21.15.19:0/3786680403 shutdown_connections 2024-05-09T01:23:25.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.598+0000 7ff5eebb6700 1 -- 172.21.15.19:0/3786680403 wait complete. 2024-05-09T01:23:25.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.599+0000 7ff5eebb6700 1 Processor -- start 2024-05-09T01:23:25.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.599+0000 7ff5eebb6700 1 -- start start 2024-05-09T01:23:25.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5eebb6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 0x7ff5e819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:25.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5eebb6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8108f00 0x7ff5e819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:25.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5eebb6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5e8109850 0x7ff5e819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:25.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5eebb6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e8114570 con 0x7ff5e8102ae0 2024-05-09T01:23:25.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5eebb6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81143f0 con 0x7ff5e8109850 2024-05-09T01:23:25.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5eebb6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81146f0 con 0x7ff5e8108f00 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ec952700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 0x7ff5e819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ec952700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 0x7ff5e819af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56854/0 (socket says 172.21.15.19:56854) 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ec952700 1 -- 172.21.15.19:0/1485236841 learned_addr learned my addr 172.21.15.19:0/1485236841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5e7fff700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8108f00 0x7ff5e819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ed153700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5e8109850 0x7ff5e819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ec952700 1 -- 172.21.15.19:0/1485236841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8108f00 msgr2=0x7ff5e819b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ec952700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8108f00 0x7ff5e819b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ec952700 1 -- 172.21.15.19:0/1485236841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5e8109850 msgr2=0x7ff5e819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ec952700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5e8109850 0x7ff5e819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ec952700 1 -- 172.21.15.19:0/1485236841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e819fdd0 con 0x7ff5e8102ae0 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.600+0000 7ff5ec952700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 0x7ff5e819af00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000b580 tx=0x7ff5d00037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.601+0000 7ff5e5ffb700 1 -- 172.21.15.19:0/1485236841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0027070 con 0x7ff5e8102ae0 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.601+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e81a0060 con 0x7ff5e8102ae0 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.601+0000 7ff5e5ffb700 1 -- 172.21.15.19:0/1485236841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d0004110 con 0x7ff5e8102ae0 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.601+0000 7ff5e5ffb700 1 -- 172.21.15.19:0/1485236841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000f460 con 0x7ff5e8102ae0 2024-05-09T01:23:25.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.601+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e81a79e0 con 0x7ff5e8102ae0 2024-05-09T01:23:25.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.602+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e80512c0 con 0x7ff5e8102ae0 2024-05-09T01:23:25.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.604+0000 7ff5e5ffb700 1 -- 172.21.15.19:0/1485236841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff5d002a070 con 0x7ff5e8102ae0 2024-05-09T01:23:25.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.605+0000 7ff5e5ffb700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff5d40b2990 0x7ff5d40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:25.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.605+0000 7ff5e7fff700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff5d40b2990 0x7ff5d40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:25.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.605+0000 7ff5e5ffb700 1 -- 172.21.15.19:0/1485236841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff5d00e38e0 con 0x7ff5e8102ae0 2024-05-09T01:23:25.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.606+0000 7ff5e7fff700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff5d40b2990 0x7ff5d40b4ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80fea00 tx=0x7ff5d80094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:25.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.609+0000 7ff5e5ffb700 1 -- 172.21.15.19:0/1485236841 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff5d0013070 con 0x7ff5e8102ae0 2024-05-09T01:23:25.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.820+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5e819c260 con 0x7ff5d40b2990 2024-05-09T01:23:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.827+0000 7ff5e5ffb700 1 -- 172.21.15.19:0/1485236841 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258746 (secure 0 0 0) 0x7ff5e819c260 con 0x7ff5d40b2990 2024-05-09T01:23:25.829 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.830+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff5d40b2990 msgr2=0x7ff5d40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.830+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff5d40b2990 0x7ff5d40b4ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80fea00 tx=0x7ff5d80094f0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.830+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 msgr2=0x7ff5e819af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.830+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 0x7ff5e819af00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000b580 tx=0x7ff5d00037c0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.831+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 shutdown_connections 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.831+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5e8109850 0x7ff5e819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.831+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff5d40b2990 0x7ff5d40b4ea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.831+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5e8102ae0 0x7ff5e819af00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.831+0000 7ff5eebb6700 1 --2- 172.21.15.19:0/1485236841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8108f00 0x7ff5e819b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.831+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 >> 172.21.15.19:0/1485236841 conn(0x7ff5e80fe540 msgr2=0x7ff5e81070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.831+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 shutdown_connections 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:25.831+0000 7ff5eebb6700 1 -- 172.21.15.19:0/1485236841 wait complete. 2024-05-09T01:23:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T01:23:26.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:26 smithi019 ceph-mon[30158]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 195 B/s, 0 objects/s recovering 2024-05-09T01:23:26.555 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-05-09T01:23:25.133581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650880,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":331,"num_read_kb":264,"num_write":365,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2953401,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382676,"kb_used_data":12676,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559148,"statfs":{"total":1151940427776,"available":1151548567552,"internally_reserved":0,"allocated":12980224,"data_stored":7123808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10599,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":9,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002157"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.820750+0000","last_change":"2024-05-09T01:22:39.836756+0000","last_active":"2024-05-09T01:22:41.820750+0000","last_peered":"2024-05-09T01:22:41.820750+0000","last_clean":"2024-05-09T01:22:41.820750+0000","last_became_active":"2024-05-09T01:22:39.835505+0000","last_became_peered":"2024-05-09T01:22:39.835505+0000","last_unstale":"2024-05-09T01:22:41.820750+0000","last_undegraded":"2024-05-09T01:22:41.820750+0000","last_fullsized":"2024-05-09T01:22:41.820750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:29.109762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.837077+0000","last_change":"2024-05-09T01:22:39.842897+0000","last_active":"2024-05-09T01:22:41.837077+0000","last_peered":"2024-05-09T01:22:41.837077+0000","last_clean":"2024-05-09T01:22:41.837077+0000","last_became_active":"2024-05-09T01:22:39.842537+0000","last_became_peered":"2024-05-09T01:22:39.842537+0000","last_unstale":"2024-05-09T01:22:41.837077+0000","last_undegraded":"2024-05-09T01:22:41.837077+0000","last_fullsized":"2024-05-09T01:22:41.837077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:20.429654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.836881+0000","last_change":"2024-05-09T01:22:39.847176+0000","last_active":"2024-05-09T01:22:41.836881+0000","last_peered":"2024-05-09T01:22:41.836881+0000","last_clean":"2024-05-09T01:22:41.836881+0000","last_became_active":"2024-05-09T01:22:39.846782+0000","last_became_peered":"2024-05-09T01:22:39.846782+0000","last_unstale":"2024-05-09T01:22:41.836881+0000","last_undegraded":"2024-05-09T01:22:41.836881+0000","last_fullsized":"2024-05-09T01:22:41.836881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:06.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303905+0000","last_change":"2024-05-09T01:22:39.842399+0000","last_active":"2024-05-09T01:23:22.303905+0000","last_peered":"2024-05-09T01:23:22.303905+0000","last_clean":"2024-05-09T01:23:22.303905+0000","last_became_active":"2024-05-09T01:22:39.842035+0000","last_became_peered":"2024-05-09T01:22:39.842035+0000","last_unstale":"2024-05-09T01:23:22.303905+0000","last_undegraded":"2024-05-09T01:23:22.303905+0000","last_fullsized":"2024-05-09T01:23:22.303905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:35.011920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303152+0000","last_change":"2024-05-09T01:22:39.838574+0000","last_active":"2024-05-09T01:23:22.303152+0000","last_peered":"2024-05-09T01:23:22.303152+0000","last_clean":"2024-05-09T01:23:22.303152+0000","last_became_active":"2024-05-09T01:22:39.838059+0000","last_became_peered":"2024-05-09T01:22:39.838059+0000","last_unstale":"2024-05-09T01:23:22.303152+0000","last_undegraded":"2024-05-09T01:23:22.303152+0000","last_fullsized":"2024-05-09T01:23:22.303152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:20.968526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.821387+0000","last_change":"2024-05-09T01:22:39.842407+0000","last_active":"2024-05-09T01:22:41.821387+0000","last_peered":"2024-05-09T01:22:41.821387+0000","last_clean":"2024-05-09T01:22:41.821387+0000","last_became_active":"2024-05-09T01:22:39.841982+0000","last_became_peered":"2024-05-09T01:22:39.841982+0000","last_unstale":"2024-05-09T01:22:41.821387+0000","last_undegraded":"2024-05-09T01:22:41.821387+0000","last_fullsized":"2024-05-09T01:22:41.821387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:17.086304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308532+0000","last_change":"2024-05-09T01:22:39.833800+0000","last_active":"2024-05-09T01:23:22.308532+0000","last_peered":"2024-05-09T01:23:22.308532+0000","last_clean":"2024-05-09T01:23:22.308532+0000","last_became_active":"2024-05-09T01:22:39.833388+0000","last_became_peered":"2024-05-09T01:22:39.833388+0000","last_unstale":"2024-05-09T01:23:22.308532+0000","last_undegraded":"2024-05-09T01:23:22.308532+0000","last_fullsized":"2024-05-09T01:23:22.308532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:35.341428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309287+0000","last_change":"2024-05-09T01:22:39.845395+0000","last_active":"2024-05-09T01:23:22.309287+0000","last_peered":"2024-05-09T01:23:22.309287+0000","last_clean":"2024-05-09T01:23:22.309287+0000","last_became_active":"2024-05-09T01:22:39.844946+0000","last_became_peered":"2024-05-09T01:22:39.844946+0000","last_unstale":"2024-05-09T01:23:22.309287+0000","last_undegraded":"2024-05-09T01:23:22.309287+0000","last_fullsized":"2024-05-09T01:23:22.309287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:44.750614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.422048+0000","last_change":"2024-05-09T01:22:39.832894+0000","last_active":"2024-05-09T01:23:22.422048+0000","last_peered":"2024-05-09T01:23:22.422048+0000","last_clean":"2024-05-09T01:23:22.422048+0000","last_became_active":"2024-05-09T01:22:39.829743+0000","last_became_peered":"2024-05-09T01:22:39.829743+0000","last_unstale":"2024-05-09T01:23:22.422048+0000","last_undegraded":"2024-05-09T01:23:22.422048+0000","last_fullsized":"2024-05-09T01:23:22.422048+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.463833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.821839+0000","last_change":"2024-05-09T01:22:39.843131+0000","last_active":"2024-05-09T01:22:41.821839+0000","last_peered":"2024-05-09T01:22:41.821839+0000","last_clean":"2024-05-09T01:22:41.821839+0000","last_became_active":"2024-05-09T01:22:39.842690+0000","last_became_peered":"2024-05-09T01:22:39.842690+0000","last_unstale":"2024-05-09T01:22:41.821839+0000","last_undegraded":"2024-05-09T01:22:41.821839+0000","last_fullsized":"2024-05-09T01:22:41.821839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:14.172365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309023+0000","last_change":"2024-05-09T01:22:39.845557+0000","last_active":"2024-05-09T01:23:22.309023+0000","last_peered":"2024-05-09T01:23:22.309023+0000","last_clean":"2024-05-09T01:23:22.309023+0000","last_became_active":"2024-05-09T01:22:39.845059+0000","last_became_peered":"2024-05-09T01:22:39.845059+0000","last_unstale":"2024-05-09T01:23:22.309023+0000","last_undegraded":"2024-05-09T01:23:22.309023+0000","last_fullsized":"2024-05-09T01:23:22.309023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:47.238080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303479+0000","last_change":"2024-05-09T01:22:39.842237+0000","last_active":"2024-05-09T01:23:22.303479+0000","last_peered":"2024-05-09T01:23:22.303479+0000","last_clean":"2024-05-09T01:23:22.303479+0000","last_became_active":"2024-05-09T01:22:39.841849+0000","last_became_peered":"2024-05-09T01:22:39.841849+0000","last_unstale":"2024-05-09T01:23:22.303479+0000","last_undegraded":"2024-05-09T01:23:22.303479+0000","last_fullsized":"2024-05-09T01:23:22.303479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:40.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308267+0000","last_change":"2024-05-09T01:22:39.839943+0000","last_active":"2024-05-09T01:23:22.308267+0000","last_peered":"2024-05-09T01:23:22.308267+0000","last_clean":"2024-05-09T01:23:22.308267+0000","last_became_active":"2024-05-09T01:22:39.839604+0000","last_became_peered":"2024-05-09T01:22:39.839604+0000","last_unstale":"2024-05-09T01:23:22.308267+0000","last_undegraded":"2024-05-09T01:23:22.308267+0000","last_fullsized":"2024-05-09T01:23:22.308267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:34.854263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.837178+0000","last_change":"2024-05-09T01:22:39.846981+0000","last_active":"2024-05-09T01:22:41.837178+0000","last_peered":"2024-05-09T01:22:41.837178+0000","last_clean":"2024-05-09T01:22:41.837178+0000","last_became_active":"2024-05-09T01:22:39.846586+0000","last_became_peered":"2024-05-09T01:22:39.846586+0000","last_unstale":"2024-05-09T01:22:41.837178+0000","last_undegraded":"2024-05-09T01:22:41.837178+0000","last_fullsized":"2024-05-09T01:22:41.837178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:50.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.836739+0000","last_change":"2024-05-09T01:22:39.848964+0000","last_active":"2024-05-09T01:22:41.836739+0000","last_peered":"2024-05-09T01:22:41.836739+0000","last_clean":"2024-05-09T01:22:41.836739+0000","last_became_active":"2024-05-09T01:22:39.848442+0000","last_became_peered":"2024-05-09T01:22:39.848442+0000","last_unstale":"2024-05-09T01:22:41.836739+0000","last_undegraded":"2024-05-09T01:22:41.836739+0000","last_fullsized":"2024-05-09T01:22:41.836739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:35.802739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421217+0000","last_change":"2024-05-09T01:22:39.840586+0000","last_active":"2024-05-09T01:23:22.421217+0000","last_peered":"2024-05-09T01:23:22.421217+0000","last_clean":"2024-05-09T01:23:22.421217+0000","last_became_active":"2024-05-09T01:22:39.839887+0000","last_became_peered":"2024-05-09T01:22:39.839887+0000","last_unstale":"2024-05-09T01:23:22.421217+0000","last_undegraded":"2024-05-09T01:23:22.421217+0000","last_fullsized":"2024-05-09T01:23:22.421217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:36.510039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.820967+0000","last_change":"2024-05-09T01:22:39.836152+0000","last_active":"2024-05-09T01:22:41.820967+0000","last_peered":"2024-05-09T01:22:41.820967+0000","last_clean":"2024-05-09T01:22:41.820967+0000","last_became_active":"2024-05-09T01:22:39.834691+0000","last_became_peered":"2024-05-09T01:22:39.834691+0000","last_unstale":"2024-05-09T01:22:41.820967+0000","last_undegraded":"2024-05-09T01:22:41.820967+0000","last_fullsized":"2024-05-09T01:22:41.820967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:31.331096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.833130+0000","last_change":"2024-05-09T01:22:39.839760+0000","last_active":"2024-05-09T01:22:41.833130+0000","last_peered":"2024-05-09T01:22:41.833130+0000","last_clean":"2024-05-09T01:22:41.833130+0000","last_became_active":"2024-05-09T01:22:39.839403+0000","last_became_peered":"2024-05-09T01:22:39.839403+0000","last_unstale":"2024-05-09T01:22:41.833130+0000","last_undegraded":"2024-05-09T01:22:41.833130+0000","last_fullsized":"2024-05-09T01:22:41.833130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:19.248368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304340+0000","last_change":"2024-05-09T01:22:39.837538+0000","last_active":"2024-05-09T01:23:22.304340+0000","last_peered":"2024-05-09T01:23:22.304340+0000","last_clean":"2024-05-09T01:23:22.304340+0000","last_became_active":"2024-05-09T01:22:39.837167+0000","last_became_peered":"2024-05-09T01:22:39.837167+0000","last_unstale":"2024-05-09T01:23:22.304340+0000","last_undegraded":"2024-05-09T01:23:22.304340+0000","last_fullsized":"2024-05-09T01:23:22.304340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:35.695077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:23:18.659229+0000","last_change":"2024-05-09T01:22:36.804555+0000","last_active":"2024-05-09T01:23:18.659229+0000","last_peered":"2024-05-09T01:23:18.659229+0000","last_clean":"2024-05-09T01:23:18.659229+0000","last_became_active":"2024-05-09T01:22:36.804266+0000","last_became_peered":"2024-05-09T01:22:36.804266+0000","last_unstale":"2024-05-09T01:23:18.659229+0000","last_undegraded":"2024-05-09T01:23:18.659229+0000","last_fullsized":"2024-05-09T01:23:18.659229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:20.027073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.314186+0000","last_change":"2024-05-09T01:22:39.842667+0000","last_active":"2024-05-09T01:23:22.314186+0000","last_peered":"2024-05-09T01:23:22.314186+0000","last_clean":"2024-05-09T01:23:22.314186+0000","last_became_active":"2024-05-09T01:22:39.842256+0000","last_became_peered":"2024-05-09T01:22:39.842256+0000","last_unstale":"2024-05-09T01:23:22.314186+0000","last_undegraded":"2024-05-09T01:23:22.314186+0000","last_fullsized":"2024-05-09T01:23:22.314186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:28.525790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.314520+0000","last_change":"2024-05-09T01:22:36.841515+0000","last_active":"2024-05-09T01:23:22.314520+0000","last_peered":"2024-05-09T01:23:22.314520+0000","last_clean":"2024-05-09T01:23:22.314520+0000","last_became_active":"2024-05-09T01:22:36.841128+0000","last_became_peered":"2024-05-09T01:22:36.841128+0000","last_unstale":"2024-05-09T01:23:22.314520+0000","last_undegraded":"2024-05-09T01:23:22.314520+0000","last_fullsized":"2024-05-09T01:23:22.314520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:13.598177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.837518+0000","last_change":"2024-05-09T01:22:39.843491+0000","last_active":"2024-05-09T01:22:41.837518+0000","last_peered":"2024-05-09T01:22:41.837518+0000","last_clean":"2024-05-09T01:22:41.837518+0000","last_became_active":"2024-05-09T01:22:39.843098+0000","last_became_peered":"2024-05-09T01:22:39.843098+0000","last_unstale":"2024-05-09T01:22:41.837518+0000","last_undegraded":"2024-05-09T01:22:41.837518+0000","last_fullsized":"2024-05-09T01:22:41.837518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:03:33.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303907+0000","last_change":"2024-05-09T01:22:36.798217+0000","last_active":"2024-05-09T01:23:22.303907+0000","last_peered":"2024-05-09T01:23:22.303907+0000","last_clean":"2024-05-09T01:23:22.303907+0000","last_became_active":"2024-05-09T01:22:36.797847+0000","last_became_peered":"2024-05-09T01:22:36.797847+0000","last_unstale":"2024-05-09T01:23:22.303907+0000","last_undegraded":"2024-05-09T01:23:22.303907+0000","last_fullsized":"2024-05-09T01:23:22.303907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:21.116535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421431+0000","last_change":"2024-05-09T01:22:39.837568+0000","last_active":"2024-05-09T01:23:22.421431+0000","last_peered":"2024-05-09T01:23:22.421431+0000","last_clean":"2024-05-09T01:23:22.421431+0000","last_became_active":"2024-05-09T01:22:39.836159+0000","last_became_peered":"2024-05-09T01:22:39.836159+0000","last_unstale":"2024-05-09T01:23:22.421431+0000","last_undegraded":"2024-05-09T01:23:22.421431+0000","last_fullsized":"2024-05-09T01:23:22.421431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:28.089830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303388+0000","last_change":"2024-05-09T01:22:36.798185+0000","last_active":"2024-05-09T01:23:22.303388+0000","last_peered":"2024-05-09T01:23:22.303388+0000","last_clean":"2024-05-09T01:23:22.303388+0000","last_became_active":"2024-05-09T01:22:36.797522+0000","last_became_peered":"2024-05-09T01:22:36.797522+0000","last_unstale":"2024-05-09T01:23:22.303388+0000","last_undegraded":"2024-05-09T01:23:22.303388+0000","last_fullsized":"2024-05-09T01:23:22.303388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:40.960006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421855+0000","last_change":"2024-05-09T01:22:39.837436+0000","last_active":"2024-05-09T01:23:22.421855+0000","last_peered":"2024-05-09T01:23:22.421855+0000","last_clean":"2024-05-09T01:23:22.421855+0000","last_became_active":"2024-05-09T01:22:39.837050+0000","last_became_peered":"2024-05-09T01:22:39.837050+0000","last_unstale":"2024-05-09T01:23:22.421855+0000","last_undegraded":"2024-05-09T01:23:22.421855+0000","last_fullsized":"2024-05-09T01:23:22.421855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:07.149311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421840+0000","last_change":"2024-05-09T01:22:36.800767+0000","last_active":"2024-05-09T01:23:22.421840+0000","last_peered":"2024-05-09T01:23:22.421840+0000","last_clean":"2024-05-09T01:23:22.421840+0000","last_became_active":"2024-05-09T01:22:36.800183+0000","last_became_peered":"2024-05-09T01:22:36.800183+0000","last_unstale":"2024-05-09T01:23:22.421840+0000","last_undegraded":"2024-05-09T01:23:22.421840+0000","last_fullsized":"2024-05-09T01:23:22.421840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:05.682889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303036+0000","last_change":"2024-05-09T01:22:39.832798+0000","last_active":"2024-05-09T01:23:22.303036+0000","last_peered":"2024-05-09T01:23:22.303036+0000","last_clean":"2024-05-09T01:23:22.303036+0000","last_became_active":"2024-05-09T01:22:39.828803+0000","last_became_peered":"2024-05-09T01:22:39.828803+0000","last_unstale":"2024-05-09T01:23:22.303036+0000","last_undegraded":"2024-05-09T01:23:22.303036+0000","last_fullsized":"2024-05-09T01:23:22.303036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:23.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:57.235199+0000","last_change":"2024-05-09T01:22:36.794171+0000","last_active":"2024-05-09T01:22:57.235199+0000","last_peered":"2024-05-09T01:22:57.235199+0000","last_clean":"2024-05-09T01:22:57.235199+0000","last_became_active":"2024-05-09T01:22:36.793713+0000","last_became_peered":"2024-05-09T01:22:36.793713+0000","last_unstale":"2024-05-09T01:22:57.235199+0000","last_undegraded":"2024-05-09T01:22:57.235199+0000","last_fullsized":"2024-05-09T01:22:57.235199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:06.654667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421893+0000","last_change":"2024-05-09T01:22:39.838266+0000","last_active":"2024-05-09T01:23:22.421893+0000","last_peered":"2024-05-09T01:23:22.421893+0000","last_clean":"2024-05-09T01:23:22.421893+0000","last_became_active":"2024-05-09T01:22:39.837911+0000","last_became_peered":"2024-05-09T01:22:39.837911+0000","last_unstale":"2024-05-09T01:23:22.421893+0000","last_undegraded":"2024-05-09T01:23:22.421893+0000","last_fullsized":"2024-05-09T01:23:22.421893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:39.414176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308540+0000","last_change":"2024-05-09T01:22:36.799766+0000","last_active":"2024-05-09T01:23:22.308540+0000","last_peered":"2024-05-09T01:23:22.308540+0000","last_clean":"2024-05-09T01:23:22.308540+0000","last_became_active":"2024-05-09T01:22:36.799388+0000","last_became_peered":"2024-05-09T01:22:36.799388+0000","last_unstale":"2024-05-09T01:23:22.308540+0000","last_undegraded":"2024-05-09T01:23:22.308540+0000","last_fullsized":"2024-05-09T01:23:22.308540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:13.401659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309449+0000","last_change":"2024-05-09T01:22:39.838493+0000","last_active":"2024-05-09T01:23:22.309449+0000","last_peered":"2024-05-09T01:23:22.309449+0000","last_clean":"2024-05-09T01:23:22.309449+0000","last_became_active":"2024-05-09T01:22:39.838086+0000","last_became_peered":"2024-05-09T01:22:39.838086+0000","last_unstale":"2024-05-09T01:23:22.309449+0000","last_undegraded":"2024-05-09T01:23:22.309449+0000","last_fullsized":"2024-05-09T01:23:22.309449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:06.340693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421503+0000","last_change":"2024-05-09T01:22:36.804872+0000","last_active":"2024-05-09T01:23:22.421503+0000","last_peered":"2024-05-09T01:23:22.421503+0000","last_clean":"2024-05-09T01:23:22.421503+0000","last_became_active":"2024-05-09T01:22:36.804439+0000","last_became_peered":"2024-05-09T01:22:36.804439+0000","last_unstale":"2024-05-09T01:23:22.421503+0000","last_undegraded":"2024-05-09T01:23:22.421503+0000","last_fullsized":"2024-05-09T01:23:22.421503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:08.584749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.832388+0000","last_change":"2024-05-09T01:22:39.836579+0000","last_active":"2024-05-09T01:22:41.832388+0000","last_peered":"2024-05-09T01:22:41.832388+0000","last_clean":"2024-05-09T01:22:41.832388+0000","last_became_active":"2024-05-09T01:22:39.836244+0000","last_became_peered":"2024-05-09T01:22:39.836244+0000","last_unstale":"2024-05-09T01:22:41.832388+0000","last_undegraded":"2024-05-09T01:22:41.832388+0000","last_fullsized":"2024-05-09T01:22:41.832388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:43.329448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:23:19.661374+0000","last_change":"2024-05-09T01:22:36.796639+0000","last_active":"2024-05-09T01:23:19.661374+0000","last_peered":"2024-05-09T01:23:19.661374+0000","last_clean":"2024-05-09T01:23:19.661374+0000","last_became_active":"2024-05-09T01:22:36.796227+0000","last_became_peered":"2024-05-09T01:22:36.796227+0000","last_unstale":"2024-05-09T01:23:19.661374+0000","last_undegraded":"2024-05-09T01:23:19.661374+0000","last_fullsized":"2024-05-09T01:23:19.661374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:56.219281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421815+0000","last_change":"2024-05-09T01:22:39.841515+0000","last_active":"2024-05-09T01:23:22.421815+0000","last_peered":"2024-05-09T01:23:22.421815+0000","last_clean":"2024-05-09T01:23:22.421815+0000","last_became_active":"2024-05-09T01:22:39.840057+0000","last_became_peered":"2024-05-09T01:22:39.840057+0000","last_unstale":"2024-05-09T01:23:22.421815+0000","last_undegraded":"2024-05-09T01:23:22.421815+0000","last_fullsized":"2024-05-09T01:23:22.421815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:43.258466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309092+0000","last_change":"2024-05-09T01:22:36.801937+0000","last_active":"2024-05-09T01:23:22.309092+0000","last_peered":"2024-05-09T01:23:22.309092+0000","last_clean":"2024-05-09T01:23:22.309092+0000","last_became_active":"2024-05-09T01:22:36.801563+0000","last_became_peered":"2024-05-09T01:22:36.801563+0000","last_unstale":"2024-05-09T01:23:22.309092+0000","last_undegraded":"2024-05-09T01:23:22.309092+0000","last_fullsized":"2024-05-09T01:23:22.309092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:00:42.170156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.837300+0000","last_change":"2024-05-09T01:22:39.849154+0000","last_active":"2024-05-09T01:22:41.837300+0000","last_peered":"2024-05-09T01:22:41.837300+0000","last_clean":"2024-05-09T01:22:41.837300+0000","last_became_active":"2024-05-09T01:22:39.848815+0000","last_became_peered":"2024-05-09T01:22:39.848815+0000","last_unstale":"2024-05-09T01:22:41.837300+0000","last_undegraded":"2024-05-09T01:22:41.837300+0000","last_fullsized":"2024-05-09T01:22:41.837300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:00.300208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421958+0000","last_change":"2024-05-09T01:22:36.843201+0000","last_active":"2024-05-09T01:23:22.421958+0000","last_peered":"2024-05-09T01:23:22.421958+0000","last_clean":"2024-05-09T01:23:22.421958+0000","last_became_active":"2024-05-09T01:22:36.842500+0000","last_became_peered":"2024-05-09T01:22:36.842500+0000","last_unstale":"2024-05-09T01:23:22.421958+0000","last_undegraded":"2024-05-09T01:23:22.421958+0000","last_fullsized":"2024-05-09T01:23:22.421958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:35.092246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421515+0000","last_change":"2024-05-09T01:22:39.835120+0000","last_active":"2024-05-09T01:23:22.421515+0000","last_peered":"2024-05-09T01:23:22.421515+0000","last_clean":"2024-05-09T01:23:22.421515+0000","last_became_active":"2024-05-09T01:22:39.834613+0000","last_became_peered":"2024-05-09T01:22:39.834613+0000","last_unstale":"2024-05-09T01:23:22.421515+0000","last_undegraded":"2024-05-09T01:23:22.421515+0000","last_fullsized":"2024-05-09T01:23:22.421515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:08.380445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'28","reported_seq":101,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:23:19.668447+0000","last_change":"2024-05-09T01:22:36.800816+0000","last_active":"2024-05-09T01:23:19.668447+0000","last_peered":"2024-05-09T01:23:19.668447+0000","last_clean":"2024-05-09T01:23:19.668447+0000","last_became_active":"2024-05-09T01:22:36.798797+0000","last_became_peered":"2024-05-09T01:22:36.798797+0000","last_unstale":"2024-05-09T01:23:19.668447+0000","last_undegraded":"2024-05-09T01:23:19.668447+0000","last_fullsized":"2024-05-09T01:23:19.668447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:37.670163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52683,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":76,"num_write":30,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303530+0000","last_change":"2024-05-09T01:22:39.831504+0000","last_active":"2024-05-09T01:23:22.303530+0000","last_peered":"2024-05-09T01:23:22.303530+0000","last_clean":"2024-05-09T01:23:22.303530+0000","last_became_active":"2024-05-09T01:22:39.831129+0000","last_became_peered":"2024-05-09T01:22:39.831129+0000","last_unstale":"2024-05-09T01:23:22.303530+0000","last_undegraded":"2024-05-09T01:23:22.303530+0000","last_fullsized":"2024-05-09T01:23:22.303530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:19.212473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304426+0000","last_change":"2024-05-09T01:22:36.803447+0000","last_active":"2024-05-09T01:23:22.304426+0000","last_peered":"2024-05-09T01:23:22.304426+0000","last_clean":"2024-05-09T01:23:22.304426+0000","last_became_active":"2024-05-09T01:22:36.802986+0000","last_became_peered":"2024-05-09T01:22:36.802986+0000","last_unstale":"2024-05-09T01:23:22.304426+0000","last_undegraded":"2024-05-09T01:23:22.304426+0000","last_fullsized":"2024-05-09T01:23:22.304426+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:28.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.307697+0000","last_change":"2024-05-09T01:22:39.836546+0000","last_active":"2024-05-09T01:23:22.307697+0000","last_peered":"2024-05-09T01:23:22.307697+0000","last_clean":"2024-05-09T01:23:22.307697+0000","last_became_active":"2024-05-09T01:22:39.836168+0000","last_became_peered":"2024-05-09T01:22:39.836168+0000","last_unstale":"2024-05-09T01:23:22.307697+0000","last_undegraded":"2024-05-09T01:23:22.307697+0000","last_fullsized":"2024-05-09T01:23:22.307697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:45.522279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421596+0000","last_change":"2024-05-09T01:22:36.798703+0000","last_active":"2024-05-09T01:23:22.421596+0000","last_peered":"2024-05-09T01:23:22.421596+0000","last_clean":"2024-05-09T01:23:22.421596+0000","last_became_active":"2024-05-09T01:22:36.798280+0000","last_became_peered":"2024-05-09T01:22:36.798280+0000","last_unstale":"2024-05-09T01:23:22.421596+0000","last_undegraded":"2024-05-09T01:23:22.421596+0000","last_fullsized":"2024-05-09T01:23:22.421596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:33.794488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308781+0000","last_change":"2024-05-09T01:22:39.831223+0000","last_active":"2024-05-09T01:23:22.308781+0000","last_peered":"2024-05-09T01:23:22.308781+0000","last_clean":"2024-05-09T01:23:22.308781+0000","last_became_active":"2024-05-09T01:22:39.830845+0000","last_became_peered":"2024-05-09T01:22:39.830845+0000","last_unstale":"2024-05-09T01:23:22.308781+0000","last_undegraded":"2024-05-09T01:23:22.308781+0000","last_fullsized":"2024-05-09T01:23:22.308781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:23.365950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303353+0000","last_change":"2024-05-09T01:22:36.802327+0000","last_active":"2024-05-09T01:23:22.303353+0000","last_peered":"2024-05-09T01:23:22.303353+0000","last_clean":"2024-05-09T01:23:22.303353+0000","last_became_active":"2024-05-09T01:22:36.801951+0000","last_became_peered":"2024-05-09T01:22:36.801951+0000","last_unstale":"2024-05-09T01:23:22.303353+0000","last_undegraded":"2024-05-09T01:23:22.303353+0000","last_fullsized":"2024-05-09T01:23:22.303353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:12.282424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308443+0000","last_change":"2024-05-09T01:22:39.833626+0000","last_active":"2024-05-09T01:23:22.308443+0000","last_peered":"2024-05-09T01:23:22.308443+0000","last_clean":"2024-05-09T01:23:22.308443+0000","last_became_active":"2024-05-09T01:22:39.833176+0000","last_became_peered":"2024-05-09T01:22:39.833176+0000","last_unstale":"2024-05-09T01:23:22.308443+0000","last_undegraded":"2024-05-09T01:23:22.308443+0000","last_fullsized":"2024-05-09T01:23:22.308443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:25.082300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:58.246463+0000","last_change":"2024-05-09T01:22:36.842209+0000","last_active":"2024-05-09T01:22:58.246463+0000","last_peered":"2024-05-09T01:22:58.246463+0000","last_clean":"2024-05-09T01:22:58.246463+0000","last_became_active":"2024-05-09T01:22:36.841749+0000","last_became_peered":"2024-05-09T01:22:36.841749+0000","last_unstale":"2024-05-09T01:22:58.246463+0000","last_undegraded":"2024-05-09T01:22:58.246463+0000","last_fullsized":"2024-05-09T01:22:58.246463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:17.819638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421554+0000","last_change":"2024-05-09T01:22:39.837529+0000","last_active":"2024-05-09T01:23:22.421554+0000","last_peered":"2024-05-09T01:23:22.421554+0000","last_clean":"2024-05-09T01:23:22.421554+0000","last_became_active":"2024-05-09T01:22:39.837155+0000","last_became_peered":"2024-05-09T01:22:39.837155+0000","last_unstale":"2024-05-09T01:23:22.421554+0000","last_undegraded":"2024-05-09T01:23:22.421554+0000","last_fullsized":"2024-05-09T01:23:22.421554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:44.485629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303529+0000","last_change":"2024-05-09T01:22:36.800073+0000","last_active":"2024-05-09T01:23:22.303529+0000","last_peered":"2024-05-09T01:23:22.303529+0000","last_clean":"2024-05-09T01:23:22.303529+0000","last_became_active":"2024-05-09T01:22:36.799463+0000","last_became_peered":"2024-05-09T01:22:36.799463+0000","last_unstale":"2024-05-09T01:23:22.303529+0000","last_undegraded":"2024-05-09T01:23:22.303529+0000","last_fullsized":"2024-05-09T01:23:22.303529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.942945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.820559+0000","last_change":"2024-05-09T01:22:39.833615+0000","last_active":"2024-05-09T01:22:41.820559+0000","last_peered":"2024-05-09T01:22:41.820559+0000","last_clean":"2024-05-09T01:22:41.820559+0000","last_became_active":"2024-05-09T01:22:39.833212+0000","last_became_peered":"2024-05-09T01:22:39.833212+0000","last_unstale":"2024-05-09T01:22:41.820559+0000","last_undegraded":"2024-05-09T01:22:41.820559+0000","last_fullsized":"2024-05-09T01:22:41.820559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:35.387442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.832934+0000","last_change":"2024-05-09T01:22:36.843143+0000","last_active":"2024-05-09T01:22:41.832934+0000","last_peered":"2024-05-09T01:22:41.832934+0000","last_clean":"2024-05-09T01:22:41.832934+0000","last_became_active":"2024-05-09T01:22:36.842759+0000","last_became_peered":"2024-05-09T01:22:36.842759+0000","last_unstale":"2024-05-09T01:22:41.832934+0000","last_undegraded":"2024-05-09T01:22:41.832934+0000","last_fullsized":"2024-05-09T01:22:41.832934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:42.666186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.821521+0000","last_change":"2024-05-09T01:22:39.838267+0000","last_active":"2024-05-09T01:22:41.821521+0000","last_peered":"2024-05-09T01:22:41.821521+0000","last_clean":"2024-05-09T01:22:41.821521+0000","last_became_active":"2024-05-09T01:22:39.837910+0000","last_became_peered":"2024-05-09T01:22:39.837910+0000","last_unstale":"2024-05-09T01:22:41.821521+0000","last_undegraded":"2024-05-09T01:22:41.821521+0000","last_fullsized":"2024-05-09T01:22:41.821521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:34.620332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.424637+0000","last_change":"2024-05-09T01:22:36.800515+0000","last_active":"2024-05-09T01:23:22.424637+0000","last_peered":"2024-05-09T01:23:22.424637+0000","last_clean":"2024-05-09T01:23:22.424637+0000","last_became_active":"2024-05-09T01:22:36.800065+0000","last_became_peered":"2024-05-09T01:22:36.800065+0000","last_unstale":"2024-05-09T01:23:22.424637+0000","last_undegraded":"2024-05-09T01:23:22.424637+0000","last_fullsized":"2024-05-09T01:23:22.424637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:50.160463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.821542+0000","last_change":"2024-05-09T01:22:39.844339+0000","last_active":"2024-05-09T01:22:41.821542+0000","last_peered":"2024-05-09T01:22:41.821542+0000","last_clean":"2024-05-09T01:22:41.821542+0000","last_became_active":"2024-05-09T01:22:39.843874+0000","last_became_peered":"2024-05-09T01:22:39.843874+0000","last_unstale":"2024-05-09T01:22:41.821542+0000","last_undegraded":"2024-05-09T01:22:41.821542+0000","last_fullsized":"2024-05-09T01:22:41.821542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:39.996128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.422009+0000","last_change":"2024-05-09T01:22:39.832188+0000","last_active":"2024-05-09T01:23:22.422009+0000","last_peered":"2024-05-09T01:23:22.422009+0000","last_clean":"2024-05-09T01:23:22.422009+0000","last_became_active":"2024-05-09T01:22:39.831302+0000","last_became_peered":"2024-05-09T01:22:39.831302+0000","last_unstale":"2024-05-09T01:23:22.422009+0000","last_undegraded":"2024-05-09T01:23:22.422009+0000","last_fullsized":"2024-05-09T01:23:22.422009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:06.703950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.672613+0000","last_change":"2024-05-09T01:22:36.841446+0000","last_active":"2024-05-09T01:23:22.672613+0000","last_peered":"2024-05-09T01:23:22.672613+0000","last_clean":"2024-05-09T01:23:22.672613+0000","last_became_active":"2024-05-09T01:22:36.841071+0000","last_became_peered":"2024-05-09T01:22:36.841071+0000","last_unstale":"2024-05-09T01:23:22.672613+0000","last_undegraded":"2024-05-09T01:23:22.672613+0000","last_fullsized":"2024-05-09T01:23:22.672613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:20.777482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309539+0000","last_change":"2024-05-09T01:22:39.846711+0000","last_active":"2024-05-09T01:23:22.309539+0000","last_peered":"2024-05-09T01:23:22.309539+0000","last_clean":"2024-05-09T01:23:22.309539+0000","last_became_active":"2024-05-09T01:22:39.846293+0000","last_became_peered":"2024-05-09T01:22:39.846293+0000","last_unstale":"2024-05-09T01:23:22.309539+0000","last_undegraded":"2024-05-09T01:23:22.309539+0000","last_fullsized":"2024-05-09T01:23:22.309539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:27.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303787+0000","last_change":"2024-05-09T01:22:36.798302+0000","last_active":"2024-05-09T01:23:22.303787+0000","last_peered":"2024-05-09T01:23:22.303787+0000","last_clean":"2024-05-09T01:23:22.303787+0000","last_became_active":"2024-05-09T01:22:36.797906+0000","last_became_peered":"2024-05-09T01:22:36.797906+0000","last_unstale":"2024-05-09T01:23:22.303787+0000","last_undegraded":"2024-05-09T01:23:22.303787+0000","last_fullsized":"2024-05-09T01:23:22.303787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:13.599240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308593+0000","last_change":"2024-05-09T01:21:12.781366+0000","last_active":"2024-05-09T01:23:22.308593+0000","last_peered":"2024-05-09T01:23:22.308593+0000","last_clean":"2024-05-09T01:23:22.308593+0000","last_became_active":"2024-05-09T01:21:12.774967+0000","last_became_peered":"2024-05-09T01:21:12.774967+0000","last_unstale":"2024-05-09T01:23:22.308593+0000","last_undegraded":"2024-05-09T01:23:22.308593+0000","last_fullsized":"2024-05-09T01:23:22.308593+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_clean_scrub_stamp":"2024-05-09T01:17:49.682508+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:08.075365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.307279+0000","last_change":"2024-05-09T01:22:39.840952+0000","last_active":"2024-05-09T01:23:22.307279+0000","last_peered":"2024-05-09T01:23:22.307279+0000","last_clean":"2024-05-09T01:23:22.307279+0000","last_became_active":"2024-05-09T01:22:39.840323+0000","last_became_peered":"2024-05-09T01:22:39.840323+0000","last_unstale":"2024-05-09T01:23:22.307279+0000","last_undegraded":"2024-05-09T01:23:22.307279+0000","last_fullsized":"2024-05-09T01:23:22.307279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:23.837123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308793+0000","last_change":"2024-05-09T01:22:36.798631+0000","last_active":"2024-05-09T01:23:22.308793+0000","last_peered":"2024-05-09T01:23:22.308793+0000","last_clean":"2024-05-09T01:23:22.308793+0000","last_became_active":"2024-05-09T01:22:36.798242+0000","last_became_peered":"2024-05-09T01:22:36.798242+0000","last_unstale":"2024-05-09T01:23:22.308793+0000","last_undegraded":"2024-05-09T01:23:22.308793+0000","last_fullsized":"2024-05-09T01:23:22.308793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:18.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303746+0000","last_change":"2024-05-09T01:22:36.843779+0000","last_active":"2024-05-09T01:23:22.303746+0000","last_peered":"2024-05-09T01:23:22.303746+0000","last_clean":"2024-05-09T01:23:22.303746+0000","last_became_active":"2024-05-09T01:22:36.842084+0000","last_became_peered":"2024-05-09T01:22:36.842084+0000","last_unstale":"2024-05-09T01:23:22.303746+0000","last_undegraded":"2024-05-09T01:23:22.303746+0000","last_fullsized":"2024-05-09T01:23:22.303746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:34.743372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308911+0000","last_change":"2024-05-09T01:22:39.842266+0000","last_active":"2024-05-09T01:23:22.308911+0000","last_peered":"2024-05-09T01:23:22.308911+0000","last_clean":"2024-05-09T01:23:22.308911+0000","last_became_active":"2024-05-09T01:22:39.841927+0000","last_became_peered":"2024-05-09T01:22:39.841927+0000","last_unstale":"2024-05-09T01:23:22.308911+0000","last_undegraded":"2024-05-09T01:23:22.308911+0000","last_fullsized":"2024-05-09T01:23:22.308911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:26.774250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'14","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:23:16.836654+0000","last_change":"2024-05-09T01:22:36.796929+0000","last_active":"2024-05-09T01:23:16.836654+0000","last_peered":"2024-05-09T01:23:16.836654+0000","last_clean":"2024-05-09T01:23:16.836654+0000","last_became_active":"2024-05-09T01:22:36.796480+0000","last_became_peered":"2024-05-09T01:22:36.796480+0000","last_unstale":"2024-05-09T01:23:16.836654+0000","last_undegraded":"2024-05-09T01:23:16.836654+0000","last_fullsized":"2024-05-09T01:23:16.836654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:45.168305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.312853+0000","last_change":"2024-05-09T01:22:39.843093+0000","last_active":"2024-05-09T01:23:22.312853+0000","last_peered":"2024-05-09T01:23:22.312853+0000","last_clean":"2024-05-09T01:23:22.312853+0000","last_became_active":"2024-05-09T01:22:39.842729+0000","last_became_peered":"2024-05-09T01:22:39.842729+0000","last_unstale":"2024-05-09T01:23:22.312853+0000","last_undegraded":"2024-05-09T01:23:22.312853+0000","last_fullsized":"2024-05-09T01:23:22.312853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:02.074843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303637+0000","last_change":"2024-05-09T01:22:36.798166+0000","last_active":"2024-05-09T01:23:22.303637+0000","last_peered":"2024-05-09T01:23:22.303637+0000","last_clean":"2024-05-09T01:23:22.303637+0000","last_became_active":"2024-05-09T01:22:36.797752+0000","last_became_peered":"2024-05-09T01:22:36.797752+0000","last_unstale":"2024-05-09T01:23:22.303637+0000","last_undegraded":"2024-05-09T01:23:22.303637+0000","last_fullsized":"2024-05-09T01:23:22.303637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:05.211720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.837273+0000","last_change":"2024-05-09T01:22:39.848489+0000","last_active":"2024-05-09T01:22:41.837273+0000","last_peered":"2024-05-09T01:22:41.837273+0000","last_clean":"2024-05-09T01:22:41.837273+0000","last_became_active":"2024-05-09T01:22:39.848111+0000","last_became_peered":"2024-05-09T01:22:39.848111+0000","last_unstale":"2024-05-09T01:22:41.837273+0000","last_undegraded":"2024-05-09T01:22:41.837273+0000","last_fullsized":"2024-05-09T01:22:41.837273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:42.605726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421234+0000","last_change":"2024-05-09T01:22:36.841999+0000","last_active":"2024-05-09T01:23:22.421234+0000","last_peered":"2024-05-09T01:23:22.421234+0000","last_clean":"2024-05-09T01:23:22.421234+0000","last_became_active":"2024-05-09T01:22:36.841664+0000","last_became_peered":"2024-05-09T01:22:36.841664+0000","last_unstale":"2024-05-09T01:23:22.421234+0000","last_undegraded":"2024-05-09T01:23:22.421234+0000","last_fullsized":"2024-05-09T01:23:22.421234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:27.110258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303011+0000","last_change":"2024-05-09T01:22:39.839553+0000","last_active":"2024-05-09T01:23:22.303011+0000","last_peered":"2024-05-09T01:23:22.303011+0000","last_clean":"2024-05-09T01:23:22.303011+0000","last_became_active":"2024-05-09T01:22:39.839189+0000","last_became_peered":"2024-05-09T01:22:39.839189+0000","last_unstale":"2024-05-09T01:23:22.303011+0000","last_undegraded":"2024-05-09T01:23:22.303011+0000","last_fullsized":"2024-05-09T01:23:22.303011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:02.794073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309848+0000","last_change":"2024-05-09T01:22:36.792516+0000","last_active":"2024-05-09T01:23:22.309848+0000","last_peered":"2024-05-09T01:23:22.309848+0000","last_clean":"2024-05-09T01:23:22.309848+0000","last_became_active":"2024-05-09T01:22:36.792129+0000","last_became_peered":"2024-05-09T01:22:36.792129+0000","last_unstale":"2024-05-09T01:23:22.309848+0000","last_undegraded":"2024-05-09T01:23:22.309848+0000","last_fullsized":"2024-05-09T01:23:22.309848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:46.596723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.820376+0000","last_change":"2024-05-09T01:22:39.841523+0000","last_active":"2024-05-09T01:22:41.820376+0000","last_peered":"2024-05-09T01:22:41.820376+0000","last_clean":"2024-05-09T01:22:41.820376+0000","last_became_active":"2024-05-09T01:22:39.840667+0000","last_became_peered":"2024-05-09T01:22:39.840667+0000","last_unstale":"2024-05-09T01:22:41.820376+0000","last_undegraded":"2024-05-09T01:22:41.820376+0000","last_fullsized":"2024-05-09T01:22:41.820376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:39.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:57.234089+0000","last_change":"2024-05-09T01:22:36.797888+0000","last_active":"2024-05-09T01:22:57.234089+0000","last_peered":"2024-05-09T01:22:57.234089+0000","last_clean":"2024-05-09T01:22:57.234089+0000","last_became_active":"2024-05-09T01:22:36.797322+0000","last_became_peered":"2024-05-09T01:22:36.797322+0000","last_unstale":"2024-05-09T01:22:57.234089+0000","last_undegraded":"2024-05-09T01:22:57.234089+0000","last_fullsized":"2024-05-09T01:22:57.234089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:15.521625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309287+0000","last_change":"2024-05-09T01:22:39.845031+0000","last_active":"2024-05-09T01:23:22.309287+0000","last_peered":"2024-05-09T01:23:22.309287+0000","last_clean":"2024-05-09T01:23:22.309287+0000","last_became_active":"2024-05-09T01:22:39.844653+0000","last_became_peered":"2024-05-09T01:22:39.844653+0000","last_unstale":"2024-05-09T01:23:22.309287+0000","last_undegraded":"2024-05-09T01:23:22.309287+0000","last_fullsized":"2024-05-09T01:23:22.309287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:14.382445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303070+0000","last_change":"2024-05-09T01:22:36.801395+0000","last_active":"2024-05-09T01:23:22.303070+0000","last_peered":"2024-05-09T01:23:22.303070+0000","last_clean":"2024-05-09T01:23:22.303070+0000","last_became_active":"2024-05-09T01:22:36.800992+0000","last_became_peered":"2024-05-09T01:22:36.800992+0000","last_unstale":"2024-05-09T01:23:22.303070+0000","last_undegraded":"2024-05-09T01:23:22.303070+0000","last_fullsized":"2024-05-09T01:23:22.303070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:16.089068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308707+0000","last_change":"2024-05-09T01:22:39.835842+0000","last_active":"2024-05-09T01:23:22.308707+0000","last_peered":"2024-05-09T01:23:22.308707+0000","last_clean":"2024-05-09T01:23:22.308707+0000","last_became_active":"2024-05-09T01:22:39.835372+0000","last_became_peered":"2024-05-09T01:22:39.835372+0000","last_unstale":"2024-05-09T01:23:22.308707+0000","last_undegraded":"2024-05-09T01:23:22.308707+0000","last_fullsized":"2024-05-09T01:23:22.308707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:13.197946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:23.318556+0000","last_change":"2024-05-09T01:23:22.332514+0000","last_active":"2024-05-09T01:23:23.318556+0000","last_peered":"2024-05-09T01:23:23.318556+0000","last_clean":"2024-05-09T01:23:23.318556+0000","last_became_active":"2024-05-09T01:23:22.312966+0000","last_became_peered":"2024-05-09T01:23:22.312966+0000","last_unstale":"2024-05-09T01:23:23.318556+0000","last_undegraded":"2024-05-09T01:23:23.318556+0000","last_fullsized":"2024-05-09T01:23:23.318556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:52.637108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309343+0000","last_change":"2024-05-09T01:22:39.845390+0000","last_active":"2024-05-09T01:23:22.309343+0000","last_peered":"2024-05-09T01:23:22.309343+0000","last_clean":"2024-05-09T01:23:22.309343+0000","last_became_active":"2024-05-09T01:22:39.844918+0000","last_became_peered":"2024-05-09T01:22:39.844918+0000","last_unstale":"2024-05-09T01:23:22.309343+0000","last_undegraded":"2024-05-09T01:23:22.309343+0000","last_fullsized":"2024-05-09T01:23:22.309343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:04.455794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421710+0000","last_change":"2024-05-09T01:22:36.803329+0000","last_active":"2024-05-09T01:23:22.421710+0000","last_peered":"2024-05-09T01:23:22.421710+0000","last_clean":"2024-05-09T01:23:22.421710+0000","last_became_active":"2024-05-09T01:22:36.802831+0000","last_became_peered":"2024-05-09T01:22:36.802831+0000","last_unstale":"2024-05-09T01:23:22.421710+0000","last_undegraded":"2024-05-09T01:23:22.421710+0000","last_fullsized":"2024-05-09T01:23:22.421710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:57.919840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309007+0000","last_change":"2024-05-09T01:22:39.837595+0000","last_active":"2024-05-09T01:23:22.309007+0000","last_peered":"2024-05-09T01:23:22.309007+0000","last_clean":"2024-05-09T01:23:22.309007+0000","last_became_active":"2024-05-09T01:22:39.837197+0000","last_became_peered":"2024-05-09T01:22:39.837197+0000","last_unstale":"2024-05-09T01:23:22.309007+0000","last_undegraded":"2024-05-09T01:23:22.309007+0000","last_fullsized":"2024-05-09T01:23:22.309007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:08.700741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.321286+0000","last_change":"2024-05-09T01:23:22.321286+0000","last_active":"2024-05-09T01:23:22.321286+0000","last_peered":"2024-05-09T01:23:22.321286+0000","last_clean":"2024-05-09T01:23:22.321286+0000","last_became_active":"2024-05-09T01:23:22.315810+0000","last_became_peered":"2024-05-09T01:23:22.315810+0000","last_unstale":"2024-05-09T01:23:22.321286+0000","last_undegraded":"2024-05-09T01:23:22.321286+0000","last_fullsized":"2024-05-09T01:23:22.321286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:43.100893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309221+0000","last_change":"2024-05-09T01:22:39.827798+0000","last_active":"2024-05-09T01:23:22.309221+0000","last_peered":"2024-05-09T01:23:22.309221+0000","last_clean":"2024-05-09T01:23:22.309221+0000","last_became_active":"2024-05-09T01:22:39.827322+0000","last_became_peered":"2024-05-09T01:22:39.827322+0000","last_unstale":"2024-05-09T01:23:22.309221+0000","last_undegraded":"2024-05-09T01:23:22.309221+0000","last_fullsized":"2024-05-09T01:23:22.309221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:52.454033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.820483+0000","last_change":"2024-05-09T01:22:39.841885+0000","last_active":"2024-05-09T01:22:41.820483+0000","last_peered":"2024-05-09T01:22:41.820483+0000","last_clean":"2024-05-09T01:22:41.820483+0000","last_became_active":"2024-05-09T01:22:39.841160+0000","last_became_peered":"2024-05-09T01:22:39.841160+0000","last_unstale":"2024-05-09T01:22:41.820483+0000","last_undegraded":"2024-05-09T01:22:41.820483+0000","last_fullsized":"2024-05-09T01:22:41.820483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:57.290465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303742+0000","last_change":"2024-05-09T01:22:39.846495+0000","last_active":"2024-05-09T01:23:22.303742+0000","last_peered":"2024-05-09T01:23:22.303742+0000","last_clean":"2024-05-09T01:23:22.303742+0000","last_became_active":"2024-05-09T01:22:39.846108+0000","last_became_peered":"2024-05-09T01:22:39.846108+0000","last_unstale":"2024-05-09T01:23:22.303742+0000","last_undegraded":"2024-05-09T01:23:22.303742+0000","last_fullsized":"2024-05-09T01:23:22.303742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:39.958174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303450+0000","last_change":"2024-05-09T01:22:39.841288+0000","last_active":"2024-05-09T01:23:22.303450+0000","last_peered":"2024-05-09T01:23:22.303450+0000","last_clean":"2024-05-09T01:23:22.303450+0000","last_became_active":"2024-05-09T01:22:39.840900+0000","last_became_peered":"2024-05-09T01:22:39.840900+0000","last_unstale":"2024-05-09T01:23:22.303450+0000","last_undegraded":"2024-05-09T01:23:22.303450+0000","last_fullsized":"2024-05-09T01:23:22.303450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:30.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.837280+0000","last_change":"2024-05-09T01:22:39.848638+0000","last_active":"2024-05-09T01:22:41.837280+0000","last_peered":"2024-05-09T01:22:41.837280+0000","last_clean":"2024-05-09T01:22:41.837280+0000","last_became_active":"2024-05-09T01:22:39.848246+0000","last_became_peered":"2024-05-09T01:22:39.848246+0000","last_unstale":"2024-05-09T01:22:41.837280+0000","last_undegraded":"2024-05-09T01:22:41.837280+0000","last_fullsized":"2024-05-09T01:22:41.837280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:12.663330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309028+0000","last_change":"2024-05-09T01:22:39.838037+0000","last_active":"2024-05-09T01:23:22.309028+0000","last_peered":"2024-05-09T01:23:22.309028+0000","last_clean":"2024-05-09T01:23:22.309028+0000","last_became_active":"2024-05-09T01:22:39.837362+0000","last_became_peered":"2024-05-09T01:22:39.837362+0000","last_unstale":"2024-05-09T01:23:22.309028+0000","last_undegraded":"2024-05-09T01:23:22.309028+0000","last_fullsized":"2024-05-09T01:23:22.309028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.781318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.820779+0000","last_change":"2024-05-09T01:22:39.830181+0000","last_active":"2024-05-09T01:22:41.820779+0000","last_peered":"2024-05-09T01:22:41.820779+0000","last_clean":"2024-05-09T01:22:41.820779+0000","last_became_active":"2024-05-09T01:22:39.829085+0000","last_became_peered":"2024-05-09T01:22:39.829085+0000","last_unstale":"2024-05-09T01:22:41.820779+0000","last_undegraded":"2024-05-09T01:22:41.820779+0000","last_fullsized":"2024-05-09T01:22:41.820779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:21.676596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304754+0000","last_change":"2024-05-09T01:22:39.840453+0000","last_active":"2024-05-09T01:23:22.304754+0000","last_peered":"2024-05-09T01:23:22.304754+0000","last_clean":"2024-05-09T01:23:22.304754+0000","last_became_active":"2024-05-09T01:22:39.840077+0000","last_became_peered":"2024-05-09T01:22:39.840077+0000","last_unstale":"2024-05-09T01:23:22.304754+0000","last_undegraded":"2024-05-09T01:23:22.304754+0000","last_fullsized":"2024-05-09T01:23:22.304754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:20.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.312731+0000","last_change":"2024-05-09T01:22:39.844554+0000","last_active":"2024-05-09T01:23:22.312731+0000","last_peered":"2024-05-09T01:23:22.312731+0000","last_clean":"2024-05-09T01:23:22.312731+0000","last_became_active":"2024-05-09T01:22:39.843640+0000","last_became_peered":"2024-05-09T01:22:39.843640+0000","last_unstale":"2024-05-09T01:23:22.312731+0000","last_undegraded":"2024-05-09T01:23:22.312731+0000","last_fullsized":"2024-05-09T01:23:22.312731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:27.801565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.422009+0000","last_change":"2024-05-09T01:22:39.839043+0000","last_active":"2024-05-09T01:23:22.422009+0000","last_peered":"2024-05-09T01:23:22.422009+0000","last_clean":"2024-05-09T01:23:22.422009+0000","last_became_active":"2024-05-09T01:22:39.838662+0000","last_became_peered":"2024-05-09T01:22:39.838662+0000","last_unstale":"2024-05-09T01:23:22.422009+0000","last_undegraded":"2024-05-09T01:23:22.422009+0000","last_fullsized":"2024-05-09T01:23:22.422009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:16.117689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.833899+0000","last_change":"2024-05-09T01:22:39.842522+0000","last_active":"2024-05-09T01:22:41.833899+0000","last_peered":"2024-05-09T01:22:41.833899+0000","last_clean":"2024-05-09T01:22:41.833899+0000","last_became_active":"2024-05-09T01:22:39.842164+0000","last_became_peered":"2024-05-09T01:22:39.842164+0000","last_unstale":"2024-05-09T01:22:41.833899+0000","last_undegraded":"2024-05-09T01:22:41.833899+0000","last_fullsized":"2024-05-09T01:22:41.833899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:54.488884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421407+0000","last_change":"2024-05-09T01:22:39.838911+0000","last_active":"2024-05-09T01:23:22.421407+0000","last_peered":"2024-05-09T01:23:22.421407+0000","last_clean":"2024-05-09T01:23:22.421407+0000","last_became_active":"2024-05-09T01:22:39.838293+0000","last_became_peered":"2024-05-09T01:22:39.838293+0000","last_unstale":"2024-05-09T01:23:22.421407+0000","last_undegraded":"2024-05-09T01:23:22.421407+0000","last_fullsized":"2024-05-09T01:23:22.421407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:32.596306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308388+0000","last_change":"2024-05-09T01:22:39.838185+0000","last_active":"2024-05-09T01:23:22.308388+0000","last_peered":"2024-05-09T01:23:22.308388+0000","last_clean":"2024-05-09T01:23:22.308388+0000","last_became_active":"2024-05-09T01:22:39.837530+0000","last_became_peered":"2024-05-09T01:22:39.837530+0000","last_unstale":"2024-05-09T01:23:22.308388+0000","last_undegraded":"2024-05-09T01:23:22.308388+0000","last_fullsized":"2024-05-09T01:23:22.308388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:22.556710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T01:22:41.820380+0000","last_change":"2024-05-09T01:22:39.834124+0000","last_active":"2024-05-09T01:22:41.820380+0000","last_peered":"2024-05-09T01:22:41.820380+0000","last_clean":"2024-05-09T01:22:41.820380+0000","last_became_active":"2024-05-09T01:22:39.833716+0000","last_became_peered":"2024-05-09T01:22:39.833716+0000","last_unstale":"2024-05-09T01:22:41.820380+0000","last_undegraded":"2024-05-09T01:22:41.820380+0000","last_fullsized":"2024-05-09T01:22:41.820380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:54:42.337900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60278,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":255,"num_read_kb":200,"num_write":240,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":190342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612632,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":876544,"data_stored":420204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1063916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972385,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":419743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":436613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1527808,"data_stored":1030306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528371,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":437127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":496205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":421190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":494192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117057,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":436561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":438549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1540096,"data_stored":1029202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T01:23:26.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T01:23:26.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:26 smithi059 ceph-mon[34351]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 195 B/s, 0 objects/s recovering 2024-05-09T01:23:26.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:26 smithi196 ceph-mon[35041]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 195 B/s, 0 objects/s recovering 2024-05-09T01:23:26.815 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:27.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:27 smithi059 ceph-mon[34351]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:23:27.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:27 smithi019 ceph-mon[30158]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:23:27.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:27 smithi196 ceph-mon[35041]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:23:28.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:28 smithi019 ceph-mon[30158]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.5 KiB/s wr, 8 op/s; 293 B/s, 2 objects/s recovering 2024-05-09T01:23:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 -- 172.21.15.19:0/3708205197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 msgr2=0x7fdb4c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3708205197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 0x7fdb4c108200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdb400099c0 tx=0x7fdb40009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 -- 172.21.15.19:0/3708205197 shutdown_connections 2024-05-09T01:23:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3708205197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb4c108740 0x7fdb4c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3708205197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb4c1034c0 0x7fdb4c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3708205197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 0x7fdb4c108200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 -- 172.21.15.19:0/3708205197 >> 172.21.15.19:0/3708205197 conn(0x7fdb4c0fedc0 msgr2=0x7fdb4c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 -- 172.21.15.19:0/3708205197 shutdown_connections 2024-05-09T01:23:28.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 -- 172.21.15.19:0/3708205197 wait complete. 2024-05-09T01:23:28.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.529+0000 7fdb5399c700 1 Processor -- start 2024-05-09T01:23:28.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb5399c700 1 -- start start 2024-05-09T01:23:28.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb5399c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb4c1034c0 0x7fdb4c19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:28.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb5399c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 0x7fdb4c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:28.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb5399c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb4c108740 0x7fdb4c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:28.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb5399c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1133c0 con 0x7fdb4c107df0 2024-05-09T01:23:28.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb5399c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c113240 con 0x7fdb4c108740 2024-05-09T01:23:28.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb5399c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c113540 con 0x7fdb4c1034c0 2024-05-09T01:23:28.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb50f37700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 0x7fdb4c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb50f37700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 0x7fdb4c19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56884/0 (socket says 172.21.15.19:56884) 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb50f37700 1 -- 172.21.15.19:0/3338900078 learned_addr learned my addr 172.21.15.19:0/3338900078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb51738700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb4c1034c0 0x7fdb4c19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb51f39700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb4c108740 0x7fdb4c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb50f37700 1 -- 172.21.15.19:0/3338900078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb4c1034c0 msgr2=0x7fdb4c19bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb50f37700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb4c1034c0 0x7fdb4c19bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb50f37700 1 -- 172.21.15.19:0/3338900078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb4c108740 msgr2=0x7fdb4c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb50f37700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb4c108740 0x7fdb4c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.530+0000 7fdb50f37700 1 -- 172.21.15.19:0/3338900078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c1a7030 con 0x7fdb4c107df0 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.531+0000 7fdb50f37700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 0x7fdb4c19c220 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdb40006010 tx=0x7fdb400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.531+0000 7fdb4a7fc700 1 -- 172.21.15.19:0/3338900078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40027070 con 0x7fdb4c107df0 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.531+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c1a72c0 con 0x7fdb4c107df0 2024-05-09T01:23:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.531+0000 7fdb4a7fc700 1 -- 172.21.15.19:0/3338900078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb40003f20 con 0x7fdb4c107df0 2024-05-09T01:23:28.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.531+0000 7fdb4a7fc700 1 -- 172.21.15.19:0/3338900078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40022c60 con 0x7fdb4c107df0 2024-05-09T01:23:28.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.531+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c1a7770 con 0x7fdb4c107df0 2024-05-09T01:23:28.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.532+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4c0512c0 con 0x7fdb4c107df0 2024-05-09T01:23:28.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.534+0000 7fdb4a7fc700 1 -- 172.21.15.19:0/3338900078 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdb4002a070 con 0x7fdb4c107df0 2024-05-09T01:23:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.534+0000 7fdb4a7fc700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb340b2910 0x7fdb340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.535+0000 7fdb51738700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb340b2910 0x7fdb340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.535+0000 7fdb4a7fc700 1 -- 172.21.15.19:0/3338900078 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdb40022e00 con 0x7fdb4c107df0 2024-05-09T01:23:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.535+0000 7fdb51738700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb340b2910 0x7fdb340b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005950 tx=0x7fdb4400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:28.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.538+0000 7fdb4a7fc700 1 -- 172.21.15.19:0/3338900078 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb40031080 con 0x7fdb4c107df0 2024-05-09T01:23:28.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:28 smithi059 ceph-mon[34351]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.5 KiB/s wr, 8 op/s; 293 B/s, 2 objects/s recovering 2024-05-09T01:23:28.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:28 smithi196 ceph-mon[35041]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.5 KiB/s wr, 8 op/s; 293 B/s, 2 objects/s recovering 2024-05-09T01:23:28.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.748+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb4c063c00 con 0x7fdb340b2910 2024-05-09T01:23:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.755+0000 7fdb4a7fc700 1 -- 172.21.15.19:0/3338900078 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258834 (secure 0 0 0) 0x7fdb4c063c00 con 0x7fdb340b2910 2024-05-09T01:23:28.757 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:23:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.758+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb340b2910 msgr2=0x7fdb340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.758+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb340b2910 0x7fdb340b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005950 tx=0x7fdb4400b410 comp rx=0 tx=0).stop 2024-05-09T01:23:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.758+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 msgr2=0x7fdb4c19c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.758+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 0x7fdb4c19c220 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdb40006010 tx=0x7fdb400047b0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.759+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 shutdown_connections 2024-05-09T01:23:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.759+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb4c108740 0x7fdb4c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.759+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb4c1034c0 0x7fdb4c19bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.759+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb340b2910 0x7fdb340b4e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.759+0000 7fdb5399c700 1 --2- 172.21.15.19:0/3338900078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c107df0 0x7fdb4c19c220 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.759+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 >> 172.21.15.19:0/3338900078 conn(0x7fdb4c0fedc0 msgr2=0x7fdb4c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.759+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 shutdown_connections 2024-05-09T01:23:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:28.759+0000 7fdb5399c700 1 -- 172.21.15.19:0/3338900078 wait complete. 2024-05-09T01:23:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T01:23:29.599 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-05-09T01:23:27.134261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655690,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":352,"num_read_kb":284,"num_write":372,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":2954185,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":295,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387004,"kb_used_data":12816,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554820,"statfs":{"total":1151940427776,"available":1151544135680,"internally_reserved":0,"allocated":13123584,"data_stored":7197834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15292,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":60,"num_write":15,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2345,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002245"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306119+0000","last_change":"2024-05-09T01:22:39.836756+0000","last_active":"2024-05-09T01:23:22.306119+0000","last_peered":"2024-05-09T01:23:22.306119+0000","last_clean":"2024-05-09T01:23:22.306119+0000","last_became_active":"2024-05-09T01:22:39.835505+0000","last_became_peered":"2024-05-09T01:22:39.835505+0000","last_unstale":"2024-05-09T01:23:22.306119+0000","last_undegraded":"2024-05-09T01:23:22.306119+0000","last_fullsized":"2024-05-09T01:23:22.306119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:29.109762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.311111+0000","last_change":"2024-05-09T01:22:39.842897+0000","last_active":"2024-05-09T01:23:22.311111+0000","last_peered":"2024-05-09T01:23:22.311111+0000","last_clean":"2024-05-09T01:23:22.311111+0000","last_became_active":"2024-05-09T01:22:39.842537+0000","last_became_peered":"2024-05-09T01:22:39.842537+0000","last_unstale":"2024-05-09T01:23:22.311111+0000","last_undegraded":"2024-05-09T01:23:22.311111+0000","last_fullsized":"2024-05-09T01:23:22.311111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:20.429654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309635+0000","last_change":"2024-05-09T01:22:39.847176+0000","last_active":"2024-05-09T01:23:22.309635+0000","last_peered":"2024-05-09T01:23:22.309635+0000","last_clean":"2024-05-09T01:23:22.309635+0000","last_became_active":"2024-05-09T01:22:39.846782+0000","last_became_peered":"2024-05-09T01:22:39.846782+0000","last_unstale":"2024-05-09T01:23:22.309635+0000","last_undegraded":"2024-05-09T01:23:22.309635+0000","last_fullsized":"2024-05-09T01:23:22.309635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:06.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303905+0000","last_change":"2024-05-09T01:22:39.842399+0000","last_active":"2024-05-09T01:23:22.303905+0000","last_peered":"2024-05-09T01:23:22.303905+0000","last_clean":"2024-05-09T01:23:22.303905+0000","last_became_active":"2024-05-09T01:22:39.842035+0000","last_became_peered":"2024-05-09T01:22:39.842035+0000","last_unstale":"2024-05-09T01:23:22.303905+0000","last_undegraded":"2024-05-09T01:23:22.303905+0000","last_fullsized":"2024-05-09T01:23:22.303905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:35.011920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303152+0000","last_change":"2024-05-09T01:22:39.838574+0000","last_active":"2024-05-09T01:23:22.303152+0000","last_peered":"2024-05-09T01:23:22.303152+0000","last_clean":"2024-05-09T01:23:22.303152+0000","last_became_active":"2024-05-09T01:22:39.838059+0000","last_became_peered":"2024-05-09T01:22:39.838059+0000","last_unstale":"2024-05-09T01:23:22.303152+0000","last_undegraded":"2024-05-09T01:23:22.303152+0000","last_fullsized":"2024-05-09T01:23:22.303152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:20.968526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306049+0000","last_change":"2024-05-09T01:22:39.842407+0000","last_active":"2024-05-09T01:23:22.306049+0000","last_peered":"2024-05-09T01:23:22.306049+0000","last_clean":"2024-05-09T01:23:22.306049+0000","last_became_active":"2024-05-09T01:22:39.841982+0000","last_became_peered":"2024-05-09T01:22:39.841982+0000","last_unstale":"2024-05-09T01:23:22.306049+0000","last_undegraded":"2024-05-09T01:23:22.306049+0000","last_fullsized":"2024-05-09T01:23:22.306049+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:17.086304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308532+0000","last_change":"2024-05-09T01:22:39.833800+0000","last_active":"2024-05-09T01:23:22.308532+0000","last_peered":"2024-05-09T01:23:22.308532+0000","last_clean":"2024-05-09T01:23:22.308532+0000","last_became_active":"2024-05-09T01:22:39.833388+0000","last_became_peered":"2024-05-09T01:22:39.833388+0000","last_unstale":"2024-05-09T01:23:22.308532+0000","last_undegraded":"2024-05-09T01:23:22.308532+0000","last_fullsized":"2024-05-09T01:23:22.308532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:35.341428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309287+0000","last_change":"2024-05-09T01:22:39.845395+0000","last_active":"2024-05-09T01:23:22.309287+0000","last_peered":"2024-05-09T01:23:22.309287+0000","last_clean":"2024-05-09T01:23:22.309287+0000","last_became_active":"2024-05-09T01:22:39.844946+0000","last_became_peered":"2024-05-09T01:22:39.844946+0000","last_unstale":"2024-05-09T01:23:22.309287+0000","last_undegraded":"2024-05-09T01:23:22.309287+0000","last_fullsized":"2024-05-09T01:23:22.309287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:44.750614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.422048+0000","last_change":"2024-05-09T01:22:39.832894+0000","last_active":"2024-05-09T01:23:22.422048+0000","last_peered":"2024-05-09T01:23:22.422048+0000","last_clean":"2024-05-09T01:23:22.422048+0000","last_became_active":"2024-05-09T01:22:39.829743+0000","last_became_peered":"2024-05-09T01:22:39.829743+0000","last_unstale":"2024-05-09T01:23:22.422048+0000","last_undegraded":"2024-05-09T01:23:22.422048+0000","last_fullsized":"2024-05-09T01:23:22.422048+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.463833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306424+0000","last_change":"2024-05-09T01:22:39.843131+0000","last_active":"2024-05-09T01:23:22.306424+0000","last_peered":"2024-05-09T01:23:22.306424+0000","last_clean":"2024-05-09T01:23:22.306424+0000","last_became_active":"2024-05-09T01:22:39.842690+0000","last_became_peered":"2024-05-09T01:22:39.842690+0000","last_unstale":"2024-05-09T01:23:22.306424+0000","last_undegraded":"2024-05-09T01:23:22.306424+0000","last_fullsized":"2024-05-09T01:23:22.306424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:14.172365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309023+0000","last_change":"2024-05-09T01:22:39.845557+0000","last_active":"2024-05-09T01:23:22.309023+0000","last_peered":"2024-05-09T01:23:22.309023+0000","last_clean":"2024-05-09T01:23:22.309023+0000","last_became_active":"2024-05-09T01:22:39.845059+0000","last_became_peered":"2024-05-09T01:22:39.845059+0000","last_unstale":"2024-05-09T01:23:22.309023+0000","last_undegraded":"2024-05-09T01:23:22.309023+0000","last_fullsized":"2024-05-09T01:23:22.309023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:47.238080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303479+0000","last_change":"2024-05-09T01:22:39.842237+0000","last_active":"2024-05-09T01:23:22.303479+0000","last_peered":"2024-05-09T01:23:22.303479+0000","last_clean":"2024-05-09T01:23:22.303479+0000","last_became_active":"2024-05-09T01:22:39.841849+0000","last_became_peered":"2024-05-09T01:22:39.841849+0000","last_unstale":"2024-05-09T01:23:22.303479+0000","last_undegraded":"2024-05-09T01:23:22.303479+0000","last_fullsized":"2024-05-09T01:23:22.303479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:40.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308267+0000","last_change":"2024-05-09T01:22:39.839943+0000","last_active":"2024-05-09T01:23:22.308267+0000","last_peered":"2024-05-09T01:23:22.308267+0000","last_clean":"2024-05-09T01:23:22.308267+0000","last_became_active":"2024-05-09T01:22:39.839604+0000","last_became_peered":"2024-05-09T01:22:39.839604+0000","last_unstale":"2024-05-09T01:23:22.308267+0000","last_undegraded":"2024-05-09T01:23:22.308267+0000","last_fullsized":"2024-05-09T01:23:22.308267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:34.854263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306370+0000","last_change":"2024-05-09T01:22:39.846981+0000","last_active":"2024-05-09T01:23:22.306370+0000","last_peered":"2024-05-09T01:23:22.306370+0000","last_clean":"2024-05-09T01:23:22.306370+0000","last_became_active":"2024-05-09T01:22:39.846586+0000","last_became_peered":"2024-05-09T01:22:39.846586+0000","last_unstale":"2024-05-09T01:23:22.306370+0000","last_undegraded":"2024-05-09T01:23:22.306370+0000","last_fullsized":"2024-05-09T01:23:22.306370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:50.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306075+0000","last_change":"2024-05-09T01:22:39.848964+0000","last_active":"2024-05-09T01:23:22.306075+0000","last_peered":"2024-05-09T01:23:22.306075+0000","last_clean":"2024-05-09T01:23:22.306075+0000","last_became_active":"2024-05-09T01:22:39.848442+0000","last_became_peered":"2024-05-09T01:22:39.848442+0000","last_unstale":"2024-05-09T01:23:22.306075+0000","last_undegraded":"2024-05-09T01:23:22.306075+0000","last_fullsized":"2024-05-09T01:23:22.306075+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:35.802739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421217+0000","last_change":"2024-05-09T01:22:39.840586+0000","last_active":"2024-05-09T01:23:22.421217+0000","last_peered":"2024-05-09T01:23:22.421217+0000","last_clean":"2024-05-09T01:23:22.421217+0000","last_became_active":"2024-05-09T01:22:39.839887+0000","last_became_peered":"2024-05-09T01:22:39.839887+0000","last_unstale":"2024-05-09T01:23:22.421217+0000","last_undegraded":"2024-05-09T01:23:22.421217+0000","last_fullsized":"2024-05-09T01:23:22.421217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:36.510039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306409+0000","last_change":"2024-05-09T01:22:39.836152+0000","last_active":"2024-05-09T01:23:22.306409+0000","last_peered":"2024-05-09T01:23:22.306409+0000","last_clean":"2024-05-09T01:23:22.306409+0000","last_became_active":"2024-05-09T01:22:39.834691+0000","last_became_peered":"2024-05-09T01:22:39.834691+0000","last_unstale":"2024-05-09T01:23:22.306409+0000","last_undegraded":"2024-05-09T01:23:22.306409+0000","last_fullsized":"2024-05-09T01:23:22.306409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:31.331096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305788+0000","last_change":"2024-05-09T01:22:39.839760+0000","last_active":"2024-05-09T01:23:22.305788+0000","last_peered":"2024-05-09T01:23:22.305788+0000","last_clean":"2024-05-09T01:23:22.305788+0000","last_became_active":"2024-05-09T01:22:39.839403+0000","last_became_peered":"2024-05-09T01:22:39.839403+0000","last_unstale":"2024-05-09T01:23:22.305788+0000","last_undegraded":"2024-05-09T01:23:22.305788+0000","last_fullsized":"2024-05-09T01:23:22.305788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:19.248368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304340+0000","last_change":"2024-05-09T01:22:39.837538+0000","last_active":"2024-05-09T01:23:22.304340+0000","last_peered":"2024-05-09T01:23:22.304340+0000","last_clean":"2024-05-09T01:23:22.304340+0000","last_became_active":"2024-05-09T01:22:39.837167+0000","last_became_peered":"2024-05-09T01:22:39.837167+0000","last_unstale":"2024-05-09T01:23:22.304340+0000","last_undegraded":"2024-05-09T01:23:22.304340+0000","last_fullsized":"2024-05-09T01:23:22.304340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:35.695077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":87,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:24.427095+0000","last_change":"2024-05-09T01:23:22.433852+0000","last_active":"2024-05-09T01:23:24.427095+0000","last_peered":"2024-05-09T01:23:24.427095+0000","last_clean":"2024-05-09T01:23:24.427095+0000","last_became_active":"2024-05-09T01:23:22.423465+0000","last_became_peered":"2024-05-09T01:23:22.423465+0000","last_unstale":"2024-05-09T01:23:24.427095+0000","last_undegraded":"2024-05-09T01:23:24.427095+0000","last_fullsized":"2024-05-09T01:23:24.427095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:58.578798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.314186+0000","last_change":"2024-05-09T01:22:39.842667+0000","last_active":"2024-05-09T01:23:22.314186+0000","last_peered":"2024-05-09T01:23:22.314186+0000","last_clean":"2024-05-09T01:23:22.314186+0000","last_became_active":"2024-05-09T01:22:39.842256+0000","last_became_peered":"2024-05-09T01:22:39.842256+0000","last_unstale":"2024-05-09T01:23:22.314186+0000","last_undegraded":"2024-05-09T01:23:22.314186+0000","last_fullsized":"2024-05-09T01:23:22.314186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:28.525790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.314520+0000","last_change":"2024-05-09T01:22:36.841515+0000","last_active":"2024-05-09T01:23:22.314520+0000","last_peered":"2024-05-09T01:23:22.314520+0000","last_clean":"2024-05-09T01:23:22.314520+0000","last_became_active":"2024-05-09T01:22:36.841128+0000","last_became_peered":"2024-05-09T01:22:36.841128+0000","last_unstale":"2024-05-09T01:23:22.314520+0000","last_undegraded":"2024-05-09T01:23:22.314520+0000","last_fullsized":"2024-05-09T01:23:22.314520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:13.598177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.307805+0000","last_change":"2024-05-09T01:22:39.843491+0000","last_active":"2024-05-09T01:23:22.307805+0000","last_peered":"2024-05-09T01:23:22.307805+0000","last_clean":"2024-05-09T01:23:22.307805+0000","last_became_active":"2024-05-09T01:22:39.843098+0000","last_became_peered":"2024-05-09T01:22:39.843098+0000","last_unstale":"2024-05-09T01:23:22.307805+0000","last_undegraded":"2024-05-09T01:23:22.307805+0000","last_fullsized":"2024-05-09T01:23:22.307805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:03:33.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303907+0000","last_change":"2024-05-09T01:22:36.798217+0000","last_active":"2024-05-09T01:23:22.303907+0000","last_peered":"2024-05-09T01:23:22.303907+0000","last_clean":"2024-05-09T01:23:22.303907+0000","last_became_active":"2024-05-09T01:22:36.797847+0000","last_became_peered":"2024-05-09T01:22:36.797847+0000","last_unstale":"2024-05-09T01:23:22.303907+0000","last_undegraded":"2024-05-09T01:23:22.303907+0000","last_fullsized":"2024-05-09T01:23:22.303907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:21.116535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421431+0000","last_change":"2024-05-09T01:22:39.837568+0000","last_active":"2024-05-09T01:23:22.421431+0000","last_peered":"2024-05-09T01:23:22.421431+0000","last_clean":"2024-05-09T01:23:22.421431+0000","last_became_active":"2024-05-09T01:22:39.836159+0000","last_became_peered":"2024-05-09T01:22:39.836159+0000","last_unstale":"2024-05-09T01:23:22.421431+0000","last_undegraded":"2024-05-09T01:23:22.421431+0000","last_fullsized":"2024-05-09T01:23:22.421431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:28.089830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303388+0000","last_change":"2024-05-09T01:22:36.798185+0000","last_active":"2024-05-09T01:23:22.303388+0000","last_peered":"2024-05-09T01:23:22.303388+0000","last_clean":"2024-05-09T01:23:22.303388+0000","last_became_active":"2024-05-09T01:22:36.797522+0000","last_became_peered":"2024-05-09T01:22:36.797522+0000","last_unstale":"2024-05-09T01:23:22.303388+0000","last_undegraded":"2024-05-09T01:23:22.303388+0000","last_fullsized":"2024-05-09T01:23:22.303388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:40.960006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421855+0000","last_change":"2024-05-09T01:22:39.837436+0000","last_active":"2024-05-09T01:23:22.421855+0000","last_peered":"2024-05-09T01:23:22.421855+0000","last_clean":"2024-05-09T01:23:22.421855+0000","last_became_active":"2024-05-09T01:22:39.837050+0000","last_became_peered":"2024-05-09T01:22:39.837050+0000","last_unstale":"2024-05-09T01:23:22.421855+0000","last_undegraded":"2024-05-09T01:23:22.421855+0000","last_fullsized":"2024-05-09T01:23:22.421855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:07.149311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421840+0000","last_change":"2024-05-09T01:22:36.800767+0000","last_active":"2024-05-09T01:23:22.421840+0000","last_peered":"2024-05-09T01:23:22.421840+0000","last_clean":"2024-05-09T01:23:22.421840+0000","last_became_active":"2024-05-09T01:22:36.800183+0000","last_became_peered":"2024-05-09T01:22:36.800183+0000","last_unstale":"2024-05-09T01:23:22.421840+0000","last_undegraded":"2024-05-09T01:23:22.421840+0000","last_fullsized":"2024-05-09T01:23:22.421840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:05.682889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303036+0000","last_change":"2024-05-09T01:22:39.832798+0000","last_active":"2024-05-09T01:23:22.303036+0000","last_peered":"2024-05-09T01:23:22.303036+0000","last_clean":"2024-05-09T01:23:22.303036+0000","last_became_active":"2024-05-09T01:22:39.828803+0000","last_became_peered":"2024-05-09T01:22:39.828803+0000","last_unstale":"2024-05-09T01:23:22.303036+0000","last_undegraded":"2024-05-09T01:23:22.303036+0000","last_fullsized":"2024-05-09T01:23:22.303036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:23.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305959+0000","last_change":"2024-05-09T01:22:36.794171+0000","last_active":"2024-05-09T01:23:22.305959+0000","last_peered":"2024-05-09T01:23:22.305959+0000","last_clean":"2024-05-09T01:23:22.305959+0000","last_became_active":"2024-05-09T01:22:36.793713+0000","last_became_peered":"2024-05-09T01:22:36.793713+0000","last_unstale":"2024-05-09T01:23:22.305959+0000","last_undegraded":"2024-05-09T01:23:22.305959+0000","last_fullsized":"2024-05-09T01:23:22.305959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:06.654667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421893+0000","last_change":"2024-05-09T01:22:39.838266+0000","last_active":"2024-05-09T01:23:22.421893+0000","last_peered":"2024-05-09T01:23:22.421893+0000","last_clean":"2024-05-09T01:23:22.421893+0000","last_became_active":"2024-05-09T01:22:39.837911+0000","last_became_peered":"2024-05-09T01:22:39.837911+0000","last_unstale":"2024-05-09T01:23:22.421893+0000","last_undegraded":"2024-05-09T01:23:22.421893+0000","last_fullsized":"2024-05-09T01:23:22.421893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:39.414176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308540+0000","last_change":"2024-05-09T01:22:36.799766+0000","last_active":"2024-05-09T01:23:22.308540+0000","last_peered":"2024-05-09T01:23:22.308540+0000","last_clean":"2024-05-09T01:23:22.308540+0000","last_became_active":"2024-05-09T01:22:36.799388+0000","last_became_peered":"2024-05-09T01:22:36.799388+0000","last_unstale":"2024-05-09T01:23:22.308540+0000","last_undegraded":"2024-05-09T01:23:22.308540+0000","last_fullsized":"2024-05-09T01:23:22.308540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:13.401659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309449+0000","last_change":"2024-05-09T01:22:39.838493+0000","last_active":"2024-05-09T01:23:22.309449+0000","last_peered":"2024-05-09T01:23:22.309449+0000","last_clean":"2024-05-09T01:23:22.309449+0000","last_became_active":"2024-05-09T01:22:39.838086+0000","last_became_peered":"2024-05-09T01:22:39.838086+0000","last_unstale":"2024-05-09T01:23:22.309449+0000","last_undegraded":"2024-05-09T01:23:22.309449+0000","last_fullsized":"2024-05-09T01:23:22.309449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:06.340693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421503+0000","last_change":"2024-05-09T01:22:36.804872+0000","last_active":"2024-05-09T01:23:22.421503+0000","last_peered":"2024-05-09T01:23:22.421503+0000","last_clean":"2024-05-09T01:23:22.421503+0000","last_became_active":"2024-05-09T01:22:36.804439+0000","last_became_peered":"2024-05-09T01:22:36.804439+0000","last_unstale":"2024-05-09T01:23:22.421503+0000","last_undegraded":"2024-05-09T01:23:22.421503+0000","last_fullsized":"2024-05-09T01:23:22.421503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:08.584749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306460+0000","last_change":"2024-05-09T01:22:39.836579+0000","last_active":"2024-05-09T01:23:22.306460+0000","last_peered":"2024-05-09T01:23:22.306460+0000","last_clean":"2024-05-09T01:23:22.306460+0000","last_became_active":"2024-05-09T01:22:39.836244+0000","last_became_peered":"2024-05-09T01:22:39.836244+0000","last_unstale":"2024-05-09T01:23:22.306460+0000","last_undegraded":"2024-05-09T01:23:22.306460+0000","last_fullsized":"2024-05-09T01:23:22.306460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:43.329448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":84,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:24.669003+0000","last_change":"2024-05-09T01:22:36.796639+0000","last_active":"2024-05-09T01:23:24.669003+0000","last_peered":"2024-05-09T01:23:24.669003+0000","last_clean":"2024-05-09T01:23:24.669003+0000","last_became_active":"2024-05-09T01:22:36.796227+0000","last_became_peered":"2024-05-09T01:22:36.796227+0000","last_unstale":"2024-05-09T01:23:24.669003+0000","last_undegraded":"2024-05-09T01:23:24.669003+0000","last_fullsized":"2024-05-09T01:23:24.669003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:56.219281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421815+0000","last_change":"2024-05-09T01:22:39.841515+0000","last_active":"2024-05-09T01:23:22.421815+0000","last_peered":"2024-05-09T01:23:22.421815+0000","last_clean":"2024-05-09T01:23:22.421815+0000","last_became_active":"2024-05-09T01:22:39.840057+0000","last_became_peered":"2024-05-09T01:22:39.840057+0000","last_unstale":"2024-05-09T01:23:22.421815+0000","last_undegraded":"2024-05-09T01:23:22.421815+0000","last_fullsized":"2024-05-09T01:23:22.421815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:43.258466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309092+0000","last_change":"2024-05-09T01:22:36.801937+0000","last_active":"2024-05-09T01:23:22.309092+0000","last_peered":"2024-05-09T01:23:22.309092+0000","last_clean":"2024-05-09T01:23:22.309092+0000","last_became_active":"2024-05-09T01:22:36.801563+0000","last_became_peered":"2024-05-09T01:22:36.801563+0000","last_unstale":"2024-05-09T01:23:22.309092+0000","last_undegraded":"2024-05-09T01:23:22.309092+0000","last_fullsized":"2024-05-09T01:23:22.309092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:00:42.170156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.307684+0000","last_change":"2024-05-09T01:22:39.849154+0000","last_active":"2024-05-09T01:23:22.307684+0000","last_peered":"2024-05-09T01:23:22.307684+0000","last_clean":"2024-05-09T01:23:22.307684+0000","last_became_active":"2024-05-09T01:22:39.848815+0000","last_became_peered":"2024-05-09T01:22:39.848815+0000","last_unstale":"2024-05-09T01:23:22.307684+0000","last_undegraded":"2024-05-09T01:23:22.307684+0000","last_fullsized":"2024-05-09T01:23:22.307684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:00.300208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421958+0000","last_change":"2024-05-09T01:22:36.843201+0000","last_active":"2024-05-09T01:23:22.421958+0000","last_peered":"2024-05-09T01:23:22.421958+0000","last_clean":"2024-05-09T01:23:22.421958+0000","last_became_active":"2024-05-09T01:22:36.842500+0000","last_became_peered":"2024-05-09T01:22:36.842500+0000","last_unstale":"2024-05-09T01:23:22.421958+0000","last_undegraded":"2024-05-09T01:23:22.421958+0000","last_fullsized":"2024-05-09T01:23:22.421958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:35.092246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421515+0000","last_change":"2024-05-09T01:22:39.835120+0000","last_active":"2024-05-09T01:23:22.421515+0000","last_peered":"2024-05-09T01:23:22.421515+0000","last_clean":"2024-05-09T01:23:22.421515+0000","last_became_active":"2024-05-09T01:22:39.834613+0000","last_became_peered":"2024-05-09T01:22:39.834613+0000","last_unstale":"2024-05-09T01:23:22.421515+0000","last_undegraded":"2024-05-09T01:23:22.421515+0000","last_fullsized":"2024-05-09T01:23:22.421515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:08.380445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":119,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:24.677132+0000","last_change":"2024-05-09T01:22:36.800816+0000","last_active":"2024-05-09T01:23:24.677132+0000","last_peered":"2024-05-09T01:23:24.677132+0000","last_clean":"2024-05-09T01:23:24.677132+0000","last_became_active":"2024-05-09T01:22:36.798797+0000","last_became_peered":"2024-05-09T01:22:36.798797+0000","last_unstale":"2024-05-09T01:23:24.677132+0000","last_undegraded":"2024-05-09T01:23:24.677132+0000","last_fullsized":"2024-05-09T01:23:24.677132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:37.670163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57493,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":62,"num_read_kb":86,"num_write":33,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303530+0000","last_change":"2024-05-09T01:22:39.831504+0000","last_active":"2024-05-09T01:23:22.303530+0000","last_peered":"2024-05-09T01:23:22.303530+0000","last_clean":"2024-05-09T01:23:22.303530+0000","last_became_active":"2024-05-09T01:22:39.831129+0000","last_became_peered":"2024-05-09T01:22:39.831129+0000","last_unstale":"2024-05-09T01:23:22.303530+0000","last_undegraded":"2024-05-09T01:23:22.303530+0000","last_fullsized":"2024-05-09T01:23:22.303530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:19.212473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304426+0000","last_change":"2024-05-09T01:22:36.803447+0000","last_active":"2024-05-09T01:23:22.304426+0000","last_peered":"2024-05-09T01:23:22.304426+0000","last_clean":"2024-05-09T01:23:22.304426+0000","last_became_active":"2024-05-09T01:22:36.802986+0000","last_became_peered":"2024-05-09T01:22:36.802986+0000","last_unstale":"2024-05-09T01:23:22.304426+0000","last_undegraded":"2024-05-09T01:23:22.304426+0000","last_fullsized":"2024-05-09T01:23:22.304426+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:28.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.307697+0000","last_change":"2024-05-09T01:22:39.836546+0000","last_active":"2024-05-09T01:23:22.307697+0000","last_peered":"2024-05-09T01:23:22.307697+0000","last_clean":"2024-05-09T01:23:22.307697+0000","last_became_active":"2024-05-09T01:22:39.836168+0000","last_became_peered":"2024-05-09T01:22:39.836168+0000","last_unstale":"2024-05-09T01:23:22.307697+0000","last_undegraded":"2024-05-09T01:23:22.307697+0000","last_fullsized":"2024-05-09T01:23:22.307697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:45.522279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421596+0000","last_change":"2024-05-09T01:22:36.798703+0000","last_active":"2024-05-09T01:23:22.421596+0000","last_peered":"2024-05-09T01:23:22.421596+0000","last_clean":"2024-05-09T01:23:22.421596+0000","last_became_active":"2024-05-09T01:22:36.798280+0000","last_became_peered":"2024-05-09T01:22:36.798280+0000","last_unstale":"2024-05-09T01:23:22.421596+0000","last_undegraded":"2024-05-09T01:23:22.421596+0000","last_fullsized":"2024-05-09T01:23:22.421596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:33.794488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308781+0000","last_change":"2024-05-09T01:22:39.831223+0000","last_active":"2024-05-09T01:23:22.308781+0000","last_peered":"2024-05-09T01:23:22.308781+0000","last_clean":"2024-05-09T01:23:22.308781+0000","last_became_active":"2024-05-09T01:22:39.830845+0000","last_became_peered":"2024-05-09T01:22:39.830845+0000","last_unstale":"2024-05-09T01:23:22.308781+0000","last_undegraded":"2024-05-09T01:23:22.308781+0000","last_fullsized":"2024-05-09T01:23:22.308781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:23.365950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303353+0000","last_change":"2024-05-09T01:22:36.802327+0000","last_active":"2024-05-09T01:23:22.303353+0000","last_peered":"2024-05-09T01:23:22.303353+0000","last_clean":"2024-05-09T01:23:22.303353+0000","last_became_active":"2024-05-09T01:22:36.801951+0000","last_became_peered":"2024-05-09T01:22:36.801951+0000","last_unstale":"2024-05-09T01:23:22.303353+0000","last_undegraded":"2024-05-09T01:23:22.303353+0000","last_fullsized":"2024-05-09T01:23:22.303353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:12.282424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308443+0000","last_change":"2024-05-09T01:22:39.833626+0000","last_active":"2024-05-09T01:23:22.308443+0000","last_peered":"2024-05-09T01:23:22.308443+0000","last_clean":"2024-05-09T01:23:22.308443+0000","last_became_active":"2024-05-09T01:22:39.833176+0000","last_became_peered":"2024-05-09T01:22:39.833176+0000","last_unstale":"2024-05-09T01:23:22.308443+0000","last_undegraded":"2024-05-09T01:23:22.308443+0000","last_fullsized":"2024-05-09T01:23:22.308443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:25.082300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.328609+0000","last_change":"2024-05-09T01:23:22.328011+0000","last_active":"2024-05-09T01:23:22.328609+0000","last_peered":"2024-05-09T01:23:22.328609+0000","last_clean":"2024-05-09T01:23:22.328609+0000","last_became_active":"2024-05-09T01:23:22.313049+0000","last_became_peered":"2024-05-09T01:23:22.313049+0000","last_unstale":"2024-05-09T01:23:22.328609+0000","last_undegraded":"2024-05-09T01:23:22.328609+0000","last_fullsized":"2024-05-09T01:23:22.328609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:17.854491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421554+0000","last_change":"2024-05-09T01:22:39.837529+0000","last_active":"2024-05-09T01:23:22.421554+0000","last_peered":"2024-05-09T01:23:22.421554+0000","last_clean":"2024-05-09T01:23:22.421554+0000","last_became_active":"2024-05-09T01:22:39.837155+0000","last_became_peered":"2024-05-09T01:22:39.837155+0000","last_unstale":"2024-05-09T01:23:22.421554+0000","last_undegraded":"2024-05-09T01:23:22.421554+0000","last_fullsized":"2024-05-09T01:23:22.421554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:44.485629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303529+0000","last_change":"2024-05-09T01:22:36.800073+0000","last_active":"2024-05-09T01:23:22.303529+0000","last_peered":"2024-05-09T01:23:22.303529+0000","last_clean":"2024-05-09T01:23:22.303529+0000","last_became_active":"2024-05-09T01:22:36.799463+0000","last_became_peered":"2024-05-09T01:22:36.799463+0000","last_unstale":"2024-05-09T01:23:22.303529+0000","last_undegraded":"2024-05-09T01:23:22.303529+0000","last_fullsized":"2024-05-09T01:23:22.303529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.942945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306568+0000","last_change":"2024-05-09T01:22:39.833615+0000","last_active":"2024-05-09T01:23:22.306568+0000","last_peered":"2024-05-09T01:23:22.306568+0000","last_clean":"2024-05-09T01:23:22.306568+0000","last_became_active":"2024-05-09T01:22:39.833212+0000","last_became_peered":"2024-05-09T01:22:39.833212+0000","last_unstale":"2024-05-09T01:23:22.306568+0000","last_undegraded":"2024-05-09T01:23:22.306568+0000","last_fullsized":"2024-05-09T01:23:22.306568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:35.387442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305607+0000","last_change":"2024-05-09T01:22:36.843143+0000","last_active":"2024-05-09T01:23:22.305607+0000","last_peered":"2024-05-09T01:23:22.305607+0000","last_clean":"2024-05-09T01:23:22.305607+0000","last_became_active":"2024-05-09T01:22:36.842759+0000","last_became_peered":"2024-05-09T01:22:36.842759+0000","last_unstale":"2024-05-09T01:23:22.305607+0000","last_undegraded":"2024-05-09T01:23:22.305607+0000","last_fullsized":"2024-05-09T01:23:22.305607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:42.666186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306198+0000","last_change":"2024-05-09T01:22:39.838267+0000","last_active":"2024-05-09T01:23:22.306198+0000","last_peered":"2024-05-09T01:23:22.306198+0000","last_clean":"2024-05-09T01:23:22.306198+0000","last_became_active":"2024-05-09T01:22:39.837910+0000","last_became_peered":"2024-05-09T01:22:39.837910+0000","last_unstale":"2024-05-09T01:23:22.306198+0000","last_undegraded":"2024-05-09T01:23:22.306198+0000","last_fullsized":"2024-05-09T01:23:22.306198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:34.620332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.424637+0000","last_change":"2024-05-09T01:22:36.800515+0000","last_active":"2024-05-09T01:23:22.424637+0000","last_peered":"2024-05-09T01:23:22.424637+0000","last_clean":"2024-05-09T01:23:22.424637+0000","last_became_active":"2024-05-09T01:22:36.800065+0000","last_became_peered":"2024-05-09T01:22:36.800065+0000","last_unstale":"2024-05-09T01:23:22.424637+0000","last_undegraded":"2024-05-09T01:23:22.424637+0000","last_fullsized":"2024-05-09T01:23:22.424637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:50.160463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.313627+0000","last_change":"2024-05-09T01:23:22.313627+0000","last_active":"2024-05-09T01:23:22.313627+0000","last_peered":"2024-05-09T01:23:22.313627+0000","last_clean":"2024-05-09T01:23:22.313627+0000","last_became_active":"2024-05-09T01:23:22.313138+0000","last_became_peered":"2024-05-09T01:23:22.313138+0000","last_unstale":"2024-05-09T01:23:22.313627+0000","last_undegraded":"2024-05-09T01:23:22.313627+0000","last_fullsized":"2024-05-09T01:23:22.313627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:07.216086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.422009+0000","last_change":"2024-05-09T01:22:39.832188+0000","last_active":"2024-05-09T01:23:22.422009+0000","last_peered":"2024-05-09T01:23:22.422009+0000","last_clean":"2024-05-09T01:23:22.422009+0000","last_became_active":"2024-05-09T01:22:39.831302+0000","last_became_peered":"2024-05-09T01:22:39.831302+0000","last_unstale":"2024-05-09T01:23:22.422009+0000","last_undegraded":"2024-05-09T01:23:22.422009+0000","last_fullsized":"2024-05-09T01:23:22.422009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:06.703950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.672613+0000","last_change":"2024-05-09T01:22:36.841446+0000","last_active":"2024-05-09T01:23:22.672613+0000","last_peered":"2024-05-09T01:23:22.672613+0000","last_clean":"2024-05-09T01:23:22.672613+0000","last_became_active":"2024-05-09T01:22:36.841071+0000","last_became_peered":"2024-05-09T01:22:36.841071+0000","last_unstale":"2024-05-09T01:23:22.672613+0000","last_undegraded":"2024-05-09T01:23:22.672613+0000","last_fullsized":"2024-05-09T01:23:22.672613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:20.777482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309539+0000","last_change":"2024-05-09T01:22:39.846711+0000","last_active":"2024-05-09T01:23:22.309539+0000","last_peered":"2024-05-09T01:23:22.309539+0000","last_clean":"2024-05-09T01:23:22.309539+0000","last_became_active":"2024-05-09T01:22:39.846293+0000","last_became_peered":"2024-05-09T01:22:39.846293+0000","last_unstale":"2024-05-09T01:23:22.309539+0000","last_undegraded":"2024-05-09T01:23:22.309539+0000","last_fullsized":"2024-05-09T01:23:22.309539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:27.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303787+0000","last_change":"2024-05-09T01:22:36.798302+0000","last_active":"2024-05-09T01:23:22.303787+0000","last_peered":"2024-05-09T01:23:22.303787+0000","last_clean":"2024-05-09T01:23:22.303787+0000","last_became_active":"2024-05-09T01:22:36.797906+0000","last_became_peered":"2024-05-09T01:22:36.797906+0000","last_unstale":"2024-05-09T01:23:22.303787+0000","last_undegraded":"2024-05-09T01:23:22.303787+0000","last_fullsized":"2024-05-09T01:23:22.303787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:13.599240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308593+0000","last_change":"2024-05-09T01:21:12.781366+0000","last_active":"2024-05-09T01:23:22.308593+0000","last_peered":"2024-05-09T01:23:22.308593+0000","last_clean":"2024-05-09T01:23:22.308593+0000","last_became_active":"2024-05-09T01:21:12.774967+0000","last_became_peered":"2024-05-09T01:21:12.774967+0000","last_unstale":"2024-05-09T01:23:22.308593+0000","last_undegraded":"2024-05-09T01:23:22.308593+0000","last_fullsized":"2024-05-09T01:23:22.308593+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_clean_scrub_stamp":"2024-05-09T01:17:49.682508+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:08.075365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.307279+0000","last_change":"2024-05-09T01:22:39.840952+0000","last_active":"2024-05-09T01:23:22.307279+0000","last_peered":"2024-05-09T01:23:22.307279+0000","last_clean":"2024-05-09T01:23:22.307279+0000","last_became_active":"2024-05-09T01:22:39.840323+0000","last_became_peered":"2024-05-09T01:22:39.840323+0000","last_unstale":"2024-05-09T01:23:22.307279+0000","last_undegraded":"2024-05-09T01:23:22.307279+0000","last_fullsized":"2024-05-09T01:23:22.307279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:23.837123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308793+0000","last_change":"2024-05-09T01:22:36.798631+0000","last_active":"2024-05-09T01:23:22.308793+0000","last_peered":"2024-05-09T01:23:22.308793+0000","last_clean":"2024-05-09T01:23:22.308793+0000","last_became_active":"2024-05-09T01:22:36.798242+0000","last_became_peered":"2024-05-09T01:22:36.798242+0000","last_unstale":"2024-05-09T01:23:22.308793+0000","last_undegraded":"2024-05-09T01:23:22.308793+0000","last_fullsized":"2024-05-09T01:23:22.308793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:18.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303746+0000","last_change":"2024-05-09T01:22:36.843779+0000","last_active":"2024-05-09T01:23:22.303746+0000","last_peered":"2024-05-09T01:23:22.303746+0000","last_clean":"2024-05-09T01:23:22.303746+0000","last_became_active":"2024-05-09T01:22:36.842084+0000","last_became_peered":"2024-05-09T01:22:36.842084+0000","last_unstale":"2024-05-09T01:23:22.303746+0000","last_undegraded":"2024-05-09T01:23:22.303746+0000","last_fullsized":"2024-05-09T01:23:22.303746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:34.743372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308911+0000","last_change":"2024-05-09T01:22:39.842266+0000","last_active":"2024-05-09T01:23:22.308911+0000","last_peered":"2024-05-09T01:23:22.308911+0000","last_clean":"2024-05-09T01:23:22.308911+0000","last_became_active":"2024-05-09T01:22:39.841927+0000","last_became_peered":"2024-05-09T01:22:39.841927+0000","last_unstale":"2024-05-09T01:23:22.308911+0000","last_undegraded":"2024-05-09T01:23:22.308911+0000","last_fullsized":"2024-05-09T01:23:22.308911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:26.774250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305605+0000","last_change":"2024-05-09T01:22:36.796929+0000","last_active":"2024-05-09T01:23:22.305605+0000","last_peered":"2024-05-09T01:23:22.305605+0000","last_clean":"2024-05-09T01:23:22.305605+0000","last_became_active":"2024-05-09T01:22:36.796480+0000","last_became_peered":"2024-05-09T01:22:36.796480+0000","last_unstale":"2024-05-09T01:23:22.305605+0000","last_undegraded":"2024-05-09T01:23:22.305605+0000","last_fullsized":"2024-05-09T01:23:22.305605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:45.168305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.312853+0000","last_change":"2024-05-09T01:22:39.843093+0000","last_active":"2024-05-09T01:23:22.312853+0000","last_peered":"2024-05-09T01:23:22.312853+0000","last_clean":"2024-05-09T01:23:22.312853+0000","last_became_active":"2024-05-09T01:22:39.842729+0000","last_became_peered":"2024-05-09T01:22:39.842729+0000","last_unstale":"2024-05-09T01:23:22.312853+0000","last_undegraded":"2024-05-09T01:23:22.312853+0000","last_fullsized":"2024-05-09T01:23:22.312853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:02.074843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303637+0000","last_change":"2024-05-09T01:22:36.798166+0000","last_active":"2024-05-09T01:23:22.303637+0000","last_peered":"2024-05-09T01:23:22.303637+0000","last_clean":"2024-05-09T01:23:22.303637+0000","last_became_active":"2024-05-09T01:22:36.797752+0000","last_became_peered":"2024-05-09T01:22:36.797752+0000","last_unstale":"2024-05-09T01:23:22.303637+0000","last_undegraded":"2024-05-09T01:23:22.303637+0000","last_fullsized":"2024-05-09T01:23:22.303637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:05.211720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.316228+0000","last_change":"2024-05-09T01:23:22.316228+0000","last_active":"2024-05-09T01:23:22.316228+0000","last_peered":"2024-05-09T01:23:22.316228+0000","last_clean":"2024-05-09T01:23:22.316228+0000","last_became_active":"2024-05-09T01:23:22.315683+0000","last_became_peered":"2024-05-09T01:23:22.315683+0000","last_unstale":"2024-05-09T01:23:22.316228+0000","last_undegraded":"2024-05-09T01:23:22.316228+0000","last_fullsized":"2024-05-09T01:23:22.316228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:04.623701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421234+0000","last_change":"2024-05-09T01:22:36.841999+0000","last_active":"2024-05-09T01:23:22.421234+0000","last_peered":"2024-05-09T01:23:22.421234+0000","last_clean":"2024-05-09T01:23:22.421234+0000","last_became_active":"2024-05-09T01:22:36.841664+0000","last_became_peered":"2024-05-09T01:22:36.841664+0000","last_unstale":"2024-05-09T01:23:22.421234+0000","last_undegraded":"2024-05-09T01:23:22.421234+0000","last_fullsized":"2024-05-09T01:23:22.421234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:27.110258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303011+0000","last_change":"2024-05-09T01:22:39.839553+0000","last_active":"2024-05-09T01:23:22.303011+0000","last_peered":"2024-05-09T01:23:22.303011+0000","last_clean":"2024-05-09T01:23:22.303011+0000","last_became_active":"2024-05-09T01:22:39.839189+0000","last_became_peered":"2024-05-09T01:22:39.839189+0000","last_unstale":"2024-05-09T01:23:22.303011+0000","last_undegraded":"2024-05-09T01:23:22.303011+0000","last_fullsized":"2024-05-09T01:23:22.303011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:02.794073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309848+0000","last_change":"2024-05-09T01:22:36.792516+0000","last_active":"2024-05-09T01:23:22.309848+0000","last_peered":"2024-05-09T01:23:22.309848+0000","last_clean":"2024-05-09T01:23:22.309848+0000","last_became_active":"2024-05-09T01:22:36.792129+0000","last_became_peered":"2024-05-09T01:22:36.792129+0000","last_unstale":"2024-05-09T01:23:22.309848+0000","last_undegraded":"2024-05-09T01:23:22.309848+0000","last_fullsized":"2024-05-09T01:23:22.309848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:46.596723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306298+0000","last_change":"2024-05-09T01:22:39.841523+0000","last_active":"2024-05-09T01:23:22.306298+0000","last_peered":"2024-05-09T01:23:22.306298+0000","last_clean":"2024-05-09T01:23:22.306298+0000","last_became_active":"2024-05-09T01:22:39.840667+0000","last_became_peered":"2024-05-09T01:22:39.840667+0000","last_unstale":"2024-05-09T01:23:22.306298+0000","last_undegraded":"2024-05-09T01:23:22.306298+0000","last_fullsized":"2024-05-09T01:23:22.306298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:39.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.332290+0000","last_change":"2024-05-09T01:23:22.331855+0000","last_active":"2024-05-09T01:23:22.332290+0000","last_peered":"2024-05-09T01:23:22.332290+0000","last_clean":"2024-05-09T01:23:22.332290+0000","last_became_active":"2024-05-09T01:23:22.313059+0000","last_became_peered":"2024-05-09T01:23:22.313059+0000","last_unstale":"2024-05-09T01:23:22.332290+0000","last_undegraded":"2024-05-09T01:23:22.332290+0000","last_fullsized":"2024-05-09T01:23:22.332290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:16.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309287+0000","last_change":"2024-05-09T01:22:39.845031+0000","last_active":"2024-05-09T01:23:22.309287+0000","last_peered":"2024-05-09T01:23:22.309287+0000","last_clean":"2024-05-09T01:23:22.309287+0000","last_became_active":"2024-05-09T01:22:39.844653+0000","last_became_peered":"2024-05-09T01:22:39.844653+0000","last_unstale":"2024-05-09T01:23:22.309287+0000","last_undegraded":"2024-05-09T01:23:22.309287+0000","last_fullsized":"2024-05-09T01:23:22.309287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:14.382445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303070+0000","last_change":"2024-05-09T01:22:36.801395+0000","last_active":"2024-05-09T01:23:22.303070+0000","last_peered":"2024-05-09T01:23:22.303070+0000","last_clean":"2024-05-09T01:23:22.303070+0000","last_became_active":"2024-05-09T01:22:36.800992+0000","last_became_peered":"2024-05-09T01:22:36.800992+0000","last_unstale":"2024-05-09T01:23:22.303070+0000","last_undegraded":"2024-05-09T01:23:22.303070+0000","last_fullsized":"2024-05-09T01:23:22.303070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:16.089068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308707+0000","last_change":"2024-05-09T01:22:39.835842+0000","last_active":"2024-05-09T01:23:22.308707+0000","last_peered":"2024-05-09T01:23:22.308707+0000","last_clean":"2024-05-09T01:23:22.308707+0000","last_became_active":"2024-05-09T01:22:39.835372+0000","last_became_peered":"2024-05-09T01:22:39.835372+0000","last_unstale":"2024-05-09T01:23:22.308707+0000","last_undegraded":"2024-05-09T01:23:22.308707+0000","last_fullsized":"2024-05-09T01:23:22.308707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:13.197946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:23.318556+0000","last_change":"2024-05-09T01:23:22.332514+0000","last_active":"2024-05-09T01:23:23.318556+0000","last_peered":"2024-05-09T01:23:23.318556+0000","last_clean":"2024-05-09T01:23:23.318556+0000","last_became_active":"2024-05-09T01:23:22.312966+0000","last_became_peered":"2024-05-09T01:23:22.312966+0000","last_unstale":"2024-05-09T01:23:23.318556+0000","last_undegraded":"2024-05-09T01:23:23.318556+0000","last_fullsized":"2024-05-09T01:23:23.318556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:52.637108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309343+0000","last_change":"2024-05-09T01:22:39.845390+0000","last_active":"2024-05-09T01:23:22.309343+0000","last_peered":"2024-05-09T01:23:22.309343+0000","last_clean":"2024-05-09T01:23:22.309343+0000","last_became_active":"2024-05-09T01:22:39.844918+0000","last_became_peered":"2024-05-09T01:22:39.844918+0000","last_unstale":"2024-05-09T01:23:22.309343+0000","last_undegraded":"2024-05-09T01:23:22.309343+0000","last_fullsized":"2024-05-09T01:23:22.309343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:04.455794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421710+0000","last_change":"2024-05-09T01:22:36.803329+0000","last_active":"2024-05-09T01:23:22.421710+0000","last_peered":"2024-05-09T01:23:22.421710+0000","last_clean":"2024-05-09T01:23:22.421710+0000","last_became_active":"2024-05-09T01:22:36.802831+0000","last_became_peered":"2024-05-09T01:22:36.802831+0000","last_unstale":"2024-05-09T01:23:22.421710+0000","last_undegraded":"2024-05-09T01:23:22.421710+0000","last_fullsized":"2024-05-09T01:23:22.421710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:57.919840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309007+0000","last_change":"2024-05-09T01:22:39.837595+0000","last_active":"2024-05-09T01:23:22.309007+0000","last_peered":"2024-05-09T01:23:22.309007+0000","last_clean":"2024-05-09T01:23:22.309007+0000","last_became_active":"2024-05-09T01:22:39.837197+0000","last_became_peered":"2024-05-09T01:22:39.837197+0000","last_unstale":"2024-05-09T01:23:22.309007+0000","last_undegraded":"2024-05-09T01:23:22.309007+0000","last_fullsized":"2024-05-09T01:23:22.309007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:08.700741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.321286+0000","last_change":"2024-05-09T01:23:22.321286+0000","last_active":"2024-05-09T01:23:22.321286+0000","last_peered":"2024-05-09T01:23:22.321286+0000","last_clean":"2024-05-09T01:23:22.321286+0000","last_became_active":"2024-05-09T01:23:22.315810+0000","last_became_peered":"2024-05-09T01:23:22.315810+0000","last_unstale":"2024-05-09T01:23:22.321286+0000","last_undegraded":"2024-05-09T01:23:22.321286+0000","last_fullsized":"2024-05-09T01:23:22.321286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:43.100893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309221+0000","last_change":"2024-05-09T01:22:39.827798+0000","last_active":"2024-05-09T01:23:22.309221+0000","last_peered":"2024-05-09T01:23:22.309221+0000","last_clean":"2024-05-09T01:23:22.309221+0000","last_became_active":"2024-05-09T01:22:39.827322+0000","last_became_peered":"2024-05-09T01:22:39.827322+0000","last_unstale":"2024-05-09T01:23:22.309221+0000","last_undegraded":"2024-05-09T01:23:22.309221+0000","last_fullsized":"2024-05-09T01:23:22.309221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:52.454033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306428+0000","last_change":"2024-05-09T01:22:39.841885+0000","last_active":"2024-05-09T01:23:22.306428+0000","last_peered":"2024-05-09T01:23:22.306428+0000","last_clean":"2024-05-09T01:23:22.306428+0000","last_became_active":"2024-05-09T01:22:39.841160+0000","last_became_peered":"2024-05-09T01:22:39.841160+0000","last_unstale":"2024-05-09T01:23:22.306428+0000","last_undegraded":"2024-05-09T01:23:22.306428+0000","last_fullsized":"2024-05-09T01:23:22.306428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:57.290465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303742+0000","last_change":"2024-05-09T01:22:39.846495+0000","last_active":"2024-05-09T01:23:22.303742+0000","last_peered":"2024-05-09T01:23:22.303742+0000","last_clean":"2024-05-09T01:23:22.303742+0000","last_became_active":"2024-05-09T01:22:39.846108+0000","last_became_peered":"2024-05-09T01:22:39.846108+0000","last_unstale":"2024-05-09T01:23:22.303742+0000","last_undegraded":"2024-05-09T01:23:22.303742+0000","last_fullsized":"2024-05-09T01:23:22.303742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:39.958174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303450+0000","last_change":"2024-05-09T01:22:39.841288+0000","last_active":"2024-05-09T01:23:22.303450+0000","last_peered":"2024-05-09T01:23:22.303450+0000","last_clean":"2024-05-09T01:23:22.303450+0000","last_became_active":"2024-05-09T01:22:39.840900+0000","last_became_peered":"2024-05-09T01:22:39.840900+0000","last_unstale":"2024-05-09T01:23:22.303450+0000","last_undegraded":"2024-05-09T01:23:22.303450+0000","last_fullsized":"2024-05-09T01:23:22.303450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:30.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.313135+0000","last_change":"2024-05-09T01:23:22.313135+0000","last_active":"2024-05-09T01:23:22.313135+0000","last_peered":"2024-05-09T01:23:22.313135+0000","last_clean":"2024-05-09T01:23:22.313135+0000","last_became_active":"2024-05-09T01:23:22.312710+0000","last_became_peered":"2024-05-09T01:23:22.312710+0000","last_unstale":"2024-05-09T01:23:22.313135+0000","last_undegraded":"2024-05-09T01:23:22.313135+0000","last_fullsized":"2024-05-09T01:23:22.313135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:47.725790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309028+0000","last_change":"2024-05-09T01:22:39.838037+0000","last_active":"2024-05-09T01:23:22.309028+0000","last_peered":"2024-05-09T01:23:22.309028+0000","last_clean":"2024-05-09T01:23:22.309028+0000","last_became_active":"2024-05-09T01:22:39.837362+0000","last_became_peered":"2024-05-09T01:22:39.837362+0000","last_unstale":"2024-05-09T01:23:22.309028+0000","last_undegraded":"2024-05-09T01:23:22.309028+0000","last_fullsized":"2024-05-09T01:23:22.309028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.781318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.310046+0000","last_change":"2024-05-09T01:23:22.310046+0000","last_active":"2024-05-09T01:23:22.310046+0000","last_peered":"2024-05-09T01:23:22.310046+0000","last_clean":"2024-05-09T01:23:22.310046+0000","last_became_active":"2024-05-09T01:23:22.309541+0000","last_became_peered":"2024-05-09T01:23:22.309541+0000","last_unstale":"2024-05-09T01:23:22.310046+0000","last_undegraded":"2024-05-09T01:23:22.310046+0000","last_fullsized":"2024-05-09T01:23:22.310046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:30.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304754+0000","last_change":"2024-05-09T01:22:39.840453+0000","last_active":"2024-05-09T01:23:22.304754+0000","last_peered":"2024-05-09T01:23:22.304754+0000","last_clean":"2024-05-09T01:23:22.304754+0000","last_became_active":"2024-05-09T01:22:39.840077+0000","last_became_peered":"2024-05-09T01:22:39.840077+0000","last_unstale":"2024-05-09T01:23:22.304754+0000","last_undegraded":"2024-05-09T01:23:22.304754+0000","last_fullsized":"2024-05-09T01:23:22.304754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:20.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.312731+0000","last_change":"2024-05-09T01:22:39.844554+0000","last_active":"2024-05-09T01:23:22.312731+0000","last_peered":"2024-05-09T01:23:22.312731+0000","last_clean":"2024-05-09T01:23:22.312731+0000","last_became_active":"2024-05-09T01:22:39.843640+0000","last_became_peered":"2024-05-09T01:22:39.843640+0000","last_unstale":"2024-05-09T01:23:22.312731+0000","last_undegraded":"2024-05-09T01:23:22.312731+0000","last_fullsized":"2024-05-09T01:23:22.312731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:27.801565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.422009+0000","last_change":"2024-05-09T01:22:39.839043+0000","last_active":"2024-05-09T01:23:22.422009+0000","last_peered":"2024-05-09T01:23:22.422009+0000","last_clean":"2024-05-09T01:23:22.422009+0000","last_became_active":"2024-05-09T01:22:39.838662+0000","last_became_peered":"2024-05-09T01:22:39.838662+0000","last_unstale":"2024-05-09T01:23:22.422009+0000","last_undegraded":"2024-05-09T01:23:22.422009+0000","last_fullsized":"2024-05-09T01:23:22.422009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:16.117689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.313903+0000","last_change":"2024-05-09T01:23:22.309945+0000","last_active":"2024-05-09T01:23:22.313903+0000","last_peered":"2024-05-09T01:23:22.313903+0000","last_clean":"2024-05-09T01:23:22.313903+0000","last_became_active":"2024-05-09T01:23:22.309543+0000","last_became_peered":"2024-05-09T01:23:22.309543+0000","last_unstale":"2024-05-09T01:23:22.313903+0000","last_undegraded":"2024-05-09T01:23:22.313903+0000","last_fullsized":"2024-05-09T01:23:22.313903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:47.464524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421407+0000","last_change":"2024-05-09T01:22:39.838911+0000","last_active":"2024-05-09T01:23:22.421407+0000","last_peered":"2024-05-09T01:23:22.421407+0000","last_clean":"2024-05-09T01:23:22.421407+0000","last_became_active":"2024-05-09T01:22:39.838293+0000","last_became_peered":"2024-05-09T01:22:39.838293+0000","last_unstale":"2024-05-09T01:23:22.421407+0000","last_undegraded":"2024-05-09T01:23:22.421407+0000","last_fullsized":"2024-05-09T01:23:22.421407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:32.596306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308388+0000","last_change":"2024-05-09T01:22:39.838185+0000","last_active":"2024-05-09T01:23:22.308388+0000","last_peered":"2024-05-09T01:23:22.308388+0000","last_clean":"2024-05-09T01:23:22.308388+0000","last_became_active":"2024-05-09T01:22:39.837530+0000","last_became_peered":"2024-05-09T01:22:39.837530+0000","last_unstale":"2024-05-09T01:23:22.308388+0000","last_undegraded":"2024-05-09T01:23:22.308388+0000","last_fullsized":"2024-05-09T01:23:22.308388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:22.556710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305947+0000","last_change":"2024-05-09T01:22:39.834124+0000","last_active":"2024-05-09T01:23:22.305947+0000","last_peered":"2024-05-09T01:23:22.305947+0000","last_clean":"2024-05-09T01:23:22.305947+0000","last_became_active":"2024-05-09T01:22:39.833716+0000","last_became_peered":"2024-05-09T01:22:39.833716+0000","last_unstale":"2024-05-09T01:23:22.305947+0000","last_undegraded":"2024-05-09T01:23:22.305947+0000","last_fullsized":"2024-05-09T01:23:22.305947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:54:42.337900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65088,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":276,"num_read_kb":220,"num_write":247,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2345,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":195264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612633,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":437592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1605632,"data_stored":1086002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":437019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":913408,"data_stored":436613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1527808,"data_stored":1030306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528371,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":437127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":496205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":937984,"data_stored":438466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":494192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117057,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":436561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":438549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1540096,"data_stored":1029202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T01:23:29.603 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T01:23:29.603 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T01:23:29.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T01:23:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:30.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:30 smithi059 ceph-mon[34351]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:23:30.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:30 smithi059 ceph-mon[34351]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 895 B/s wr, 6 op/s; 293 B/s, 2 objects/s recovering 2024-05-09T01:23:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:30 smithi019 ceph-mon[30158]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:23:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:30 smithi019 ceph-mon[30158]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 895 B/s wr, 6 op/s; 293 B/s, 2 objects/s recovering 2024-05-09T01:23:30.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:30 smithi196 ceph-mon[35041]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:23:30.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:30 smithi196 ceph-mon[35041]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 895 B/s wr, 6 op/s; 293 B/s, 2 objects/s recovering 2024-05-09T01:23:31.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.599+0000 7f36d0080700 1 -- 172.21.15.19:0/3520684169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c81059b0 msgr2=0x7f36c8105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:31.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.599+0000 7f36d0080700 1 --2- 172.21.15.19:0/3520684169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c81059b0 0x7f36c8105dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009990 tx=0x7f36c4009ca0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.599+0000 7f36d0080700 1 -- 172.21.15.19:0/3520684169 shutdown_connections 2024-05-09T01:23:31.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.599+0000 7f36d0080700 1 --2- 172.21.15.19:0/3520684169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36c81049e0 0x7f36c810c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.599+0000 7f36d0080700 1 --2- 172.21.15.19:0/3520684169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c81059b0 0x7f36c8105dc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.599+0000 7f36d0080700 1 --2- 172.21.15.19:0/3520684169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c8103f60 0x7f36c8104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.599+0000 7f36d0080700 1 -- 172.21.15.19:0/3520684169 >> 172.21.15.19:0/3520684169 conn(0x7f36c80ff9d0 msgr2=0x7f36c8101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:31.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.599+0000 7f36d0080700 1 -- 172.21.15.19:0/3520684169 shutdown_connections 2024-05-09T01:23:31.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.599+0000 7f36d0080700 1 -- 172.21.15.19:0/3520684169 wait complete. 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36d0080700 1 Processor -- start 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36d0080700 1 -- start start 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36d0080700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c8103f60 0x7f36c819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36d0080700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36c81049e0 0x7f36c819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36d0080700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c81059b0 0x7f36c81a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36d0080700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36c810fed0 con 0x7f36c8103f60 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36d0080700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f36c810fd50 con 0x7f36c81049e0 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36d0080700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8110050 con 0x7f36c81059b0 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36cde1c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c8103f60 0x7f36c819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36cde1c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c8103f60 0x7f36c819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44970/0 (socket says 172.21.15.19:44970) 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36cde1c700 1 -- 172.21.15.19:0/129165519 learned_addr learned my addr 172.21.15.19:0/129165519 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.600+0000 7f36cd61b700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36c81049e0 0x7f36c819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36ce61d700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c81059b0 0x7f36c81a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36cde1c700 1 -- 172.21.15.19:0/129165519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c81059b0 msgr2=0x7f36c81a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36cde1c700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c81059b0 0x7f36c81a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36cde1c700 1 -- 172.21.15.19:0/129165519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36c81049e0 msgr2=0x7f36c819b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36cde1c700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36c81049e0 0x7f36c819b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36cde1c700 1 -- 172.21.15.19:0/129165519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c81a2630 con 0x7f36c8103f60 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36cde1c700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c8103f60 0x7f36c819aad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f36c4000c00 tx=0x7f36c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36beffd700 1 -- 172.21.15.19:0/129165519 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4027070 con 0x7f36c8103f60 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36beffd700 1 -- 172.21.15.19:0/129165519 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c4004000 con 0x7f36c8103f60 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c81a28c0 con 0x7f36c8103f60 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36beffd700 1 -- 172.21.15.19:0/129165519 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4022d80 con 0x7f36c8103f60 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.601+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c81a2d10 con 0x7f36c8103f60 2024-05-09T01:23:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.602+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c80512c0 con 0x7f36c8103f60 2024-05-09T01:23:31.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.604+0000 7f36beffd700 1 -- 172.21.15.19:0/129165519 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f36c402a070 con 0x7f36c8103f60 2024-05-09T01:23:31.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.605+0000 7f36beffd700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36b40b2910 0x7f36b40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:31.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.605+0000 7f36cd61b700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36b40b2910 0x7f36b40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:31.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.605+0000 7f36beffd700 1 -- 172.21.15.19:0/129165519 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f36c40e3750 con 0x7f36c8103f60 2024-05-09T01:23:31.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.605+0000 7f36cd61b700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36b40b2910 0x7f36b40b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f36c819bb80 tx=0x7f36b800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:31.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.608+0000 7f36beffd700 1 -- 172.21.15.19:0/129165519 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36c4013070 con 0x7f36c8103f60 2024-05-09T01:23:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.918+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f36c802e5f0 con 0x7f36c8103f60 2024-05-09T01:23:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.921+0000 7f36beffd700 1 -- 172.21.15.19:0/129165519 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f36c40041b0 con 0x7f36c8103f60 2024-05-09T01:23:31.919 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-05-09T01:23:31.919 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCTJTxmBiLRNhAA+vIMeW7tmpGXC6YAUW0XLg== 2024-05-09T01:23:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36b40b2910 msgr2=0x7f36b40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36b40b2910 0x7f36b40b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f36c819bb80 tx=0x7f36b800b410 comp rx=0 tx=0).stop 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c8103f60 msgr2=0x7f36c819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c8103f60 0x7f36c819aad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f36c4000c00 tx=0x7f36c40047b0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 shutdown_connections 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c81059b0 0x7f36c81a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36b40b2910 0x7f36b40b4e20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36c8103f60 0x7f36c819aad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 --2- 172.21.15.19:0/129165519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36c81049e0 0x7f36c819b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.923+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 >> 172.21.15.19:0/129165519 conn(0x7f36c80ff9d0 msgr2=0x7f36c81086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.924+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 shutdown_connections 2024-05-09T01:23:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:31.924+0000 7f36d0080700 1 -- 172.21.15.19:0/129165519 wait complete. 2024-05-09T01:23:32.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:32 smithi019 ceph-mon[30158]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s; 238 B/s, 2 objects/s recovering 2024-05-09T01:23:32.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:32 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/129165519' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T01:23:32.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:32 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/129165519' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T01:23:32.564 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:23:32.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T01:23:32.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T01:23:32.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:32 smithi059 ceph-mon[34351]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s; 238 B/s, 2 objects/s recovering 2024-05-09T01:23:32.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:32 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/129165519' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T01:23:32.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:32 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/129165519' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T01:23:32.604 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T01:23:32.604 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T01:23:32.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T01:23:32.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:32 smithi196 ceph-mon[35041]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s; 238 B/s, 2 objects/s recovering 2024-05-09T01:23:32.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:32 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/129165519' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T01:23:32.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:32 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/129165519' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T01:23:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.558+0000 7faab369d700 1 -- 172.21.15.19:0/2320844433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac108740 msgr2=0x7faaac1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.558+0000 7faab369d700 1 --2- 172.21.15.19:0/2320844433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac108740 0x7faaac1100a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faaa400b0d0 tx=0x7faaa400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:34 smithi019 ceph-mon[30158]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 216 B/s, 1 objects/s recovering 2024-05-09T01:23:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.558+0000 7faab369d700 1 -- 172.21.15.19:0/2320844433 shutdown_connections 2024-05-09T01:23:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.558+0000 7faab369d700 1 --2- 172.21.15.19:0/2320844433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac108740 0x7faaac1100a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.558+0000 7faab369d700 1 --2- 172.21.15.19:0/2320844433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaac1034c0 0x7faaac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.558+0000 7faab369d700 1 --2- 172.21.15.19:0/2320844433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaac107df0 0x7faaac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.558+0000 7faab369d700 1 -- 172.21.15.19:0/2320844433 >> 172.21.15.19:0/2320844433 conn(0x7faaac0fedc0 msgr2=0x7faaac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.558+0000 7faab369d700 1 -- 172.21.15.19:0/2320844433 shutdown_connections 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.558+0000 7faab369d700 1 -- 172.21.15.19:0/2320844433 wait complete. 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.559+0000 7faab369d700 1 Processor -- start 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.559+0000 7faab369d700 1 -- start start 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.559+0000 7faab369d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac1034c0 0x7faaac19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.559+0000 7faab369d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaac107df0 0x7faaac19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.559+0000 7faab369d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaac108740 0x7faaac1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab369d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1133e0 con 0x7faaac1034c0 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab369d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faaac113260 con 0x7faaac107df0 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab369d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faaac113560 con 0x7faaac108740 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1439700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac1034c0 0x7faaac19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1439700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac1034c0 0x7faaac19d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44994/0 (socket says 172.21.15.19:44994) 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1439700 1 -- 172.21.15.19:0/2732193953 learned_addr learned my addr 172.21.15.19:0/2732193953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1c3a700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaac108740 0x7faaac1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab0c38700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaac107df0 0x7faaac19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1439700 1 -- 172.21.15.19:0/2732193953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaac108740 msgr2=0x7faaac1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1439700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaac108740 0x7faaac1a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1439700 1 -- 172.21.15.19:0/2732193953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaac107df0 msgr2=0x7faaac19d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1439700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaac107df0 0x7faaac19d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1439700 1 -- 172.21.15.19:0/2732193953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac1a1ee0 con 0x7faaac1034c0 2024-05-09T01:23:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab1439700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac1034c0 0x7faaac19d010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faaa800d950 tx=0x7faaa800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faaa27fc700 1 -- 172.21.15.19:0/2732193953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8016070 con 0x7faaac1034c0 2024-05-09T01:23:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faaa27fc700 1 -- 172.21.15.19:0/2732193953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa800fd40 con 0x7faaac1034c0 2024-05-09T01:23:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.560+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaac1a21d0 con 0x7faaac1034c0 2024-05-09T01:23:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.561+0000 7faaa27fc700 1 -- 172.21.15.19:0/2732193953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa800e3b0 con 0x7faaac1034c0 2024-05-09T01:23:34.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.561+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaac1a9af0 con 0x7faaac1034c0 2024-05-09T01:23:34.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.562+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaac0512c0 con 0x7faaac1034c0 2024-05-09T01:23:34.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.563+0000 7faaa27fc700 1 -- 172.21.15.19:0/2732193953 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7faaa8019070 con 0x7faaac1034c0 2024-05-09T01:23:34.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.564+0000 7faaa27fc700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faa980b2910 0x7faa980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:34.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.564+0000 7faab0c38700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faa980b2910 0x7faa980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:34.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.564+0000 7faaa27fc700 1 -- 172.21.15.19:0/2732193953 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7faaa80e1e10 con 0x7faaac1034c0 2024-05-09T01:23:34.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.565+0000 7faab0c38700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faa980b2910 0x7faa980b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faaac06c430 tx=0x7faa9c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:34.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.568+0000 7faaa27fc700 1 -- 172.21.15.19:0/2732193953 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faaa80a99b0 con 0x7faaac1034c0 2024-05-09T01:23:34.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:34 smithi196 ceph-mon[35041]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 216 B/s, 1 objects/s recovering 2024-05-09T01:23:34.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.844+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7faaac02e5f0 con 0x7faaac1034c0 2024-05-09T01:23:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:34 smithi059 ceph-mon[34351]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 216 B/s, 1 objects/s recovering 2024-05-09T01:23:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.850+0000 7faaa27fc700 1 -- 172.21.15.19:0/2732193953 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288647 (secure 0 0 0) 0x7faaa80a92b0 con 0x7faaac1034c0 2024-05-09T01:23:34.852 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:23:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faa980b2910 msgr2=0x7faa980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faa980b2910 0x7faa980b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faaac06c430 tx=0x7faa9c009190 comp rx=0 tx=0).stop 2024-05-09T01:23:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac1034c0 msgr2=0x7faaac19d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac1034c0 0x7faaac19d010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faaa800d950 tx=0x7faaa800dc60 comp rx=0 tx=0).stop 2024-05-09T01:23:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 shutdown_connections 2024-05-09T01:23:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaac108740 0x7faaac1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faa980b2910 0x7faa980b4e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaac1034c0 0x7faaac19d010 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 --2- 172.21.15.19:0/2732193953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaac107df0 0x7faaac19d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:34.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 >> 172.21.15.19:0/2732193953 conn(0x7faaac0fedc0 msgr2=0x7faaac10ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:34.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 shutdown_connections 2024-05-09T01:23:34.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:34.854+0000 7faab369d700 1 -- 172.21.15.19:0/2732193953 wait complete. 2024-05-09T01:23:35.457 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:35 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2732193953' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T01:23:35.487 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":2331192605},{"type":"v1","addr":"172.21.15.19:6801","nonce":2331192605}]},"active_addr":"172.21.15.19:6801/2331192605","active_change":"2024-05-09T01:15:21.027314+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1679793166}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1315263502}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2117409099}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2153532114}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1098745903}]}]} 2024-05-09T01:23:35.495 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T01:23:35.495 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T01:23:35.495 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:23:35.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:35 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2732193953' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T01:23:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:35.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:35 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2732193953' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T01:23:36.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:36 smithi019 ceph-mon[30158]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 8 op/s; 195 B/s, 1 objects/s recovering 2024-05-09T01:23:36.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:36 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:23:36.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:36 smithi196 ceph-mon[35041]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 8 op/s; 195 B/s, 1 objects/s recovering 2024-05-09T01:23:36.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:36 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:23:36.830 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:36 smithi059 ceph-mon[34351]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 8 op/s; 195 B/s, 1 objects/s recovering 2024-05-09T01:23:36.830 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:36 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:23:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.429+0000 7f0eafc25700 1 -- 172.21.15.19:0/3277165838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 msgr2=0x7f0ea8103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.429+0000 7f0eafc25700 1 --2- 172.21.15.19:0/3277165838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 0x7f0ea8103950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800b0d0 tx=0x7f0e9800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.429+0000 7f0eafc25700 1 -- 172.21.15.19:0/3277165838 shutdown_connections 2024-05-09T01:23:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.429+0000 7f0eafc25700 1 --2- 172.21.15.19:0/3277165838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ea8108770 0x7f0ea81100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.429+0000 7f0eafc25700 1 --2- 172.21.15.19:0/3277165838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 0x7f0ea8103950 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.429+0000 7f0eafc25700 1 --2- 172.21.15.19:0/3277165838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea8107e20 0x7f0ea8108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.429+0000 7f0eafc25700 1 -- 172.21.15.19:0/3277165838 >> 172.21.15.19:0/3277165838 conn(0x7f0ea8078e50 msgr2=0x7f0ea8079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.429+0000 7f0eafc25700 1 -- 172.21.15.19:0/3277165838 shutdown_connections 2024-05-09T01:23:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.430+0000 7f0eafc25700 1 -- 172.21.15.19:0/3277165838 wait complete. 2024-05-09T01:23:37.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0eafc25700 1 Processor -- start 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0eafc25700 1 -- start start 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0eafc25700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 0x7f0ea819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0eafc25700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea8107e20 0x7f0ea819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0eafc25700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ea8108770 0x7f0ea8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0eafc25700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81133a0 con 0x7f0ea8103540 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0eafc25700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8113220 con 0x7f0ea8108770 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0eafc25700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8113520 con 0x7f0ea8107e20 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0ead9c1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 0x7f0ea819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0ead9c1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 0x7f0ea819f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45010/0 (socket says 172.21.15.19:45010) 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0ead9c1700 1 -- 172.21.15.19:0/4267612707 learned_addr learned my addr 172.21.15.19:0/4267612707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0ead1c0700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea8107e20 0x7f0ea819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.431+0000 7f0eae1c2700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ea8108770 0x7f0ea8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0ead9c1700 1 -- 172.21.15.19:0/4267612707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea8107e20 msgr2=0x7f0ea819fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0ead9c1700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea8107e20 0x7f0ea819fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0ead9c1700 1 -- 172.21.15.19:0/4267612707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ea8108770 msgr2=0x7f0ea8199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0ead9c1700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ea8108770 0x7f0ea8199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0ead9c1700 1 -- 172.21.15.19:0/4267612707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea819a270 con 0x7f0ea8103540 2024-05-09T01:23:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0ead9c1700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 0x7f0ea819f9b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800d010 tx=0x7f0e98007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0ea6ffd700 1 -- 172.21.15.19:0/4267612707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9801f070 con 0x7f0ea8103540 2024-05-09T01:23:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea819a500 con 0x7f0ea8103540 2024-05-09T01:23:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0ea6ffd700 1 -- 172.21.15.19:0/4267612707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e98007e50 con 0x7f0ea8103540 2024-05-09T01:23:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0ea6ffd700 1 -- 172.21.15.19:0/4267612707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9801b4a0 con 0x7f0ea8103540 2024-05-09T01:23:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.432+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea81a6760 con 0x7f0ea8103540 2024-05-09T01:23:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.433+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea81049a0 con 0x7f0ea8103540 2024-05-09T01:23:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.435+0000 7f0ea6ffd700 1 -- 172.21.15.19:0/4267612707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0e98022070 con 0x7f0ea8103540 2024-05-09T01:23:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.435+0000 7f0ea6ffd700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e900b2910 0x7f0e900b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.436+0000 7f0ead1c0700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e900b2910 0x7f0e900b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.436+0000 7f0ea6ffd700 1 -- 172.21.15.19:0/4267612707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0e9800f020 con 0x7f0ea8103540 2024-05-09T01:23:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.436+0000 7f0ead1c0700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e900b2910 0x7f0e900b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ea819b270 tx=0x7f0e9c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.439+0000 7f0ea6ffd700 1 -- 172.21.15.19:0/4267612707 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e98029020 con 0x7f0ea8103540 2024-05-09T01:23:37.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.670+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ea819afe0 con 0x7f0ea8103540 2024-05-09T01:23:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.672+0000 7f0ea6ffd700 1 -- 172.21.15.19:0/4267612707 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22716 (secure 0 0 0) 0x7f0e980ab570 con 0x7f0ea8103540 2024-05-09T01:23:37.670 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:23:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:23:22.298486+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.674+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e900b2910 msgr2=0x7f0e900b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.674+0000 7f0eafc25700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e900b2910 0x7f0e900b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ea819b270 tx=0x7f0e9c0073d0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.674+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 msgr2=0x7f0ea819f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.674+0000 7f0eafc25700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 0x7f0ea819f9b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800d010 tx=0x7f0e98007a20 comp rx=0 tx=0).stop 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.675+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 shutdown_connections 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.675+0000 7f0eafc25700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ea8108770 0x7f0ea8199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.675+0000 7f0eafc25700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e900b2910 0x7f0e900b4e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.675+0000 7f0eafc25700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8103540 0x7f0ea819f9b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.675+0000 7f0eafc25700 1 --2- 172.21.15.19:0/4267612707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ea8107e20 0x7f0ea819fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.675+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 >> 172.21.15.19:0/4267612707 conn(0x7f0ea8078e50 msgr2=0x7f0ea81019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.675+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 shutdown_connections 2024-05-09T01:23:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:37.675+0000 7f0eafc25700 1 -- 172.21.15.19:0/4267612707 wait complete. 2024-05-09T01:23:38.458 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:38 smithi019 ceph-mon[30158]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 10 op/s; 65 B/s, 1 objects/s recovering 2024-05-09T01:23:38.459 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.459 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.459 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.459 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:38 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.459 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:38 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/4267612707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:23:38.488 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T01:23:38.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:23:38.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:38 smithi196 ceph-mon[35041]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 10 op/s; 65 B/s, 1 objects/s recovering 2024-05-09T01:23:38.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:38 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:38 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/4267612707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:23:38.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:38 smithi059 ceph-mon[34351]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 10 op/s; 65 B/s, 1 objects/s recovering 2024-05-09T01:23:38.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:38 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:38.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:38 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/4267612707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:23:39.074 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:40.472 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:40 smithi019 ceph-mon[30158]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:23:40.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:40 smithi196 ceph-mon[35041]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:23:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.707+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3638235320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20107e10 msgr2=0x7f9d20108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.707+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3638235320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20107e10 0x7f9d20108220 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d080099c0 tx=0x7f9d08009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.707+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3638235320 shutdown_connections 2024-05-09T01:23:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.707+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3638235320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20108760 0x7f9d201100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.707+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3638235320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d201034e0 0x7f9d201038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.707+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3638235320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20107e10 0x7f9d20108220 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.707+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3638235320 >> 172.21.15.19:0/3638235320 conn(0x7f9d200fef30 msgr2=0x7f9d20101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.707+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3638235320 shutdown_connections 2024-05-09T01:23:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.707+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3638235320 wait complete. 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.708+0000 7f9d25a4a700 1 Processor -- start 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.708+0000 7f9d25a4a700 1 -- start start 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d25a4a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d201034e0 0x7f9d2019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d25a4a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20107e10 0x7f9d2019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d25a4a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20108760 0x7f9d2019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d25a4a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d201133e0 con 0x7f9d20108760 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d25a4a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20113260 con 0x7f9d20107e10 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d25a4a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20113560 con 0x7f9d201034e0 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20108760 0x7f9d2019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20108760 0x7f9d2019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42602/0 (socket says 172.21.15.19:42602) 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1f7fe700 1 -- 172.21.15.19:0/3985450454 learned_addr learned my addr 172.21.15.19:0/3985450454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1effd700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d201034e0 0x7f9d2019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d167fc700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20107e10 0x7f9d2019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1f7fe700 1 -- 172.21.15.19:0/3985450454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d201034e0 msgr2=0x7f9d2019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1f7fe700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d201034e0 0x7f9d2019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1f7fe700 1 -- 172.21.15.19:0/3985450454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20107e10 msgr2=0x7f9d2019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1f7fe700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20107e10 0x7f9d2019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1f7fe700 1 -- 172.21.15.19:0/3985450454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2019fd30 con 0x7f9d20108760 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.709+0000 7f9d1f7fe700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20108760 0x7f9d2019f690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800ef30 tx=0x7f9d1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.710+0000 7f9d24a48700 1 -- 172.21.15.19:0/3985450454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d180092e0 con 0x7f9d20108760 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.710+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2019ffc0 con 0x7f9d20108760 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.710+0000 7f9d24a48700 1 -- 172.21.15.19:0/3985450454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d18004bb0 con 0x7f9d20108760 2024-05-09T01:23:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.710+0000 7f9d24a48700 1 -- 172.21.15.19:0/3985450454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18004020 con 0x7f9d20108760 2024-05-09T01:23:40.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.710+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d201a04d0 con 0x7f9d20108760 2024-05-09T01:23:40.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.712+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d20104990 con 0x7f9d20108760 2024-05-09T01:23:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.713+0000 7f9d24a48700 1 -- 172.21.15.19:0/3985450454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9d1801b030 con 0x7f9d20108760 2024-05-09T01:23:40.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.714+0000 7f9d24a48700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9d0c0b2910 0x7f9d0c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:40.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.714+0000 7f9d1effd700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9d0c0b2910 0x7f9d0c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:40.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.714+0000 7f9d24a48700 1 -- 172.21.15.19:0/3985450454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9d180e1e10 con 0x7f9d20108760 2024-05-09T01:23:40.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.715+0000 7f9d1effd700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9d0c0b2910 0x7f9d0c0b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009c60 tx=0x7f9d10009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:40.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.721+0000 7f9d24a48700 1 -- 172.21.15.19:0/3985450454 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d18022020 con 0x7f9d20108760 2024-05-09T01:23:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:40 smithi059 ceph-mon[34351]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:23:40.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.972+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d2019c1c0 con 0x7f9d20108760 2024-05-09T01:23:40.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.974+0000 7f9d24a48700 1 -- 172.21.15.19:0/3985450454 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22716 (secure 0 0 0) 0x7f9d18010040 con 0x7f9d20108760 2024-05-09T01:23:40.973 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:23:40.973 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:23:22.298486+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:23:40.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9d0c0b2910 msgr2=0x7f9d0c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:40.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9d0c0b2910 0x7f9d0c0b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009c60 tx=0x7f9d10009260 comp rx=0 tx=0).stop 2024-05-09T01:23:40.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20108760 msgr2=0x7f9d2019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:40.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20108760 0x7f9d2019f690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800ef30 tx=0x7f9d1800c3a0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 shutdown_connections 2024-05-09T01:23:40.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9d0c0b2910 0x7f9d0c0b4e20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d20108760 0x7f9d2019f690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d201034e0 0x7f9d2019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 --2- 172.21.15.19:0/3985450454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20107e10 0x7f9d2019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:40.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 >> 172.21.15.19:0/3985450454 conn(0x7f9d200fef30 msgr2=0x7f9d2006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:40.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 shutdown_connections 2024-05-09T01:23:40.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:40.977+0000 7f9d25a4a700 1 -- 172.21.15.19:0/3985450454 wait complete. 2024-05-09T01:23:41.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T01:23:41.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T01:23:41.725 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T01:23:41.725 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T01:23:41.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T01:23:41.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T01:23:41.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T01:23:41.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T01:23:41.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T01:23:41.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T01:23:41.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T01:23:41.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T01:23:42.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:41 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:42.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:41 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:42.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:41 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3985450454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:23:42.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:41 smithi059 ceph-mon[34351]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:23:42.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:41 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:42.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:41 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:42.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:41 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3985450454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:23:42.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:41 smithi019 ceph-mon[30158]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:23:42.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:41 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:42.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:41 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:42.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:41 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3985450454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:23:42.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:41 smithi196 ceph-mon[35041]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:23:42.516 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:42.520 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:42.723 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:42.723 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:42.724 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:42.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:23:42.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:23:42.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:42 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:43.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:23:43.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:23:43.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:42 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:43.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:23:43.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:23:43.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:42 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:23:44.103 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:43 smithi019 ceph-mon[30158]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:23:44.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:43 smithi196 ceph-mon[35041]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:23:44.347 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:43 smithi059 ceph-mon[34351]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:23:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:46.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:46 smithi019 ceph-mon[30158]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:23:46.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:46 smithi196 ceph-mon[35041]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:23:46.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:46 smithi059 ceph-mon[34351]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:23:48.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:48 smithi196 ceph-mon[35041]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:23:48.479 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:48 smithi019 ceph-mon[30158]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:23:48.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:48 smithi059 ceph-mon[34351]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:23:50.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:50 smithi059 ceph-mon[34351]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:23:50.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:50 smithi019 ceph-mon[30158]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:23:50.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:50 smithi196 ceph-mon[35041]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:23:52.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:52 smithi019 ceph-mon[30158]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:23:52.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.538+0000 7f7489fd3700 1 -- 172.21.15.19:0/3837940713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74841034c0 msgr2=0x7f74841038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:52.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.538+0000 7f7489fd3700 1 --2- 172.21.15.19:0/3837940713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74841034c0 0x7f74841038d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f747400b0d0 tx=0x7f747400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:23:52.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.538+0000 7f7489fd3700 1 -- 172.21.15.19:0/3837940713 shutdown_connections 2024-05-09T01:23:52.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.538+0000 7f7489fd3700 1 --2- 172.21.15.19:0/3837940713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7484108740 0x7f74841100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:52.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.538+0000 7f7489fd3700 1 --2- 172.21.15.19:0/3837940713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74841034c0 0x7f74841038d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:52.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.538+0000 7f7489fd3700 1 --2- 172.21.15.19:0/3837940713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484107df0 0x7f7484108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:52.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.538+0000 7f7489fd3700 1 -- 172.21.15.19:0/3837940713 >> 172.21.15.19:0/3837940713 conn(0x7f74840fedf0 msgr2=0x7f7484101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:52.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.538+0000 7f7489fd3700 1 -- 172.21.15.19:0/3837940713 shutdown_connections 2024-05-09T01:23:52.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.538+0000 7f7489fd3700 1 -- 172.21.15.19:0/3837940713 wait complete. 2024-05-09T01:23:52.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.539+0000 7f7489fd3700 1 Processor -- start 2024-05-09T01:23:52.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.539+0000 7f7489fd3700 1 -- start start 2024-05-09T01:23:52.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7489fd3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74841034c0 0x7f748419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:52.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7489fd3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7484107df0 0x7f748419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:52.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7489fd3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484108740 0x7f748419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:52.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7489fd3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7484113440 con 0x7f7484108740 2024-05-09T01:23:52.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7489fd3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74841132c0 con 0x7f7484107df0 2024-05-09T01:23:52.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7489fd3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74841135c0 con 0x7f74841034c0 2024-05-09T01:23:52.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7483fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484108740 0x7f748419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7483fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484108740 0x7f748419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42464/0 (socket says 172.21.15.19:42464) 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7483fff700 1 -- 172.21.15.19:0/1455394160 learned_addr learned my addr 172.21.15.19:0/1455394160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f74837fe700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74841034c0 0x7f748419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7482ffd700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7484107df0 0x7f748419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7483fff700 1 -- 172.21.15.19:0/1455394160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74841034c0 msgr2=0x7f748419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7483fff700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74841034c0 0x7f748419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7483fff700 1 -- 172.21.15.19:0/1455394160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7484107df0 msgr2=0x7f748419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7483fff700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7484107df0 0x7f748419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f7483fff700 1 -- 172.21.15.19:0/1455394160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748419fcc0 con 0x7f7484108740 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.540+0000 7f74837fe700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74841034c0 0x7f748419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.541+0000 7f7483fff700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484108740 0x7f748419f6d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f747c00b550 tx=0x7f747c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.541+0000 7f7480ff9700 1 -- 172.21.15.19:0/1455394160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c015020 con 0x7f7484108740 2024-05-09T01:23:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.541+0000 7f7480ff9700 1 -- 172.21.15.19:0/1455394160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f747c011ce0 con 0x7f7484108740 2024-05-09T01:23:52.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.541+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748419ffb0 con 0x7f7484108740 2024-05-09T01:23:52.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.541+0000 7f7480ff9700 1 -- 172.21.15.19:0/1455394160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c010d40 con 0x7f7484108740 2024-05-09T01:23:52.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.541+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74841a04a0 con 0x7f7484108740 2024-05-09T01:23:52.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.542+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7484063f10 con 0x7f7484108740 2024-05-09T01:23:52.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.543+0000 7f7480ff9700 1 -- 172.21.15.19:0/1455394160 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f747c018020 con 0x7f7484108740 2024-05-09T01:23:52.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.544+0000 7f7480ff9700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2910 0x7f746c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:52.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.544+0000 7f74837fe700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2910 0x7f746c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.544+0000 7f7480ff9700 1 -- 172.21.15.19:0/1455394160 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f747c0e2120 con 0x7f7484108740 2024-05-09T01:23:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.545+0000 7f7480ff9700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] conn(0x7f746c0bc5a0 0x7f746c0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.545+0000 7f7480ff9700 1 -- 172.21.15.19:0/1455394160 --> [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f746c0bf160 con 0x7f746c0bc5a0 2024-05-09T01:23:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.545+0000 7f7480ff9700 1 -- 172.21.15.19:0/1455394160 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f747c0e2570 con 0x7f7484108740 2024-05-09T01:23:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.545+0000 7f7482ffd700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] conn(0x7f746c0bc5a0 0x7f746c0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.545+0000 7f74837fe700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2910 0x7f746c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f747400d010 tx=0x7f747400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.545+0000 7f7482ffd700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] conn(0x7f746c0bc5a0 0x7f746c0bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:52.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.546+0000 7f7480ff9700 1 -- 172.21.15.19:0/1455394160 <== osd.9 v2:172.21.15.19:6826/3240481605 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f746c0bf160 con 0x7f746c0bc5a0 2024-05-09T01:23:52.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.562+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 --> [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7484064310 con 0x7f746c0bc5a0 2024-05-09T01:23:52.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.562+0000 7f7480ff9700 1 -- 172.21.15.19:0/1455394160 <== osd.9 v2:172.21.15.19:6826/3240481605 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7484064310 con 0x7f746c0bc5a0 2024-05-09T01:23:52.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] conn(0x7f746c0bc5a0 msgr2=0x7f746c0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:52.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] conn(0x7f746c0bc5a0 0x7f746c0bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:52.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2910 msgr2=0x7f746c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:52.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2910 0x7f746c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f747400d010 tx=0x7f747400b040 comp rx=0 tx=0).stop 2024-05-09T01:23:52.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484108740 msgr2=0x7f748419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:52.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 --2- 172.21.15.19:0/1455394160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484108740 0x7f748419f6d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f747c00b550 tx=0x7f747c00b910 comp rx=0 tx=0).stop 2024-05-09T01:23:52.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7483fff700 1 -- 172.21.15.19:0/1455394160 reap_dead start 2024-05-09T01:23:52.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 shutdown_connections 2024-05-09T01:23:52.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 >> 172.21.15.19:0/1455394160 conn(0x7f74840fedf0 msgr2=0x7f7484075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:52.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 shutdown_connections 2024-05-09T01:23:52.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:52.563+0000 7f7489fd3700 1 -- 172.21.15.19:0/1455394160 wait complete. 2024-05-09T01:23:52.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:52 smithi059 ceph-mon[34351]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:23:52.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:52 smithi196 ceph-mon[35041]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:23:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.424+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/3151757930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e24102b30 msgr2=0x7f0e24102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.424+0000 7f0e2b8e4700 1 --2- 172.21.15.19:0/3151757930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e24102b30 0x7f0e24102f40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0e200099c0 tx=0x7f0e20009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.425+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/3151757930 shutdown_connections 2024-05-09T01:23:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.425+0000 7f0e2b8e4700 1 --2- 172.21.15.19:0/3151757930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e241098a0 0x7f0e24111240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.425+0000 7f0e2b8e4700 1 --2- 172.21.15.19:0/3151757930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e24102b30 0x7f0e24102f40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.425+0000 7f0e2b8e4700 1 --2- 172.21.15.19:0/3151757930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e24108f50 0x7f0e24109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.425+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/3151757930 >> 172.21.15.19:0/3151757930 conn(0x7f0e240fe6f0 msgr2=0x7f0e24100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.425+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/3151757930 shutdown_connections 2024-05-09T01:23:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.425+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/3151757930 wait complete. 2024-05-09T01:23:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.425+0000 7f0e2b8e4700 1 Processor -- start 2024-05-09T01:23:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e2b8e4700 1 -- start start 2024-05-09T01:23:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e2b8e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e24102b30 0x7f0e2419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e2b8e4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e24108f50 0x7f0e2419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e2b8e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e241098a0 0x7f0e2419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e2b8e4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e24114530 con 0x7f0e24102b30 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e2b8e4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e241143b0 con 0x7f0e241098a0 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e2b8e4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e241146b0 con 0x7f0e24108f50 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29680700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e24102b30 0x7f0e2419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29680700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e24102b30 0x7f0e2419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42484/0 (socket says 172.21.15.19:42484) 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29680700 1 -- 172.21.15.19:0/2934511621 learned_addr learned my addr 172.21.15.19:0/2934511621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e28e7f700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e24108f50 0x7f0e2419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29e81700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e241098a0 0x7f0e2419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29680700 1 -- 172.21.15.19:0/2934511621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e24108f50 msgr2=0x7f0e2419b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29680700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e24108f50 0x7f0e2419b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29680700 1 -- 172.21.15.19:0/2934511621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e241098a0 msgr2=0x7f0e2419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29680700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e241098a0 0x7f0e2419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29680700 1 -- 172.21.15.19:0/2934511621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2419fe10 con 0x7f0e24102b30 2024-05-09T01:23:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.426+0000 7f0e29e81700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e241098a0 0x7f0e2419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:23:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.427+0000 7f0e29680700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e24102b30 0x7f0e2419af40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000b580 tx=0x7f0e200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.427+0000 7f0e1a7fc700 1 -- 172.21.15.19:0/2934511621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20027070 con 0x7f0e24102b30 2024-05-09T01:23:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.427+0000 7f0e1a7fc700 1 -- 172.21.15.19:0/2934511621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e20003f20 con 0x7f0e24102b30 2024-05-09T01:23:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.427+0000 7f0e1a7fc700 1 -- 172.21.15.19:0/2934511621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20022d70 con 0x7f0e24102b30 2024-05-09T01:23:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.427+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e241a00a0 con 0x7f0e24102b30 2024-05-09T01:23:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.427+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e241a0470 con 0x7f0e24102b30 2024-05-09T01:23:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.428+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e24063ed0 con 0x7f0e24102b30 2024-05-09T01:23:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.430+0000 7f0e1a7fc700 1 -- 172.21.15.19:0/2934511621 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0e20022500 con 0x7f0e24102b30 2024-05-09T01:23:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.431+0000 7f0e1a7fc700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e100b2a60 0x7f0e100b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.431+0000 7f0e28e7f700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e100b2a60 0x7f0e100b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.431+0000 7f0e1a7fc700 1 -- 172.21.15.19:0/2934511621 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0e200e3590 con 0x7f0e24102b30 2024-05-09T01:23:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.431+0000 7f0e1a7fc700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] conn(0x7f0e100bc6f0 0x7f0e100bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.431+0000 7f0e1a7fc700 1 -- 172.21.15.19:0/2934511621 --> [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e200043d0 con 0x7f0e100bc6f0 2024-05-09T01:23:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.431+0000 7f0e1a7fc700 1 -- 172.21.15.19:0/2934511621 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f0e200e3940 con 0x7f0e24102b30 2024-05-09T01:23:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.432+0000 7f0e28e7f700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e100b2a60 0x7f0e100b4f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e24107890 tx=0x7f0e1400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.432+0000 7f0e29e81700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] conn(0x7f0e100bc6f0 0x7f0e100bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.432+0000 7f0e29e81700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] conn(0x7f0e100bc6f0 0x7f0e100bebe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.433+0000 7f0e1a7fc700 1 -- 172.21.15.19:0/2934511621 <== osd.10 v2:172.21.15.59:6824/1290773390 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e200043d0 con 0x7f0e100bc6f0 2024-05-09T01:23:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.450+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 --> [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e240642d0 con 0x7f0e100bc6f0 2024-05-09T01:23:53.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e1a7fc700 1 -- 172.21.15.19:0/2934511621 <== osd.10 v2:172.21.15.59:6824/1290773390 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0e240642d0 con 0x7f0e100bc6f0 2024-05-09T01:23:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 >> [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] conn(0x7f0e100bc6f0 msgr2=0x7f0e100bebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] conn(0x7f0e100bc6f0 0x7f0e100bebe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:53.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e100b2a60 msgr2=0x7f0e100b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e100b2a60 0x7f0e100b4f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e24107890 tx=0x7f0e1400b410 comp rx=0 tx=0).stop 2024-05-09T01:23:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e24102b30 msgr2=0x7f0e2419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 --2- 172.21.15.19:0/2934511621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e24102b30 0x7f0e2419af40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000b580 tx=0x7f0e200047b0 comp rx=0 tx=0).stop 2024-05-09T01:23:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e29e81700 1 -- 172.21.15.19:0/2934511621 reap_dead start 2024-05-09T01:23:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 shutdown_connections 2024-05-09T01:23:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 >> 172.21.15.19:0/2934511621 conn(0x7f0e240fe6f0 msgr2=0x7f0e2410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 shutdown_connections 2024-05-09T01:23:53.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:53.451+0000 7f0e2b8e4700 1 -- 172.21.15.19:0/2934511621 wait complete. 2024-05-09T01:23:54.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:54 smithi019 ceph-mon[30158]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-09T01:23:54.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:54 smithi059 ceph-mon[34351]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-09T01:23:54.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:54 smithi196 ceph-mon[35041]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-09T01:23:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.972+0000 7fe55436d700 1 -- 172.21.15.19:0/108527179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe54c107df0 msgr2=0x7fe54c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.972+0000 7fe55436d700 1 --2- 172.21.15.19:0/108527179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe54c107df0 0x7fe54c108200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0099c0 tx=0x7fe53c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.972+0000 7fe55436d700 1 -- 172.21.15.19:0/108527179 shutdown_connections 2024-05-09T01:23:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.972+0000 7fe55436d700 1 --2- 172.21.15.19:0/108527179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe54c108740 0x7fe54c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.972+0000 7fe55436d700 1 --2- 172.21.15.19:0/108527179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c1034c0 0x7fe54c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.972+0000 7fe55436d700 1 --2- 172.21.15.19:0/108527179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe54c107df0 0x7fe54c108200 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.972+0000 7fe55436d700 1 -- 172.21.15.19:0/108527179 >> 172.21.15.19:0/108527179 conn(0x7fe54c0fedf0 msgr2=0x7fe54c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.973+0000 7fe55436d700 1 -- 172.21.15.19:0/108527179 shutdown_connections 2024-05-09T01:23:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.973+0000 7fe55436d700 1 -- 172.21.15.19:0/108527179 wait complete. 2024-05-09T01:23:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55436d700 1 Processor -- start 2024-05-09T01:23:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55436d700 1 -- start start 2024-05-09T01:23:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55436d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c1034c0 0x7fe54c19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55436d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe54c107df0 0x7fe54c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55436d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe54c108740 0x7fe54c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55436d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1133a0 con 0x7fe54c108740 2024-05-09T01:23:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55436d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c113220 con 0x7fe54c107df0 2024-05-09T01:23:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55436d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c113520 con 0x7fe54c1034c0 2024-05-09T01:23:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55290a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe54c108740 0x7fe54c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe552109700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c1034c0 0x7fe54c19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55290a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe54c108740 0x7fe54c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42542/0 (socket says 172.21.15.19:42542) 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe552109700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c1034c0 0x7fe54c19aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:43082/0 (socket says 172.21.15.19:43082) 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe55290a700 1 -- 172.21.15.19:0/498954640 learned_addr learned my addr 172.21.15.19:0/498954640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.974+0000 7fe551908700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe54c107df0 0x7fe54c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.975+0000 7fe55290a700 1 -- 172.21.15.19:0/498954640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c1034c0 msgr2=0x7fe54c19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.975+0000 7fe55290a700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c1034c0 0x7fe54c19aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.975+0000 7fe55290a700 1 -- 172.21.15.19:0/498954640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe54c107df0 msgr2=0x7fe54c19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.975+0000 7fe55290a700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe54c107df0 0x7fe54c19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.975+0000 7fe55290a700 1 -- 172.21.15.19:0/498954640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c19fd60 con 0x7fe54c108740 2024-05-09T01:23:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.975+0000 7fe55290a700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe54c108740 0x7fe54c19f6c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe54400c970 tx=0x7fe54400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.976+0000 7fe5437fe700 1 -- 172.21.15.19:0/498954640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54400ce60 con 0x7fe54c108740 2024-05-09T01:23:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.976+0000 7fe5437fe700 1 -- 172.21.15.19:0/498954640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe544004830 con 0x7fe54c108740 2024-05-09T01:23:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.976+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c1a0050 con 0x7fe54c108740 2024-05-09T01:23:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.976+0000 7fe5437fe700 1 -- 172.21.15.19:0/498954640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5440054c0 con 0x7fe54c108740 2024-05-09T01:23:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.976+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1a7970 con 0x7fe54c108740 2024-05-09T01:23:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.977+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe54c1016a0 con 0x7fe54c108740 2024-05-09T01:23:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.979+0000 7fe5437fe700 1 -- 172.21.15.19:0/498954640 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe5440049e0 con 0x7fe54c108740 2024-05-09T01:23:54.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.980+0000 7fe5437fe700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5380b2990 0x7fe5380b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:54.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.980+0000 7fe552109700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5380b2990 0x7fe5380b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:54.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.980+0000 7fe5437fe700 1 -- 172.21.15.19:0/498954640 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe5440e2250 con 0x7fe54c108740 2024-05-09T01:23:54.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.980+0000 7fe5437fe700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] conn(0x7fe5380bc680 0x7fe5380beb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.980+0000 7fe5437fe700 1 -- 172.21.15.19:0/498954640 --> [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5380bf240 con 0x7fe5380bc680 2024-05-09T01:23:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.980+0000 7fe5437fe700 1 -- 172.21.15.19:0/498954640 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fe5440e2600 con 0x7fe54c108740 2024-05-09T01:23:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.980+0000 7fe551908700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] conn(0x7fe5380bc680 0x7fe5380beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.980+0000 7fe552109700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5380b2990 0x7fe5380b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe5480060b0 tx=0x7fe54800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.981+0000 7fe551908700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] conn(0x7fe5380bc680 0x7fe5380beb70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.981+0000 7fe5437fe700 1 -- 172.21.15.19:0/498954640 <== osd.3 v2:172.21.15.19:6810/713020110 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe5380bf240 con 0x7fe5380bc680 2024-05-09T01:23:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.998+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 --> [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe54c104970 con 0x7fe5380bc680 2024-05-09T01:23:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.998+0000 7fe5437fe700 1 -- 172.21.15.19:0/498954640 <== osd.3 v2:172.21.15.19:6810/713020110 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe54c104970 con 0x7fe5380bc680 2024-05-09T01:23:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] conn(0x7fe5380bc680 msgr2=0x7fe5380beb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] conn(0x7fe5380bc680 0x7fe5380beb70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5380b2990 msgr2=0x7fe5380b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5380b2990 0x7fe5380b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe5480060b0 tx=0x7fe54800c040 comp rx=0 tx=0).stop 2024-05-09T01:23:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe54c108740 msgr2=0x7fe54c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 --2- 172.21.15.19:0/498954640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe54c108740 0x7fe54c19f6c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe54400c970 tx=0x7fe54400cc80 comp rx=0 tx=0).stop 2024-05-09T01:23:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55290a700 1 -- 172.21.15.19:0/498954640 reap_dead start 2024-05-09T01:23:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 shutdown_connections 2024-05-09T01:23:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 >> 172.21.15.19:0/498954640 conn(0x7fe54c0fedf0 msgr2=0x7fe54c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 shutdown_connections 2024-05-09T01:23:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.999+0000 7fe55436d700 1 -- 172.21.15.19:0/498954640 wait complete. 2024-05-09T01:23:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.081+0000 7fe17c794700 1 -- 172.21.15.19:0/913268554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1741034c0 msgr2=0x7fe1741038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.081+0000 7fe17c794700 1 --2- 172.21.15.19:0/913268554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1741034c0 0x7fe1741038d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe17000b0d0 tx=0x7fe17000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:23:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.081+0000 7fe17c794700 1 -- 172.21.15.19:0/913268554 shutdown_connections 2024-05-09T01:23:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.081+0000 7fe17c794700 1 --2- 172.21.15.19:0/913268554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe174108740 0x7fe1741100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.081+0000 7fe17c794700 1 --2- 172.21.15.19:0/913268554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1741034c0 0x7fe1741038d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.081+0000 7fe17c794700 1 --2- 172.21.15.19:0/913268554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe174107df0 0x7fe174108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.081+0000 7fe17c794700 1 -- 172.21.15.19:0/913268554 >> 172.21.15.19:0/913268554 conn(0x7fe1740fedf0 msgr2=0x7fe174101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 -- 172.21.15.19:0/913268554 shutdown_connections 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 -- 172.21.15.19:0/913268554 wait complete. 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 Processor -- start 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 -- start start 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1741034c0 0x7fe17419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174107df0 0x7fe17419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe174108740 0x7fe17419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741133f0 con 0x7fe174107df0 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe174113270 con 0x7fe174108740 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.082+0000 7fe17c794700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe174113570 con 0x7fe1741034c0 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe179d2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174107df0 0x7fe17419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe179d2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174107df0 0x7fe17419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42494/0 (socket says 172.21.15.19:42494) 2024-05-09T01:23:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe179d2f700 1 -- 172.21.15.19:0/944267630 learned_addr learned my addr 172.21.15.19:0/944267630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe17a530700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1741034c0 0x7fe17419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe17ad31700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe174108740 0x7fe17419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe179d2f700 1 -- 172.21.15.19:0/944267630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1741034c0 msgr2=0x7fe17419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe179d2f700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1741034c0 0x7fe17419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe179d2f700 1 -- 172.21.15.19:0/944267630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe174108740 msgr2=0x7fe17419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe179d2f700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe174108740 0x7fe17419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe179d2f700 1 -- 172.21.15.19:0/944267630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17419fcb0 con 0x7fe174107df0 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe17ad31700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe174108740 0x7fe17419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe179d2f700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174107df0 0x7fe17419b430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe16400c9e0 tx=0x7fe16400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe16b7fe700 1 -- 172.21.15.19:0/944267630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164007aa0 con 0x7fe174107df0 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe16b7fe700 1 -- 172.21.15.19:0/944267630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe164007c40 con 0x7fe174107df0 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17419ffa0 con 0x7fe174107df0 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe16b7fe700 1 -- 172.21.15.19:0/944267630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164018660 con 0x7fe174107df0 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.083+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1741a04e0 con 0x7fe174107df0 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.084+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe174063f50 con 0x7fe174107df0 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.086+0000 7fe16b7fe700 1 -- 172.21.15.19:0/944267630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe164018800 con 0x7fe174107df0 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.087+0000 7fe16b7fe700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe1600b2910 0x7fe1600b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.087+0000 7fe17a530700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe1600b2910 0x7fe1600b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.087+0000 7fe16b7fe700 1 -- 172.21.15.19:0/944267630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe1640e2800 con 0x7fe174107df0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.087+0000 7fe16b7fe700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] conn(0x7fe1600bc5a0 0x7fe1600bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.087+0000 7fe16b7fe700 1 -- 172.21.15.19:0/944267630 --> [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe16401fe50 con 0x7fe1600bc5a0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.087+0000 7fe16b7fe700 1 -- 172.21.15.19:0/944267630 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fe1640e2c50 con 0x7fe174107df0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.087+0000 7fe17ad31700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] conn(0x7fe1600bc5a0 0x7fe1600bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.087+0000 7fe17a530700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe1600b2910 0x7fe1600b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe170009900 tx=0x7fe17000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.088+0000 7fe17ad31700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] conn(0x7fe1600bc5a0 0x7fe1600bea90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.088+0000 7fe16b7fe700 1 -- 172.21.15.19:0/944267630 <== osd.6 v2:172.21.15.19:6818/3027498508 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe16401fe50 con 0x7fe1600bc5a0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.105+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 --> [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe1740643d0 con 0x7fe1600bc5a0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.105+0000 7fe16b7fe700 1 -- 172.21.15.19:0/944267630 <== osd.6 v2:172.21.15.19:6818/3027498508 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe1740643d0 con 0x7fe1600bc5a0 2024-05-09T01:23:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] conn(0x7fe1600bc5a0 msgr2=0x7fe1600bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] conn(0x7fe1600bc5a0 0x7fe1600bea90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe1600b2910 msgr2=0x7fe1600b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe1600b2910 0x7fe1600b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe170009900 tx=0x7fe17000b040 comp rx=0 tx=0).stop 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174107df0 msgr2=0x7fe17419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 --2- 172.21.15.19:0/944267630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174107df0 0x7fe17419b430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe16400c9e0 tx=0x7fe16400ccf0 comp rx=0 tx=0).stop 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17ad31700 1 -- 172.21.15.19:0/944267630 reap_dead start 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 shutdown_connections 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 >> 172.21.15.19:0/944267630 conn(0x7fe1740fedf0 msgr2=0x7fe174075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 shutdown_connections 2024-05-09T01:23:55.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.106+0000 7fe17c794700 1 -- 172.21.15.19:0/944267630 wait complete. 2024-05-09T01:23:56.107 INFO:teuthology.orchestra.run.smithi019.stdout:193273528377 2024-05-09T01:23:56.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T01:23:56.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:56 smithi019 ceph-mon[30158]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-09T01:23:56.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.862+0000 7f6383b41700 1 -- 172.21.15.19:0/567679499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f637c109860 msgr2=0x7f637c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:56.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.862+0000 7f6383b41700 1 --2- 172.21.15.19:0/567679499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f637c109860 0x7f637c1111c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f637400b0d0 tx=0x7f637400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.862+0000 7f6383b41700 1 -- 172.21.15.19:0/567679499 shutdown_connections 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.862+0000 7f6383b41700 1 --2- 172.21.15.19:0/567679499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f637c109860 0x7f637c1111c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.862+0000 7f6383b41700 1 --2- 172.21.15.19:0/567679499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f637c1045e0 0x7f637c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.862+0000 7f6383b41700 1 --2- 172.21.15.19:0/567679499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f637c108f10 0x7f637c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.862+0000 7f6383b41700 1 -- 172.21.15.19:0/567679499 >> 172.21.15.19:0/567679499 conn(0x7f637c0ffe70 msgr2=0x7f637c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.862+0000 7f6383b41700 1 -- 172.21.15.19:0/567679499 shutdown_connections 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.862+0000 7f6383b41700 1 -- 172.21.15.19:0/567679499 wait complete. 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.863+0000 7f6383b41700 1 Processor -- start 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.863+0000 7f6383b41700 1 -- start start 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f6383b41700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f637c1045e0 0x7f637c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f6383b41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f637c108f10 0x7f637c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f6383b41700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f637c109860 0x7f637c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f6383b41700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f637c114490 con 0x7f637c109860 2024-05-09T01:23:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f6383b41700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f637c114310 con 0x7f637c108f10 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f6383b41700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f637c114610 con 0x7f637c1045e0 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63820de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f637c109860 0x7f637c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63820de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f637c109860 0x7f637c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42516/0 (socket says 172.21.15.19:42516) 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63820de700 1 -- 172.21.15.19:0/3625776221 learned_addr learned my addr 172.21.15.19:0/3625776221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63818dd700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f637c1045e0 0x7f637c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63810dc700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f637c108f10 0x7f637c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63820de700 1 -- 172.21.15.19:0/3625776221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f637c1045e0 msgr2=0x7f637c19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63820de700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f637c1045e0 0x7f637c19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63820de700 1 -- 172.21.15.19:0/3625776221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f637c108f10 msgr2=0x7f637c19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63820de700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f637c108f10 0x7f637c19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.864+0000 7f63820de700 1 -- 172.21.15.19:0/3625776221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c19fd10 con 0x7f637c109860 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.865+0000 7f63820de700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f637c109860 0x7f637c19f670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f63740095c0 tx=0x7f6374007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.865+0000 7f636effd700 1 -- 172.21.15.19:0/3625776221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637401f070 con 0x7f637c109860 2024-05-09T01:23:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.865+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637c19ffa0 con 0x7f637c109860 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.865+0000 7f636effd700 1 -- 172.21.15.19:0/3625776221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6374003c00 con 0x7f637c109860 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.865+0000 7f636effd700 1 -- 172.21.15.19:0/3625776221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637401b6a0 con 0x7f637c109860 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.865+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637c1a7920 con 0x7f637c109860 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.866+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f637c063ed0 con 0x7f637c109860 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.868+0000 7f636effd700 1 -- 172.21.15.19:0/3625776221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6374022070 con 0x7f637c109860 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.868+0000 7f636effd700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f63680b2910 0x7f63680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.868+0000 7f636effd700 1 -- 172.21.15.19:0/3625776221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f637400f020 con 0x7f637c109860 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.868+0000 7f63818dd700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f63680b2910 0x7f63680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.869+0000 7f636effd700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] conn(0x7f63680bc5a0 0x7f63680bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.869+0000 7f636effd700 1 -- 172.21.15.19:0/3625776221 --> [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f637401de80 con 0x7f63680bc5a0 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.869+0000 7f636effd700 1 -- 172.21.15.19:0/3625776221 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f63740e3cc0 con 0x7f637c109860 2024-05-09T01:23:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.869+0000 7f63818dd700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f63680b2910 0x7f63680b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6378005950 tx=0x7f637800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:56.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.869+0000 7f63810dc700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] conn(0x7f63680bc5a0 0x7f63680bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:56.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.869+0000 7f63810dc700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] conn(0x7f63680bc5a0 0x7f63680bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:56.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.870+0000 7f636effd700 1 -- 172.21.15.19:0/3625776221 <== osd.4 v2:172.21.15.59:6808/2419703664 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f637401de80 con 0x7f63680bc5a0 2024-05-09T01:23:56.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.887+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 --> [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f637c0642d0 con 0x7f63680bc5a0 2024-05-09T01:23:56.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f636effd700 1 -- 172.21.15.19:0/3625776221 <== osd.4 v2:172.21.15.59:6808/2419703664 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f637c0642d0 con 0x7f63680bc5a0 2024-05-09T01:23:56.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 >> [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] conn(0x7f63680bc5a0 msgr2=0x7f63680bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:56.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] conn(0x7f63680bc5a0 0x7f63680bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:56.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f63680b2910 msgr2=0x7f63680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:56.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f63680b2910 0x7f63680b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6378005950 tx=0x7f637800a300 comp rx=0 tx=0).stop 2024-05-09T01:23:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f637c109860 msgr2=0x7f637c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 --2- 172.21.15.19:0/3625776221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f637c109860 0x7f637c19f670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f63740095c0 tx=0x7f6374007a20 comp rx=0 tx=0).stop 2024-05-09T01:23:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f63820de700 1 -- 172.21.15.19:0/3625776221 reap_dead start 2024-05-09T01:23:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 shutdown_connections 2024-05-09T01:23:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 >> 172.21.15.19:0/3625776221 conn(0x7f637c0ffe70 msgr2=0x7f637c108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 shutdown_connections 2024-05-09T01:23:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:54.888+0000 7f6383b41700 1 -- 172.21.15.19:0/3625776221 wait complete. 2024-05-09T01:23:56.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:56 smithi059 ceph-mon[34351]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-09T01:23:56.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:56 smithi196 ceph-mon[35041]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-09T01:23:57.166 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:23:57.877 INFO:teuthology.orchestra.run.smithi019.stdout:266287972392 2024-05-09T01:23:57.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T01:23:58.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:23:58 smithi059 ceph-mon[34351]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:23:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:23:58 smithi019 ceph-mon[30158]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:23:58.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:23:58 smithi196 ceph-mon[35041]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:23:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.656+0000 7f656768e700 1 -- 172.21.15.19:0/4213982292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6560108740 msgr2=0x7f65601100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.656+0000 7f656768e700 1 --2- 172.21.15.19:0/4213982292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6560108740 0x7f65601100a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f65580099c0 tx=0x7f6558009cd0 comp rx=0 tx=0).stop 2024-05-09T01:23:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.656+0000 7f656768e700 1 -- 172.21.15.19:0/4213982292 shutdown_connections 2024-05-09T01:23:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.656+0000 7f656768e700 1 --2- 172.21.15.19:0/4213982292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6560108740 0x7f65601100a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.656+0000 7f656768e700 1 --2- 172.21.15.19:0/4213982292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65601034c0 0x7f65601038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.656+0000 7f656768e700 1 --2- 172.21.15.19:0/4213982292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6560107df0 0x7f6560108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.656+0000 7f656768e700 1 -- 172.21.15.19:0/4213982292 >> 172.21.15.19:0/4213982292 conn(0x7f65600fedc0 msgr2=0x7f65601011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.656+0000 7f656768e700 1 -- 172.21.15.19:0/4213982292 shutdown_connections 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.656+0000 7f656768e700 1 -- 172.21.15.19:0/4213982292 wait complete. 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.657+0000 7f656768e700 1 Processor -- start 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.657+0000 7f656768e700 1 -- start start 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.657+0000 7f656768e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601034c0 0x7f6560199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.657+0000 7f656768e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6560107df0 0x7f656019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.657+0000 7f656768e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6560108740 0x7f656019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.657+0000 7f656768e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65601133c0 con 0x7f6560108740 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.657+0000 7f656768e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6560113240 con 0x7f65601034c0 2024-05-09T01:23:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.657+0000 7f656768e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6560113540 con 0x7f6560107df0 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.657+0000 7f6565c2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6560108740 0x7f656019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6565c2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6560108740 0x7f656019e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55844/0 (socket says 172.21.15.19:55844) 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6565c2b700 1 -- 172.21.15.19:0/612942685 learned_addr learned my addr 172.21.15.19:0/612942685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f656542a700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601034c0 0x7f6560199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6564c29700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6560107df0 0x7f656019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6565c2b700 1 -- 172.21.15.19:0/612942685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6560107df0 msgr2=0x7f656019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6565c2b700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6560107df0 0x7f656019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6565c2b700 1 -- 172.21.15.19:0/612942685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601034c0 msgr2=0x7f6560199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6565c2b700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601034c0 0x7f6560199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6565c2b700 1 -- 172.21.15.19:0/612942685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656019ebe0 con 0x7f6560108740 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6564c29700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6560107df0 0x7f656019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:23:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f6565c2b700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6560108740 0x7f656019e540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f655800bff0 tx=0x7f65580036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f655e7fc700 1 -- 172.21.15.19:0/612942685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558004830 con 0x7f6560108740 2024-05-09T01:23:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656019ee70 con 0x7f6560108740 2024-05-09T01:23:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f655e7fc700 1 -- 172.21.15.19:0/612942685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6558003bb0 con 0x7f6560108740 2024-05-09T01:23:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f655e7fc700 1 -- 172.21.15.19:0/612942685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580053a0 con 0x7f6560108740 2024-05-09T01:23:59.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.658+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656019f2f0 con 0x7f6560108740 2024-05-09T01:23:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.659+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6560063f50 con 0x7f6560108740 2024-05-09T01:23:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.661+0000 7f655e7fc700 1 -- 172.21.15.19:0/612942685 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6558004060 con 0x7f6560108740 2024-05-09T01:23:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.662+0000 7f655e7fc700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65480b2910 0x7f65480b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.662+0000 7f656542a700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65480b2910 0x7f65480b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.662+0000 7f655e7fc700 1 -- 172.21.15.19:0/612942685 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f65580e3d80 con 0x7f6560108740 2024-05-09T01:23:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.662+0000 7f655e7fc700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] conn(0x7f65480bc5a0 0x7f65480bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:23:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.662+0000 7f655e7fc700 1 -- 172.21.15.19:0/612942685 --> [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f655802de50 con 0x7f65480bc5a0 2024-05-09T01:23:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.662+0000 7f655e7fc700 1 -- 172.21.15.19:0/612942685 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f65580e41d0 con 0x7f6560108740 2024-05-09T01:23:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.662+0000 7f656542a700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65480b2910 0x7f65480b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65500060b0 tx=0x7f655000c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.662+0000 7f6564c29700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] conn(0x7f65480bc5a0 0x7f65480bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:23:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.663+0000 7f6564c29700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] conn(0x7f65480bc5a0 0x7f65480bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:23:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.664+0000 7f655e7fc700 1 -- 172.21.15.19:0/612942685 <== osd.1 v2:172.21.15.59:6800/2880894912 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f655802de50 con 0x7f65480bc5a0 2024-05-09T01:23:59.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.684+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 --> [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65600643d0 con 0x7f65480bc5a0 2024-05-09T01:23:59.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.685+0000 7f655e7fc700 1 -- 172.21.15.19:0/612942685 <== osd.1 v2:172.21.15.59:6800/2880894912 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f65600643d0 con 0x7f65480bc5a0 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.685+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 >> [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] conn(0x7f65480bc5a0 msgr2=0x7f65480bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.685+0000 7f656768e700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] conn(0x7f65480bc5a0 0x7f65480bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.686+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65480b2910 msgr2=0x7f65480b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.686+0000 7f656768e700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65480b2910 0x7f65480b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65500060b0 tx=0x7f655000c040 comp rx=0 tx=0).stop 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.686+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6560108740 msgr2=0x7f656019e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.686+0000 7f656768e700 1 --2- 172.21.15.19:0/612942685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6560108740 0x7f656019e540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f655800bff0 tx=0x7f65580036f0 comp rx=0 tx=0).stop 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.686+0000 7f6565c2b700 1 -- 172.21.15.19:0/612942685 reap_dead start 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.686+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 shutdown_connections 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.686+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 >> 172.21.15.19:0/612942685 conn(0x7f65600fedc0 msgr2=0x7f6560107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.686+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 shutdown_connections 2024-05-09T01:23:59.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:23:59.686+0000 7f656768e700 1 -- 172.21.15.19:0/612942685 wait complete. 2024-05-09T01:24:00.273 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:00 smithi019 ceph-mon[30158]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:00.405 INFO:teuthology.orchestra.run.smithi019.stdout:287762808867 2024-05-09T01:24:00.406 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T01:24:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 -- 172.21.15.19:0/617272097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee1c076450 msgr2=0x7fee1c1106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 --2- 172.21.15.19:0/617272097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee1c076450 0x7fee1c1106b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee04009990 tx=0x7fee04009ca0 comp rx=0 tx=0).stop 2024-05-09T01:24:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 -- 172.21.15.19:0/617272097 shutdown_connections 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 --2- 172.21.15.19:0/617272097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee1c110bf0 0x7fee1c113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 --2- 172.21.15.19:0/617272097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee1c075b00 0x7fee1c075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 --2- 172.21.15.19:0/617272097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee1c076450 0x7fee1c1106b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 -- 172.21.15.19:0/617272097 >> 172.21.15.19:0/617272097 conn(0x7fee1c0ffe70 msgr2=0x7fee1c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 -- 172.21.15.19:0/617272097 shutdown_connections 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 -- 172.21.15.19:0/617272097 wait complete. 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.398+0000 7fee21c24700 1 Processor -- start 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee21c24700 1 -- start start 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee21c24700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee1c075b00 0x7fee1c19d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee21c24700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee1c076450 0x7fee1c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee21c24700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee1c110bf0 0x7fee1c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee21c24700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c116700 con 0x7fee1c110bf0 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee21c24700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c116580 con 0x7fee1c076450 2024-05-09T01:24:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee21c24700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c116880 con 0x7fee1c075b00 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee1bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee1c110bf0 0x7fee1c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee1bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee1c110bf0 0x7fee1c1a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55872/0 (socket says 172.21.15.19:55872) 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee1bfff700 1 -- 172.21.15.19:0/3106231055 learned_addr learned my addr 172.21.15.19:0/3106231055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee1b7fe700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee1c075b00 0x7fee1c19d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.399+0000 7fee1affd700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee1c076450 0x7fee1c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee1bfff700 1 -- 172.21.15.19:0/3106231055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee1c075b00 msgr2=0x7fee1c19d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee1bfff700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee1c075b00 0x7fee1c19d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee1bfff700 1 -- 172.21.15.19:0/3106231055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee1c076450 msgr2=0x7fee1c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee1bfff700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee1c076450 0x7fee1c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee1bfff700 1 -- 172.21.15.19:0/3106231055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c1a1fa0 con 0x7fee1c110bf0 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee1affd700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee1c076450 0x7fee1c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee1bfff700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee1c110bf0 0x7fee1c1a1900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee140094f0 tx=0x7fee1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee18ff9700 1 -- 172.21.15.19:0/3106231055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400ddd0 con 0x7fee1c110bf0 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee18ff9700 1 -- 172.21.15.19:0/3106231055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee14009e20 con 0x7fee1c110bf0 2024-05-09T01:24:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1c1a2290 con 0x7fee1c110bf0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee18ff9700 1 -- 172.21.15.19:0/3106231055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14015400 con 0x7fee1c110bf0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.400+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1c1a9bb0 con 0x7fee1c110bf0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.401+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee1c063f10 con 0x7fee1c110bf0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.403+0000 7fee18ff9700 1 -- 172.21.15.19:0/3106231055 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fee14004dd0 con 0x7fee1c110bf0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.404+0000 7fee18ff9700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee080b2910 0x7fee080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.404+0000 7fee1b7fe700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee080b2910 0x7fee080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.404+0000 7fee18ff9700 1 -- 172.21.15.19:0/3106231055 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fee140e2950 con 0x7fee1c110bf0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.404+0000 7fee18ff9700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] conn(0x7fee080bc5a0 0x7fee080bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.404+0000 7fee18ff9700 1 -- 172.21.15.19:0/3106231055 --> [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee1400be50 con 0x7fee080bc5a0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.404+0000 7fee18ff9700 1 -- 172.21.15.19:0/3106231055 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fee140e2da0 con 0x7fee1c110bf0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.404+0000 7fee1b7fe700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee080b2910 0x7fee080b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee0c005950 tx=0x7fee0c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.404+0000 7fee1affd700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] conn(0x7fee080bc5a0 0x7fee080bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.405+0000 7fee1affd700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] conn(0x7fee080bc5a0 0x7fee080bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.406+0000 7fee18ff9700 1 -- 172.21.15.19:0/3106231055 <== osd.2 v2:172.21.15.196:6800/1599368048 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fee1400be50 con 0x7fee080bc5a0 2024-05-09T01:24:00.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.423+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 --> [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee1c064310 con 0x7fee080bc5a0 2024-05-09T01:24:00.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.423+0000 7fee18ff9700 1 -- 172.21.15.19:0/3106231055 <== osd.2 v2:172.21.15.196:6800/1599368048 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fee1c064310 con 0x7fee080bc5a0 2024-05-09T01:24:00.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 >> [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] conn(0x7fee080bc5a0 msgr2=0x7fee080bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:00.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] conn(0x7fee080bc5a0 0x7fee080bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:00.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee080b2910 msgr2=0x7fee080b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:00.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee080b2910 0x7fee080b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee0c005950 tx=0x7fee0c00b410 comp rx=0 tx=0).stop 2024-05-09T01:24:00.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee1c110bf0 msgr2=0x7fee1c1a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:00.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 --2- 172.21.15.19:0/3106231055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee1c110bf0 0x7fee1c1a1900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee140094f0 tx=0x7fee1400d3b0 comp rx=0 tx=0).stop 2024-05-09T01:24:00.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee1bfff700 1 -- 172.21.15.19:0/3106231055 reap_dead start 2024-05-09T01:24:00.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 shutdown_connections 2024-05-09T01:24:00.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 >> 172.21.15.19:0/3106231055 conn(0x7fee1c0ffe70 msgr2=0x7fee1c101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:00.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 shutdown_connections 2024-05-09T01:24:00.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:00.424+0000 7fee21c24700 1 -- 172.21.15.19:0/3106231055 wait complete. 2024-05-09T01:24:00.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:00 smithi059 ceph-mon[34351]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:00 smithi196 ceph-mon[35041]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:01.116 INFO:teuthology.orchestra.run.smithi019.stdout:141733920835 2024-05-09T01:24:01.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T01:24:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.448+0000 7f53d94d4700 1 -- 172.21.15.19:0/276472459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4103540 msgr2=0x7f53d4103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.448+0000 7f53d94d4700 1 --2- 172.21.15.19:0/276472459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4103540 0x7f53d4103950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f53c40099c0 tx=0x7f53c4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.448+0000 7f53d94d4700 1 -- 172.21.15.19:0/276472459 shutdown_connections 2024-05-09T01:24:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.448+0000 7f53d94d4700 1 --2- 172.21.15.19:0/276472459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d4108770 0x7f53d41100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.448+0000 7f53d94d4700 1 --2- 172.21.15.19:0/276472459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4103540 0x7f53d4103950 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.448+0000 7f53d94d4700 1 --2- 172.21.15.19:0/276472459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d4107e20 0x7f53d4108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.448+0000 7f53d94d4700 1 -- 172.21.15.19:0/276472459 >> 172.21.15.19:0/276472459 conn(0x7f53d4078e50 msgr2=0x7f53d4079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.448+0000 7f53d94d4700 1 -- 172.21.15.19:0/276472459 shutdown_connections 2024-05-09T01:24:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.448+0000 7f53d94d4700 1 -- 172.21.15.19:0/276472459 wait complete. 2024-05-09T01:24:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d94d4700 1 Processor -- start 2024-05-09T01:24:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d94d4700 1 -- start start 2024-05-09T01:24:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d94d4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d4103540 0x7f53d419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d94d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4107e20 0x7f53d419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d94d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d4108770 0x7f53d419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d94d4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41133a0 con 0x7f53d4107e20 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d94d4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4113220 con 0x7f53d4108770 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d94d4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4113520 con 0x7f53d4103540 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4107e20 0x7f53d419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4107e20 0x7f53d419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55908/0 (socket says 172.21.15.19:55908) 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d27fc700 1 -- 172.21.15.19:0/1990966843 learned_addr learned my addr 172.21.15.19:0/1990966843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.449+0000 7f53d37fe700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d4108770 0x7f53d419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d2ffd700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d4103540 0x7f53d419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d27fc700 1 -- 172.21.15.19:0/1990966843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d4103540 msgr2=0x7f53d419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d27fc700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d4103540 0x7f53d419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d27fc700 1 -- 172.21.15.19:0/1990966843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d4108770 msgr2=0x7f53d419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d27fc700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d4108770 0x7f53d419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d27fc700 1 -- 172.21.15.19:0/1990966843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d41a7010 con 0x7f53d4107e20 2024-05-09T01:24:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d37fe700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d4108770 0x7f53d419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:24:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d27fc700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4107e20 0x7f53d419c1e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f53c800d980 tx=0x7f53c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53bbfff700 1 -- 172.21.15.19:0/1990966843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8017070 con 0x7f53d4107e20 2024-05-09T01:24:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d41a7300 con 0x7f53d4107e20 2024-05-09T01:24:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53bbfff700 1 -- 172.21.15.19:0/1990966843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c800fd40 con 0x7f53d4107e20 2024-05-09T01:24:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53bbfff700 1 -- 172.21.15.19:0/1990966843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c800e3b0 con 0x7f53d4107e20 2024-05-09T01:24:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.450+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d41a7890 con 0x7f53d4107e20 2024-05-09T01:24:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.451+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53d4063ed0 con 0x7f53d4107e20 2024-05-09T01:24:01.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.454+0000 7f53bbfff700 1 -- 172.21.15.19:0/1990966843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f53c8010610 con 0x7f53d4107e20 2024-05-09T01:24:01.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.454+0000 7f53bbfff700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f53bc0b2910 0x7f53bc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:01.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.455+0000 7f53d2ffd700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f53bc0b2910 0x7f53bc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:01.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.455+0000 7f53bbfff700 1 -- 172.21.15.19:0/1990966843 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f53c80e2400 con 0x7f53d4107e20 2024-05-09T01:24:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.455+0000 7f53bbfff700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] conn(0x7f53bc0bc5a0 0x7f53bc0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.455+0000 7f53bbfff700 1 -- 172.21.15.19:0/1990966843 --> [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53c8016ec0 con 0x7f53bc0bc5a0 2024-05-09T01:24:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.455+0000 7f53bbfff700 1 -- 172.21.15.19:0/1990966843 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f53c80e2850 con 0x7f53d4107e20 2024-05-09T01:24:01.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.456+0000 7f53d2ffd700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f53bc0b2910 0x7f53bc0b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009990 tx=0x7f53c40036e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:01.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.456+0000 7f53d37fe700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] conn(0x7f53bc0bc5a0 0x7f53bc0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:01.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.457+0000 7f53d37fe700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] conn(0x7f53bc0bc5a0 0x7f53bc0bea90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:01.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.458+0000 7f53bbfff700 1 -- 172.21.15.19:0/1990966843 <== osd.8 v2:172.21.15.196:6816/2419396882 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53c8016ec0 con 0x7f53bc0bc5a0 2024-05-09T01:24:01.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.475+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 --> [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53d4064300 con 0x7f53bc0bc5a0 2024-05-09T01:24:01.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.475+0000 7f53bbfff700 1 -- 172.21.15.19:0/1990966843 <== osd.8 v2:172.21.15.196:6816/2419396882 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f53d4064300 con 0x7f53bc0bc5a0 2024-05-09T01:24:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 >> [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] conn(0x7f53bc0bc5a0 msgr2=0x7f53bc0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] conn(0x7f53bc0bc5a0 0x7f53bc0bea90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f53bc0b2910 msgr2=0x7f53bc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f53bc0b2910 0x7f53bc0b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009990 tx=0x7f53c40036e0 comp rx=0 tx=0).stop 2024-05-09T01:24:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4107e20 msgr2=0x7f53d419c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 --2- 172.21.15.19:0/1990966843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4107e20 0x7f53d419c1e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f53c800d980 tx=0x7f53c800dc90 comp rx=0 tx=0).stop 2024-05-09T01:24:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d37fe700 1 -- 172.21.15.19:0/1990966843 reap_dead start 2024-05-09T01:24:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 shutdown_connections 2024-05-09T01:24:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 >> 172.21.15.19:0/1990966843 conn(0x7f53d4078e50 msgr2=0x7f53d4101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 shutdown_connections 2024-05-09T01:24:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.476+0000 7f53d94d4700 1 -- 172.21.15.19:0/1990966843 wait complete. 2024-05-09T01:24:02.222 INFO:teuthology.orchestra.run.smithi019.stdout:77309411406 2024-05-09T01:24:02.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T01:24:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.382+0000 7fa93c382700 1 -- 172.21.15.19:0/2803712960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa934108f00 msgr2=0x7fa934109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.382+0000 7fa93c382700 1 --2- 172.21.15.19:0/2803712960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa934108f00 0x7fa934109310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa9240099c0 tx=0x7fa924009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.382+0000 7fa93c382700 1 -- 172.21.15.19:0/2803712960 shutdown_connections 2024-05-09T01:24:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.382+0000 7fa93c382700 1 --2- 172.21.15.19:0/2803712960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934109850 0x7fa9341111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.382+0000 7fa93c382700 1 --2- 172.21.15.19:0/2803712960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9341045d0 0x7fa9341049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.382+0000 7fa93c382700 1 --2- 172.21.15.19:0/2803712960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa934108f00 0x7fa934109310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.382+0000 7fa93c382700 1 -- 172.21.15.19:0/2803712960 >> 172.21.15.19:0/2803712960 conn(0x7fa9340ffea0 msgr2=0x7fa9341022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.383+0000 7fa93c382700 1 -- 172.21.15.19:0/2803712960 shutdown_connections 2024-05-09T01:24:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.383+0000 7fa93c382700 1 -- 172.21.15.19:0/2803712960 wait complete. 2024-05-09T01:24:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.383+0000 7fa93c382700 1 Processor -- start 2024-05-09T01:24:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.383+0000 7fa93c382700 1 -- start start 2024-05-09T01:24:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93c382700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9341045d0 0x7fa934076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93c382700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934108f00 0x7fa934077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93c382700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa934109850 0x7fa934077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93c382700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341144b0 con 0x7fa934109850 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93c382700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa934114330 con 0x7fa9341045d0 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93c382700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa934114630 con 0x7fa934108f00 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a91f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa934109850 0x7fa934077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a91f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa934109850 0x7fa934077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55920/0 (socket says 172.21.15.19:55920) 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a91f700 1 -- 172.21.15.19:0/2515845306 learned_addr learned my addr 172.21.15.19:0/2515845306 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a11e700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9341045d0 0x7fa934076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93991d700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934108f00 0x7fa934077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a91f700 1 -- 172.21.15.19:0/2515845306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934108f00 msgr2=0x7fa934077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a91f700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934108f00 0x7fa934077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a91f700 1 -- 172.21.15.19:0/2515845306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9341045d0 msgr2=0x7fa934076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a91f700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9341045d0 0x7fa934076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a91f700 1 -- 172.21.15.19:0/2515845306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9341a91a0 con 0x7fa934109850 2024-05-09T01:24:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.384+0000 7fa93a91f700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa934109850 0x7fa934077650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00e940 tx=0x7fa92c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.385+0000 7fa92b7fe700 1 -- 172.21.15.19:0/2515845306 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c004a30 con 0x7fa934109850 2024-05-09T01:24:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.385+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9341a9490 con 0x7fa934109850 2024-05-09T01:24:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.385+0000 7fa92b7fe700 1 -- 172.21.15.19:0/2515845306 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa92c0054d0 con 0x7fa934109850 2024-05-09T01:24:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.385+0000 7fa92b7fe700 1 -- 172.21.15.19:0/2515845306 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00ccf0 con 0x7fa934109850 2024-05-09T01:24:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.385+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9341a99f0 con 0x7fa934109850 2024-05-09T01:24:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.386+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa918001030 con 0x7fa934109850 2024-05-09T01:24:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.388+0000 7fa92b7fe700 1 -- 172.21.15.19:0/2515845306 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa92c004bd0 con 0x7fa934109850 2024-05-09T01:24:02.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.388+0000 7fa92b7fe700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9200b2910 0x7fa9200b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.389+0000 7fa93a11e700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9200b2910 0x7fa9200b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.389+0000 7fa92b7fe700 1 -- 172.21.15.19:0/2515845306 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa92c0e29a0 con 0x7fa934109850 2024-05-09T01:24:02.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.389+0000 7fa92b7fe700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] conn(0x7fa9200bc5a0 0x7fa9200bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.389+0000 7fa92b7fe700 1 -- 172.21.15.19:0/2515845306 --> [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa92c00be50 con 0x7fa9200bc5a0 2024-05-09T01:24:02.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.389+0000 7fa92b7fe700 1 -- 172.21.15.19:0/2515845306 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fa92c0e2df0 con 0x7fa934109850 2024-05-09T01:24:02.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.389+0000 7fa93991d700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] conn(0x7fa9200bc5a0 0x7fa9200bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.389+0000 7fa93a11e700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9200b2910 0x7fa9200b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa930009550 tx=0x7fa9300091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:02.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.389+0000 7fa93991d700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] conn(0x7fa9200bc5a0 0x7fa9200bea90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:02.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.391+0000 7fa92b7fe700 1 -- 172.21.15.19:0/2515845306 <== osd.0 v2:172.21.15.19:6802/2513726191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa92c00be50 con 0x7fa9200bc5a0 2024-05-09T01:24:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.407+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 --> [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa918002e60 con 0x7fa9200bc5a0 2024-05-09T01:24:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa92b7fe700 1 -- 172.21.15.19:0/2515845306 <== osd.0 v2:172.21.15.19:6802/2513726191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa918002e60 con 0x7fa9200bc5a0 2024-05-09T01:24:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] conn(0x7fa9200bc5a0 msgr2=0x7fa9200bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] conn(0x7fa9200bc5a0 0x7fa9200bea90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9200b2910 msgr2=0x7fa9200b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9200b2910 0x7fa9200b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa930009550 tx=0x7fa9300091b0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa934109850 msgr2=0x7fa934077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 --2- 172.21.15.19:0/2515845306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa934109850 0x7fa934077650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00e940 tx=0x7fa92c00ec50 comp rx=0 tx=0).stop 2024-05-09T01:24:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93a91f700 1 -- 172.21.15.19:0/2515845306 reap_dead start 2024-05-09T01:24:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 shutdown_connections 2024-05-09T01:24:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 >> 172.21.15.19:0/2515845306 conn(0x7fa9340ffea0 msgr2=0x7fa93410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 shutdown_connections 2024-05-09T01:24:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:02.408+0000 7fa93c382700 1 -- 172.21.15.19:0/2515845306 wait complete. 2024-05-09T01:24:02.497 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:02 smithi196 ceph-mon[35041]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:02.515 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:02 smithi019 ceph-mon[30158]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:02.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:02 smithi059 ceph-mon[34351]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.336+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1055899111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb881034c0 msgr2=0x7fcb881038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.336+0000 7fcb8fe33700 1 --2- 172.21.15.19:0/1055899111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb881034c0 0x7fcb881038d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb840099c0 tx=0x7fcb84009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.336+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1055899111 shutdown_connections 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.336+0000 7fcb8fe33700 1 --2- 172.21.15.19:0/1055899111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb88108740 0x7fcb881100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.336+0000 7fcb8fe33700 1 --2- 172.21.15.19:0/1055899111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb881034c0 0x7fcb881038d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.336+0000 7fcb8fe33700 1 --2- 172.21.15.19:0/1055899111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88107df0 0x7fcb88108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.336+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1055899111 >> 172.21.15.19:0/1055899111 conn(0x7fcb880fedc0 msgr2=0x7fcb881011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.336+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1055899111 shutdown_connections 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.336+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1055899111 wait complete. 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.337+0000 7fcb8fe33700 1 Processor -- start 2024-05-09T01:24:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.337+0000 7fcb8fe33700 1 -- start start 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8fe33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb881034c0 0x7fcb8819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8fe33700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb88107df0 0x7fcb8819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8fe33700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88108740 0x7fcb881a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8fe33700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb88113370 con 0x7fcb88107df0 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8fe33700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881131f0 con 0x7fcb881034c0 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8fe33700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881134f0 con 0x7fcb88108740 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8d3ce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb88107df0 0x7fcb8819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8d3ce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb88107df0 0x7fcb8819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55892/0 (socket says 172.21.15.19:55892) 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8dbcf700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb881034c0 0x7fcb8819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8d3ce700 1 -- 172.21.15.19:0/1515668603 learned_addr learned my addr 172.21.15.19:0/1515668603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8e3d0700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88108740 0x7fcb881a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8dbcf700 1 -- 172.21.15.19:0/1515668603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88108740 msgr2=0x7fcb881a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8dbcf700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88108740 0x7fcb881a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8dbcf700 1 -- 172.21.15.19:0/1515668603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb88107df0 msgr2=0x7fcb8819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8dbcf700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb88107df0 0x7fcb8819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.338+0000 7fcb8dbcf700 1 -- 172.21.15.19:0/1515668603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb881a1f10 con 0x7fcb881034c0 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.339+0000 7fcb8dbcf700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb881034c0 0x7fcb8819d040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcb840098e0 tx=0x7fcb840045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.339+0000 7fcb7effd700 1 -- 172.21.15.19:0/1515668603 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84026070 con 0x7fcb881034c0 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.339+0000 7fcb7effd700 1 -- 172.21.15.19:0/1515668603 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb8400f460 con 0x7fcb881034c0 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.339+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb881a21a0 con 0x7fcb881034c0 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.339+0000 7fcb7effd700 1 -- 172.21.15.19:0/1515668603 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84009e60 con 0x7fcb881034c0 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.339+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb881a9b20 con 0x7fcb881034c0 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.340+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb881a26d0 con 0x7fcb881034c0 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.342+0000 7fcb7effd700 1 -- 172.21.15.19:0/1515668603 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcb84029070 con 0x7fcb881034c0 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.343+0000 7fcb7effd700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb740b2990 0x7fcb740b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.343+0000 7fcb8d3ce700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb740b2990 0x7fcb740b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.343+0000 7fcb7effd700 1 -- 172.21.15.19:0/1515668603 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcb840e34c0 con 0x7fcb881034c0 2024-05-09T01:24:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.343+0000 7fcb7effd700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] conn(0x7fcb740bc620 0x7fcb740beb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.343+0000 7fcb7effd700 1 -- 172.21.15.19:0/1515668603 --> [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb840095d0 con 0x7fcb740bc620 2024-05-09T01:24:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.343+0000 7fcb7effd700 1 -- 172.21.15.19:0/1515668603 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fcb840e3870 con 0x7fcb881034c0 2024-05-09T01:24:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.343+0000 7fcb8d3ce700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb740b2990 0x7fcb740b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb88102070 tx=0x7fcb7800b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.343+0000 7fcb8e3d0700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] conn(0x7fcb740bc620 0x7fcb740beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.344+0000 7fcb8e3d0700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] conn(0x7fcb740bc620 0x7fcb740beb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.345+0000 7fcb7effd700 1 -- 172.21.15.19:0/1515668603 <== osd.7 v2:172.21.15.59:6816/2903314697 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcb840095d0 con 0x7fcb740bc620 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.363+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 --> [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcb881a26d0 con 0x7fcb740bc620 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.363+0000 7fcb7effd700 1 -- 172.21.15.19:0/1515668603 <== osd.7 v2:172.21.15.59:6816/2903314697 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcb881a26d0 con 0x7fcb740bc620 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 >> [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] conn(0x7fcb740bc620 msgr2=0x7fcb740beb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] conn(0x7fcb740bc620 0x7fcb740beb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb740b2990 msgr2=0x7fcb740b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb740b2990 0x7fcb740b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb88102070 tx=0x7fcb7800b480 comp rx=0 tx=0).stop 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb881034c0 msgr2=0x7fcb8819d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 --2- 172.21.15.19:0/1515668603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb881034c0 0x7fcb8819d040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcb840098e0 tx=0x7fcb840045e0 comp rx=0 tx=0).stop 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8e3d0700 1 -- 172.21.15.19:0/1515668603 reap_dead start 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 shutdown_connections 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 >> 172.21.15.19:0/1515668603 conn(0x7fcb880fedc0 msgr2=0x7fcb881063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 shutdown_connections 2024-05-09T01:24:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:01.364+0000 7fcb8fe33700 1 -- 172.21.15.19:0/1515668603 wait complete. 2024-05-09T01:24:03.597 INFO:teuthology.orchestra.run.smithi019.stdout:214748364852 2024-05-09T01:24:03.598 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T01:24:04.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:04 smithi019 ceph-mon[30158]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:04.566 INFO:teuthology.orchestra.run.smithi019.stdout:55834574929 2024-05-09T01:24:04.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T01:24:04.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:04 smithi059 ceph-mon[34351]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:04.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:04 smithi196 ceph-mon[35041]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:05.462 INFO:teuthology.orchestra.run.smithi019.stdout:34359738457 2024-05-09T01:24:05.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T01:24:05.905 INFO:teuthology.orchestra.run.smithi019.stdout:115964117064 2024-05-09T01:24:05.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T01:24:06.370 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:06 smithi019 ceph-mon[30158]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-05-09T01:24:06.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:06 smithi059 ceph-mon[34351]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-05-09T01:24:06.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:06 smithi196 ceph-mon[35041]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-05-09T01:24:07.385 INFO:teuthology.orchestra.run.smithi019.stdout:240518168624 2024-05-09T01:24:07.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T01:24:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.585+0000 7feec9ba7700 1 -- 172.21.15.19:0/2226901068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec41034c0 msgr2=0x7feec41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.585+0000 7feec9ba7700 1 --2- 172.21.15.19:0/2226901068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec41034c0 0x7feec41038d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feeb40099c0 tx=0x7feeb4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.585+0000 7feec9ba7700 1 -- 172.21.15.19:0/2226901068 shutdown_connections 2024-05-09T01:24:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.586+0000 7feec9ba7700 1 --2- 172.21.15.19:0/2226901068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec4108740 0x7feec41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.586+0000 7feec9ba7700 1 --2- 172.21.15.19:0/2226901068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec41034c0 0x7feec41038d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.586+0000 7feec9ba7700 1 --2- 172.21.15.19:0/2226901068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4107df0 0x7feec4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.586+0000 7feec9ba7700 1 -- 172.21.15.19:0/2226901068 >> 172.21.15.19:0/2226901068 conn(0x7feec40fedc0 msgr2=0x7feec41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.586+0000 7feec9ba7700 1 -- 172.21.15.19:0/2226901068 shutdown_connections 2024-05-09T01:24:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.586+0000 7feec9ba7700 1 -- 172.21.15.19:0/2226901068 wait complete. 2024-05-09T01:24:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.586+0000 7feec9ba7700 1 Processor -- start 2024-05-09T01:24:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec9ba7700 1 -- start start 2024-05-09T01:24:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec9ba7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec41034c0 0x7feec419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec9ba7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec4107df0 0x7feec419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec9ba7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4108740 0x7feec419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec9ba7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feec4113400 con 0x7feec4107df0 2024-05-09T01:24:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec9ba7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feec4113280 con 0x7feec4108740 2024-05-09T01:24:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec9ba7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feec4113580 con 0x7feec41034c0 2024-05-09T01:24:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec4107df0 0x7feec419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec4107df0 0x7feec419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55942/0 (socket says 172.21.15.19:55942) 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec2ffd700 1 -- 172.21.15.19:0/867860617 learned_addr learned my addr 172.21.15.19:0/867860617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec3fff700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4108740 0x7feec419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec37fe700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec41034c0 0x7feec419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec2ffd700 1 -- 172.21.15.19:0/867860617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec41034c0 msgr2=0x7feec419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec2ffd700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec41034c0 0x7feec419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec2ffd700 1 -- 172.21.15.19:0/867860617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4108740 msgr2=0x7feec419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec2ffd700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4108740 0x7feec419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec2ffd700 1 -- 172.21.15.19:0/867860617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec419fd30 con 0x7feec4107df0 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.587+0000 7feec37fe700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec41034c0 0x7feec419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:24:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.588+0000 7feec2ffd700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec4107df0 0x7feec419b400 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feeb800b500 tx=0x7feeb800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.588+0000 7feec0ff9700 1 -- 172.21.15.19:0/867860617 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8015020 con 0x7feec4107df0 2024-05-09T01:24:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.588+0000 7feec0ff9700 1 -- 172.21.15.19:0/867860617 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb8011ce0 con 0x7feec4107df0 2024-05-09T01:24:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.588+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec41a0020 con 0x7feec4107df0 2024-05-09T01:24:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.588+0000 7feec0ff9700 1 -- 172.21.15.19:0/867860617 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8010d40 con 0x7feec4107df0 2024-05-09T01:24:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.588+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec41a7940 con 0x7feec4107df0 2024-05-09T01:24:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.589+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feec4063f50 con 0x7feec4107df0 2024-05-09T01:24:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.590+0000 7feec0ff9700 1 -- 172.21.15.19:0/867860617 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7feeb8018020 con 0x7feec4107df0 2024-05-09T01:24:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.591+0000 7feec0ff9700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2990 0x7feeac0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.591+0000 7feec37fe700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2990 0x7feeac0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.591+0000 7feec0ff9700 1 -- 172.21.15.19:0/867860617 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7feeb80e2170 con 0x7feec4107df0 2024-05-09T01:24:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.591+0000 7feec0ff9700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] conn(0x7feeac0bc620 0x7feeac0beb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.591+0000 7feec0ff9700 1 -- 172.21.15.19:0/867860617 --> [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feeb801be50 con 0x7feeac0bc620 2024-05-09T01:24:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.592+0000 7feec0ff9700 1 -- 172.21.15.19:0/867860617 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7feeb80e2520 con 0x7feec4107df0 2024-05-09T01:24:07.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.592+0000 7feec37fe700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2990 0x7feeac0b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feeb4006010 tx=0x7feeb4005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:07.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.592+0000 7feec3fff700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] conn(0x7feeac0bc620 0x7feeac0beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.592+0000 7feec3fff700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] conn(0x7feeac0bc620 0x7feeac0beb10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:07.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.593+0000 7feec0ff9700 1 -- 172.21.15.19:0/867860617 <== osd.11 v2:172.21.15.196:6824/1918691791 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feeb801be50 con 0x7feeac0bc620 2024-05-09T01:24:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.610+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 --> [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feec4064350 con 0x7feeac0bc620 2024-05-09T01:24:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.610+0000 7feec0ff9700 1 -- 172.21.15.19:0/867860617 <== osd.11 v2:172.21.15.196:6824/1918691791 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feec4064350 con 0x7feeac0bc620 2024-05-09T01:24:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 >> [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] conn(0x7feeac0bc620 msgr2=0x7feeac0beb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] conn(0x7feeac0bc620 0x7feeac0beb10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2990 msgr2=0x7feeac0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2990 0x7feeac0b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feeb4006010 tx=0x7feeb4005940 comp rx=0 tx=0).stop 2024-05-09T01:24:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec4107df0 msgr2=0x7feec419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 --2- 172.21.15.19:0/867860617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec4107df0 0x7feec419b400 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feeb800b500 tx=0x7feeb800b810 comp rx=0 tx=0).stop 2024-05-09T01:24:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec3fff700 1 -- 172.21.15.19:0/867860617 reap_dead start 2024-05-09T01:24:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 shutdown_connections 2024-05-09T01:24:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 >> 172.21.15.19:0/867860617 conn(0x7feec40fedc0 msgr2=0x7feec406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 shutdown_connections 2024-05-09T01:24:07.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.611+0000 7feec9ba7700 1 -- 172.21.15.19:0/867860617 wait complete. 2024-05-09T01:24:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.822+0000 7f11ead0c700 1 -- 172.21.15.19:0/2233821537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e4107df0 msgr2=0x7f11e4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.822+0000 7f11ead0c700 1 --2- 172.21.15.19:0/2233821537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e4107df0 0x7f11e4108200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f11d00099c0 tx=0x7f11d0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.822+0000 7f11ead0c700 1 -- 172.21.15.19:0/2233821537 shutdown_connections 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.822+0000 7f11ead0c700 1 --2- 172.21.15.19:0/2233821537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e4108740 0x7f11e41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.822+0000 7f11ead0c700 1 --2- 172.21.15.19:0/2233821537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e41034c0 0x7f11e41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.822+0000 7f11ead0c700 1 --2- 172.21.15.19:0/2233821537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e4107df0 0x7f11e4108200 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.822+0000 7f11ead0c700 1 -- 172.21.15.19:0/2233821537 >> 172.21.15.19:0/2233821537 conn(0x7f11e40fedc0 msgr2=0x7f11e41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.822+0000 7f11ead0c700 1 -- 172.21.15.19:0/2233821537 shutdown_connections 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.822+0000 7f11ead0c700 1 -- 172.21.15.19:0/2233821537 wait complete. 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11ead0c700 1 Processor -- start 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11ead0c700 1 -- start start 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11ead0c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e41034c0 0x7f11e419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11ead0c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e4107df0 0x7f11e419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11ead0c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e4108740 0x7f11e419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11ead0c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11e4113350 con 0x7f11e4107df0 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11ead0c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41131d0 con 0x7f11e4108740 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11ead0c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41134d0 con 0x7f11e41034c0 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e4107df0 0x7f11e419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e4107df0 0x7f11e419c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55952/0 (socket says 172.21.15.19:55952) 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e3fff700 1 -- 172.21.15.19:0/1888563252 learned_addr learned my addr 172.21.15.19:0/1888563252 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e92a9700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e4108740 0x7f11e419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e8aa8700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e41034c0 0x7f11e419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e3fff700 1 -- 172.21.15.19:0/1888563252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e41034c0 msgr2=0x7f11e419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e3fff700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e41034c0 0x7f11e419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e3fff700 1 -- 172.21.15.19:0/1888563252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e4108740 msgr2=0x7f11e419c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e3fff700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e4108740 0x7f11e419c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.823+0000 7f11e3fff700 1 -- 172.21.15.19:0/1888563252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e41a6f50 con 0x7f11e4107df0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.824+0000 7f11e3fff700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e4107df0 0x7f11e419c1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f11d00098e0 tx=0x7f11d00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.824+0000 7f11e1ffb700 1 -- 172.21.15.19:0/1888563252 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0027070 con 0x7f11e4107df0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.824+0000 7f11e1ffb700 1 -- 172.21.15.19:0/1888563252 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d0003f20 con 0x7f11e4107df0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.824+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e41a71e0 con 0x7f11e4107df0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.824+0000 7f11e1ffb700 1 -- 172.21.15.19:0/1888563252 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0022c70 con 0x7f11e4107df0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.824+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e41a7690 con 0x7f11e4107df0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.825+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f11e4063e90 con 0x7f11e4107df0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.827+0000 7f11e1ffb700 1 -- 172.21.15.19:0/1888563252 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f11d002a070 con 0x7f11e4107df0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.827+0000 7f11e1ffb700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f11d40b2990 0x7f11d40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.827+0000 7f11e8aa8700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f11d40b2990 0x7f11d40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.827+0000 7f11e1ffb700 1 -- 172.21.15.19:0/1888563252 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f11d00e3c10 con 0x7f11e4107df0 2024-05-09T01:24:07.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.828+0000 7f11e1ffb700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] conn(0x7f11d40bc620 0x7f11d40beb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:07.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.828+0000 7f11e1ffb700 1 -- 172.21.15.19:0/1888563252 --> [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11d40bf1e0 con 0x7f11d40bc620 2024-05-09T01:24:07.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.828+0000 7f11e1ffb700 1 -- 172.21.15.19:0/1888563252 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f11d00e4060 con 0x7f11e4107df0 2024-05-09T01:24:07.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.828+0000 7f11e8aa8700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f11d40b2990 0x7f11d40b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f11d800ac50 tx=0x7f11d800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:07.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.828+0000 7f11e92a9700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] conn(0x7f11d40bc620 0x7f11d40beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:07.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.828+0000 7f11e92a9700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] conn(0x7f11d40bc620 0x7f11d40beb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:07.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.829+0000 7f11e1ffb700 1 -- 172.21.15.19:0/1888563252 <== osd.5 v2:172.21.15.196:6808/1450867392 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11d40bf1e0 con 0x7f11d40bc620 2024-05-09T01:24:07.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.851+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 --> [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f11e40642c0 con 0x7f11d40bc620 2024-05-09T01:24:07.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.852+0000 7f11e1ffb700 1 -- 172.21.15.19:0/1888563252 <== osd.5 v2:172.21.15.196:6808/1450867392 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f11e40642c0 con 0x7f11d40bc620 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.852+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 >> [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] conn(0x7f11d40bc620 msgr2=0x7f11d40beb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.852+0000 7f11ead0c700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] conn(0x7f11d40bc620 0x7f11d40beb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.852+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f11d40b2990 msgr2=0x7f11d40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.852+0000 7f11ead0c700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f11d40b2990 0x7f11d40b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f11d800ac50 tx=0x7f11d800a380 comp rx=0 tx=0).stop 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.852+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e4107df0 msgr2=0x7f11e419c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.852+0000 7f11ead0c700 1 --2- 172.21.15.19:0/1888563252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e4107df0 0x7f11e419c1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f11d00098e0 tx=0x7f11d00057c0 comp rx=0 tx=0).stop 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.852+0000 7f11e92a9700 1 -- 172.21.15.19:0/1888563252 reap_dead start 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.853+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 shutdown_connections 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.853+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 >> 172.21.15.19:0/1888563252 conn(0x7f11e40fedc0 msgr2=0x7f11e41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.853+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 shutdown_connections 2024-05-09T01:24:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:07.853+0000 7f11ead0c700 1 -- 172.21.15.19:0/1888563252 wait complete. 2024-05-09T01:24:08.449 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:08 smithi019 ceph-mon[30158]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T01:24:08.476 INFO:teuthology.orchestra.run.smithi019.stdout:313532612642 2024-05-09T01:24:08.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T01:24:08.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:08 smithi059 ceph-mon[34351]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T01:24:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:08.689 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:08.691 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:08.691 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:08 smithi196 ceph-mon[35041]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T01:24:09.096 INFO:teuthology.orchestra.run.smithi019.stdout:167503724608 2024-05-09T01:24:09.097 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T01:24:09.122 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:09.176 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:09.226 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:09.226 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:10.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:10 smithi019 ceph-mon[30158]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:10.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:10 smithi059 ceph-mon[34351]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:10.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:10 smithi196 ceph-mon[35041]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:10.718 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:11.846 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:12.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:12 smithi019 ceph-mon[30158]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:12.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:12 smithi059 ceph-mon[34351]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:12.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:12 smithi196 ceph-mon[35041]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:14.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:14 smithi019 ceph-mon[30158]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:14.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:14 smithi059 ceph-mon[34351]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:14.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:14 smithi196 ceph-mon[35041]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:16.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:16 smithi019 ceph-mon[30158]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:24:16.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:16 smithi059 ceph-mon[34351]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:24:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:16 smithi196 ceph-mon[35041]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:24:18.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:18 smithi019 ceph-mon[30158]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:24:18.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:18 smithi059 ceph-mon[34351]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:24:18.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:18 smithi196 ceph-mon[35041]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:24:19.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:19 smithi019 ceph-mon[30158]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:19.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:19 smithi196 ceph-mon[35041]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:20.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:19 smithi059 ceph-mon[34351]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:21.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.591+0000 7fbb3be8f700 1 -- 172.21.15.19:0/3276443963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34107e20 msgr2=0x7fbb34108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:21.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.591+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/3276443963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34107e20 0x7fbb34108230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb240099c0 tx=0x7fbb24009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.591+0000 7fbb3be8f700 1 -- 172.21.15.19:0/3276443963 shutdown_connections 2024-05-09T01:24:21.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.591+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/3276443963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb34108770 0x7fbb341100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.591+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/3276443963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb34103c20 0x7fbb34104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.591+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/3276443963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34107e20 0x7fbb34108230 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.591+0000 7fbb3be8f700 1 -- 172.21.15.19:0/3276443963 >> 172.21.15.19:0/3276443963 conn(0x7fbb340fe6f0 msgr2=0x7fbb34100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.591+0000 7fbb3be8f700 1 -- 172.21.15.19:0/3276443963 shutdown_connections 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.591+0000 7fbb3be8f700 1 -- 172.21.15.19:0/3276443963 wait complete. 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3be8f700 1 Processor -- start 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3be8f700 1 -- start start 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3be8f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb34103c20 0x7fbb3419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3be8f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb34107e20 0x7fbb3419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3be8f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34108770 0x7fbb3419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3be8f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34113390 con 0x7fbb34108770 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3be8f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34113210 con 0x7fbb34107e20 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3be8f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34113510 con 0x7fbb34103c20 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3a42c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34108770 0x7fbb3419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3a42c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34108770 0x7fbb3419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57750/0 (socket says 172.21.15.19:57750) 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb39c2b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb34103c20 0x7fbb3419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:21.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3a42c700 1 -- 172.21.15.19:0/2254365119 learned_addr learned my addr 172.21.15.19:0/2254365119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.592+0000 7fbb3942a700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb34107e20 0x7fbb3419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb3a42c700 1 -- 172.21.15.19:0/2254365119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb34103c20 msgr2=0x7fbb3419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb3a42c700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb34103c20 0x7fbb3419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb3a42c700 1 -- 172.21.15.19:0/2254365119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb34107e20 msgr2=0x7fbb3419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb3a42c700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb34107e20 0x7fbb3419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb3a42c700 1 -- 172.21.15.19:0/2254365119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3419fcf0 con 0x7fbb34108770 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb3a42c700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34108770 0x7fbb3419f700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007ee0 tx=0x7fbb2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb2affd700 1 -- 172.21.15.19:0/2254365119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c004bb0 con 0x7fbb34108770 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb2affd700 1 -- 172.21.15.19:0/2254365119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb2c004d50 con 0x7fbb34108770 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb2affd700 1 -- 172.21.15.19:0/2254365119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c005920 con 0x7fbb34108770 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3419ffe0 con 0x7fbb34108770 2024-05-09T01:24:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.593+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb341a04f0 con 0x7fbb34108770 2024-05-09T01:24:21.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.594+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb340512c0 con 0x7fbb34108770 2024-05-09T01:24:21.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.596+0000 7fbb2affd700 1 -- 172.21.15.19:0/2254365119 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbb2c01a020 con 0x7fbb34108770 2024-05-09T01:24:21.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.596+0000 7fbb2affd700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb200b2990 0x7fbb200b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:21.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.596+0000 7fbb39c2b700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb200b2990 0x7fbb200b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:21.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.596+0000 7fbb2affd700 1 -- 172.21.15.19:0/2254365119 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbb2c0e1fb0 con 0x7fbb34108770 2024-05-09T01:24:21.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.597+0000 7fbb39c2b700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb200b2990 0x7fbb200b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbb3406b110 tx=0x7fbb3000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:21.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.600+0000 7fbb2affd700 1 -- 172.21.15.19:0/2254365119 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb2c010070 con 0x7fbb34108770 2024-05-09T01:24:21.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.817+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbb3402e5f0 con 0x7fbb34108770 2024-05-09T01:24:21.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.817+0000 7fbb2affd700 1 -- 172.21.15.19:0/2254365119 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbb2c0a9b50 con 0x7fbb34108770 2024-05-09T01:24:21.815 INFO:teuthology.orchestra.run.smithi019.stdout:77309411410 2024-05-09T01:24:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb200b2990 msgr2=0x7fbb200b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb200b2990 0x7fbb200b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbb3406b110 tx=0x7fbb3000a380 comp rx=0 tx=0).stop 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34108770 msgr2=0x7fbb3419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34108770 0x7fbb3419f700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007ee0 tx=0x7fbb2c00d3b0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 shutdown_connections 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb200b2990 0x7fbb200b4ea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb34108770 0x7fbb3419f700 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb34103c20 0x7fbb3419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 --2- 172.21.15.19:0/2254365119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb34107e20 0x7fbb3419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.821+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 >> 172.21.15.19:0/2254365119 conn(0x7fbb340fe6f0 msgr2=0x7fbb34102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.822+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 shutdown_connections 2024-05-09T01:24:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.822+0000 7fbb3be8f700 1 -- 172.21.15.19:0/2254365119 wait complete. 2024-05-09T01:24:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.943+0000 7fab9900d700 1 -- 172.21.15.19:0/3215920133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab94107e10 msgr2=0x7fab94108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.943+0000 7fab9900d700 1 --2- 172.21.15.19:0/3215920133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab94107e10 0x7fab94108220 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fab880099c0 tx=0x7fab88009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.943+0000 7fab9900d700 1 -- 172.21.15.19:0/3215920133 shutdown_connections 2024-05-09T01:24:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.943+0000 7fab9900d700 1 --2- 172.21.15.19:0/3215920133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab94108760 0x7fab941100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.943+0000 7fab9900d700 1 --2- 172.21.15.19:0/3215920133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab941034e0 0x7fab941038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.943+0000 7fab9900d700 1 --2- 172.21.15.19:0/3215920133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab94107e10 0x7fab94108220 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.943+0000 7fab9900d700 1 -- 172.21.15.19:0/3215920133 >> 172.21.15.19:0/3215920133 conn(0x7fab940fef30 msgr2=0x7fab94101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.944+0000 7fab9900d700 1 -- 172.21.15.19:0/3215920133 shutdown_connections 2024-05-09T01:24:21.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.944+0000 7fab9900d700 1 -- 172.21.15.19:0/3215920133 wait complete. 2024-05-09T01:24:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.944+0000 7fab9900d700 1 Processor -- start 2024-05-09T01:24:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9900d700 1 -- start start 2024-05-09T01:24:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9900d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab941034e0 0x7fab9419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9900d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab94107e10 0x7fab9419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9900d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab94108760 0x7fab9419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9900d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab941133e0 con 0x7fab941034e0 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9900d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab94113260 con 0x7fab94107e10 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9900d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab94113560 con 0x7fab94108760 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab92d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab941034e0 0x7fab9419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab92d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab941034e0 0x7fab9419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57760/0 (socket says 172.21.15.19:57760) 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab92d9d700 1 -- 172.21.15.19:0/390187575 learned_addr learned my addr 172.21.15.19:0/390187575 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9259c700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab94107e10 0x7fab9419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9359e700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab94108760 0x7fab9419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab92d9d700 1 -- 172.21.15.19:0/390187575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab94108760 msgr2=0x7fab9419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab92d9d700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab94108760 0x7fab9419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab92d9d700 1 -- 172.21.15.19:0/390187575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab94107e10 msgr2=0x7fab9419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab92d9d700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab94107e10 0x7fab9419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab92d9d700 1 -- 172.21.15.19:0/390187575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9419fd40 con 0x7fab941034e0 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9259c700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab94107e10 0x7fab9419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:24:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.945+0000 7fab9359e700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab94108760 0x7fab9419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:24:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.946+0000 7fab92d9d700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab941034e0 0x7fab9419af20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab8400c9b0 tx=0x7fab8400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.946+0000 7fab7bfff700 1 -- 172.21.15.19:0/390187575 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84007b00 con 0x7fab941034e0 2024-05-09T01:24:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.946+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab941a0030 con 0x7fab941034e0 2024-05-09T01:24:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.946+0000 7fab7bfff700 1 -- 172.21.15.19:0/390187575 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab84007ca0 con 0x7fab941034e0 2024-05-09T01:24:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.946+0000 7fab7bfff700 1 -- 172.21.15.19:0/390187575 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84018660 con 0x7fab941034e0 2024-05-09T01:24:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.946+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab941a0590 con 0x7fab941034e0 2024-05-09T01:24:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.947+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab940512c0 con 0x7fab941034e0 2024-05-09T01:24:21.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.949+0000 7fab7bfff700 1 -- 172.21.15.19:0/390187575 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fab8400f450 con 0x7fab941034e0 2024-05-09T01:24:21.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.949+0000 7fab7bfff700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab7c0b2910 0x7fab7c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:21.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.950+0000 7fab9259c700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab7c0b2910 0x7fab7c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:21.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.950+0000 7fab7bfff700 1 -- 172.21.15.19:0/390187575 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fab840e2660 con 0x7fab941034e0 2024-05-09T01:24:21.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.950+0000 7fab9259c700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab7c0b2910 0x7fab7c0b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fab88006010 tx=0x7fab8801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:21.953+0000 7fab7bfff700 1 -- 172.21.15.19:0/390187575 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab84014030 con 0x7fab941034e0 2024-05-09T01:24:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.184+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fab9419c280 con 0x7fab941034e0 2024-05-09T01:24:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.185+0000 7fab7bfff700 1 -- 172.21.15.19:0/390187575 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fab840aa290 con 0x7fab941034e0 2024-05-09T01:24:22.183 INFO:teuthology.orchestra.run.smithi019.stdout:287762808873 2024-05-09T01:24:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab7c0b2910 msgr2=0x7fab7c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab7c0b2910 0x7fab7c0b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fab88006010 tx=0x7fab8801a040 comp rx=0 tx=0).stop 2024-05-09T01:24:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab941034e0 msgr2=0x7fab9419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab941034e0 0x7fab9419af20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab8400c9b0 tx=0x7fab8400ccc0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 shutdown_connections 2024-05-09T01:24:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab94108760 0x7fab9419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab7c0b2910 0x7fab7c0b4e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab941034e0 0x7fab9419af20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 --2- 172.21.15.19:0/390187575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab94107e10 0x7fab9419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 >> 172.21.15.19:0/390187575 conn(0x7fab940fef30 msgr2=0x7fab9406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 shutdown_connections 2024-05-09T01:24:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.187+0000 7fab9900d700 1 -- 172.21.15.19:0/390187575 wait complete. 2024-05-09T01:24:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.390+0000 7fb0efad4700 1 -- 172.21.15.19:0/1878430399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e81034c0 msgr2=0x7fb0e81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.390+0000 7fb0efad4700 1 --2- 172.21.15.19:0/1878430399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e81034c0 0x7fb0e81038d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40099c0 tx=0x7fb0e4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.390+0000 7fb0efad4700 1 -- 172.21.15.19:0/1878430399 shutdown_connections 2024-05-09T01:24:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.390+0000 7fb0efad4700 1 --2- 172.21.15.19:0/1878430399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0e8108740 0x7fb0e81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.390+0000 7fb0efad4700 1 --2- 172.21.15.19:0/1878430399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e81034c0 0x7fb0e81038d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.390+0000 7fb0efad4700 1 --2- 172.21.15.19:0/1878430399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0e8107df0 0x7fb0e8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.390+0000 7fb0efad4700 1 -- 172.21.15.19:0/1878430399 >> 172.21.15.19:0/1878430399 conn(0x7fb0e80fedf0 msgr2=0x7fb0e8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.390+0000 7fb0efad4700 1 -- 172.21.15.19:0/1878430399 shutdown_connections 2024-05-09T01:24:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.390+0000 7fb0efad4700 1 -- 172.21.15.19:0/1878430399 wait complete. 2024-05-09T01:24:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0efad4700 1 Processor -- start 2024-05-09T01:24:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0efad4700 1 -- start start 2024-05-09T01:24:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0efad4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0e81034c0 0x7fb0e819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0efad4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e8107df0 0x7fb0e819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0efad4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0e8108740 0x7fb0e819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0efad4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8113370 con 0x7fb0e8107df0 2024-05-09T01:24:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0efad4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81131f0 con 0x7fb0e81034c0 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0efad4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81134f0 con 0x7fb0e8108740 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0ed870700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0e81034c0 0x7fb0e819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0ed06f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e8107df0 0x7fb0e819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0ed870700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0e81034c0 0x7fb0e819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:51268/0 (socket says 172.21.15.19:51268) 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0ed870700 1 -- 172.21.15.19:0/122174475 learned_addr learned my addr 172.21.15.19:0/122174475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.391+0000 7fb0ee071700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0e8108740 0x7fb0e819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0ed06f700 1 -- 172.21.15.19:0/122174475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0e8108740 msgr2=0x7fb0e819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0ed06f700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0e8108740 0x7fb0e819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0ed06f700 1 -- 172.21.15.19:0/122174475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0e81034c0 msgr2=0x7fb0e819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0ed06f700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0e81034c0 0x7fb0e819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0ed06f700 1 -- 172.21.15.19:0/122174475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e819fce0 con 0x7fb0e8107df0 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0ed870700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0e81034c0 0x7fb0e819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0ed06f700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e8107df0 0x7fb0e819b3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8076b40 tx=0x7fb0d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0deffd700 1 -- 172.21.15.19:0/122174475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8016070 con 0x7fb0e8107df0 2024-05-09T01:24:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e819ffd0 con 0x7fb0e8107df0 2024-05-09T01:24:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0deffd700 1 -- 172.21.15.19:0/122174475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d800fd40 con 0x7fb0e8107df0 2024-05-09T01:24:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0deffd700 1 -- 172.21.15.19:0/122174475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800e3b0 con 0x7fb0e8107df0 2024-05-09T01:24:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.392+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81a78f0 con 0x7fb0e8107df0 2024-05-09T01:24:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.393+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e80512c0 con 0x7fb0e8107df0 2024-05-09T01:24:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.396+0000 7fb0deffd700 1 -- 172.21.15.19:0/122174475 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb0d8019070 con 0x7fb0e8107df0 2024-05-09T01:24:22.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:22 smithi019 ceph-mon[30158]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:22.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:22 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2254365119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T01:24:22.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:22 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/390187575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T01:24:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.397+0000 7fb0deffd700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0d40b2900 0x7fb0d40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:22.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.397+0000 7fb0ed870700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0d40b2900 0x7fb0d40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:22.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.397+0000 7fb0deffd700 1 -- 172.21.15.19:0/122174475 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb0d80e2070 con 0x7fb0e8107df0 2024-05-09T01:24:22.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.398+0000 7fb0ed870700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0d40b2900 0x7fb0d40b4e10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4003800 tx=0x7fb0e401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.401+0000 7fb0deffd700 1 -- 172.21.15.19:0/122174475 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb0d80a9d10 con 0x7fb0e8107df0 2024-05-09T01:24:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.270+0000 7f55213e2700 1 -- 172.21.15.19:0/1182398762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 msgr2=0x7f551c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.270+0000 7f55213e2700 1 --2- 172.21.15.19:0/1182398762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 0x7f551c1038d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f550c0099c0 tx=0x7f550c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.270+0000 7f55213e2700 1 -- 172.21.15.19:0/1182398762 shutdown_connections 2024-05-09T01:24:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.270+0000 7f55213e2700 1 --2- 172.21.15.19:0/1182398762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f551c108740 0x7f551c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.270+0000 7f55213e2700 1 --2- 172.21.15.19:0/1182398762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 0x7f551c1038d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.270+0000 7f55213e2700 1 --2- 172.21.15.19:0/1182398762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c107df0 0x7f551c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.270+0000 7f55213e2700 1 -- 172.21.15.19:0/1182398762 >> 172.21.15.19:0/1182398762 conn(0x7f551c0fedc0 msgr2=0x7f551c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.270+0000 7f55213e2700 1 -- 172.21.15.19:0/1182398762 shutdown_connections 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.270+0000 7f55213e2700 1 -- 172.21.15.19:0/1182398762 wait complete. 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.271+0000 7f55213e2700 1 Processor -- start 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.271+0000 7f55213e2700 1 -- start start 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.271+0000 7f55213e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 0x7f551c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f55213e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c107df0 0x7f551c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f55213e2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f551c108740 0x7f551c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f55213e2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f551c113370 con 0x7f551c1034c0 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f55213e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1131f0 con 0x7f551c107df0 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f55213e2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1134f0 con 0x7f551c108740 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f551affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 0x7f551c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f551affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 0x7f551c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57784/0 (socket says 172.21.15.19:57784) 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f551affd700 1 -- 172.21.15.19:0/2174988121 learned_addr learned my addr 172.21.15.19:0/2174988121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f551b7fe700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f551c108740 0x7f551c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:22.514 INFO:teuthology.orchestra.run.smithi019.stdout:55834574934 2024-05-09T01:24:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.272+0000 7f55127fc700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c107df0 0x7f551c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f551affd700 1 -- 172.21.15.19:0/2174988121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f551c108740 msgr2=0x7f551c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f551affd700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f551c108740 0x7f551c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f551affd700 1 -- 172.21.15.19:0/2174988121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c107df0 msgr2=0x7f551c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f551affd700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c107df0 0x7f551c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f551affd700 1 -- 172.21.15.19:0/2174988121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c1a9180 con 0x7f551c1034c0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f551affd700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 0x7f551c199a80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b580 tx=0x7f550c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f5518ff9700 1 -- 172.21.15.19:0/2174988121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c027070 con 0x7f551c1034c0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551c1a9410 con 0x7f551c1034c0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f5518ff9700 1 -- 172.21.15.19:0/2174988121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550c003f20 con 0x7f551c1034c0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f5518ff9700 1 -- 172.21.15.19:0/2174988121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c022d10 con 0x7f551c1034c0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.273+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551c1a98c0 con 0x7f551c1034c0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.274+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f551c104970 con 0x7f551c1034c0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.276+0000 7f5518ff9700 1 -- 172.21.15.19:0/2174988121 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f550c003a70 con 0x7f551c1034c0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.277+0000 7f5518ff9700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55080b2880 0x7f55080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.277+0000 7f55127fc700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55080b2880 0x7f55080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.277+0000 7f5518ff9700 1 -- 172.21.15.19:0/2174988121 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f550c0e2e60 con 0x7f551c1034c0 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.277+0000 7f55127fc700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55080b2880 0x7f55080b4d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f551c106930 tx=0x7f550400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.282+0000 7f5518ff9700 1 -- 172.21.15.19:0/2174988121 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f550c013070 con 0x7f551c1034c0 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.504+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f551c02e5f0 con 0x7f551c1034c0 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.504+0000 7f5518ff9700 1 -- 172.21.15.19:0/2174988121 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f550c0aabf0 con 0x7f551c1034c0 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55080b2880 msgr2=0x7f55080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55080b2880 0x7f55080b4d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f551c106930 tx=0x7f550400a300 comp rx=0 tx=0).stop 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 msgr2=0x7f551c199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 0x7f551c199a80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b580 tx=0x7f550c0047b0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 shutdown_connections 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f551c108740 0x7f551c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55080b2880 0x7f55080b4d90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1034c0 0x7f551c199a80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 --2- 172.21.15.19:0/2174988121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c107df0 0x7f551c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 >> 172.21.15.19:0/2174988121 conn(0x7f551c0fedc0 msgr2=0x7f551c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:22.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 shutdown_connections 2024-05-09T01:24:22.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.507+0000 7f55213e2700 1 -- 172.21.15.19:0/2174988121 wait complete. 2024-05-09T01:24:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:22 smithi059 ceph-mon[34351]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:22 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2254365119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T01:24:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:22 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/390187575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T01:24:22.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.626+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb0e802e5f0 con 0x7fb0e8107df0 2024-05-09T01:24:22.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.626+0000 7fb0deffd700 1 -- 172.21.15.19:0/122174475 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb0d80a9610 con 0x7fb0e8107df0 2024-05-09T01:24:22.624 INFO:teuthology.orchestra.run.smithi019.stdout:266287972398 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0d40b2900 msgr2=0x7fb0d40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0d40b2900 0x7fb0d40b4e10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4003800 tx=0x7fb0e401a040 comp rx=0 tx=0).stop 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e8107df0 msgr2=0x7fb0e819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e8107df0 0x7fb0e819b3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8076b40 tx=0x7fb0d800dc30 comp rx=0 tx=0).stop 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 shutdown_connections 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0e8108740 0x7fb0e819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0e81034c0 0x7fb0e819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0d40b2900 0x7fb0d40b4e10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 --2- 172.21.15.19:0/122174475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e8107df0 0x7fb0e819b3b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 >> 172.21.15.19:0/122174475 conn(0x7fb0e80fedf0 msgr2=0x7fb0e8075e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 shutdown_connections 2024-05-09T01:24:22.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:22.629+0000 7fb0efad4700 1 -- 172.21.15.19:0/122174475 wait complete. 2024-05-09T01:24:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:22 smithi196 ceph-mon[35041]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:22.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:22 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2254365119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T01:24:22.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:22 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/390187575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T01:24:23.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:23 smithi019 ceph-mon[30158]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T01:24:23.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:23 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2174988121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T01:24:23.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:23 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/122174475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T01:24:23.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.573+0000 7f7412769700 1 -- 172.21.15.19:0/2301081860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c108f20 msgr2=0x7f740c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:23.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.573+0000 7f7412769700 1 --2- 172.21.15.19:0/2301081860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c108f20 0x7f740c109330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f73f40099c0 tx=0x7f73f4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.573+0000 7f7412769700 1 -- 172.21.15.19:0/2301081860 shutdown_connections 2024-05-09T01:24:23.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.573+0000 7f7412769700 1 --2- 172.21.15.19:0/2301081860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f740c109870 0x7f740c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.573+0000 7f7412769700 1 --2- 172.21.15.19:0/2301081860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c1045f0 0x7f740c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.573+0000 7f7412769700 1 --2- 172.21.15.19:0/2301081860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c108f20 0x7f740c109330 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.573+0000 7f7412769700 1 -- 172.21.15.19:0/2301081860 >> 172.21.15.19:0/2301081860 conn(0x7f740c0fffe0 msgr2=0x7f740c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:23.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.573+0000 7f7412769700 1 -- 172.21.15.19:0/2301081860 shutdown_connections 2024-05-09T01:24:23.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.574+0000 7f7412769700 1 -- 172.21.15.19:0/2301081860 wait complete. 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.574+0000 7f7412769700 1 Processor -- start 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.574+0000 7f7412769700 1 -- start start 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7412769700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f740c1045f0 0x7f740c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7412769700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c108f20 0x7f740c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7412769700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c109870 0x7f740c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7412769700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f740c114570 con 0x7f740c109870 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7412769700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1143f0 con 0x7f740c1045f0 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7412769700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1146f0 con 0x7f740c108f20 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7410d06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c109870 0x7f740c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:23.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7410d06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c109870 0x7f740c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57816/0 (socket says 172.21.15.19:57816) 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7410d06700 1 -- 172.21.15.19:0/2756904209 learned_addr learned my addr 172.21.15.19:0/2756904209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f740bfff700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f740c1045f0 0x7f740c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7403fff700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c108f20 0x7f740c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7410d06700 1 -- 172.21.15.19:0/2756904209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c108f20 msgr2=0x7f740c19b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7410d06700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c108f20 0x7f740c19b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7410d06700 1 -- 172.21.15.19:0/2756904209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f740c1045f0 msgr2=0x7f740c19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7410d06700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f740c1045f0 0x7f740c19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7410d06700 1 -- 172.21.15.19:0/2756904209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c19fdf0 con 0x7f740c109870 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.575+0000 7f7410d06700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c109870 0x7f740c19f750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f740400b970 tx=0x7f740400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.576+0000 7f7409ffb700 1 -- 172.21.15.19:0/2756904209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404019070 con 0x7f740c109870 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.576+0000 7f7409ffb700 1 -- 172.21.15.19:0/2756904209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7404014440 con 0x7f740c109870 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.576+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740c1a00e0 con 0x7f740c109870 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.576+0000 7f7409ffb700 1 -- 172.21.15.19:0/2756904209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740400ce60 con 0x7f740c109870 2024-05-09T01:24:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.576+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740c1a7a00 con 0x7f740c109870 2024-05-09T01:24:23.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.577+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f740c0512c0 con 0x7f740c109870 2024-05-09T01:24:23.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.578+0000 7f7409ffb700 1 -- 172.21.15.19:0/2756904209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f740401c070 con 0x7f740c109870 2024-05-09T01:24:23.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.579+0000 7f7409ffb700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f73f80b2880 0x7f73f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.579+0000 7f740bfff700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f73f80b2880 0x7f73f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.580+0000 7f7409ffb700 1 -- 172.21.15.19:0/2756904209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f74040e1b50 con 0x7f740c109870 2024-05-09T01:24:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.580+0000 7f740bfff700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f73f80b2880 0x7f73f80b4d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73fc006fd0 tx=0x7f73fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:23.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.583+0000 7f7409ffb700 1 -- 172.21.15.19:0/2756904209 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74040a9850 con 0x7f740c109870 2024-05-09T01:24:23.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:23 smithi059 ceph-mon[34351]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T01:24:23.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:23 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2174988121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T01:24:23.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:23 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/122174475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T01:24:23.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:23 smithi196 ceph-mon[35041]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T01:24:23.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:23 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2174988121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T01:24:23.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:23 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/122174475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T01:24:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.804+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f740c19c280 con 0x7f740c109870 2024-05-09T01:24:23.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.805+0000 7f7409ffb700 1 -- 172.21.15.19:0/2756904209 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f74040a9150 con 0x7f740c109870 2024-05-09T01:24:23.803 INFO:teuthology.orchestra.run.smithi019.stdout:34359738461 2024-05-09T01:24:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f73f80b2880 msgr2=0x7f73f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f73f80b2880 0x7f73f80b4d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73fc006fd0 tx=0x7f73fc008040 comp rx=0 tx=0).stop 2024-05-09T01:24:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c109870 msgr2=0x7f740c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c109870 0x7f740c19f750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f740400b970 tx=0x7f740400bd30 comp rx=0 tx=0).stop 2024-05-09T01:24:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 shutdown_connections 2024-05-09T01:24:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f73f80b2880 0x7f73f80b4d90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f740c109870 0x7f740c19f750 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f740c1045f0 0x7f740c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 --2- 172.21.15.19:0/2756904209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c108f20 0x7f740c19b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:23.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 >> 172.21.15.19:0/2756904209 conn(0x7f740c0fffe0 msgr2=0x7f740c107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:23.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 shutdown_connections 2024-05-09T01:24:23.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:23.807+0000 7f7412769700 1 -- 172.21.15.19:0/2756904209 wait complete. 2024-05-09T01:24:24.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff97f7fe700 1 -- 172.21.15.19:0/1107786987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978012650 con 0x7ff980108970 2024-05-09T01:24:24.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff986c7f700 1 -- 172.21.15.19:0/1107786987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980108970 msgr2=0x7ff9801102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:24.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1107786987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980108970 0x7ff9801102d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b0d0 tx=0x7ff97800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff986c7f700 1 -- 172.21.15.19:0/1107786987 shutdown_connections 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1107786987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980108970 0x7ff9801102d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1107786987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff980103c10 0x7ff980104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1107786987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff980108020 0x7ff980108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff986c7f700 1 -- 172.21.15.19:0/1107786987 >> 172.21.15.19:0/1107786987 conn(0x7ff9800fe540 msgr2=0x7ff980100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff986c7f700 1 -- 172.21.15.19:0/1107786987 shutdown_connections 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.010+0000 7ff986c7f700 1 -- 172.21.15.19:0/1107786987 wait complete. 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.011+0000 7ff986c7f700 1 Processor -- start 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.011+0000 7ff986c7f700 1 -- start start 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.011+0000 7ff986c7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980103c10 0x7ff980199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.011+0000 7ff986c7f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff980108020 0x7ff98019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:24.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.011+0000 7ff986c7f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff980108970 0x7ff98019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.011+0000 7ff986c7f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff980113640 con 0x7ff980103c10 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.011+0000 7ff986c7f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801134c0 con 0x7ff980108970 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.011+0000 7ff986c7f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801137c0 con 0x7ff980108020 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.011+0000 7ff984a1b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980103c10 0x7ff980199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff984a1b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980103c10 0x7ff980199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57822/0 (socket says 172.21.15.19:57822) 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff984a1b700 1 -- 172.21.15.19:0/1507465142 learned_addr learned my addr 172.21.15.19:0/1507465142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff98521c700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff980108970 0x7ff98019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff97ffff700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff980108020 0x7ff98019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff984a1b700 1 -- 172.21.15.19:0/1507465142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff980108020 msgr2=0x7ff98019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff984a1b700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff980108020 0x7ff98019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff984a1b700 1 -- 172.21.15.19:0/1507465142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff980108970 msgr2=0x7ff98019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff984a1b700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff980108970 0x7ff98019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff984a1b700 1 -- 172.21.15.19:0/1507465142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98019eda0 con 0x7ff980103c10 2024-05-09T01:24:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff98521c700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff980108970 0x7ff98019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:24:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff984a1b700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980103c10 0x7ff980199ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff97000d980 tx=0x7ff97000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff97dffb700 1 -- 172.21.15.19:0/1507465142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970016070 con 0x7ff980103c10 2024-05-09T01:24:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98019f090 con 0x7ff980103c10 2024-05-09T01:24:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff97dffb700 1 -- 172.21.15.19:0/1507465142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9700044d0 con 0x7ff980103c10 2024-05-09T01:24:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.012+0000 7ff97dffb700 1 -- 172.21.15.19:0/1507465142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970004d00 con 0x7ff980103c10 2024-05-09T01:24:24.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.013+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff980102870 con 0x7ff980103c10 2024-05-09T01:24:24.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.014+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff980104c30 con 0x7ff980103c10 2024-05-09T01:24:24.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.015+0000 7ff97dffb700 1 -- 172.21.15.19:0/1507465142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff970004020 con 0x7ff980103c10 2024-05-09T01:24:24.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.016+0000 7ff97dffb700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff96c0b2880 0x7ff96c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:24.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.016+0000 7ff97ffff700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff96c0b2880 0x7ff96c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:24.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.016+0000 7ff97dffb700 1 -- 172.21.15.19:0/1507465142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff9700e2100 con 0x7ff980103c10 2024-05-09T01:24:24.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.016+0000 7ff97ffff700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff96c0b2880 0x7ff96c0b4d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff968005fd0 tx=0x7ff968005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:24.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.021+0000 7ff97dffb700 1 -- 172.21.15.19:0/1507465142 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff97001f020 con 0x7ff980103c10 2024-05-09T01:24:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.239+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff9800512c0 con 0x7ff980103c10 2024-05-09T01:24:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.239+0000 7ff97dffb700 1 -- 172.21.15.19:0/1507465142 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff9800512c0 con 0x7ff980103c10 2024-05-09T01:24:24.237 INFO:teuthology.orchestra.run.smithi019.stdout:193273528383 2024-05-09T01:24:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff96c0b2880 msgr2=0x7ff96c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff96c0b2880 0x7ff96c0b4d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff968005fd0 tx=0x7ff968005db0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980103c10 msgr2=0x7ff980199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980103c10 0x7ff980199ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff97000d980 tx=0x7ff97000dd40 comp rx=0 tx=0).stop 2024-05-09T01:24:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 shutdown_connections 2024-05-09T01:24:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff980108970 0x7ff98019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff96c0b2880 0x7ff96c0b4d90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff980103c10 0x7ff980199ed0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 --2- 172.21.15.19:0/1507465142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff980108020 0x7ff98019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.241+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 >> 172.21.15.19:0/1507465142 conn(0x7ff9800fe540 msgr2=0x7ff980107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.242+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 shutdown_connections 2024-05-09T01:24:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:24.242+0000 7ff986c7f700 1 -- 172.21.15.19:0/1507465142 wait complete. 2024-05-09T01:24:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:24 smithi059 ceph-mon[34351]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:24.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:24 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2756904209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T01:24:24.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:24 smithi019 ceph-mon[30158]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:24.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:24 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2756904209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T01:24:24.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:24 smithi196 ceph-mon[35041]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:24.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:24 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2756904209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T01:24:25.211 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738461 for osd.0 2024-05-09T01:24:25.212 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.379+0000 7fba7de69700 1 -- 172.21.15.19:0/660006216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78109860 msgr2=0x7fba781111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.379+0000 7fba7de69700 1 --2- 172.21.15.19:0/660006216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78109860 0x7fba781111c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fba7000b0d0 tx=0x7fba7000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.380+0000 7fba7de69700 1 -- 172.21.15.19:0/660006216 shutdown_connections 2024-05-09T01:24:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.380+0000 7fba7de69700 1 --2- 172.21.15.19:0/660006216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78109860 0x7fba781111c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.380+0000 7fba7de69700 1 --2- 172.21.15.19:0/660006216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba781045e0 0x7fba781049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.380+0000 7fba7de69700 1 --2- 172.21.15.19:0/660006216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba78108f10 0x7fba78109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.380+0000 7fba7de69700 1 -- 172.21.15.19:0/660006216 >> 172.21.15.19:0/660006216 conn(0x7fba780ffe70 msgr2=0x7fba781022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.380+0000 7fba7de69700 1 -- 172.21.15.19:0/660006216 shutdown_connections 2024-05-09T01:24:25.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.380+0000 7fba7de69700 1 -- 172.21.15.19:0/660006216 wait complete. 2024-05-09T01:24:25.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.380+0000 7fba7de69700 1 Processor -- start 2024-05-09T01:24:25.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba7de69700 1 -- start start 2024-05-09T01:24:25.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba7de69700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781045e0 0x7fba78075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:25.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba7de69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108f10 0x7fba78076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:25.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba7de69700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba78109860 0x7fba78076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba7de69700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba78114490 con 0x7fba781045e0 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba7de69700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fba78114310 con 0x7fba78108f10 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba7de69700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba78114610 con 0x7fba78109860 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba777fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781045e0 0x7fba78075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba777fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781045e0 0x7fba78075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57842/0 (socket says 172.21.15.19:57842) 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba777fe700 1 -- 172.21.15.19:0/2533485568 learned_addr learned my addr 172.21.15.19:0/2533485568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba77fff700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba78109860 0x7fba78076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba76ffd700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108f10 0x7fba78076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba777fe700 1 -- 172.21.15.19:0/2533485568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba78109860 msgr2=0x7fba78076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba777fe700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba78109860 0x7fba78076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba777fe700 1 -- 172.21.15.19:0/2533485568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108f10 msgr2=0x7fba78076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba777fe700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108f10 0x7fba78076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba777fe700 1 -- 172.21.15.19:0/2533485568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba781a9180 con 0x7fba781045e0 2024-05-09T01:24:25.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba76ffd700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108f10 0x7fba78076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:24:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.381+0000 7fba77fff700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba78109860 0x7fba78076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:24:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.382+0000 7fba777fe700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781045e0 0x7fba78075ac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fba6800d980 tx=0x7fba6800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.382+0000 7fba74ff9700 1 -- 172.21.15.19:0/2533485568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68016070 con 0x7fba781045e0 2024-05-09T01:24:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.382+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba781a9470 con 0x7fba781045e0 2024-05-09T01:24:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.382+0000 7fba74ff9700 1 -- 172.21.15.19:0/2533485568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba680044d0 con 0x7fba781045e0 2024-05-09T01:24:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.382+0000 7fba74ff9700 1 -- 172.21.15.19:0/2533485568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68004d00 con 0x7fba781045e0 2024-05-09T01:24:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.382+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba781a99b0 con 0x7fba781045e0 2024-05-09T01:24:25.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.383+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba780512c0 con 0x7fba781045e0 2024-05-09T01:24:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.385+0000 7fba74ff9700 1 -- 172.21.15.19:0/2533485568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fba68004020 con 0x7fba781045e0 2024-05-09T01:24:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.386+0000 7fba74ff9700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba600b2880 0x7fba600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:25.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.386+0000 7fba76ffd700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba600b2880 0x7fba600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.386+0000 7fba74ff9700 1 -- 172.21.15.19:0/2533485568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fba680e2670 con 0x7fba781045e0 2024-05-09T01:24:25.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.386+0000 7fba76ffd700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba600b2880 0x7fba600b4d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fba781089c0 tx=0x7fba6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:25.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.390+0000 7fba74ff9700 1 -- 172.21.15.19:0/2533485568 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba6801f020 con 0x7fba781045e0 2024-05-09T01:24:25.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:25 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1507465142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T01:24:25.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:25 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1507465142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T01:24:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.614+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fba7802e5f0 con 0x7fba781045e0 2024-05-09T01:24:25.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.614+0000 7fba74ff9700 1 -- 172.21.15.19:0/2533485568 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fba680aa370 con 0x7fba781045e0 2024-05-09T01:24:25.612 INFO:teuthology.orchestra.run.smithi019.stdout:115964117070 2024-05-09T01:24:25.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba600b2880 msgr2=0x7fba600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba600b2880 0x7fba600b4d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fba781089c0 tx=0x7fba6c009260 comp rx=0 tx=0).stop 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781045e0 msgr2=0x7fba78075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781045e0 0x7fba78075ac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fba6800d980 tx=0x7fba6800dd40 comp rx=0 tx=0).stop 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 shutdown_connections 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba78109860 0x7fba78076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba600b2880 0x7fba600b4d90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781045e0 0x7fba78075ac0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 --2- 172.21.15.19:0/2533485568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108f10 0x7fba78076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.616+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 >> 172.21.15.19:0/2533485568 conn(0x7fba780ffe70 msgr2=0x7fba78106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.617+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 shutdown_connections 2024-05-09T01:24:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:25.617+0000 7fba7de69700 1 -- 172.21.15.19:0/2533485568 wait complete. 2024-05-09T01:24:25.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:25 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1507465142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T01:24:26.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.120+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3603239723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 msgr2=0x7f5b481100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.120+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3603239723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 0x7f5b481100a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000b0d0 tx=0x7f5b4000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.120+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3603239723 shutdown_connections 2024-05-09T01:24:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.120+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3603239723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 0x7f5b481100a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.120+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3603239723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b481034c0 0x7f5b481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.120+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3603239723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b48107df0 0x7f5b48108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.120+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3603239723 >> 172.21.15.19:0/3603239723 conn(0x7f5b480fedc0 msgr2=0x7f5b481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.120+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3603239723 shutdown_connections 2024-05-09T01:24:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.120+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3603239723 wait complete. 2024-05-09T01:24:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.121+0000 7f5b4d34f700 1 Processor -- start 2024-05-09T01:24:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.121+0000 7f5b4d34f700 1 -- start start 2024-05-09T01:24:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b4d34f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b481034c0 0x7f5b4819bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b4d34f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b48107df0 0x7f5b4819c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b4d34f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 0x7f5b4819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b4d34f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48113370 con 0x7f5b48108740 2024-05-09T01:24:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b4d34f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481131f0 con 0x7f5b48107df0 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b4d34f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481134f0 con 0x7f5b481034c0 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b477fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 0x7f5b4819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b477fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 0x7f5b4819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57860/0 (socket says 172.21.15.19:57860) 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b477fe700 1 -- 172.21.15.19:0/3289297997 learned_addr learned my addr 172.21.15.19:0/3289297997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b46ffd700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b481034c0 0x7f5b4819bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b3e7fc700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b48107df0 0x7f5b4819c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b477fe700 1 -- 172.21.15.19:0/3289297997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b481034c0 msgr2=0x7f5b4819bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b477fe700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b481034c0 0x7f5b4819bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b477fe700 1 -- 172.21.15.19:0/3289297997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b48107df0 msgr2=0x7f5b4819c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b477fe700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b48107df0 0x7f5b4819c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b477fe700 1 -- 172.21.15.19:0/3289297997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b481a6e50 con 0x7f5b48108740 2024-05-09T01:24:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b477fe700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 0x7f5b4819c630 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009fd0 tx=0x7f5b40007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.122+0000 7f5b44ff9700 1 -- 172.21.15.19:0/3289297997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4001f070 con 0x7f5b48108740 2024-05-09T01:24:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.123+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b481a70e0 con 0x7f5b48108740 2024-05-09T01:24:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.123+0000 7f5b44ff9700 1 -- 172.21.15.19:0/3289297997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b40007e50 con 0x7f5b48108740 2024-05-09T01:24:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.123+0000 7f5b44ff9700 1 -- 172.21.15.19:0/3289297997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4001b6a0 con 0x7f5b48108740 2024-05-09T01:24:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.123+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b481a7530 con 0x7f5b48108740 2024-05-09T01:24:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.124+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b480512c0 con 0x7f5b48108740 2024-05-09T01:24:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.126+0000 7f5b44ff9700 1 -- 172.21.15.19:0/3289297997 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5b40022070 con 0x7f5b48108740 2024-05-09T01:24:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.127+0000 7f5b44ff9700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5b340b2880 0x7f5b340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.128+0000 7f5b44ff9700 1 -- 172.21.15.19:0/3289297997 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5b4000f020 con 0x7f5b48108740 2024-05-09T01:24:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.128+0000 7f5b46ffd700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5b340b2880 0x7f5b340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.128+0000 7f5b46ffd700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5b340b2880 0x7f5b340b4d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800ac50 tx=0x7f5b3800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:26.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.133+0000 7f5b44ff9700 1 -- 172.21.15.19:0/3289297997 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b40029020 con 0x7f5b48108740 2024-05-09T01:24:26.335 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:26 smithi019 ceph-mon[30158]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T01:24:26.336 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:26 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2533485568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T01:24:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5b4802e7b0 con 0x7f5b48108740 2024-05-09T01:24:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f77201ed700 1 -- 172.21.15.19:0/3118923480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 msgr2=0x7f7718108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f77201ed700 1 --2- 172.21.15.19:0/3118923480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 0x7f7718108220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f77080099c0 tx=0x7f7708009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f77201ed700 1 -- 172.21.15.19:0/3118923480 shutdown_connections 2024-05-09T01:24:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f77201ed700 1 --2- 172.21.15.19:0/3118923480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7718108760 0x7f77181100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f77201ed700 1 --2- 172.21.15.19:0/3118923480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181034e0 0x7f77181038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f77201ed700 1 --2- 172.21.15.19:0/3118923480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 0x7f7718108220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f77201ed700 1 -- 172.21.15.19:0/3118923480 >> 172.21.15.19:0/3118923480 conn(0x7f77180fef30 msgr2=0x7f7718101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:26.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f77201ed700 1 -- 172.21.15.19:0/3118923480 shutdown_connections 2024-05-09T01:24:26.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f77201ed700 1 -- 172.21.15.19:0/3118923480 wait complete. 2024-05-09T01:24:26.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.356+0000 7f5b44ff9700 1 -- 172.21.15.19:0/3289297997 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5b400ab220 con 0x7f5b48108740 2024-05-09T01:24:26.355 INFO:teuthology.orchestra.run.smithi019.stdout:141733920841 2024-05-09T01:24:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f77201ed700 1 Processor -- start 2024-05-09T01:24:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f77201ed700 1 -- start start 2024-05-09T01:24:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f77201ed700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181034e0 0x7f77180ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f77201ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 0x7f77180ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f77201ed700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7718108760 0x7f77180ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f77201ed700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7718113370 con 0x7f7718107e10 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f77201ed700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77181131f0 con 0x7f77181034e0 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f77201ed700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77181134f0 con 0x7f7718108760 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f771d788700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 0x7f77180ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f771d788700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 0x7f77180ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57874/0 (socket says 172.21.15.19:57874) 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f771d788700 1 -- 172.21.15.19:0/3688983959 learned_addr learned my addr 172.21.15.19:0/3688983959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.357+0000 7f771df89700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181034e0 0x7f77180ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f771e78a700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7718108760 0x7f77180ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f771d788700 1 -- 172.21.15.19:0/3688983959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7718108760 msgr2=0x7f77180ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f771d788700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7718108760 0x7f77180ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f771d788700 1 -- 172.21.15.19:0/3688983959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181034e0 msgr2=0x7f77180ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f771d788700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181034e0 0x7f77180ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f771d788700 1 -- 172.21.15.19:0/3688983959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77181a91f0 con 0x7f7718107e10 2024-05-09T01:24:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f771d788700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 0x7f77180ff950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77080098e0 tx=0x7f77080057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f770effd700 1 -- 172.21.15.19:0/3688983959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708027070 con 0x7f7718107e10 2024-05-09T01:24:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77181a9480 con 0x7f7718107e10 2024-05-09T01:24:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f770effd700 1 -- 172.21.15.19:0/3688983959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7708003f20 con 0x7f7718107e10 2024-05-09T01:24:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f770effd700 1 -- 172.21.15.19:0/3688983959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708022d70 con 0x7f7718107e10 2024-05-09T01:24:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.358+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77181a98e0 con 0x7f7718107e10 2024-05-09T01:24:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5b340b2880 msgr2=0x7f5b340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5b340b2880 0x7f5b340b4d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800ac50 tx=0x7f5b3800a380 comp rx=0 tx=0).stop 2024-05-09T01:24:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 msgr2=0x7f5b4819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 0x7f5b4819c630 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009fd0 tx=0x7f5b40007a20 comp rx=0 tx=0).stop 2024-05-09T01:24:26.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 shutdown_connections 2024-05-09T01:24:26.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5b340b2880 0x7f5b340b4d90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b48108740 0x7f5b4819c630 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77180512c0 con 0x7f7718107e10 2024-05-09T01:24:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b481034c0 0x7f5b4819bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.359+0000 7f5b4d34f700 1 --2- 172.21.15.19:0/3289297997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b48107df0 0x7f5b4819c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.360+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 >> 172.21.15.19:0/3289297997 conn(0x7f5b480fedc0 msgr2=0x7f5b48107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.360+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 shutdown_connections 2024-05-09T01:24:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.360+0000 7f5b4d34f700 1 -- 172.21.15.19:0/3289297997 wait complete. 2024-05-09T01:24:26.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.361+0000 7f770effd700 1 -- 172.21.15.19:0/3688983959 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f770802a070 con 0x7f7718107e10 2024-05-09T01:24:26.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.362+0000 7f770effd700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f77040b2880 0x7f77040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:26.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.362+0000 7f771df89700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f77040b2880 0x7f77040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:26.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.362+0000 7f770effd700 1 -- 172.21.15.19:0/3688983959 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f77080e2fe0 con 0x7f7718107e10 2024-05-09T01:24:26.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.363+0000 7f771df89700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f77040b2880 0x7f77040b4d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7714005950 tx=0x7f771400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.367+0000 7f770effd700 1 -- 172.21.15.19:0/3688983959 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7708013070 con 0x7f7718107e10 2024-05-09T01:24:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.596+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f771802e5f0 con 0x7f7718107e10 2024-05-09T01:24:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.597+0000 7f770effd700 1 -- 172.21.15.19:0/3688983959 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f77080aad70 con 0x7f7718107e10 2024-05-09T01:24:26.595 INFO:teuthology.orchestra.run.smithi019.stdout:313532612645 2024-05-09T01:24:26.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:26 smithi059 ceph-mon[34351]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T01:24:26.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:26 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2533485568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T01:24:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f77040b2880 msgr2=0x7f77040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f77040b2880 0x7f77040b4d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7714005950 tx=0x7f771400b410 comp rx=0 tx=0).stop 2024-05-09T01:24:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 msgr2=0x7f77180ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 0x7f77180ff950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77080098e0 tx=0x7f77080057c0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 shutdown_connections 2024-05-09T01:24:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7718108760 0x7f77180ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181034e0 0x7f77180ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f77040b2880 0x7f77040b4d90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 --2- 172.21.15.19:0/3688983959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7718107e10 0x7f77180ff950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 >> 172.21.15.19:0/3688983959 conn(0x7f77180fef30 msgr2=0x7f771806c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 shutdown_connections 2024-05-09T01:24:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:26.599+0000 7f77201ed700 1 -- 172.21.15.19:0/3688983959 wait complete. 2024-05-09T01:24:26.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:26 smithi196 ceph-mon[35041]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T01:24:26.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:26 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2533485568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T01:24:27.250 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972392 got 266287972398 for osd.9 2024-05-09T01:24:27.250 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:27.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:27 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3289297997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T01:24:27.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:27 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3688983959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T01:24:27.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:27 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3289297997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T01:24:27.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:27 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3688983959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T01:24:27.598 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808867 got 287762808873 for osd.10 2024-05-09T01:24:27.599 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:27.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:27 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3289297997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T01:24:27.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:27 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3688983959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T01:24:28.200 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920835 got 141733920841 for osd.4 2024-05-09T01:24:28.200 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:28.596 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:28 smithi019 ceph-mon[30158]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:24:28.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:28 smithi059 ceph-mon[34351]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:24:28.680 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528377 got 193273528383 for osd.6 2024-05-09T01:24:28.680 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:28.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:28 smithi196 ceph-mon[35041]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:24:29.712 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411410 for osd.2 2024-05-09T01:24:29.713 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:30.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.374+0000 7f495d7c3700 1 -- 172.21.15.19:0/42678861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49581045d0 msgr2=0x7f49581049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.374+0000 7f495d7c3700 1 --2- 172.21.15.19:0/42678861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49581045d0 0x7f49581049e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f49480099c0 tx=0x7f4948009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.374+0000 7f495d7c3700 1 -- 172.21.15.19:0/42678861 shutdown_connections 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.374+0000 7f495d7c3700 1 --2- 172.21.15.19:0/42678861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4958109850 0x7f49581111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.374+0000 7f495d7c3700 1 --2- 172.21.15.19:0/42678861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49581045d0 0x7f49581049e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.374+0000 7f495d7c3700 1 --2- 172.21.15.19:0/42678861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4958108f00 0x7f4958109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.374+0000 7f495d7c3700 1 -- 172.21.15.19:0/42678861 >> 172.21.15.19:0/42678861 conn(0x7f49580ffea0 msgr2=0x7f49581022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.374+0000 7f495d7c3700 1 -- 172.21.15.19:0/42678861 shutdown_connections 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.374+0000 7f495d7c3700 1 -- 172.21.15.19:0/42678861 wait complete. 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f495d7c3700 1 Processor -- start 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f495d7c3700 1 -- start start 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f495d7c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49581045d0 0x7f495819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f495d7c3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4958108f00 0x7f495819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f495d7c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4958109850 0x7f495819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:30.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f495d7c3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4958114480 con 0x7f4958109850 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f495d7c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4958114300 con 0x7f49581045d0 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f495d7c3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4958114600 con 0x7f4958108f00 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f49577fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4958109850 0x7f495819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f4956ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49581045d0 0x7f495819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.375+0000 7f49577fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4958109850 0x7f495819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42180/0 (socket says 172.21.15.19:42180) 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f49577fe700 1 -- 172.21.15.19:0/2015085985 learned_addr learned my addr 172.21.15.19:0/2015085985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f494e7fc700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4958108f00 0x7f495819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f49577fe700 1 -- 172.21.15.19:0/2015085985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4958108f00 msgr2=0x7f495819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f49577fe700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4958108f00 0x7f495819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f49577fe700 1 -- 172.21.15.19:0/2015085985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49581045d0 msgr2=0x7f495819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f49577fe700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49581045d0 0x7f495819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f49577fe700 1 -- 172.21.15.19:0/2015085985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495819fce0 con 0x7f4958109850 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f4956ffd700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49581045d0 0x7f495819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:24:30.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f49577fe700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4958109850 0x7f495819f640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f495000ada0 tx=0x7f495000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:30.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f4954ff9700 1 -- 172.21.15.19:0/2015085985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950017020 con 0x7f4958109850 2024-05-09T01:24:30.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495819ffd0 con 0x7f4958109850 2024-05-09T01:24:30.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f4954ff9700 1 -- 172.21.15.19:0/2015085985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f495000cd80 con 0x7f4958109850 2024-05-09T01:24:30.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f4954ff9700 1 -- 172.21.15.19:0/2015085985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950005260 con 0x7f4958109850 2024-05-09T01:24:30.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.376+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49581a78f0 con 0x7f4958109850 2024-05-09T01:24:30.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.377+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49580512c0 con 0x7f4958109850 2024-05-09T01:24:30.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.379+0000 7f4954ff9700 1 -- 172.21.15.19:0/2015085985 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f495000a090 con 0x7f4958109850 2024-05-09T01:24:30.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.380+0000 7f4954ff9700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f49440b2880 0x7f49440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:30.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.380+0000 7f4956ffd700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f49440b2880 0x7f49440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:30.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.380+0000 7f4954ff9700 1 -- 172.21.15.19:0/2015085985 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f49500e1f10 con 0x7f4958109850 2024-05-09T01:24:30.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.380+0000 7f4956ffd700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f49440b2880 0x7f49440b4d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4948006010 tx=0x7f494801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:30.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.383+0000 7f4954ff9700 1 -- 172.21.15.19:0/2015085985 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4950021020 con 0x7f4958109850 2024-05-09T01:24:30.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:30 smithi019 ceph-mon[30158]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:24:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.596+0000 7fbe6bca5700 1 -- 172.21.15.19:0/101931665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64108740 msgr2=0x7fbe641100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.596+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/101931665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64108740 0x7fbe641100a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00b0d0 tx=0x7fbe5c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.596+0000 7fbe6bca5700 1 -- 172.21.15.19:0/101931665 shutdown_connections 2024-05-09T01:24:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.596+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/101931665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64108740 0x7fbe641100a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.596+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/101931665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe641034c0 0x7fbe641038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.596+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/101931665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe64107df0 0x7fbe64108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.596+0000 7fbe6bca5700 1 -- 172.21.15.19:0/101931665 >> 172.21.15.19:0/101931665 conn(0x7fbe640fedc0 msgr2=0x7fbe641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.596+0000 7fbe6bca5700 1 -- 172.21.15.19:0/101931665 shutdown_connections 2024-05-09T01:24:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.596+0000 7fbe6bca5700 1 -- 172.21.15.19:0/101931665 wait complete. 2024-05-09T01:24:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe6bca5700 1 Processor -- start 2024-05-09T01:24:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe6bca5700 1 -- start start 2024-05-09T01:24:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe6bca5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe641034c0 0x7fbe6419bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe6bca5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64107df0 0x7fbe6419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe6bca5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe64108740 0x7fbe6419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe6bca5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64113370 con 0x7fbe64107df0 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe6bca5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe641131f0 con 0x7fbe641034c0 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe6bca5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbe641134f0 con 0x7fbe64108740 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe69240700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64107df0 0x7fbe6419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe69240700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64107df0 0x7fbe6419c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42198/0 (socket says 172.21.15.19:42198) 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe69240700 1 -- 172.21.15.19:0/2186456118 learned_addr learned my addr 172.21.15.19:0/2186456118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.597+0000 7fbe69a41700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe641034c0 0x7fbe6419bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe6a242700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe64108740 0x7fbe6419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe69240700 1 -- 172.21.15.19:0/2186456118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe64108740 msgr2=0x7fbe6419c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe69240700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe64108740 0x7fbe6419c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe69240700 1 -- 172.21.15.19:0/2186456118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe641034c0 msgr2=0x7fbe6419bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe69240700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe641034c0 0x7fbe6419bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe69240700 1 -- 172.21.15.19:0/2186456118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe641a6ed0 con 0x7fbe64107df0 2024-05-09T01:24:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe6a242700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe64108740 0x7fbe6419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:24:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe69240700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64107df0 0x7fbe6419c130 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c9e0 tx=0x7fbe5800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe56ffd700 1 -- 172.21.15.19:0/2186456118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58007aa0 con 0x7fbe64107df0 2024-05-09T01:24:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe56ffd700 1 -- 172.21.15.19:0/2186456118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe58007c40 con 0x7fbe64107df0 2024-05-09T01:24:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe641a71c0 con 0x7fbe64107df0 2024-05-09T01:24:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe56ffd700 1 -- 172.21.15.19:0/2186456118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58018660 con 0x7fbe64107df0 2024-05-09T01:24:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.598+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe641a7700 con 0x7fbe64107df0 2024-05-09T01:24:30.599 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:30 smithi059 ceph-mon[34351]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:24:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.599+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe640512c0 con 0x7fbe64107df0 2024-05-09T01:24:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.601+0000 7fbe56ffd700 1 -- 172.21.15.19:0/2186456118 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbe58018800 con 0x7fbe64107df0 2024-05-09T01:24:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.602+0000 7fbe56ffd700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbe500b2880 0x7fbe500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.602+0000 7fbe69a41700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbe500b2880 0x7fbe500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.602+0000 7fbe56ffd700 1 -- 172.21.15.19:0/2186456118 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbe580e21b0 con 0x7fbe64107df0 2024-05-09T01:24:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.602+0000 7fbe69a41700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbe500b2880 0x7fbe500b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe60005950 tx=0x7fbe6000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.603+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f495802e5f0 con 0x7f4958109850 2024-05-09T01:24:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.607+0000 7fbe56ffd700 1 -- 172.21.15.19:0/2186456118 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe58014030 con 0x7fbe64107df0 2024-05-09T01:24:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.607+0000 7f4954ff9700 1 -- 172.21.15.19:0/2015085985 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49500a9c40 con 0x7f4958109850 2024-05-09T01:24:30.605 INFO:teuthology.orchestra.run.smithi019.stdout:240518168630 2024-05-09T01:24:30.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f49440b2880 msgr2=0x7f49440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f49440b2880 0x7f49440b4d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4948006010 tx=0x7f494801a040 comp rx=0 tx=0).stop 2024-05-09T01:24:30.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4958109850 msgr2=0x7f495819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4958109850 0x7f495819f640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f495000ada0 tx=0x7f495000c330 comp rx=0 tx=0).stop 2024-05-09T01:24:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 shutdown_connections 2024-05-09T01:24:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f49440b2880 0x7f49440b4d90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4958109850 0x7f495819f640 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49581045d0 0x7f495819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 --2- 172.21.15.19:0/2015085985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4958108f00 0x7f495819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.609+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 >> 172.21.15.19:0/2015085985 conn(0x7f49580ffea0 msgr2=0x7f49581022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.610+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 shutdown_connections 2024-05-09T01:24:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.610+0000 7f495d7c3700 1 -- 172.21.15.19:0/2015085985 wait complete. 2024-05-09T01:24:30.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:30 smithi196 ceph-mon[35041]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:24:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.836+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbe6402e5f0 con 0x7fbe64107df0 2024-05-09T01:24:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.836+0000 7fbe56ffd700 1 -- 172.21.15.19:0/2186456118 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbe580a9e50 con 0x7fbe64107df0 2024-05-09T01:24:30.834 INFO:teuthology.orchestra.run.smithi019.stdout:214748364858 2024-05-09T01:24:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbe500b2880 msgr2=0x7fbe500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbe500b2880 0x7fbe500b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe60005950 tx=0x7fbe6000b410 comp rx=0 tx=0).stop 2024-05-09T01:24:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64107df0 msgr2=0x7fbe6419c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64107df0 0x7fbe6419c130 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c9e0 tx=0x7fbe5800ccf0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 shutdown_connections 2024-05-09T01:24:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe64108740 0x7fbe6419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe641034c0 0x7fbe6419bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbe500b2880 0x7fbe500b4d90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 --2- 172.21.15.19:0/2186456118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe64107df0 0x7fbe6419c130 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:30.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 >> 172.21.15.19:0/2186456118 conn(0x7fbe640fedc0 msgr2=0x7fbe64107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:30.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 shutdown_connections 2024-05-09T01:24:30.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:30.839+0000 7fbe6bca5700 1 -- 172.21.15.19:0/2186456118 wait complete. 2024-05-09T01:24:31.013 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117064 got 115964117070 for osd.3 2024-05-09T01:24:31.013 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:31.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:31 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2015085985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T01:24:31.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:31 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2186456118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T01:24:31.473 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574934 for osd.1 2024-05-09T01:24:31.474 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:31.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:31 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2015085985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T01:24:31.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:31 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2186456118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T01:24:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:31.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:31 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2015085985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T01:24:31.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:31 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2186456118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T01:24:32.029 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612642 got 313532612645 for osd.11 2024-05-09T01:24:32.029 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:32.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:32 smithi196 ceph-mon[35041]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T01:24:32.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:32 smithi059 ceph-mon[34351]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T01:24:32.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:32 smithi019 ceph-mon[30158]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T01:24:32.669 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364852 got 214748364858 for osd.7 2024-05-09T01:24:32.669 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:33.743 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168624 got 240518168630 for osd.8 2024-05-09T01:24:33.744 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:34.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:34 smithi019 ceph-mon[30158]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T01:24:34.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:34 smithi059 ceph-mon[34351]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T01:24:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.687+0000 7f8e52411700 1 -- 172.21.15.19:0/2714621706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c108740 msgr2=0x7f8e4c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.687+0000 7f8e52411700 1 --2- 172.21.15.19:0/2714621706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c108740 0x7f8e4c1100a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400b0d0 tx=0x7f8e4400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.687+0000 7f8e52411700 1 -- 172.21.15.19:0/2714621706 shutdown_connections 2024-05-09T01:24:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.687+0000 7f8e52411700 1 --2- 172.21.15.19:0/2714621706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c108740 0x7f8e4c1100a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.687+0000 7f8e52411700 1 --2- 172.21.15.19:0/2714621706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c1034c0 0x7f8e4c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.687+0000 7f8e52411700 1 --2- 172.21.15.19:0/2714621706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c107df0 0x7f8e4c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.687+0000 7f8e52411700 1 -- 172.21.15.19:0/2714621706 >> 172.21.15.19:0/2714621706 conn(0x7f8e4c0fedc0 msgr2=0x7f8e4c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.687+0000 7f8e52411700 1 -- 172.21.15.19:0/2714621706 shutdown_connections 2024-05-09T01:24:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.687+0000 7f8e52411700 1 -- 172.21.15.19:0/2714621706 wait complete. 2024-05-09T01:24:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e52411700 1 Processor -- start 2024-05-09T01:24:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e52411700 1 -- start start 2024-05-09T01:24:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e52411700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c1034c0 0x7f8e4c19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e52411700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c107df0 0x7f8e4c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e52411700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c108740 0x7f8e4c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e52411700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c113370 con 0x7f8e4c1034c0 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e52411700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1131f0 con 0x7f8e4c107df0 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e52411700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1134f0 con 0x7f8e4c108740 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c1034c0 0x7f8e4c19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c1034c0 0x7f8e4c19cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42216/0 (socket says 172.21.15.19:42216) 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e4bfff700 1 -- 172.21.15.19:0/1144490244 learned_addr learned my addr 172.21.15.19:0/1144490244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e43fff700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c107df0 0x7f8e4c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e509ae700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c108740 0x7f8e4c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e4bfff700 1 -- 172.21.15.19:0/1144490244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c108740 msgr2=0x7f8e4c1a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e4bfff700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c108740 0x7f8e4c1a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e4bfff700 1 -- 172.21.15.19:0/1144490244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c107df0 msgr2=0x7f8e4c19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e4bfff700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c107df0 0x7f8e4c19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.688+0000 7f8e4bfff700 1 -- 172.21.15.19:0/1144490244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c1a1e70 con 0x7f8e4c1034c0 2024-05-09T01:24:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.689+0000 7f8e4bfff700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c1034c0 0x7f8e4c19cfa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00d980 tx=0x7f8e3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:34.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.689+0000 7f8e49ffb700 1 -- 172.21.15.19:0/1144490244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c016070 con 0x7f8e4c1034c0 2024-05-09T01:24:34.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.689+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c1a2160 con 0x7f8e4c1034c0 2024-05-09T01:24:34.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.689+0000 7f8e49ffb700 1 -- 172.21.15.19:0/1144490244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e3c004830 con 0x7f8e4c1034c0 2024-05-09T01:24:34.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.689+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1a9a80 con 0x7f8e4c1034c0 2024-05-09T01:24:34.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.689+0000 7f8e49ffb700 1 -- 172.21.15.19:0/1144490244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c005540 con 0x7f8e4c1034c0 2024-05-09T01:24:34.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.691+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e2c005330 con 0x7f8e4c1034c0 2024-05-09T01:24:34.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.692+0000 7f8e49ffb700 1 -- 172.21.15.19:0/1144490244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8e3c004220 con 0x7f8e4c1034c0 2024-05-09T01:24:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.692+0000 7f8e49ffb700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e380b2900 0x7f8e380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.693+0000 7f8e43fff700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e380b2900 0x7f8e380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.693+0000 7f8e49ffb700 1 -- 172.21.15.19:0/1144490244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8e3c0e1e50 con 0x7f8e4c1034c0 2024-05-09T01:24:34.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.693+0000 7f8e43fff700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e380b2900 0x7f8e380b4e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c06c310 tx=0x7f8e3400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:34.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:34 smithi196 ceph-mon[35041]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T01:24:34.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.698+0000 7f8e49ffb700 1 -- 172.21.15.19:0/1144490244 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e3c01f020 con 0x7f8e4c1034c0 2024-05-09T01:24:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.915+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8e2c005620 con 0x7f8e4c1034c0 2024-05-09T01:24:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.915+0000 7f8e49ffb700 1 -- 172.21.15.19:0/1144490244 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e3c0a9b50 con 0x7f8e4c1034c0 2024-05-09T01:24:34.913 INFO:teuthology.orchestra.run.smithi019.stdout:167503724613 2024-05-09T01:24:34.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.917+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e380b2900 msgr2=0x7f8e380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.917+0000 7f8e52411700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e380b2900 0x7f8e380b4e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c06c310 tx=0x7f8e3400b410 comp rx=0 tx=0).stop 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.917+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c1034c0 msgr2=0x7f8e4c19cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.917+0000 7f8e52411700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c1034c0 0x7f8e4c19cfa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00d980 tx=0x7f8e3c00dc90 comp rx=0 tx=0).stop 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.918+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 shutdown_connections 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.918+0000 7f8e52411700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c108740 0x7f8e4c1a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.918+0000 7f8e52411700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e380b2900 0x7f8e380b4e10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.918+0000 7f8e52411700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c1034c0 0x7f8e4c19cfa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.918+0000 7f8e52411700 1 --2- 172.21.15.19:0/1144490244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c107df0 0x7f8e4c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.918+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 >> 172.21.15.19:0/1144490244 conn(0x7f8e4c0fedc0 msgr2=0x7f8e4c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.918+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 shutdown_connections 2024-05-09T01:24:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:34.918+0000 7f8e52411700 1 -- 172.21.15.19:0/1144490244 wait complete. 2024-05-09T01:24:35.539 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:35 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1144490244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T01:24:35.540 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724608 got 167503724613 for osd.5 2024-05-09T01:24:35.540 DEBUG:teuthology.parallel:result is None 2024-05-09T01:24:35.540 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T01:24:35.540 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T01:24:35.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:35 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1144490244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T01:24:35.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:35 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1144490244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T01:24:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:36.575 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:36 smithi019 ceph-mon[30158]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:24:36.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:36 smithi059 ceph-mon[34351]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:24:36.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:36 smithi196 ceph-mon[35041]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:24:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.425+0000 7f17761b7700 1 -- 172.21.15.19:0/2413699878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770108740 msgr2=0x7f17701100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.425+0000 7f17761b7700 1 --2- 172.21.15.19:0/2413699878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770108740 0x7f17701100a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f176800b0d0 tx=0x7f176800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.425+0000 7f17761b7700 1 -- 172.21.15.19:0/2413699878 shutdown_connections 2024-05-09T01:24:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.425+0000 7f17761b7700 1 --2- 172.21.15.19:0/2413699878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770108740 0x7f17701100a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.425+0000 7f17761b7700 1 --2- 172.21.15.19:0/2413699878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17701034c0 0x7f17701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.425+0000 7f17761b7700 1 --2- 172.21.15.19:0/2413699878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1770107df0 0x7f1770108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.425+0000 7f17761b7700 1 -- 172.21.15.19:0/2413699878 >> 172.21.15.19:0/2413699878 conn(0x7f17700fedc0 msgr2=0x7f17701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.425+0000 7f17761b7700 1 -- 172.21.15.19:0/2413699878 shutdown_connections 2024-05-09T01:24:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.425+0000 7f17761b7700 1 -- 172.21.15.19:0/2413699878 wait complete. 2024-05-09T01:24:37.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f17761b7700 1 Processor -- start 2024-05-09T01:24:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f17761b7700 1 -- start start 2024-05-09T01:24:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f17761b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17701034c0 0x7f177019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f17761b7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770107df0 0x7f177019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f17761b7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1770108740 0x7f177019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f17761b7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1770113370 con 0x7f1770107df0 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f17761b7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17701131f0 con 0x7f17701034c0 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f17761b7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17701134f0 con 0x7f1770108740 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f176effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770107df0 0x7f177019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f176effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770107df0 0x7f177019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42238/0 (socket says 172.21.15.19:42238) 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f176effd700 1 -- 172.21.15.19:0/3193505218 learned_addr learned my addr 172.21.15.19:0/3193505218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.426+0000 7f176f7fe700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17701034c0 0x7f177019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176ffff700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1770108740 0x7f177019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176effd700 1 -- 172.21.15.19:0/3193505218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1770108740 msgr2=0x7f177019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176effd700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1770108740 0x7f177019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176effd700 1 -- 172.21.15.19:0/3193505218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17701034c0 msgr2=0x7f177019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176effd700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17701034c0 0x7f177019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176effd700 1 -- 172.21.15.19:0/3193505218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17701a6fe0 con 0x7f1770107df0 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176ffff700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1770108740 0x7f177019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:24:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176effd700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770107df0 0x7f177019c1d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f176400d980 tx=0x7f176400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176cff9700 1 -- 172.21.15.19:0/3193505218 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764017070 con 0x7f1770107df0 2024-05-09T01:24:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176cff9700 1 -- 172.21.15.19:0/3193505218 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1764009de0 con 0x7f1770107df0 2024-05-09T01:24:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17701a72d0 con 0x7f1770107df0 2024-05-09T01:24:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f176cff9700 1 -- 172.21.15.19:0/3193505218 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176400f3c0 con 0x7f1770107df0 2024-05-09T01:24:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.427+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17701a7860 con 0x7f1770107df0 2024-05-09T01:24:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.428+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17700512c0 con 0x7f1770107df0 2024-05-09T01:24:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.430+0000 7f176cff9700 1 -- 172.21.15.19:0/3193505218 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1764009930 con 0x7f1770107df0 2024-05-09T01:24:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.430+0000 7f176cff9700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17580b2900 0x7f17580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.431+0000 7f176f7fe700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17580b2900 0x7f17580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.431+0000 7f176cff9700 1 -- 172.21.15.19:0/3193505218 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f17640e1fd0 con 0x7f1770107df0 2024-05-09T01:24:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.431+0000 7f176f7fe700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17580b2900 0x7f17580b4e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1760005950 tx=0x7f176000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.434+0000 7f176cff9700 1 -- 172.21.15.19:0/3193505218 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17640a9d50 con 0x7f1770107df0 2024-05-09T01:24:37.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.642+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1770063c00 con 0x7f17580b2900 2024-05-09T01:24:37.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.649+0000 7f176cff9700 1 -- 172.21.15.19:0/3193505218 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258879 (secure 0 0 0) 0x7f1770063c00 con 0x7f17580b2900 2024-05-09T01:24:37.651 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:24:37.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17580b2900 msgr2=0x7f17580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:37.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17580b2900 0x7f17580b4e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1760005950 tx=0x7f176000a300 comp rx=0 tx=0).stop 2024-05-09T01:24:37.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770107df0 msgr2=0x7f177019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:37.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770107df0 0x7f177019c1d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f176400d980 tx=0x7f176400dc90 comp rx=0 tx=0).stop 2024-05-09T01:24:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 shutdown_connections 2024-05-09T01:24:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1770108740 0x7f177019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17701034c0 0x7f177019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17580b2900 0x7f17580b4e10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 --2- 172.21.15.19:0/3193505218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770107df0 0x7f177019c1d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 >> 172.21.15.19:0/3193505218 conn(0x7f17700fedc0 msgr2=0x7f1770107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 shutdown_connections 2024-05-09T01:24:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:37.653+0000 7f17761b7700 1 -- 172.21.15.19:0/3193505218 wait complete. 2024-05-09T01:24:37.654 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T01:24:38.310 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-05-09T01:24:37.154270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660619,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1057,"num_read_kb":643,"num_write":384,"num_write_kb":2299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":2954185,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":817,"ondisk_log_size":817,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386940,"kb_used_data":12828,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554884,"statfs":{"total":1151940427776,"available":1151544201216,"internally_reserved":0,"allocated":13135872,"data_stored":7212621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":71,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003678"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158309+0000","last_change":"2024-05-09T01:22:39.836756+0000","last_active":"2024-05-09T01:24:21.158309+0000","last_peered":"2024-05-09T01:24:21.158309+0000","last_clean":"2024-05-09T01:24:21.158309+0000","last_became_active":"2024-05-09T01:22:39.835505+0000","last_became_peered":"2024-05-09T01:22:39.835505+0000","last_unstale":"2024-05-09T01:24:21.158309+0000","last_undegraded":"2024-05-09T01:24:21.158309+0000","last_fullsized":"2024-05-09T01:24:21.158309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:29.109762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148038+0000","last_change":"2024-05-09T01:22:39.842897+0000","last_active":"2024-05-09T01:24:21.148038+0000","last_peered":"2024-05-09T01:24:21.148038+0000","last_clean":"2024-05-09T01:24:21.148038+0000","last_became_active":"2024-05-09T01:22:39.842537+0000","last_became_peered":"2024-05-09T01:22:39.842537+0000","last_unstale":"2024-05-09T01:24:21.148038+0000","last_undegraded":"2024-05-09T01:24:21.148038+0000","last_fullsized":"2024-05-09T01:24:21.148038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:20.429654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157311+0000","last_change":"2024-05-09T01:22:39.847176+0000","last_active":"2024-05-09T01:24:21.157311+0000","last_peered":"2024-05-09T01:24:21.157311+0000","last_clean":"2024-05-09T01:24:21.157311+0000","last_became_active":"2024-05-09T01:22:39.846782+0000","last_became_peered":"2024-05-09T01:22:39.846782+0000","last_unstale":"2024-05-09T01:24:21.157311+0000","last_undegraded":"2024-05-09T01:24:21.157311+0000","last_fullsized":"2024-05-09T01:24:21.157311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:06.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134296+0000","last_change":"2024-05-09T01:22:39.842399+0000","last_active":"2024-05-09T01:24:21.134296+0000","last_peered":"2024-05-09T01:24:21.134296+0000","last_clean":"2024-05-09T01:24:21.134296+0000","last_became_active":"2024-05-09T01:22:39.842035+0000","last_became_peered":"2024-05-09T01:22:39.842035+0000","last_unstale":"2024-05-09T01:24:21.134296+0000","last_undegraded":"2024-05-09T01:24:21.134296+0000","last_fullsized":"2024-05-09T01:24:21.134296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:35.011920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.124470+0000","last_change":"2024-05-09T01:22:39.838574+0000","last_active":"2024-05-09T01:24:21.124470+0000","last_peered":"2024-05-09T01:24:21.124470+0000","last_clean":"2024-05-09T01:24:21.124470+0000","last_became_active":"2024-05-09T01:22:39.838059+0000","last_became_peered":"2024-05-09T01:22:39.838059+0000","last_unstale":"2024-05-09T01:24:21.124470+0000","last_undegraded":"2024-05-09T01:24:21.124470+0000","last_fullsized":"2024-05-09T01:24:21.124470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:20.968526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153227+0000","last_change":"2024-05-09T01:22:39.842407+0000","last_active":"2024-05-09T01:24:21.153227+0000","last_peered":"2024-05-09T01:24:21.153227+0000","last_clean":"2024-05-09T01:24:21.153227+0000","last_became_active":"2024-05-09T01:22:39.841982+0000","last_became_peered":"2024-05-09T01:22:39.841982+0000","last_unstale":"2024-05-09T01:24:21.153227+0000","last_undegraded":"2024-05-09T01:24:21.153227+0000","last_fullsized":"2024-05-09T01:24:21.153227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:17.086304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155073+0000","last_change":"2024-05-09T01:22:39.833800+0000","last_active":"2024-05-09T01:24:21.155073+0000","last_peered":"2024-05-09T01:24:21.155073+0000","last_clean":"2024-05-09T01:24:21.155073+0000","last_became_active":"2024-05-09T01:22:39.833388+0000","last_became_peered":"2024-05-09T01:22:39.833388+0000","last_unstale":"2024-05-09T01:24:21.155073+0000","last_undegraded":"2024-05-09T01:24:21.155073+0000","last_fullsized":"2024-05-09T01:24:21.155073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:35.341428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142076+0000","last_change":"2024-05-09T01:22:39.845395+0000","last_active":"2024-05-09T01:24:21.142076+0000","last_peered":"2024-05-09T01:24:21.142076+0000","last_clean":"2024-05-09T01:24:21.142076+0000","last_became_active":"2024-05-09T01:22:39.844946+0000","last_became_peered":"2024-05-09T01:22:39.844946+0000","last_unstale":"2024-05-09T01:24:21.142076+0000","last_undegraded":"2024-05-09T01:24:21.142076+0000","last_fullsized":"2024-05-09T01:24:21.142076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:44.750614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159615+0000","last_change":"2024-05-09T01:22:39.832894+0000","last_active":"2024-05-09T01:24:21.159615+0000","last_peered":"2024-05-09T01:24:21.159615+0000","last_clean":"2024-05-09T01:24:21.159615+0000","last_became_active":"2024-05-09T01:22:39.829743+0000","last_became_peered":"2024-05-09T01:22:39.829743+0000","last_unstale":"2024-05-09T01:24:21.159615+0000","last_undegraded":"2024-05-09T01:24:21.159615+0000","last_fullsized":"2024-05-09T01:24:21.159615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.463833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155782+0000","last_change":"2024-05-09T01:22:39.843131+0000","last_active":"2024-05-09T01:24:21.155782+0000","last_peered":"2024-05-09T01:24:21.155782+0000","last_clean":"2024-05-09T01:24:21.155782+0000","last_became_active":"2024-05-09T01:22:39.842690+0000","last_became_peered":"2024-05-09T01:22:39.842690+0000","last_unstale":"2024-05-09T01:24:21.155782+0000","last_undegraded":"2024-05-09T01:24:21.155782+0000","last_fullsized":"2024-05-09T01:24:21.155782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:14.172365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166118+0000","last_change":"2024-05-09T01:22:39.845557+0000","last_active":"2024-05-09T01:24:21.166118+0000","last_peered":"2024-05-09T01:24:21.166118+0000","last_clean":"2024-05-09T01:24:21.166118+0000","last_became_active":"2024-05-09T01:22:39.845059+0000","last_became_peered":"2024-05-09T01:22:39.845059+0000","last_unstale":"2024-05-09T01:24:21.166118+0000","last_undegraded":"2024-05-09T01:24:21.166118+0000","last_fullsized":"2024-05-09T01:24:21.166118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:47.238080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.128844+0000","last_change":"2024-05-09T01:22:39.842237+0000","last_active":"2024-05-09T01:24:21.128844+0000","last_peered":"2024-05-09T01:24:21.128844+0000","last_clean":"2024-05-09T01:24:21.128844+0000","last_became_active":"2024-05-09T01:22:39.841849+0000","last_became_peered":"2024-05-09T01:22:39.841849+0000","last_unstale":"2024-05-09T01:24:21.128844+0000","last_undegraded":"2024-05-09T01:24:21.128844+0000","last_fullsized":"2024-05-09T01:24:21.128844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:40.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150079+0000","last_change":"2024-05-09T01:22:39.839943+0000","last_active":"2024-05-09T01:24:21.150079+0000","last_peered":"2024-05-09T01:24:21.150079+0000","last_clean":"2024-05-09T01:24:21.150079+0000","last_became_active":"2024-05-09T01:22:39.839604+0000","last_became_peered":"2024-05-09T01:22:39.839604+0000","last_unstale":"2024-05-09T01:24:21.150079+0000","last_undegraded":"2024-05-09T01:24:21.150079+0000","last_fullsized":"2024-05-09T01:24:21.150079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:34.854263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.144875+0000","last_change":"2024-05-09T01:22:39.846981+0000","last_active":"2024-05-09T01:24:21.144875+0000","last_peered":"2024-05-09T01:24:21.144875+0000","last_clean":"2024-05-09T01:24:21.144875+0000","last_became_active":"2024-05-09T01:22:39.846586+0000","last_became_peered":"2024-05-09T01:22:39.846586+0000","last_unstale":"2024-05-09T01:24:21.144875+0000","last_undegraded":"2024-05-09T01:24:21.144875+0000","last_fullsized":"2024-05-09T01:24:21.144875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:50.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151118+0000","last_change":"2024-05-09T01:22:39.848964+0000","last_active":"2024-05-09T01:24:21.151118+0000","last_peered":"2024-05-09T01:24:21.151118+0000","last_clean":"2024-05-09T01:24:21.151118+0000","last_became_active":"2024-05-09T01:22:39.848442+0000","last_became_peered":"2024-05-09T01:22:39.848442+0000","last_unstale":"2024-05-09T01:24:21.151118+0000","last_undegraded":"2024-05-09T01:24:21.151118+0000","last_fullsized":"2024-05-09T01:24:21.151118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:35.802739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163088+0000","last_change":"2024-05-09T01:22:39.840586+0000","last_active":"2024-05-09T01:24:21.163088+0000","last_peered":"2024-05-09T01:24:21.163088+0000","last_clean":"2024-05-09T01:24:21.163088+0000","last_became_active":"2024-05-09T01:22:39.839887+0000","last_became_peered":"2024-05-09T01:22:39.839887+0000","last_unstale":"2024-05-09T01:24:21.163088+0000","last_undegraded":"2024-05-09T01:24:21.163088+0000","last_fullsized":"2024-05-09T01:24:21.163088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:36.510039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158490+0000","last_change":"2024-05-09T01:22:39.836152+0000","last_active":"2024-05-09T01:24:21.158490+0000","last_peered":"2024-05-09T01:24:21.158490+0000","last_clean":"2024-05-09T01:24:21.158490+0000","last_became_active":"2024-05-09T01:22:39.834691+0000","last_became_peered":"2024-05-09T01:22:39.834691+0000","last_unstale":"2024-05-09T01:24:21.158490+0000","last_undegraded":"2024-05-09T01:24:21.158490+0000","last_fullsized":"2024-05-09T01:24:21.158490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:31.331096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163367+0000","last_change":"2024-05-09T01:22:39.839760+0000","last_active":"2024-05-09T01:24:21.163367+0000","last_peered":"2024-05-09T01:24:21.163367+0000","last_clean":"2024-05-09T01:24:21.163367+0000","last_became_active":"2024-05-09T01:22:39.839403+0000","last_became_peered":"2024-05-09T01:22:39.839403+0000","last_unstale":"2024-05-09T01:24:21.163367+0000","last_undegraded":"2024-05-09T01:24:21.163367+0000","last_fullsized":"2024-05-09T01:24:21.163367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:19.248368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142656+0000","last_change":"2024-05-09T01:22:39.837538+0000","last_active":"2024-05-09T01:24:21.142656+0000","last_peered":"2024-05-09T01:24:21.142656+0000","last_clean":"2024-05-09T01:24:21.142656+0000","last_became_active":"2024-05-09T01:22:39.837167+0000","last_became_peered":"2024-05-09T01:22:39.837167+0000","last_unstale":"2024-05-09T01:24:21.142656+0000","last_undegraded":"2024-05-09T01:24:21.142656+0000","last_fullsized":"2024-05-09T01:24:21.142656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:35.695077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:34.527510+0000","last_change":"2024-05-09T01:23:22.433852+0000","last_active":"2024-05-09T01:24:34.527510+0000","last_peered":"2024-05-09T01:24:34.527510+0000","last_clean":"2024-05-09T01:24:34.527510+0000","last_became_active":"2024-05-09T01:23:22.423465+0000","last_became_peered":"2024-05-09T01:23:22.423465+0000","last_unstale":"2024-05-09T01:24:34.527510+0000","last_undegraded":"2024-05-09T01:24:34.527510+0000","last_fullsized":"2024-05-09T01:24:34.527510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:58.578798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.135977+0000","last_change":"2024-05-09T01:22:39.842667+0000","last_active":"2024-05-09T01:24:21.135977+0000","last_peered":"2024-05-09T01:24:21.135977+0000","last_clean":"2024-05-09T01:24:21.135977+0000","last_became_active":"2024-05-09T01:22:39.842256+0000","last_became_peered":"2024-05-09T01:22:39.842256+0000","last_unstale":"2024-05-09T01:24:21.135977+0000","last_undegraded":"2024-05-09T01:24:21.135977+0000","last_fullsized":"2024-05-09T01:24:21.135977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:28.525790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.314520+0000","last_change":"2024-05-09T01:22:36.841515+0000","last_active":"2024-05-09T01:23:22.314520+0000","last_peered":"2024-05-09T01:23:22.314520+0000","last_clean":"2024-05-09T01:23:22.314520+0000","last_became_active":"2024-05-09T01:22:36.841128+0000","last_became_peered":"2024-05-09T01:22:36.841128+0000","last_unstale":"2024-05-09T01:23:22.314520+0000","last_undegraded":"2024-05-09T01:23:22.314520+0000","last_fullsized":"2024-05-09T01:23:22.314520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:13.598177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157522+0000","last_change":"2024-05-09T01:22:39.843491+0000","last_active":"2024-05-09T01:24:21.157522+0000","last_peered":"2024-05-09T01:24:21.157522+0000","last_clean":"2024-05-09T01:24:21.157522+0000","last_became_active":"2024-05-09T01:22:39.843098+0000","last_became_peered":"2024-05-09T01:22:39.843098+0000","last_unstale":"2024-05-09T01:24:21.157522+0000","last_undegraded":"2024-05-09T01:24:21.157522+0000","last_fullsized":"2024-05-09T01:24:21.157522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:03:33.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303907+0000","last_change":"2024-05-09T01:22:36.798217+0000","last_active":"2024-05-09T01:23:22.303907+0000","last_peered":"2024-05-09T01:23:22.303907+0000","last_clean":"2024-05-09T01:23:22.303907+0000","last_became_active":"2024-05-09T01:22:36.797847+0000","last_became_peered":"2024-05-09T01:22:36.797847+0000","last_unstale":"2024-05-09T01:23:22.303907+0000","last_undegraded":"2024-05-09T01:23:22.303907+0000","last_fullsized":"2024-05-09T01:23:22.303907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:21.116535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164555+0000","last_change":"2024-05-09T01:22:39.837568+0000","last_active":"2024-05-09T01:24:21.164555+0000","last_peered":"2024-05-09T01:24:21.164555+0000","last_clean":"2024-05-09T01:24:21.164555+0000","last_became_active":"2024-05-09T01:22:39.836159+0000","last_became_peered":"2024-05-09T01:22:39.836159+0000","last_unstale":"2024-05-09T01:24:21.164555+0000","last_undegraded":"2024-05-09T01:24:21.164555+0000","last_fullsized":"2024-05-09T01:24:21.164555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:28.089830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303388+0000","last_change":"2024-05-09T01:22:36.798185+0000","last_active":"2024-05-09T01:23:22.303388+0000","last_peered":"2024-05-09T01:23:22.303388+0000","last_clean":"2024-05-09T01:23:22.303388+0000","last_became_active":"2024-05-09T01:22:36.797522+0000","last_became_peered":"2024-05-09T01:22:36.797522+0000","last_unstale":"2024-05-09T01:23:22.303388+0000","last_undegraded":"2024-05-09T01:23:22.303388+0000","last_fullsized":"2024-05-09T01:23:22.303388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:40.960006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153830+0000","last_change":"2024-05-09T01:22:39.837436+0000","last_active":"2024-05-09T01:24:21.153830+0000","last_peered":"2024-05-09T01:24:21.153830+0000","last_clean":"2024-05-09T01:24:21.153830+0000","last_became_active":"2024-05-09T01:22:39.837050+0000","last_became_peered":"2024-05-09T01:22:39.837050+0000","last_unstale":"2024-05-09T01:24:21.153830+0000","last_undegraded":"2024-05-09T01:24:21.153830+0000","last_fullsized":"2024-05-09T01:24:21.153830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:07.149311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:31.773761+0000","last_change":"2024-05-09T01:22:36.800767+0000","last_active":"2024-05-09T01:24:31.773761+0000","last_peered":"2024-05-09T01:24:31.773761+0000","last_clean":"2024-05-09T01:24:31.773761+0000","last_became_active":"2024-05-09T01:22:36.800183+0000","last_became_peered":"2024-05-09T01:22:36.800183+0000","last_unstale":"2024-05-09T01:24:31.773761+0000","last_undegraded":"2024-05-09T01:24:31.773761+0000","last_fullsized":"2024-05-09T01:24:31.773761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:05.682889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.131244+0000","last_change":"2024-05-09T01:22:39.832798+0000","last_active":"2024-05-09T01:24:21.131244+0000","last_peered":"2024-05-09T01:24:21.131244+0000","last_clean":"2024-05-09T01:24:21.131244+0000","last_became_active":"2024-05-09T01:22:39.828803+0000","last_became_peered":"2024-05-09T01:22:39.828803+0000","last_unstale":"2024-05-09T01:24:21.131244+0000","last_undegraded":"2024-05-09T01:24:21.131244+0000","last_fullsized":"2024-05-09T01:24:21.131244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:23.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305959+0000","last_change":"2024-05-09T01:22:36.794171+0000","last_active":"2024-05-09T01:23:22.305959+0000","last_peered":"2024-05-09T01:23:22.305959+0000","last_clean":"2024-05-09T01:23:22.305959+0000","last_became_active":"2024-05-09T01:22:36.793713+0000","last_became_peered":"2024-05-09T01:22:36.793713+0000","last_unstale":"2024-05-09T01:23:22.305959+0000","last_undegraded":"2024-05-09T01:23:22.305959+0000","last_fullsized":"2024-05-09T01:23:22.305959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:06.654667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153309+0000","last_change":"2024-05-09T01:22:39.838266+0000","last_active":"2024-05-09T01:24:21.153309+0000","last_peered":"2024-05-09T01:24:21.153309+0000","last_clean":"2024-05-09T01:24:21.153309+0000","last_became_active":"2024-05-09T01:22:39.837911+0000","last_became_peered":"2024-05-09T01:22:39.837911+0000","last_unstale":"2024-05-09T01:24:21.153309+0000","last_undegraded":"2024-05-09T01:24:21.153309+0000","last_fullsized":"2024-05-09T01:24:21.153309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:39.414176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308540+0000","last_change":"2024-05-09T01:22:36.799766+0000","last_active":"2024-05-09T01:23:22.308540+0000","last_peered":"2024-05-09T01:23:22.308540+0000","last_clean":"2024-05-09T01:23:22.308540+0000","last_became_active":"2024-05-09T01:22:36.799388+0000","last_became_peered":"2024-05-09T01:22:36.799388+0000","last_unstale":"2024-05-09T01:23:22.308540+0000","last_undegraded":"2024-05-09T01:23:22.308540+0000","last_fullsized":"2024-05-09T01:23:22.308540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:13.401659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141635+0000","last_change":"2024-05-09T01:22:39.838493+0000","last_active":"2024-05-09T01:24:21.141635+0000","last_peered":"2024-05-09T01:24:21.141635+0000","last_clean":"2024-05-09T01:24:21.141635+0000","last_became_active":"2024-05-09T01:22:39.838086+0000","last_became_peered":"2024-05-09T01:22:39.838086+0000","last_unstale":"2024-05-09T01:24:21.141635+0000","last_undegraded":"2024-05-09T01:24:21.141635+0000","last_fullsized":"2024-05-09T01:24:21.141635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:06.340693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421503+0000","last_change":"2024-05-09T01:22:36.804872+0000","last_active":"2024-05-09T01:23:22.421503+0000","last_peered":"2024-05-09T01:23:22.421503+0000","last_clean":"2024-05-09T01:23:22.421503+0000","last_became_active":"2024-05-09T01:22:36.804439+0000","last_became_peered":"2024-05-09T01:22:36.804439+0000","last_unstale":"2024-05-09T01:23:22.421503+0000","last_undegraded":"2024-05-09T01:23:22.421503+0000","last_fullsized":"2024-05-09T01:23:22.421503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:08.584749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.161879+0000","last_change":"2024-05-09T01:22:39.836579+0000","last_active":"2024-05-09T01:24:21.161879+0000","last_peered":"2024-05-09T01:24:21.161879+0000","last_clean":"2024-05-09T01:24:21.161879+0000","last_became_active":"2024-05-09T01:22:39.836244+0000","last_became_peered":"2024-05-09T01:22:39.836244+0000","last_unstale":"2024-05-09T01:24:21.161879+0000","last_undegraded":"2024-05-09T01:24:21.161879+0000","last_fullsized":"2024-05-09T01:24:21.161879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:43.329448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":225,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:34.777657+0000","last_change":"2024-05-09T01:22:36.796639+0000","last_active":"2024-05-09T01:24:34.777657+0000","last_peered":"2024-05-09T01:24:34.777657+0000","last_clean":"2024-05-09T01:24:34.777657+0000","last_became_active":"2024-05-09T01:22:36.796227+0000","last_became_peered":"2024-05-09T01:22:36.796227+0000","last_unstale":"2024-05-09T01:24:34.777657+0000","last_undegraded":"2024-05-09T01:24:34.777657+0000","last_fullsized":"2024-05-09T01:24:34.777657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:56.219281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.162605+0000","last_change":"2024-05-09T01:22:39.841515+0000","last_active":"2024-05-09T01:24:21.162605+0000","last_peered":"2024-05-09T01:24:21.162605+0000","last_clean":"2024-05-09T01:24:21.162605+0000","last_became_active":"2024-05-09T01:22:39.840057+0000","last_became_peered":"2024-05-09T01:22:39.840057+0000","last_unstale":"2024-05-09T01:24:21.162605+0000","last_undegraded":"2024-05-09T01:24:21.162605+0000","last_fullsized":"2024-05-09T01:24:21.162605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:43.258466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309092+0000","last_change":"2024-05-09T01:22:36.801937+0000","last_active":"2024-05-09T01:23:22.309092+0000","last_peered":"2024-05-09T01:23:22.309092+0000","last_clean":"2024-05-09T01:23:22.309092+0000","last_became_active":"2024-05-09T01:22:36.801563+0000","last_became_peered":"2024-05-09T01:22:36.801563+0000","last_unstale":"2024-05-09T01:23:22.309092+0000","last_undegraded":"2024-05-09T01:23:22.309092+0000","last_fullsized":"2024-05-09T01:23:22.309092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:00:42.170156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151261+0000","last_change":"2024-05-09T01:22:39.849154+0000","last_active":"2024-05-09T01:24:21.151261+0000","last_peered":"2024-05-09T01:24:21.151261+0000","last_clean":"2024-05-09T01:24:21.151261+0000","last_became_active":"2024-05-09T01:22:39.848815+0000","last_became_peered":"2024-05-09T01:22:39.848815+0000","last_unstale":"2024-05-09T01:24:21.151261+0000","last_undegraded":"2024-05-09T01:24:21.151261+0000","last_fullsized":"2024-05-09T01:24:21.151261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:00.300208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421958+0000","last_change":"2024-05-09T01:22:36.843201+0000","last_active":"2024-05-09T01:23:22.421958+0000","last_peered":"2024-05-09T01:23:22.421958+0000","last_clean":"2024-05-09T01:23:22.421958+0000","last_became_active":"2024-05-09T01:22:36.842500+0000","last_became_peered":"2024-05-09T01:22:36.842500+0000","last_unstale":"2024-05-09T01:23:22.421958+0000","last_undegraded":"2024-05-09T01:23:22.421958+0000","last_fullsized":"2024-05-09T01:23:22.421958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:35.092246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159455+0000","last_change":"2024-05-09T01:22:39.835120+0000","last_active":"2024-05-09T01:24:21.159455+0000","last_peered":"2024-05-09T01:24:21.159455+0000","last_clean":"2024-05-09T01:24:21.159455+0000","last_became_active":"2024-05-09T01:22:39.834613+0000","last_became_peered":"2024-05-09T01:22:39.834613+0000","last_unstale":"2024-05-09T01:24:21.159455+0000","last_undegraded":"2024-05-09T01:24:21.159455+0000","last_fullsized":"2024-05-09T01:24:21.159455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:08.380445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":265,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:34.790670+0000","last_change":"2024-05-09T01:22:36.800816+0000","last_active":"2024-05-09T01:24:34.790670+0000","last_peered":"2024-05-09T01:24:34.790670+0000","last_clean":"2024-05-09T01:24:34.790670+0000","last_became_active":"2024-05-09T01:22:36.798797+0000","last_became_peered":"2024-05-09T01:22:36.798797+0000","last_unstale":"2024-05-09T01:24:34.790670+0000","last_undegraded":"2024-05-09T01:24:34.790670+0000","last_fullsized":"2024-05-09T01:24:34.790670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:37.670163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61999,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":204,"num_read_kb":162,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134404+0000","last_change":"2024-05-09T01:22:39.831504+0000","last_active":"2024-05-09T01:24:21.134404+0000","last_peered":"2024-05-09T01:24:21.134404+0000","last_clean":"2024-05-09T01:24:21.134404+0000","last_became_active":"2024-05-09T01:22:39.831129+0000","last_became_peered":"2024-05-09T01:22:39.831129+0000","last_unstale":"2024-05-09T01:24:21.134404+0000","last_undegraded":"2024-05-09T01:24:21.134404+0000","last_fullsized":"2024-05-09T01:24:21.134404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:19.212473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304426+0000","last_change":"2024-05-09T01:22:36.803447+0000","last_active":"2024-05-09T01:23:22.304426+0000","last_peered":"2024-05-09T01:23:22.304426+0000","last_clean":"2024-05-09T01:23:22.304426+0000","last_became_active":"2024-05-09T01:22:36.802986+0000","last_became_peered":"2024-05-09T01:22:36.802986+0000","last_unstale":"2024-05-09T01:23:22.304426+0000","last_undegraded":"2024-05-09T01:23:22.304426+0000","last_fullsized":"2024-05-09T01:23:22.304426+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:28.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163194+0000","last_change":"2024-05-09T01:22:39.836546+0000","last_active":"2024-05-09T01:24:21.163194+0000","last_peered":"2024-05-09T01:24:21.163194+0000","last_clean":"2024-05-09T01:24:21.163194+0000","last_became_active":"2024-05-09T01:22:39.836168+0000","last_became_peered":"2024-05-09T01:22:39.836168+0000","last_unstale":"2024-05-09T01:24:21.163194+0000","last_undegraded":"2024-05-09T01:24:21.163194+0000","last_fullsized":"2024-05-09T01:24:21.163194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:45.522279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421596+0000","last_change":"2024-05-09T01:22:36.798703+0000","last_active":"2024-05-09T01:23:22.421596+0000","last_peered":"2024-05-09T01:23:22.421596+0000","last_clean":"2024-05-09T01:23:22.421596+0000","last_became_active":"2024-05-09T01:22:36.798280+0000","last_became_peered":"2024-05-09T01:22:36.798280+0000","last_unstale":"2024-05-09T01:23:22.421596+0000","last_undegraded":"2024-05-09T01:23:22.421596+0000","last_fullsized":"2024-05-09T01:23:22.421596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:33.794488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.165878+0000","last_change":"2024-05-09T01:22:39.831223+0000","last_active":"2024-05-09T01:24:21.165878+0000","last_peered":"2024-05-09T01:24:21.165878+0000","last_clean":"2024-05-09T01:24:21.165878+0000","last_became_active":"2024-05-09T01:22:39.830845+0000","last_became_peered":"2024-05-09T01:22:39.830845+0000","last_unstale":"2024-05-09T01:24:21.165878+0000","last_undegraded":"2024-05-09T01:24:21.165878+0000","last_fullsized":"2024-05-09T01:24:21.165878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:23.365950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:31.802080+0000","last_change":"2024-05-09T01:22:36.802327+0000","last_active":"2024-05-09T01:24:31.802080+0000","last_peered":"2024-05-09T01:24:31.802080+0000","last_clean":"2024-05-09T01:24:31.802080+0000","last_became_active":"2024-05-09T01:22:36.801951+0000","last_became_peered":"2024-05-09T01:22:36.801951+0000","last_unstale":"2024-05-09T01:24:31.802080+0000","last_undegraded":"2024-05-09T01:24:31.802080+0000","last_fullsized":"2024-05-09T01:24:31.802080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:12.282424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154077+0000","last_change":"2024-05-09T01:22:39.833626+0000","last_active":"2024-05-09T01:24:21.154077+0000","last_peered":"2024-05-09T01:24:21.154077+0000","last_clean":"2024-05-09T01:24:21.154077+0000","last_became_active":"2024-05-09T01:22:39.833176+0000","last_became_peered":"2024-05-09T01:22:39.833176+0000","last_unstale":"2024-05-09T01:24:21.154077+0000","last_undegraded":"2024-05-09T01:24:21.154077+0000","last_fullsized":"2024-05-09T01:24:21.154077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:25.082300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.328609+0000","last_change":"2024-05-09T01:23:22.328011+0000","last_active":"2024-05-09T01:23:22.328609+0000","last_peered":"2024-05-09T01:23:22.328609+0000","last_clean":"2024-05-09T01:23:22.328609+0000","last_became_active":"2024-05-09T01:23:22.313049+0000","last_became_peered":"2024-05-09T01:23:22.313049+0000","last_unstale":"2024-05-09T01:23:22.328609+0000","last_undegraded":"2024-05-09T01:23:22.328609+0000","last_fullsized":"2024-05-09T01:23:22.328609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:17.854491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153164+0000","last_change":"2024-05-09T01:22:39.837529+0000","last_active":"2024-05-09T01:24:21.153164+0000","last_peered":"2024-05-09T01:24:21.153164+0000","last_clean":"2024-05-09T01:24:21.153164+0000","last_became_active":"2024-05-09T01:22:39.837155+0000","last_became_peered":"2024-05-09T01:22:39.837155+0000","last_unstale":"2024-05-09T01:24:21.153164+0000","last_undegraded":"2024-05-09T01:24:21.153164+0000","last_fullsized":"2024-05-09T01:24:21.153164+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:44.485629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303529+0000","last_change":"2024-05-09T01:22:36.800073+0000","last_active":"2024-05-09T01:23:22.303529+0000","last_peered":"2024-05-09T01:23:22.303529+0000","last_clean":"2024-05-09T01:23:22.303529+0000","last_became_active":"2024-05-09T01:22:36.799463+0000","last_became_peered":"2024-05-09T01:22:36.799463+0000","last_unstale":"2024-05-09T01:23:22.303529+0000","last_undegraded":"2024-05-09T01:23:22.303529+0000","last_fullsized":"2024-05-09T01:23:22.303529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.942945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150617+0000","last_change":"2024-05-09T01:22:39.833615+0000","last_active":"2024-05-09T01:24:21.150617+0000","last_peered":"2024-05-09T01:24:21.150617+0000","last_clean":"2024-05-09T01:24:21.150617+0000","last_became_active":"2024-05-09T01:22:39.833212+0000","last_became_peered":"2024-05-09T01:22:39.833212+0000","last_unstale":"2024-05-09T01:24:21.150617+0000","last_undegraded":"2024-05-09T01:24:21.150617+0000","last_fullsized":"2024-05-09T01:24:21.150617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:35.387442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305607+0000","last_change":"2024-05-09T01:22:36.843143+0000","last_active":"2024-05-09T01:23:22.305607+0000","last_peered":"2024-05-09T01:23:22.305607+0000","last_clean":"2024-05-09T01:23:22.305607+0000","last_became_active":"2024-05-09T01:22:36.842759+0000","last_became_peered":"2024-05-09T01:22:36.842759+0000","last_unstale":"2024-05-09T01:23:22.305607+0000","last_undegraded":"2024-05-09T01:23:22.305607+0000","last_fullsized":"2024-05-09T01:23:22.305607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:42.666186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.133027+0000","last_change":"2024-05-09T01:22:39.838267+0000","last_active":"2024-05-09T01:24:21.133027+0000","last_peered":"2024-05-09T01:24:21.133027+0000","last_clean":"2024-05-09T01:24:21.133027+0000","last_became_active":"2024-05-09T01:22:39.837910+0000","last_became_peered":"2024-05-09T01:22:39.837910+0000","last_unstale":"2024-05-09T01:24:21.133027+0000","last_undegraded":"2024-05-09T01:24:21.133027+0000","last_fullsized":"2024-05-09T01:24:21.133027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:34.620332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:32.524889+0000","last_change":"2024-05-09T01:22:36.800515+0000","last_active":"2024-05-09T01:24:32.524889+0000","last_peered":"2024-05-09T01:24:32.524889+0000","last_clean":"2024-05-09T01:24:32.524889+0000","last_became_active":"2024-05-09T01:22:36.800065+0000","last_became_peered":"2024-05-09T01:22:36.800065+0000","last_unstale":"2024-05-09T01:24:32.524889+0000","last_undegraded":"2024-05-09T01:24:32.524889+0000","last_fullsized":"2024-05-09T01:24:32.524889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:50.160463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145372+0000","last_change":"2024-05-09T01:23:22.313627+0000","last_active":"2024-05-09T01:24:21.145372+0000","last_peered":"2024-05-09T01:24:21.145372+0000","last_clean":"2024-05-09T01:24:21.145372+0000","last_became_active":"2024-05-09T01:23:22.313138+0000","last_became_peered":"2024-05-09T01:23:22.313138+0000","last_unstale":"2024-05-09T01:24:21.145372+0000","last_undegraded":"2024-05-09T01:24:21.145372+0000","last_fullsized":"2024-05-09T01:24:21.145372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:07.216086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153329+0000","last_change":"2024-05-09T01:22:39.832188+0000","last_active":"2024-05-09T01:24:21.153329+0000","last_peered":"2024-05-09T01:24:21.153329+0000","last_clean":"2024-05-09T01:24:21.153329+0000","last_became_active":"2024-05-09T01:22:39.831302+0000","last_became_peered":"2024-05-09T01:22:39.831302+0000","last_unstale":"2024-05-09T01:24:21.153329+0000","last_undegraded":"2024-05-09T01:24:21.153329+0000","last_fullsized":"2024-05-09T01:24:21.153329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:06.703950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:32.803104+0000","last_change":"2024-05-09T01:22:36.841446+0000","last_active":"2024-05-09T01:24:32.803104+0000","last_peered":"2024-05-09T01:24:32.803104+0000","last_clean":"2024-05-09T01:24:32.803104+0000","last_became_active":"2024-05-09T01:22:36.841071+0000","last_became_peered":"2024-05-09T01:22:36.841071+0000","last_unstale":"2024-05-09T01:24:32.803104+0000","last_undegraded":"2024-05-09T01:24:32.803104+0000","last_fullsized":"2024-05-09T01:24:32.803104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:20.777482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163139+0000","last_change":"2024-05-09T01:22:39.846711+0000","last_active":"2024-05-09T01:24:21.163139+0000","last_peered":"2024-05-09T01:24:21.163139+0000","last_clean":"2024-05-09T01:24:21.163139+0000","last_became_active":"2024-05-09T01:22:39.846293+0000","last_became_peered":"2024-05-09T01:22:39.846293+0000","last_unstale":"2024-05-09T01:24:21.163139+0000","last_undegraded":"2024-05-09T01:24:21.163139+0000","last_fullsized":"2024-05-09T01:24:21.163139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:27.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303787+0000","last_change":"2024-05-09T01:22:36.798302+0000","last_active":"2024-05-09T01:23:22.303787+0000","last_peered":"2024-05-09T01:23:22.303787+0000","last_clean":"2024-05-09T01:23:22.303787+0000","last_became_active":"2024-05-09T01:22:36.797906+0000","last_became_peered":"2024-05-09T01:22:36.797906+0000","last_unstale":"2024-05-09T01:23:22.303787+0000","last_undegraded":"2024-05-09T01:23:22.303787+0000","last_fullsized":"2024-05-09T01:23:22.303787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:13.599240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308697+0000","last_change":"2024-05-09T01:21:12.781366+0000","last_active":"2024-05-09T01:23:22.308697+0000","last_peered":"2024-05-09T01:23:22.308697+0000","last_clean":"2024-05-09T01:23:22.308697+0000","last_became_active":"2024-05-09T01:21:12.774967+0000","last_became_peered":"2024-05-09T01:21:12.774967+0000","last_unstale":"2024-05-09T01:23:22.308697+0000","last_undegraded":"2024-05-09T01:23:22.308697+0000","last_fullsized":"2024-05-09T01:23:22.308697+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_clean_scrub_stamp":"2024-05-09T01:17:49.682508+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:08.075365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.152872+0000","last_change":"2024-05-09T01:22:39.840952+0000","last_active":"2024-05-09T01:24:21.152872+0000","last_peered":"2024-05-09T01:24:21.152872+0000","last_clean":"2024-05-09T01:24:21.152872+0000","last_became_active":"2024-05-09T01:22:39.840323+0000","last_became_peered":"2024-05-09T01:22:39.840323+0000","last_unstale":"2024-05-09T01:24:21.152872+0000","last_undegraded":"2024-05-09T01:24:21.152872+0000","last_fullsized":"2024-05-09T01:24:21.152872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:23.837123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308793+0000","last_change":"2024-05-09T01:22:36.798631+0000","last_active":"2024-05-09T01:23:22.308793+0000","last_peered":"2024-05-09T01:23:22.308793+0000","last_clean":"2024-05-09T01:23:22.308793+0000","last_became_active":"2024-05-09T01:22:36.798242+0000","last_became_peered":"2024-05-09T01:22:36.798242+0000","last_unstale":"2024-05-09T01:23:22.308793+0000","last_undegraded":"2024-05-09T01:23:22.308793+0000","last_fullsized":"2024-05-09T01:23:22.308793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:18.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303746+0000","last_change":"2024-05-09T01:22:36.843779+0000","last_active":"2024-05-09T01:23:22.303746+0000","last_peered":"2024-05-09T01:23:22.303746+0000","last_clean":"2024-05-09T01:23:22.303746+0000","last_became_active":"2024-05-09T01:22:36.842084+0000","last_became_peered":"2024-05-09T01:22:36.842084+0000","last_unstale":"2024-05-09T01:23:22.303746+0000","last_undegraded":"2024-05-09T01:23:22.303746+0000","last_fullsized":"2024-05-09T01:23:22.303746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:34.743372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145885+0000","last_change":"2024-05-09T01:22:39.842266+0000","last_active":"2024-05-09T01:24:21.145885+0000","last_peered":"2024-05-09T01:24:21.145885+0000","last_clean":"2024-05-09T01:24:21.145885+0000","last_became_active":"2024-05-09T01:22:39.841927+0000","last_became_peered":"2024-05-09T01:22:39.841927+0000","last_unstale":"2024-05-09T01:24:21.145885+0000","last_undegraded":"2024-05-09T01:24:21.145885+0000","last_fullsized":"2024-05-09T01:24:21.145885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:26.774250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'17","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:23.838395+0000","last_change":"2024-05-09T01:22:36.796929+0000","last_active":"2024-05-09T01:23:23.838395+0000","last_peered":"2024-05-09T01:23:23.838395+0000","last_clean":"2024-05-09T01:23:23.838395+0000","last_became_active":"2024-05-09T01:22:36.796480+0000","last_became_peered":"2024-05-09T01:22:36.796480+0000","last_unstale":"2024-05-09T01:23:23.838395+0000","last_undegraded":"2024-05-09T01:23:23.838395+0000","last_fullsized":"2024-05-09T01:23:23.838395+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:45.168305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.139985+0000","last_change":"2024-05-09T01:22:39.843093+0000","last_active":"2024-05-09T01:24:21.139985+0000","last_peered":"2024-05-09T01:24:21.139985+0000","last_clean":"2024-05-09T01:24:21.139985+0000","last_became_active":"2024-05-09T01:22:39.842729+0000","last_became_peered":"2024-05-09T01:22:39.842729+0000","last_unstale":"2024-05-09T01:24:21.139985+0000","last_undegraded":"2024-05-09T01:24:21.139985+0000","last_fullsized":"2024-05-09T01:24:21.139985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:02.074843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303637+0000","last_change":"2024-05-09T01:22:36.798166+0000","last_active":"2024-05-09T01:23:22.303637+0000","last_peered":"2024-05-09T01:23:22.303637+0000","last_clean":"2024-05-09T01:23:22.303637+0000","last_became_active":"2024-05-09T01:22:36.797752+0000","last_became_peered":"2024-05-09T01:22:36.797752+0000","last_unstale":"2024-05-09T01:23:22.303637+0000","last_undegraded":"2024-05-09T01:23:22.303637+0000","last_fullsized":"2024-05-09T01:23:22.303637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:05.211720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141557+0000","last_change":"2024-05-09T01:23:22.316228+0000","last_active":"2024-05-09T01:24:21.141557+0000","last_peered":"2024-05-09T01:24:21.141557+0000","last_clean":"2024-05-09T01:24:21.141557+0000","last_became_active":"2024-05-09T01:23:22.315683+0000","last_became_peered":"2024-05-09T01:23:22.315683+0000","last_unstale":"2024-05-09T01:24:21.141557+0000","last_undegraded":"2024-05-09T01:24:21.141557+0000","last_fullsized":"2024-05-09T01:24:21.141557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:04.623701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421234+0000","last_change":"2024-05-09T01:22:36.841999+0000","last_active":"2024-05-09T01:23:22.421234+0000","last_peered":"2024-05-09T01:23:22.421234+0000","last_clean":"2024-05-09T01:23:22.421234+0000","last_became_active":"2024-05-09T01:22:36.841664+0000","last_became_peered":"2024-05-09T01:22:36.841664+0000","last_unstale":"2024-05-09T01:23:22.421234+0000","last_undegraded":"2024-05-09T01:23:22.421234+0000","last_fullsized":"2024-05-09T01:23:22.421234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:27.110258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.109425+0000","last_change":"2024-05-09T01:22:39.839553+0000","last_active":"2024-05-09T01:24:21.109425+0000","last_peered":"2024-05-09T01:24:21.109425+0000","last_clean":"2024-05-09T01:24:21.109425+0000","last_became_active":"2024-05-09T01:22:39.839189+0000","last_became_peered":"2024-05-09T01:22:39.839189+0000","last_unstale":"2024-05-09T01:24:21.109425+0000","last_undegraded":"2024-05-09T01:24:21.109425+0000","last_fullsized":"2024-05-09T01:24:21.109425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:02.794073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309848+0000","last_change":"2024-05-09T01:22:36.792516+0000","last_active":"2024-05-09T01:23:22.309848+0000","last_peered":"2024-05-09T01:23:22.309848+0000","last_clean":"2024-05-09T01:23:22.309848+0000","last_became_active":"2024-05-09T01:22:36.792129+0000","last_became_peered":"2024-05-09T01:22:36.792129+0000","last_unstale":"2024-05-09T01:23:22.309848+0000","last_undegraded":"2024-05-09T01:23:22.309848+0000","last_fullsized":"2024-05-09T01:23:22.309848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:46.596723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148698+0000","last_change":"2024-05-09T01:22:39.841523+0000","last_active":"2024-05-09T01:24:21.148698+0000","last_peered":"2024-05-09T01:24:21.148698+0000","last_clean":"2024-05-09T01:24:21.148698+0000","last_became_active":"2024-05-09T01:22:39.840667+0000","last_became_peered":"2024-05-09T01:22:39.840667+0000","last_unstale":"2024-05-09T01:24:21.148698+0000","last_undegraded":"2024-05-09T01:24:21.148698+0000","last_fullsized":"2024-05-09T01:24:21.148698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:39.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.332290+0000","last_change":"2024-05-09T01:23:22.331855+0000","last_active":"2024-05-09T01:23:22.332290+0000","last_peered":"2024-05-09T01:23:22.332290+0000","last_clean":"2024-05-09T01:23:22.332290+0000","last_became_active":"2024-05-09T01:23:22.313059+0000","last_became_peered":"2024-05-09T01:23:22.313059+0000","last_unstale":"2024-05-09T01:23:22.332290+0000","last_undegraded":"2024-05-09T01:23:22.332290+0000","last_fullsized":"2024-05-09T01:23:22.332290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:16.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140461+0000","last_change":"2024-05-09T01:22:39.845031+0000","last_active":"2024-05-09T01:24:21.140461+0000","last_peered":"2024-05-09T01:24:21.140461+0000","last_clean":"2024-05-09T01:24:21.140461+0000","last_became_active":"2024-05-09T01:22:39.844653+0000","last_became_peered":"2024-05-09T01:22:39.844653+0000","last_unstale":"2024-05-09T01:24:21.140461+0000","last_undegraded":"2024-05-09T01:24:21.140461+0000","last_fullsized":"2024-05-09T01:24:21.140461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:14.382445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303070+0000","last_change":"2024-05-09T01:22:36.801395+0000","last_active":"2024-05-09T01:23:22.303070+0000","last_peered":"2024-05-09T01:23:22.303070+0000","last_clean":"2024-05-09T01:23:22.303070+0000","last_became_active":"2024-05-09T01:22:36.800992+0000","last_became_peered":"2024-05-09T01:22:36.800992+0000","last_unstale":"2024-05-09T01:23:22.303070+0000","last_undegraded":"2024-05-09T01:23:22.303070+0000","last_fullsized":"2024-05-09T01:23:22.303070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:16.089068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.165901+0000","last_change":"2024-05-09T01:22:39.835842+0000","last_active":"2024-05-09T01:24:21.165901+0000","last_peered":"2024-05-09T01:24:21.165901+0000","last_clean":"2024-05-09T01:24:21.165901+0000","last_became_active":"2024-05-09T01:22:39.835372+0000","last_became_peered":"2024-05-09T01:22:39.835372+0000","last_unstale":"2024-05-09T01:24:21.165901+0000","last_undegraded":"2024-05-09T01:24:21.165901+0000","last_fullsized":"2024-05-09T01:24:21.165901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:13.197946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:32.436785+0000","last_change":"2024-05-09T01:23:22.332514+0000","last_active":"2024-05-09T01:24:32.436785+0000","last_peered":"2024-05-09T01:24:32.436785+0000","last_clean":"2024-05-09T01:24:32.436785+0000","last_became_active":"2024-05-09T01:23:22.312966+0000","last_became_peered":"2024-05-09T01:23:22.312966+0000","last_unstale":"2024-05-09T01:24:32.436785+0000","last_undegraded":"2024-05-09T01:24:32.436785+0000","last_fullsized":"2024-05-09T01:24:32.436785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:52.637108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166101+0000","last_change":"2024-05-09T01:22:39.845390+0000","last_active":"2024-05-09T01:24:21.166101+0000","last_peered":"2024-05-09T01:24:21.166101+0000","last_clean":"2024-05-09T01:24:21.166101+0000","last_became_active":"2024-05-09T01:22:39.844918+0000","last_became_peered":"2024-05-09T01:22:39.844918+0000","last_unstale":"2024-05-09T01:24:21.166101+0000","last_undegraded":"2024-05-09T01:24:21.166101+0000","last_fullsized":"2024-05-09T01:24:21.166101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:04.455794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421710+0000","last_change":"2024-05-09T01:22:36.803329+0000","last_active":"2024-05-09T01:23:22.421710+0000","last_peered":"2024-05-09T01:23:22.421710+0000","last_clean":"2024-05-09T01:23:22.421710+0000","last_became_active":"2024-05-09T01:22:36.802831+0000","last_became_peered":"2024-05-09T01:22:36.802831+0000","last_unstale":"2024-05-09T01:23:22.421710+0000","last_undegraded":"2024-05-09T01:23:22.421710+0000","last_fullsized":"2024-05-09T01:23:22.421710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:57.919840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154637+0000","last_change":"2024-05-09T01:22:39.837595+0000","last_active":"2024-05-09T01:24:21.154637+0000","last_peered":"2024-05-09T01:24:21.154637+0000","last_clean":"2024-05-09T01:24:21.154637+0000","last_became_active":"2024-05-09T01:22:39.837197+0000","last_became_peered":"2024-05-09T01:22:39.837197+0000","last_unstale":"2024-05-09T01:24:21.154637+0000","last_undegraded":"2024-05-09T01:24:21.154637+0000","last_fullsized":"2024-05-09T01:24:21.154637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:08.700741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.321286+0000","last_change":"2024-05-09T01:23:22.321286+0000","last_active":"2024-05-09T01:23:22.321286+0000","last_peered":"2024-05-09T01:23:22.321286+0000","last_clean":"2024-05-09T01:23:22.321286+0000","last_became_active":"2024-05-09T01:23:22.315810+0000","last_became_peered":"2024-05-09T01:23:22.315810+0000","last_unstale":"2024-05-09T01:23:22.321286+0000","last_undegraded":"2024-05-09T01:23:22.321286+0000","last_fullsized":"2024-05-09T01:23:22.321286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:43.100893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.165205+0000","last_change":"2024-05-09T01:22:39.827798+0000","last_active":"2024-05-09T01:24:21.165205+0000","last_peered":"2024-05-09T01:24:21.165205+0000","last_clean":"2024-05-09T01:24:21.165205+0000","last_became_active":"2024-05-09T01:22:39.827322+0000","last_became_peered":"2024-05-09T01:22:39.827322+0000","last_unstale":"2024-05-09T01:24:21.165205+0000","last_undegraded":"2024-05-09T01:24:21.165205+0000","last_fullsized":"2024-05-09T01:24:21.165205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:52.454033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.149595+0000","last_change":"2024-05-09T01:22:39.841885+0000","last_active":"2024-05-09T01:24:21.149595+0000","last_peered":"2024-05-09T01:24:21.149595+0000","last_clean":"2024-05-09T01:24:21.149595+0000","last_became_active":"2024-05-09T01:22:39.841160+0000","last_became_peered":"2024-05-09T01:22:39.841160+0000","last_unstale":"2024-05-09T01:24:21.149595+0000","last_undegraded":"2024-05-09T01:24:21.149595+0000","last_fullsized":"2024-05-09T01:24:21.149595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:57.290465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145960+0000","last_change":"2024-05-09T01:22:39.846495+0000","last_active":"2024-05-09T01:24:21.145960+0000","last_peered":"2024-05-09T01:24:21.145960+0000","last_clean":"2024-05-09T01:24:21.145960+0000","last_became_active":"2024-05-09T01:22:39.846108+0000","last_became_peered":"2024-05-09T01:22:39.846108+0000","last_unstale":"2024-05-09T01:24:21.145960+0000","last_undegraded":"2024-05-09T01:24:21.145960+0000","last_fullsized":"2024-05-09T01:24:21.145960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:39.958174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.131907+0000","last_change":"2024-05-09T01:22:39.841288+0000","last_active":"2024-05-09T01:24:21.131907+0000","last_peered":"2024-05-09T01:24:21.131907+0000","last_clean":"2024-05-09T01:24:21.131907+0000","last_became_active":"2024-05-09T01:22:39.840900+0000","last_became_peered":"2024-05-09T01:22:39.840900+0000","last_unstale":"2024-05-09T01:24:21.131907+0000","last_undegraded":"2024-05-09T01:24:21.131907+0000","last_fullsized":"2024-05-09T01:24:21.131907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:30.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141204+0000","last_change":"2024-05-09T01:23:22.313135+0000","last_active":"2024-05-09T01:24:21.141204+0000","last_peered":"2024-05-09T01:24:21.141204+0000","last_clean":"2024-05-09T01:24:21.141204+0000","last_became_active":"2024-05-09T01:23:22.312710+0000","last_became_peered":"2024-05-09T01:23:22.312710+0000","last_unstale":"2024-05-09T01:24:21.141204+0000","last_undegraded":"2024-05-09T01:24:21.141204+0000","last_fullsized":"2024-05-09T01:24:21.141204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:47.725790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153527+0000","last_change":"2024-05-09T01:22:39.838037+0000","last_active":"2024-05-09T01:24:21.153527+0000","last_peered":"2024-05-09T01:24:21.153527+0000","last_clean":"2024-05-09T01:24:21.153527+0000","last_became_active":"2024-05-09T01:22:39.837362+0000","last_became_peered":"2024-05-09T01:22:39.837362+0000","last_unstale":"2024-05-09T01:24:21.153527+0000","last_undegraded":"2024-05-09T01:24:21.153527+0000","last_fullsized":"2024-05-09T01:24:21.153527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.781318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146345+0000","last_change":"2024-05-09T01:23:22.310046+0000","last_active":"2024-05-09T01:24:21.146345+0000","last_peered":"2024-05-09T01:24:21.146345+0000","last_clean":"2024-05-09T01:24:21.146345+0000","last_became_active":"2024-05-09T01:23:22.309541+0000","last_became_peered":"2024-05-09T01:23:22.309541+0000","last_unstale":"2024-05-09T01:24:21.146345+0000","last_undegraded":"2024-05-09T01:24:21.146345+0000","last_fullsized":"2024-05-09T01:24:21.146345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:30.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.161164+0000","last_change":"2024-05-09T01:22:39.840453+0000","last_active":"2024-05-09T01:24:21.161164+0000","last_peered":"2024-05-09T01:24:21.161164+0000","last_clean":"2024-05-09T01:24:21.161164+0000","last_became_active":"2024-05-09T01:22:39.840077+0000","last_became_peered":"2024-05-09T01:22:39.840077+0000","last_unstale":"2024-05-09T01:24:21.161164+0000","last_undegraded":"2024-05-09T01:24:21.161164+0000","last_fullsized":"2024-05-09T01:24:21.161164+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:20.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.160729+0000","last_change":"2024-05-09T01:22:39.844554+0000","last_active":"2024-05-09T01:24:21.160729+0000","last_peered":"2024-05-09T01:24:21.160729+0000","last_clean":"2024-05-09T01:24:21.160729+0000","last_became_active":"2024-05-09T01:22:39.843640+0000","last_became_peered":"2024-05-09T01:22:39.843640+0000","last_unstale":"2024-05-09T01:24:21.160729+0000","last_undegraded":"2024-05-09T01:24:21.160729+0000","last_fullsized":"2024-05-09T01:24:21.160729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:27.801565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140606+0000","last_change":"2024-05-09T01:22:39.839043+0000","last_active":"2024-05-09T01:24:21.140606+0000","last_peered":"2024-05-09T01:24:21.140606+0000","last_clean":"2024-05-09T01:24:21.140606+0000","last_became_active":"2024-05-09T01:22:39.838662+0000","last_became_peered":"2024-05-09T01:22:39.838662+0000","last_unstale":"2024-05-09T01:24:21.140606+0000","last_undegraded":"2024-05-09T01:24:21.140606+0000","last_fullsized":"2024-05-09T01:24:21.140606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:16.117689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.138867+0000","last_change":"2024-05-09T01:23:22.309945+0000","last_active":"2024-05-09T01:24:21.138867+0000","last_peered":"2024-05-09T01:24:21.138867+0000","last_clean":"2024-05-09T01:24:21.138867+0000","last_became_active":"2024-05-09T01:23:22.309543+0000","last_became_peered":"2024-05-09T01:23:22.309543+0000","last_unstale":"2024-05-09T01:24:21.138867+0000","last_undegraded":"2024-05-09T01:24:21.138867+0000","last_fullsized":"2024-05-09T01:24:21.138867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:47.464524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164415+0000","last_change":"2024-05-09T01:22:39.838911+0000","last_active":"2024-05-09T01:24:21.164415+0000","last_peered":"2024-05-09T01:24:21.164415+0000","last_clean":"2024-05-09T01:24:21.164415+0000","last_became_active":"2024-05-09T01:22:39.838293+0000","last_became_peered":"2024-05-09T01:22:39.838293+0000","last_unstale":"2024-05-09T01:24:21.164415+0000","last_undegraded":"2024-05-09T01:24:21.164415+0000","last_fullsized":"2024-05-09T01:24:21.164415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:32.596306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163381+0000","last_change":"2024-05-09T01:22:39.838185+0000","last_active":"2024-05-09T01:24:21.163381+0000","last_peered":"2024-05-09T01:24:21.163381+0000","last_clean":"2024-05-09T01:24:21.163381+0000","last_became_active":"2024-05-09T01:22:39.837530+0000","last_became_peered":"2024-05-09T01:22:39.837530+0000","last_unstale":"2024-05-09T01:24:21.163381+0000","last_undegraded":"2024-05-09T01:24:21.163381+0000","last_fullsized":"2024-05-09T01:24:21.163381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:22.556710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150810+0000","last_change":"2024-05-09T01:22:39.834124+0000","last_active":"2024-05-09T01:24:21.150810+0000","last_peered":"2024-05-09T01:24:21.150810+0000","last_clean":"2024-05-09T01:24:21.150810+0000","last_became_active":"2024-05-09T01:22:39.833716+0000","last_became_peered":"2024-05-09T01:22:39.833716+0000","last_unstale":"2024-05-09T01:24:21.150810+0000","last_undegraded":"2024-05-09T01:24:21.150810+0000","last_fullsized":"2024-05-09T01:24:21.150810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:54:42.337900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70017,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":981,"num_read_kb":579,"num_write":259,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2345,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612648,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":437733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1090649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":437019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168631,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":913408,"data_stored":436613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364859,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1030588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528386,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":437268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":500852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":438607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":498698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117072,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":436561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":438831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1540096,"data_stored":1029202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T01:24:38.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T01:24:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:38.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:38 smithi059 ceph-mon[34351]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T01:24:38.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:38 smithi019 ceph-mon[30158]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T01:24:38.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:38 smithi196 ceph-mon[35041]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T01:24:39.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:39 smithi059 ceph-mon[34351]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:24:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:39 smithi019 ceph-mon[30158]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:24:39.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:39 smithi196 ceph-mon[35041]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:24:40.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.162+0000 7ffbc5290700 1 -- 172.21.15.19:0/2925675410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0102790 msgr2=0x7ffbc0102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:40.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.162+0000 7ffbc5290700 1 --2- 172.21.15.19:0/2925675410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0102790 0x7ffbc0102ba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009990 tx=0x7ffbb4009ca0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.162+0000 7ffbc5290700 1 -- 172.21.15.19:0/2925675410 shutdown_connections 2024-05-09T01:24:40.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.162+0000 7ffbc5290700 1 --2- 172.21.15.19:0/2925675410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc01030e0 0x7ffbc0111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.162+0000 7ffbc5290700 1 --2- 172.21.15.19:0/2925675410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc0109300 0x7ffbc0109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.162+0000 7ffbc5290700 1 --2- 172.21.15.19:0/2925675410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0102790 0x7ffbc0102ba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.162+0000 7ffbc5290700 1 -- 172.21.15.19:0/2925675410 >> 172.21.15.19:0/2925675410 conn(0x7ffbc00fe520 msgr2=0x7ffbc0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:40.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.162+0000 7ffbc5290700 1 -- 172.21.15.19:0/2925675410 shutdown_connections 2024-05-09T01:24:40.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.162+0000 7ffbc5290700 1 -- 172.21.15.19:0/2925675410 wait complete. 2024-05-09T01:24:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.163+0000 7ffbc5290700 1 Processor -- start 2024-05-09T01:24:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.163+0000 7ffbc5290700 1 -- start start 2024-05-09T01:24:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbc5290700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc0102790 0x7ffbc0103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbc5290700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc01030e0 0x7ffbc0103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbc5290700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0109300 0x7ffbc01043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbc5290700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc01145a0 con 0x7ffbc0109300 2024-05-09T01:24:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbc5290700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0114420 con 0x7ffbc01030e0 2024-05-09T01:24:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbc5290700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0114720 con 0x7ffbc0102790 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0109300 0x7ffbc01043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbeffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc0102790 0x7ffbc0103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0109300 0x7ffbc01043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46822/0 (socket says 172.21.15.19:46822) 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbf7fe700 1 -- 172.21.15.19:0/3316401889 learned_addr learned my addr 172.21.15.19:0/3316401889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbe7fc700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc01030e0 0x7ffbc0103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbf7fe700 1 -- 172.21.15.19:0/3316401889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc0102790 msgr2=0x7ffbc0103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbf7fe700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc0102790 0x7ffbc0103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbf7fe700 1 -- 172.21.15.19:0/3316401889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc01030e0 msgr2=0x7ffbc0103eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbf7fe700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc01030e0 0x7ffbc0103eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbf7fe700 1 -- 172.21.15.19:0/3316401889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc01a9220 con 0x7ffbc0109300 2024-05-09T01:24:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.164+0000 7ffbbf7fe700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0109300 0x7ffbc01043f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007d70 tx=0x7ffbb800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:40.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.165+0000 7ffba7fff700 1 -- 172.21.15.19:0/3316401889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8016020 con 0x7ffbc0109300 2024-05-09T01:24:40.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.165+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc01a9510 con 0x7ffbc0109300 2024-05-09T01:24:40.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.165+0000 7ffba7fff700 1 -- 172.21.15.19:0/3316401889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb800ac90 con 0x7ffbc0109300 2024-05-09T01:24:40.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.165+0000 7ffba7fff700 1 -- 172.21.15.19:0/3316401889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80115e0 con 0x7ffbc0109300 2024-05-09T01:24:40.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.165+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc01a9a50 con 0x7ffbc0109300 2024-05-09T01:24:40.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.166+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc00512c0 con 0x7ffbc0109300 2024-05-09T01:24:40.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.167+0000 7ffba7fff700 1 -- 172.21.15.19:0/3316401889 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffbb8019020 con 0x7ffbc0109300 2024-05-09T01:24:40.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.168+0000 7ffba7fff700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffba80b2900 0x7ffba80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:40.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.168+0000 7ffbbeffd700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffba80b2900 0x7ffba80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:40.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.168+0000 7ffba7fff700 1 -- 172.21.15.19:0/3316401889 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ffbb80e2190 con 0x7ffbc0109300 2024-05-09T01:24:40.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.169+0000 7ffbbeffd700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffba80b2900 0x7ffba80b4e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffbc006ca30 tx=0x7ffbb000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:40.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.173+0000 7ffba7fff700 1 -- 172.21.15.19:0/3316401889 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffbb8020020 con 0x7ffbc0109300 2024-05-09T01:24:40.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.384+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbc0063c00 con 0x7ffba80b2900 2024-05-09T01:24:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.392+0000 7ffba7fff700 1 -- 172.21.15.19:0/3316401889 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258880 (secure 0 0 0) 0x7ffbc0063c00 con 0x7ffba80b2900 2024-05-09T01:24:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:24:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffba80b2900 msgr2=0x7ffba80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffba80b2900 0x7ffba80b4e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffbc006ca30 tx=0x7ffbb000b410 comp rx=0 tx=0).stop 2024-05-09T01:24:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0109300 msgr2=0x7ffbc01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0109300 0x7ffbc01043f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007d70 tx=0x7ffbb800a340 comp rx=0 tx=0).stop 2024-05-09T01:24:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 shutdown_connections 2024-05-09T01:24:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ffba80b2900 0x7ffba80b4e10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc0109300 0x7ffbc01043f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc0102790 0x7ffbc0103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 --2- 172.21.15.19:0/3316401889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc01030e0 0x7ffbc0103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 >> 172.21.15.19:0/3316401889 conn(0x7ffbc00fe520 msgr2=0x7ffbc0107260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 shutdown_connections 2024-05-09T01:24:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:40.395+0000 7ffbc5290700 1 -- 172.21.15.19:0/3316401889 wait complete. 2024-05-09T01:24:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T01:24:40.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:40 smithi059 ceph-mon[34351]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:40.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:40 smithi019 ceph-mon[30158]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:40 smithi196 ceph-mon[35041]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:41.008 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-05-09T01:24:39.154890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660619,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1077,"num_read_kb":643,"num_write":384,"num_write_kb":2299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":2954185,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":817,"ondisk_log_size":817,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386940,"kb_used_data":12828,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554884,"statfs":{"total":1151940427776,"available":1151544201216,"internally_reserved":0,"allocated":13135872,"data_stored":7212621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003674"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158309+0000","last_change":"2024-05-09T01:22:39.836756+0000","last_active":"2024-05-09T01:24:21.158309+0000","last_peered":"2024-05-09T01:24:21.158309+0000","last_clean":"2024-05-09T01:24:21.158309+0000","last_became_active":"2024-05-09T01:22:39.835505+0000","last_became_peered":"2024-05-09T01:22:39.835505+0000","last_unstale":"2024-05-09T01:24:21.158309+0000","last_undegraded":"2024-05-09T01:24:21.158309+0000","last_fullsized":"2024-05-09T01:24:21.158309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:29.109762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148038+0000","last_change":"2024-05-09T01:22:39.842897+0000","last_active":"2024-05-09T01:24:21.148038+0000","last_peered":"2024-05-09T01:24:21.148038+0000","last_clean":"2024-05-09T01:24:21.148038+0000","last_became_active":"2024-05-09T01:22:39.842537+0000","last_became_peered":"2024-05-09T01:22:39.842537+0000","last_unstale":"2024-05-09T01:24:21.148038+0000","last_undegraded":"2024-05-09T01:24:21.148038+0000","last_fullsized":"2024-05-09T01:24:21.148038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:20.429654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157311+0000","last_change":"2024-05-09T01:22:39.847176+0000","last_active":"2024-05-09T01:24:21.157311+0000","last_peered":"2024-05-09T01:24:21.157311+0000","last_clean":"2024-05-09T01:24:21.157311+0000","last_became_active":"2024-05-09T01:22:39.846782+0000","last_became_peered":"2024-05-09T01:22:39.846782+0000","last_unstale":"2024-05-09T01:24:21.157311+0000","last_undegraded":"2024-05-09T01:24:21.157311+0000","last_fullsized":"2024-05-09T01:24:21.157311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:06.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134296+0000","last_change":"2024-05-09T01:22:39.842399+0000","last_active":"2024-05-09T01:24:21.134296+0000","last_peered":"2024-05-09T01:24:21.134296+0000","last_clean":"2024-05-09T01:24:21.134296+0000","last_became_active":"2024-05-09T01:22:39.842035+0000","last_became_peered":"2024-05-09T01:22:39.842035+0000","last_unstale":"2024-05-09T01:24:21.134296+0000","last_undegraded":"2024-05-09T01:24:21.134296+0000","last_fullsized":"2024-05-09T01:24:21.134296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:35.011920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.124470+0000","last_change":"2024-05-09T01:22:39.838574+0000","last_active":"2024-05-09T01:24:21.124470+0000","last_peered":"2024-05-09T01:24:21.124470+0000","last_clean":"2024-05-09T01:24:21.124470+0000","last_became_active":"2024-05-09T01:22:39.838059+0000","last_became_peered":"2024-05-09T01:22:39.838059+0000","last_unstale":"2024-05-09T01:24:21.124470+0000","last_undegraded":"2024-05-09T01:24:21.124470+0000","last_fullsized":"2024-05-09T01:24:21.124470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:20.968526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153227+0000","last_change":"2024-05-09T01:22:39.842407+0000","last_active":"2024-05-09T01:24:21.153227+0000","last_peered":"2024-05-09T01:24:21.153227+0000","last_clean":"2024-05-09T01:24:21.153227+0000","last_became_active":"2024-05-09T01:22:39.841982+0000","last_became_peered":"2024-05-09T01:22:39.841982+0000","last_unstale":"2024-05-09T01:24:21.153227+0000","last_undegraded":"2024-05-09T01:24:21.153227+0000","last_fullsized":"2024-05-09T01:24:21.153227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:17.086304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155073+0000","last_change":"2024-05-09T01:22:39.833800+0000","last_active":"2024-05-09T01:24:21.155073+0000","last_peered":"2024-05-09T01:24:21.155073+0000","last_clean":"2024-05-09T01:24:21.155073+0000","last_became_active":"2024-05-09T01:22:39.833388+0000","last_became_peered":"2024-05-09T01:22:39.833388+0000","last_unstale":"2024-05-09T01:24:21.155073+0000","last_undegraded":"2024-05-09T01:24:21.155073+0000","last_fullsized":"2024-05-09T01:24:21.155073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:35.341428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142076+0000","last_change":"2024-05-09T01:22:39.845395+0000","last_active":"2024-05-09T01:24:21.142076+0000","last_peered":"2024-05-09T01:24:21.142076+0000","last_clean":"2024-05-09T01:24:21.142076+0000","last_became_active":"2024-05-09T01:22:39.844946+0000","last_became_peered":"2024-05-09T01:22:39.844946+0000","last_unstale":"2024-05-09T01:24:21.142076+0000","last_undegraded":"2024-05-09T01:24:21.142076+0000","last_fullsized":"2024-05-09T01:24:21.142076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:44.750614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159615+0000","last_change":"2024-05-09T01:22:39.832894+0000","last_active":"2024-05-09T01:24:21.159615+0000","last_peered":"2024-05-09T01:24:21.159615+0000","last_clean":"2024-05-09T01:24:21.159615+0000","last_became_active":"2024-05-09T01:22:39.829743+0000","last_became_peered":"2024-05-09T01:22:39.829743+0000","last_unstale":"2024-05-09T01:24:21.159615+0000","last_undegraded":"2024-05-09T01:24:21.159615+0000","last_fullsized":"2024-05-09T01:24:21.159615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.463833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155782+0000","last_change":"2024-05-09T01:22:39.843131+0000","last_active":"2024-05-09T01:24:21.155782+0000","last_peered":"2024-05-09T01:24:21.155782+0000","last_clean":"2024-05-09T01:24:21.155782+0000","last_became_active":"2024-05-09T01:22:39.842690+0000","last_became_peered":"2024-05-09T01:22:39.842690+0000","last_unstale":"2024-05-09T01:24:21.155782+0000","last_undegraded":"2024-05-09T01:24:21.155782+0000","last_fullsized":"2024-05-09T01:24:21.155782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:14.172365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166118+0000","last_change":"2024-05-09T01:22:39.845557+0000","last_active":"2024-05-09T01:24:21.166118+0000","last_peered":"2024-05-09T01:24:21.166118+0000","last_clean":"2024-05-09T01:24:21.166118+0000","last_became_active":"2024-05-09T01:22:39.845059+0000","last_became_peered":"2024-05-09T01:22:39.845059+0000","last_unstale":"2024-05-09T01:24:21.166118+0000","last_undegraded":"2024-05-09T01:24:21.166118+0000","last_fullsized":"2024-05-09T01:24:21.166118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:47.238080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.128844+0000","last_change":"2024-05-09T01:22:39.842237+0000","last_active":"2024-05-09T01:24:21.128844+0000","last_peered":"2024-05-09T01:24:21.128844+0000","last_clean":"2024-05-09T01:24:21.128844+0000","last_became_active":"2024-05-09T01:22:39.841849+0000","last_became_peered":"2024-05-09T01:22:39.841849+0000","last_unstale":"2024-05-09T01:24:21.128844+0000","last_undegraded":"2024-05-09T01:24:21.128844+0000","last_fullsized":"2024-05-09T01:24:21.128844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:40.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150079+0000","last_change":"2024-05-09T01:22:39.839943+0000","last_active":"2024-05-09T01:24:21.150079+0000","last_peered":"2024-05-09T01:24:21.150079+0000","last_clean":"2024-05-09T01:24:21.150079+0000","last_became_active":"2024-05-09T01:22:39.839604+0000","last_became_peered":"2024-05-09T01:22:39.839604+0000","last_unstale":"2024-05-09T01:24:21.150079+0000","last_undegraded":"2024-05-09T01:24:21.150079+0000","last_fullsized":"2024-05-09T01:24:21.150079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:34.854263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.144875+0000","last_change":"2024-05-09T01:22:39.846981+0000","last_active":"2024-05-09T01:24:21.144875+0000","last_peered":"2024-05-09T01:24:21.144875+0000","last_clean":"2024-05-09T01:24:21.144875+0000","last_became_active":"2024-05-09T01:22:39.846586+0000","last_became_peered":"2024-05-09T01:22:39.846586+0000","last_unstale":"2024-05-09T01:24:21.144875+0000","last_undegraded":"2024-05-09T01:24:21.144875+0000","last_fullsized":"2024-05-09T01:24:21.144875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:50.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151118+0000","last_change":"2024-05-09T01:22:39.848964+0000","last_active":"2024-05-09T01:24:21.151118+0000","last_peered":"2024-05-09T01:24:21.151118+0000","last_clean":"2024-05-09T01:24:21.151118+0000","last_became_active":"2024-05-09T01:22:39.848442+0000","last_became_peered":"2024-05-09T01:22:39.848442+0000","last_unstale":"2024-05-09T01:24:21.151118+0000","last_undegraded":"2024-05-09T01:24:21.151118+0000","last_fullsized":"2024-05-09T01:24:21.151118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:35.802739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163088+0000","last_change":"2024-05-09T01:22:39.840586+0000","last_active":"2024-05-09T01:24:21.163088+0000","last_peered":"2024-05-09T01:24:21.163088+0000","last_clean":"2024-05-09T01:24:21.163088+0000","last_became_active":"2024-05-09T01:22:39.839887+0000","last_became_peered":"2024-05-09T01:22:39.839887+0000","last_unstale":"2024-05-09T01:24:21.163088+0000","last_undegraded":"2024-05-09T01:24:21.163088+0000","last_fullsized":"2024-05-09T01:24:21.163088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:36.510039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158490+0000","last_change":"2024-05-09T01:22:39.836152+0000","last_active":"2024-05-09T01:24:21.158490+0000","last_peered":"2024-05-09T01:24:21.158490+0000","last_clean":"2024-05-09T01:24:21.158490+0000","last_became_active":"2024-05-09T01:22:39.834691+0000","last_became_peered":"2024-05-09T01:22:39.834691+0000","last_unstale":"2024-05-09T01:24:21.158490+0000","last_undegraded":"2024-05-09T01:24:21.158490+0000","last_fullsized":"2024-05-09T01:24:21.158490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:31.331096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163367+0000","last_change":"2024-05-09T01:22:39.839760+0000","last_active":"2024-05-09T01:24:21.163367+0000","last_peered":"2024-05-09T01:24:21.163367+0000","last_clean":"2024-05-09T01:24:21.163367+0000","last_became_active":"2024-05-09T01:22:39.839403+0000","last_became_peered":"2024-05-09T01:22:39.839403+0000","last_unstale":"2024-05-09T01:24:21.163367+0000","last_undegraded":"2024-05-09T01:24:21.163367+0000","last_fullsized":"2024-05-09T01:24:21.163367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:19.248368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142656+0000","last_change":"2024-05-09T01:22:39.837538+0000","last_active":"2024-05-09T01:24:21.142656+0000","last_peered":"2024-05-09T01:24:21.142656+0000","last_clean":"2024-05-09T01:24:21.142656+0000","last_became_active":"2024-05-09T01:22:39.837167+0000","last_became_peered":"2024-05-09T01:22:39.837167+0000","last_unstale":"2024-05-09T01:24:21.142656+0000","last_undegraded":"2024-05-09T01:24:21.142656+0000","last_fullsized":"2024-05-09T01:24:21.142656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:35.695077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:34.527510+0000","last_change":"2024-05-09T01:23:22.433852+0000","last_active":"2024-05-09T01:24:34.527510+0000","last_peered":"2024-05-09T01:24:34.527510+0000","last_clean":"2024-05-09T01:24:34.527510+0000","last_became_active":"2024-05-09T01:23:22.423465+0000","last_became_peered":"2024-05-09T01:23:22.423465+0000","last_unstale":"2024-05-09T01:24:34.527510+0000","last_undegraded":"2024-05-09T01:24:34.527510+0000","last_fullsized":"2024-05-09T01:24:34.527510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:58.578798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.135977+0000","last_change":"2024-05-09T01:22:39.842667+0000","last_active":"2024-05-09T01:24:21.135977+0000","last_peered":"2024-05-09T01:24:21.135977+0000","last_clean":"2024-05-09T01:24:21.135977+0000","last_became_active":"2024-05-09T01:22:39.842256+0000","last_became_peered":"2024-05-09T01:22:39.842256+0000","last_unstale":"2024-05-09T01:24:21.135977+0000","last_undegraded":"2024-05-09T01:24:21.135977+0000","last_fullsized":"2024-05-09T01:24:21.135977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:28.525790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.314581+0000","last_change":"2024-05-09T01:22:36.841515+0000","last_active":"2024-05-09T01:23:22.314581+0000","last_peered":"2024-05-09T01:23:22.314581+0000","last_clean":"2024-05-09T01:23:22.314581+0000","last_became_active":"2024-05-09T01:22:36.841128+0000","last_became_peered":"2024-05-09T01:22:36.841128+0000","last_unstale":"2024-05-09T01:23:22.314581+0000","last_undegraded":"2024-05-09T01:23:22.314581+0000","last_fullsized":"2024-05-09T01:23:22.314581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:13.598177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157522+0000","last_change":"2024-05-09T01:22:39.843491+0000","last_active":"2024-05-09T01:24:21.157522+0000","last_peered":"2024-05-09T01:24:21.157522+0000","last_clean":"2024-05-09T01:24:21.157522+0000","last_became_active":"2024-05-09T01:22:39.843098+0000","last_became_peered":"2024-05-09T01:22:39.843098+0000","last_unstale":"2024-05-09T01:24:21.157522+0000","last_undegraded":"2024-05-09T01:24:21.157522+0000","last_fullsized":"2024-05-09T01:24:21.157522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:03:33.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303907+0000","last_change":"2024-05-09T01:22:36.798217+0000","last_active":"2024-05-09T01:23:22.303907+0000","last_peered":"2024-05-09T01:23:22.303907+0000","last_clean":"2024-05-09T01:23:22.303907+0000","last_became_active":"2024-05-09T01:22:36.797847+0000","last_became_peered":"2024-05-09T01:22:36.797847+0000","last_unstale":"2024-05-09T01:23:22.303907+0000","last_undegraded":"2024-05-09T01:23:22.303907+0000","last_fullsized":"2024-05-09T01:23:22.303907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:21.116535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164555+0000","last_change":"2024-05-09T01:22:39.837568+0000","last_active":"2024-05-09T01:24:21.164555+0000","last_peered":"2024-05-09T01:24:21.164555+0000","last_clean":"2024-05-09T01:24:21.164555+0000","last_became_active":"2024-05-09T01:22:39.836159+0000","last_became_peered":"2024-05-09T01:22:39.836159+0000","last_unstale":"2024-05-09T01:24:21.164555+0000","last_undegraded":"2024-05-09T01:24:21.164555+0000","last_fullsized":"2024-05-09T01:24:21.164555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:28.089830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303388+0000","last_change":"2024-05-09T01:22:36.798185+0000","last_active":"2024-05-09T01:23:22.303388+0000","last_peered":"2024-05-09T01:23:22.303388+0000","last_clean":"2024-05-09T01:23:22.303388+0000","last_became_active":"2024-05-09T01:22:36.797522+0000","last_became_peered":"2024-05-09T01:22:36.797522+0000","last_unstale":"2024-05-09T01:23:22.303388+0000","last_undegraded":"2024-05-09T01:23:22.303388+0000","last_fullsized":"2024-05-09T01:23:22.303388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:40.960006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153830+0000","last_change":"2024-05-09T01:22:39.837436+0000","last_active":"2024-05-09T01:24:21.153830+0000","last_peered":"2024-05-09T01:24:21.153830+0000","last_clean":"2024-05-09T01:24:21.153830+0000","last_became_active":"2024-05-09T01:22:39.837050+0000","last_became_peered":"2024-05-09T01:22:39.837050+0000","last_unstale":"2024-05-09T01:24:21.153830+0000","last_undegraded":"2024-05-09T01:24:21.153830+0000","last_fullsized":"2024-05-09T01:24:21.153830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:07.149311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:36.781025+0000","last_change":"2024-05-09T01:22:36.800767+0000","last_active":"2024-05-09T01:24:36.781025+0000","last_peered":"2024-05-09T01:24:36.781025+0000","last_clean":"2024-05-09T01:24:36.781025+0000","last_became_active":"2024-05-09T01:22:36.800183+0000","last_became_peered":"2024-05-09T01:22:36.800183+0000","last_unstale":"2024-05-09T01:24:36.781025+0000","last_undegraded":"2024-05-09T01:24:36.781025+0000","last_fullsized":"2024-05-09T01:24:36.781025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:05.682889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.131244+0000","last_change":"2024-05-09T01:22:39.832798+0000","last_active":"2024-05-09T01:24:21.131244+0000","last_peered":"2024-05-09T01:24:21.131244+0000","last_clean":"2024-05-09T01:24:21.131244+0000","last_became_active":"2024-05-09T01:22:39.828803+0000","last_became_peered":"2024-05-09T01:22:39.828803+0000","last_unstale":"2024-05-09T01:24:21.131244+0000","last_undegraded":"2024-05-09T01:24:21.131244+0000","last_fullsized":"2024-05-09T01:24:21.131244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:23.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305959+0000","last_change":"2024-05-09T01:22:36.794171+0000","last_active":"2024-05-09T01:23:22.305959+0000","last_peered":"2024-05-09T01:23:22.305959+0000","last_clean":"2024-05-09T01:23:22.305959+0000","last_became_active":"2024-05-09T01:22:36.793713+0000","last_became_peered":"2024-05-09T01:22:36.793713+0000","last_unstale":"2024-05-09T01:23:22.305959+0000","last_undegraded":"2024-05-09T01:23:22.305959+0000","last_fullsized":"2024-05-09T01:23:22.305959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:06.654667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153309+0000","last_change":"2024-05-09T01:22:39.838266+0000","last_active":"2024-05-09T01:24:21.153309+0000","last_peered":"2024-05-09T01:24:21.153309+0000","last_clean":"2024-05-09T01:24:21.153309+0000","last_became_active":"2024-05-09T01:22:39.837911+0000","last_became_peered":"2024-05-09T01:22:39.837911+0000","last_unstale":"2024-05-09T01:24:21.153309+0000","last_undegraded":"2024-05-09T01:24:21.153309+0000","last_fullsized":"2024-05-09T01:24:21.153309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:39.414176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308540+0000","last_change":"2024-05-09T01:22:36.799766+0000","last_active":"2024-05-09T01:23:22.308540+0000","last_peered":"2024-05-09T01:23:22.308540+0000","last_clean":"2024-05-09T01:23:22.308540+0000","last_became_active":"2024-05-09T01:22:36.799388+0000","last_became_peered":"2024-05-09T01:22:36.799388+0000","last_unstale":"2024-05-09T01:23:22.308540+0000","last_undegraded":"2024-05-09T01:23:22.308540+0000","last_fullsized":"2024-05-09T01:23:22.308540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:13.401659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141635+0000","last_change":"2024-05-09T01:22:39.838493+0000","last_active":"2024-05-09T01:24:21.141635+0000","last_peered":"2024-05-09T01:24:21.141635+0000","last_clean":"2024-05-09T01:24:21.141635+0000","last_became_active":"2024-05-09T01:22:39.838086+0000","last_became_peered":"2024-05-09T01:22:39.838086+0000","last_unstale":"2024-05-09T01:24:21.141635+0000","last_undegraded":"2024-05-09T01:24:21.141635+0000","last_fullsized":"2024-05-09T01:24:21.141635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:06.340693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421503+0000","last_change":"2024-05-09T01:22:36.804872+0000","last_active":"2024-05-09T01:23:22.421503+0000","last_peered":"2024-05-09T01:23:22.421503+0000","last_clean":"2024-05-09T01:23:22.421503+0000","last_became_active":"2024-05-09T01:22:36.804439+0000","last_became_peered":"2024-05-09T01:22:36.804439+0000","last_unstale":"2024-05-09T01:23:22.421503+0000","last_undegraded":"2024-05-09T01:23:22.421503+0000","last_fullsized":"2024-05-09T01:23:22.421503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:08.584749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.161879+0000","last_change":"2024-05-09T01:22:39.836579+0000","last_active":"2024-05-09T01:24:21.161879+0000","last_peered":"2024-05-09T01:24:21.161879+0000","last_clean":"2024-05-09T01:24:21.161879+0000","last_became_active":"2024-05-09T01:22:39.836244+0000","last_became_peered":"2024-05-09T01:22:39.836244+0000","last_unstale":"2024-05-09T01:24:21.161879+0000","last_undegraded":"2024-05-09T01:24:21.161879+0000","last_fullsized":"2024-05-09T01:24:21.161879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:43.329448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":225,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:34.777657+0000","last_change":"2024-05-09T01:22:36.796639+0000","last_active":"2024-05-09T01:24:34.777657+0000","last_peered":"2024-05-09T01:24:34.777657+0000","last_clean":"2024-05-09T01:24:34.777657+0000","last_became_active":"2024-05-09T01:22:36.796227+0000","last_became_peered":"2024-05-09T01:22:36.796227+0000","last_unstale":"2024-05-09T01:24:34.777657+0000","last_undegraded":"2024-05-09T01:24:34.777657+0000","last_fullsized":"2024-05-09T01:24:34.777657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:56.219281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.162605+0000","last_change":"2024-05-09T01:22:39.841515+0000","last_active":"2024-05-09T01:24:21.162605+0000","last_peered":"2024-05-09T01:24:21.162605+0000","last_clean":"2024-05-09T01:24:21.162605+0000","last_became_active":"2024-05-09T01:22:39.840057+0000","last_became_peered":"2024-05-09T01:22:39.840057+0000","last_unstale":"2024-05-09T01:24:21.162605+0000","last_undegraded":"2024-05-09T01:24:21.162605+0000","last_fullsized":"2024-05-09T01:24:21.162605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:43.258466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309092+0000","last_change":"2024-05-09T01:22:36.801937+0000","last_active":"2024-05-09T01:23:22.309092+0000","last_peered":"2024-05-09T01:23:22.309092+0000","last_clean":"2024-05-09T01:23:22.309092+0000","last_became_active":"2024-05-09T01:22:36.801563+0000","last_became_peered":"2024-05-09T01:22:36.801563+0000","last_unstale":"2024-05-09T01:23:22.309092+0000","last_undegraded":"2024-05-09T01:23:22.309092+0000","last_fullsized":"2024-05-09T01:23:22.309092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:00:42.170156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151261+0000","last_change":"2024-05-09T01:22:39.849154+0000","last_active":"2024-05-09T01:24:21.151261+0000","last_peered":"2024-05-09T01:24:21.151261+0000","last_clean":"2024-05-09T01:24:21.151261+0000","last_became_active":"2024-05-09T01:22:39.848815+0000","last_became_peered":"2024-05-09T01:22:39.848815+0000","last_unstale":"2024-05-09T01:24:21.151261+0000","last_undegraded":"2024-05-09T01:24:21.151261+0000","last_fullsized":"2024-05-09T01:24:21.151261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:00.300208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421958+0000","last_change":"2024-05-09T01:22:36.843201+0000","last_active":"2024-05-09T01:23:22.421958+0000","last_peered":"2024-05-09T01:23:22.421958+0000","last_clean":"2024-05-09T01:23:22.421958+0000","last_became_active":"2024-05-09T01:22:36.842500+0000","last_became_peered":"2024-05-09T01:22:36.842500+0000","last_unstale":"2024-05-09T01:23:22.421958+0000","last_undegraded":"2024-05-09T01:23:22.421958+0000","last_fullsized":"2024-05-09T01:23:22.421958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:35.092246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159455+0000","last_change":"2024-05-09T01:22:39.835120+0000","last_active":"2024-05-09T01:24:21.159455+0000","last_peered":"2024-05-09T01:24:21.159455+0000","last_clean":"2024-05-09T01:24:21.159455+0000","last_became_active":"2024-05-09T01:22:39.834613+0000","last_became_peered":"2024-05-09T01:22:39.834613+0000","last_unstale":"2024-05-09T01:24:21.159455+0000","last_undegraded":"2024-05-09T01:24:21.159455+0000","last_fullsized":"2024-05-09T01:24:21.159455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:08.380445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":265,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:34.790670+0000","last_change":"2024-05-09T01:22:36.800816+0000","last_active":"2024-05-09T01:24:34.790670+0000","last_peered":"2024-05-09T01:24:34.790670+0000","last_clean":"2024-05-09T01:24:34.790670+0000","last_became_active":"2024-05-09T01:22:36.798797+0000","last_became_peered":"2024-05-09T01:22:36.798797+0000","last_unstale":"2024-05-09T01:24:34.790670+0000","last_undegraded":"2024-05-09T01:24:34.790670+0000","last_fullsized":"2024-05-09T01:24:34.790670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:37.670163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61999,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":204,"num_read_kb":162,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134404+0000","last_change":"2024-05-09T01:22:39.831504+0000","last_active":"2024-05-09T01:24:21.134404+0000","last_peered":"2024-05-09T01:24:21.134404+0000","last_clean":"2024-05-09T01:24:21.134404+0000","last_became_active":"2024-05-09T01:22:39.831129+0000","last_became_peered":"2024-05-09T01:22:39.831129+0000","last_unstale":"2024-05-09T01:24:21.134404+0000","last_undegraded":"2024-05-09T01:24:21.134404+0000","last_fullsized":"2024-05-09T01:24:21.134404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:19.212473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304426+0000","last_change":"2024-05-09T01:22:36.803447+0000","last_active":"2024-05-09T01:23:22.304426+0000","last_peered":"2024-05-09T01:23:22.304426+0000","last_clean":"2024-05-09T01:23:22.304426+0000","last_became_active":"2024-05-09T01:22:36.802986+0000","last_became_peered":"2024-05-09T01:22:36.802986+0000","last_unstale":"2024-05-09T01:23:22.304426+0000","last_undegraded":"2024-05-09T01:23:22.304426+0000","last_fullsized":"2024-05-09T01:23:22.304426+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:28.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163194+0000","last_change":"2024-05-09T01:22:39.836546+0000","last_active":"2024-05-09T01:24:21.163194+0000","last_peered":"2024-05-09T01:24:21.163194+0000","last_clean":"2024-05-09T01:24:21.163194+0000","last_became_active":"2024-05-09T01:22:39.836168+0000","last_became_peered":"2024-05-09T01:22:39.836168+0000","last_unstale":"2024-05-09T01:24:21.163194+0000","last_undegraded":"2024-05-09T01:24:21.163194+0000","last_fullsized":"2024-05-09T01:24:21.163194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:45.522279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421596+0000","last_change":"2024-05-09T01:22:36.798703+0000","last_active":"2024-05-09T01:23:22.421596+0000","last_peered":"2024-05-09T01:23:22.421596+0000","last_clean":"2024-05-09T01:23:22.421596+0000","last_became_active":"2024-05-09T01:22:36.798280+0000","last_became_peered":"2024-05-09T01:22:36.798280+0000","last_unstale":"2024-05-09T01:23:22.421596+0000","last_undegraded":"2024-05-09T01:23:22.421596+0000","last_fullsized":"2024-05-09T01:23:22.421596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:33.794488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.165878+0000","last_change":"2024-05-09T01:22:39.831223+0000","last_active":"2024-05-09T01:24:21.165878+0000","last_peered":"2024-05-09T01:24:21.165878+0000","last_clean":"2024-05-09T01:24:21.165878+0000","last_became_active":"2024-05-09T01:22:39.830845+0000","last_became_peered":"2024-05-09T01:22:39.830845+0000","last_unstale":"2024-05-09T01:24:21.165878+0000","last_undegraded":"2024-05-09T01:24:21.165878+0000","last_fullsized":"2024-05-09T01:24:21.165878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:23.365950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:36.811267+0000","last_change":"2024-05-09T01:22:36.802327+0000","last_active":"2024-05-09T01:24:36.811267+0000","last_peered":"2024-05-09T01:24:36.811267+0000","last_clean":"2024-05-09T01:24:36.811267+0000","last_became_active":"2024-05-09T01:22:36.801951+0000","last_became_peered":"2024-05-09T01:22:36.801951+0000","last_unstale":"2024-05-09T01:24:36.811267+0000","last_undegraded":"2024-05-09T01:24:36.811267+0000","last_fullsized":"2024-05-09T01:24:36.811267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:12.282424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154077+0000","last_change":"2024-05-09T01:22:39.833626+0000","last_active":"2024-05-09T01:24:21.154077+0000","last_peered":"2024-05-09T01:24:21.154077+0000","last_clean":"2024-05-09T01:24:21.154077+0000","last_became_active":"2024-05-09T01:22:39.833176+0000","last_became_peered":"2024-05-09T01:22:39.833176+0000","last_unstale":"2024-05-09T01:24:21.154077+0000","last_undegraded":"2024-05-09T01:24:21.154077+0000","last_fullsized":"2024-05-09T01:24:21.154077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:25.082300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.328609+0000","last_change":"2024-05-09T01:23:22.328011+0000","last_active":"2024-05-09T01:23:22.328609+0000","last_peered":"2024-05-09T01:23:22.328609+0000","last_clean":"2024-05-09T01:23:22.328609+0000","last_became_active":"2024-05-09T01:23:22.313049+0000","last_became_peered":"2024-05-09T01:23:22.313049+0000","last_unstale":"2024-05-09T01:23:22.328609+0000","last_undegraded":"2024-05-09T01:23:22.328609+0000","last_fullsized":"2024-05-09T01:23:22.328609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:17.854491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153164+0000","last_change":"2024-05-09T01:22:39.837529+0000","last_active":"2024-05-09T01:24:21.153164+0000","last_peered":"2024-05-09T01:24:21.153164+0000","last_clean":"2024-05-09T01:24:21.153164+0000","last_became_active":"2024-05-09T01:22:39.837155+0000","last_became_peered":"2024-05-09T01:22:39.837155+0000","last_unstale":"2024-05-09T01:24:21.153164+0000","last_undegraded":"2024-05-09T01:24:21.153164+0000","last_fullsized":"2024-05-09T01:24:21.153164+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:44.485629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303529+0000","last_change":"2024-05-09T01:22:36.800073+0000","last_active":"2024-05-09T01:23:22.303529+0000","last_peered":"2024-05-09T01:23:22.303529+0000","last_clean":"2024-05-09T01:23:22.303529+0000","last_became_active":"2024-05-09T01:22:36.799463+0000","last_became_peered":"2024-05-09T01:22:36.799463+0000","last_unstale":"2024-05-09T01:23:22.303529+0000","last_undegraded":"2024-05-09T01:23:22.303529+0000","last_fullsized":"2024-05-09T01:23:22.303529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.942945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150617+0000","last_change":"2024-05-09T01:22:39.833615+0000","last_active":"2024-05-09T01:24:21.150617+0000","last_peered":"2024-05-09T01:24:21.150617+0000","last_clean":"2024-05-09T01:24:21.150617+0000","last_became_active":"2024-05-09T01:22:39.833212+0000","last_became_peered":"2024-05-09T01:22:39.833212+0000","last_unstale":"2024-05-09T01:24:21.150617+0000","last_undegraded":"2024-05-09T01:24:21.150617+0000","last_fullsized":"2024-05-09T01:24:21.150617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:35.387442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305607+0000","last_change":"2024-05-09T01:22:36.843143+0000","last_active":"2024-05-09T01:23:22.305607+0000","last_peered":"2024-05-09T01:23:22.305607+0000","last_clean":"2024-05-09T01:23:22.305607+0000","last_became_active":"2024-05-09T01:22:36.842759+0000","last_became_peered":"2024-05-09T01:22:36.842759+0000","last_unstale":"2024-05-09T01:23:22.305607+0000","last_undegraded":"2024-05-09T01:23:22.305607+0000","last_fullsized":"2024-05-09T01:23:22.305607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:42.666186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.133027+0000","last_change":"2024-05-09T01:22:39.838267+0000","last_active":"2024-05-09T01:24:21.133027+0000","last_peered":"2024-05-09T01:24:21.133027+0000","last_clean":"2024-05-09T01:24:21.133027+0000","last_became_active":"2024-05-09T01:22:39.837910+0000","last_became_peered":"2024-05-09T01:22:39.837910+0000","last_unstale":"2024-05-09T01:24:21.133027+0000","last_undegraded":"2024-05-09T01:24:21.133027+0000","last_fullsized":"2024-05-09T01:24:21.133027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:34.620332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:37.532596+0000","last_change":"2024-05-09T01:22:36.800515+0000","last_active":"2024-05-09T01:24:37.532596+0000","last_peered":"2024-05-09T01:24:37.532596+0000","last_clean":"2024-05-09T01:24:37.532596+0000","last_became_active":"2024-05-09T01:22:36.800065+0000","last_became_peered":"2024-05-09T01:22:36.800065+0000","last_unstale":"2024-05-09T01:24:37.532596+0000","last_undegraded":"2024-05-09T01:24:37.532596+0000","last_fullsized":"2024-05-09T01:24:37.532596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:50.160463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145372+0000","last_change":"2024-05-09T01:23:22.313627+0000","last_active":"2024-05-09T01:24:21.145372+0000","last_peered":"2024-05-09T01:24:21.145372+0000","last_clean":"2024-05-09T01:24:21.145372+0000","last_became_active":"2024-05-09T01:23:22.313138+0000","last_became_peered":"2024-05-09T01:23:22.313138+0000","last_unstale":"2024-05-09T01:24:21.145372+0000","last_undegraded":"2024-05-09T01:24:21.145372+0000","last_fullsized":"2024-05-09T01:24:21.145372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:07.216086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153329+0000","last_change":"2024-05-09T01:22:39.832188+0000","last_active":"2024-05-09T01:24:21.153329+0000","last_peered":"2024-05-09T01:24:21.153329+0000","last_clean":"2024-05-09T01:24:21.153329+0000","last_became_active":"2024-05-09T01:22:39.831302+0000","last_became_peered":"2024-05-09T01:22:39.831302+0000","last_unstale":"2024-05-09T01:24:21.153329+0000","last_undegraded":"2024-05-09T01:24:21.153329+0000","last_fullsized":"2024-05-09T01:24:21.153329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:06.703950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:32.803104+0000","last_change":"2024-05-09T01:22:36.841446+0000","last_active":"2024-05-09T01:24:32.803104+0000","last_peered":"2024-05-09T01:24:32.803104+0000","last_clean":"2024-05-09T01:24:32.803104+0000","last_became_active":"2024-05-09T01:22:36.841071+0000","last_became_peered":"2024-05-09T01:22:36.841071+0000","last_unstale":"2024-05-09T01:24:32.803104+0000","last_undegraded":"2024-05-09T01:24:32.803104+0000","last_fullsized":"2024-05-09T01:24:32.803104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:20.777482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163139+0000","last_change":"2024-05-09T01:22:39.846711+0000","last_active":"2024-05-09T01:24:21.163139+0000","last_peered":"2024-05-09T01:24:21.163139+0000","last_clean":"2024-05-09T01:24:21.163139+0000","last_became_active":"2024-05-09T01:22:39.846293+0000","last_became_peered":"2024-05-09T01:22:39.846293+0000","last_unstale":"2024-05-09T01:24:21.163139+0000","last_undegraded":"2024-05-09T01:24:21.163139+0000","last_fullsized":"2024-05-09T01:24:21.163139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:27.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303787+0000","last_change":"2024-05-09T01:22:36.798302+0000","last_active":"2024-05-09T01:23:22.303787+0000","last_peered":"2024-05-09T01:23:22.303787+0000","last_clean":"2024-05-09T01:23:22.303787+0000","last_became_active":"2024-05-09T01:22:36.797906+0000","last_became_peered":"2024-05-09T01:22:36.797906+0000","last_unstale":"2024-05-09T01:23:22.303787+0000","last_undegraded":"2024-05-09T01:23:22.303787+0000","last_fullsized":"2024-05-09T01:23:22.303787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:13.599240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308697+0000","last_change":"2024-05-09T01:21:12.781366+0000","last_active":"2024-05-09T01:23:22.308697+0000","last_peered":"2024-05-09T01:23:22.308697+0000","last_clean":"2024-05-09T01:23:22.308697+0000","last_became_active":"2024-05-09T01:21:12.774967+0000","last_became_peered":"2024-05-09T01:21:12.774967+0000","last_unstale":"2024-05-09T01:23:22.308697+0000","last_undegraded":"2024-05-09T01:23:22.308697+0000","last_fullsized":"2024-05-09T01:23:22.308697+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_clean_scrub_stamp":"2024-05-09T01:17:49.682508+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:08.075365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.152872+0000","last_change":"2024-05-09T01:22:39.840952+0000","last_active":"2024-05-09T01:24:21.152872+0000","last_peered":"2024-05-09T01:24:21.152872+0000","last_clean":"2024-05-09T01:24:21.152872+0000","last_became_active":"2024-05-09T01:22:39.840323+0000","last_became_peered":"2024-05-09T01:22:39.840323+0000","last_unstale":"2024-05-09T01:24:21.152872+0000","last_undegraded":"2024-05-09T01:24:21.152872+0000","last_fullsized":"2024-05-09T01:24:21.152872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:23.837123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308860+0000","last_change":"2024-05-09T01:22:36.798631+0000","last_active":"2024-05-09T01:23:22.308860+0000","last_peered":"2024-05-09T01:23:22.308860+0000","last_clean":"2024-05-09T01:23:22.308860+0000","last_became_active":"2024-05-09T01:22:36.798242+0000","last_became_peered":"2024-05-09T01:22:36.798242+0000","last_unstale":"2024-05-09T01:23:22.308860+0000","last_undegraded":"2024-05-09T01:23:22.308860+0000","last_fullsized":"2024-05-09T01:23:22.308860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:18.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303746+0000","last_change":"2024-05-09T01:22:36.843779+0000","last_active":"2024-05-09T01:23:22.303746+0000","last_peered":"2024-05-09T01:23:22.303746+0000","last_clean":"2024-05-09T01:23:22.303746+0000","last_became_active":"2024-05-09T01:22:36.842084+0000","last_became_peered":"2024-05-09T01:22:36.842084+0000","last_unstale":"2024-05-09T01:23:22.303746+0000","last_undegraded":"2024-05-09T01:23:22.303746+0000","last_fullsized":"2024-05-09T01:23:22.303746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:34.743372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145885+0000","last_change":"2024-05-09T01:22:39.842266+0000","last_active":"2024-05-09T01:24:21.145885+0000","last_peered":"2024-05-09T01:24:21.145885+0000","last_clean":"2024-05-09T01:24:21.145885+0000","last_became_active":"2024-05-09T01:22:39.841927+0000","last_became_peered":"2024-05-09T01:22:39.841927+0000","last_unstale":"2024-05-09T01:24:21.145885+0000","last_undegraded":"2024-05-09T01:24:21.145885+0000","last_fullsized":"2024-05-09T01:24:21.145885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:26.774250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'17","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:23.838395+0000","last_change":"2024-05-09T01:22:36.796929+0000","last_active":"2024-05-09T01:23:23.838395+0000","last_peered":"2024-05-09T01:23:23.838395+0000","last_clean":"2024-05-09T01:23:23.838395+0000","last_became_active":"2024-05-09T01:22:36.796480+0000","last_became_peered":"2024-05-09T01:22:36.796480+0000","last_unstale":"2024-05-09T01:23:23.838395+0000","last_undegraded":"2024-05-09T01:23:23.838395+0000","last_fullsized":"2024-05-09T01:23:23.838395+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:45.168305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.139985+0000","last_change":"2024-05-09T01:22:39.843093+0000","last_active":"2024-05-09T01:24:21.139985+0000","last_peered":"2024-05-09T01:24:21.139985+0000","last_clean":"2024-05-09T01:24:21.139985+0000","last_became_active":"2024-05-09T01:22:39.842729+0000","last_became_peered":"2024-05-09T01:22:39.842729+0000","last_unstale":"2024-05-09T01:24:21.139985+0000","last_undegraded":"2024-05-09T01:24:21.139985+0000","last_fullsized":"2024-05-09T01:24:21.139985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:02.074843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303637+0000","last_change":"2024-05-09T01:22:36.798166+0000","last_active":"2024-05-09T01:23:22.303637+0000","last_peered":"2024-05-09T01:23:22.303637+0000","last_clean":"2024-05-09T01:23:22.303637+0000","last_became_active":"2024-05-09T01:22:36.797752+0000","last_became_peered":"2024-05-09T01:22:36.797752+0000","last_unstale":"2024-05-09T01:23:22.303637+0000","last_undegraded":"2024-05-09T01:23:22.303637+0000","last_fullsized":"2024-05-09T01:23:22.303637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:05.211720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141557+0000","last_change":"2024-05-09T01:23:22.316228+0000","last_active":"2024-05-09T01:24:21.141557+0000","last_peered":"2024-05-09T01:24:21.141557+0000","last_clean":"2024-05-09T01:24:21.141557+0000","last_became_active":"2024-05-09T01:23:22.315683+0000","last_became_peered":"2024-05-09T01:23:22.315683+0000","last_unstale":"2024-05-09T01:24:21.141557+0000","last_undegraded":"2024-05-09T01:24:21.141557+0000","last_fullsized":"2024-05-09T01:24:21.141557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:04.623701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421234+0000","last_change":"2024-05-09T01:22:36.841999+0000","last_active":"2024-05-09T01:23:22.421234+0000","last_peered":"2024-05-09T01:23:22.421234+0000","last_clean":"2024-05-09T01:23:22.421234+0000","last_became_active":"2024-05-09T01:22:36.841664+0000","last_became_peered":"2024-05-09T01:22:36.841664+0000","last_unstale":"2024-05-09T01:23:22.421234+0000","last_undegraded":"2024-05-09T01:23:22.421234+0000","last_fullsized":"2024-05-09T01:23:22.421234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:27.110258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.109425+0000","last_change":"2024-05-09T01:22:39.839553+0000","last_active":"2024-05-09T01:24:21.109425+0000","last_peered":"2024-05-09T01:24:21.109425+0000","last_clean":"2024-05-09T01:24:21.109425+0000","last_became_active":"2024-05-09T01:22:39.839189+0000","last_became_peered":"2024-05-09T01:22:39.839189+0000","last_unstale":"2024-05-09T01:24:21.109425+0000","last_undegraded":"2024-05-09T01:24:21.109425+0000","last_fullsized":"2024-05-09T01:24:21.109425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:02.794073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309848+0000","last_change":"2024-05-09T01:22:36.792516+0000","last_active":"2024-05-09T01:23:22.309848+0000","last_peered":"2024-05-09T01:23:22.309848+0000","last_clean":"2024-05-09T01:23:22.309848+0000","last_became_active":"2024-05-09T01:22:36.792129+0000","last_became_peered":"2024-05-09T01:22:36.792129+0000","last_unstale":"2024-05-09T01:23:22.309848+0000","last_undegraded":"2024-05-09T01:23:22.309848+0000","last_fullsized":"2024-05-09T01:23:22.309848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:46.596723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148698+0000","last_change":"2024-05-09T01:22:39.841523+0000","last_active":"2024-05-09T01:24:21.148698+0000","last_peered":"2024-05-09T01:24:21.148698+0000","last_clean":"2024-05-09T01:24:21.148698+0000","last_became_active":"2024-05-09T01:22:39.840667+0000","last_became_peered":"2024-05-09T01:22:39.840667+0000","last_unstale":"2024-05-09T01:24:21.148698+0000","last_undegraded":"2024-05-09T01:24:21.148698+0000","last_fullsized":"2024-05-09T01:24:21.148698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:39.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.332290+0000","last_change":"2024-05-09T01:23:22.331855+0000","last_active":"2024-05-09T01:23:22.332290+0000","last_peered":"2024-05-09T01:23:22.332290+0000","last_clean":"2024-05-09T01:23:22.332290+0000","last_became_active":"2024-05-09T01:23:22.313059+0000","last_became_peered":"2024-05-09T01:23:22.313059+0000","last_unstale":"2024-05-09T01:23:22.332290+0000","last_undegraded":"2024-05-09T01:23:22.332290+0000","last_fullsized":"2024-05-09T01:23:22.332290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:16.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140461+0000","last_change":"2024-05-09T01:22:39.845031+0000","last_active":"2024-05-09T01:24:21.140461+0000","last_peered":"2024-05-09T01:24:21.140461+0000","last_clean":"2024-05-09T01:24:21.140461+0000","last_became_active":"2024-05-09T01:22:39.844653+0000","last_became_peered":"2024-05-09T01:22:39.844653+0000","last_unstale":"2024-05-09T01:24:21.140461+0000","last_undegraded":"2024-05-09T01:24:21.140461+0000","last_fullsized":"2024-05-09T01:24:21.140461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:14.382445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303070+0000","last_change":"2024-05-09T01:22:36.801395+0000","last_active":"2024-05-09T01:23:22.303070+0000","last_peered":"2024-05-09T01:23:22.303070+0000","last_clean":"2024-05-09T01:23:22.303070+0000","last_became_active":"2024-05-09T01:22:36.800992+0000","last_became_peered":"2024-05-09T01:22:36.800992+0000","last_unstale":"2024-05-09T01:23:22.303070+0000","last_undegraded":"2024-05-09T01:23:22.303070+0000","last_fullsized":"2024-05-09T01:23:22.303070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:16.089068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.165901+0000","last_change":"2024-05-09T01:22:39.835842+0000","last_active":"2024-05-09T01:24:21.165901+0000","last_peered":"2024-05-09T01:24:21.165901+0000","last_clean":"2024-05-09T01:24:21.165901+0000","last_became_active":"2024-05-09T01:22:39.835372+0000","last_became_peered":"2024-05-09T01:22:39.835372+0000","last_unstale":"2024-05-09T01:24:21.165901+0000","last_undegraded":"2024-05-09T01:24:21.165901+0000","last_fullsized":"2024-05-09T01:24:21.165901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:13.197946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:37.446224+0000","last_change":"2024-05-09T01:23:22.332514+0000","last_active":"2024-05-09T01:24:37.446224+0000","last_peered":"2024-05-09T01:24:37.446224+0000","last_clean":"2024-05-09T01:24:37.446224+0000","last_became_active":"2024-05-09T01:23:22.312966+0000","last_became_peered":"2024-05-09T01:23:22.312966+0000","last_unstale":"2024-05-09T01:24:37.446224+0000","last_undegraded":"2024-05-09T01:24:37.446224+0000","last_fullsized":"2024-05-09T01:24:37.446224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:52.637108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166101+0000","last_change":"2024-05-09T01:22:39.845390+0000","last_active":"2024-05-09T01:24:21.166101+0000","last_peered":"2024-05-09T01:24:21.166101+0000","last_clean":"2024-05-09T01:24:21.166101+0000","last_became_active":"2024-05-09T01:22:39.844918+0000","last_became_peered":"2024-05-09T01:22:39.844918+0000","last_unstale":"2024-05-09T01:24:21.166101+0000","last_undegraded":"2024-05-09T01:24:21.166101+0000","last_fullsized":"2024-05-09T01:24:21.166101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:04.455794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421710+0000","last_change":"2024-05-09T01:22:36.803329+0000","last_active":"2024-05-09T01:23:22.421710+0000","last_peered":"2024-05-09T01:23:22.421710+0000","last_clean":"2024-05-09T01:23:22.421710+0000","last_became_active":"2024-05-09T01:22:36.802831+0000","last_became_peered":"2024-05-09T01:22:36.802831+0000","last_unstale":"2024-05-09T01:23:22.421710+0000","last_undegraded":"2024-05-09T01:23:22.421710+0000","last_fullsized":"2024-05-09T01:23:22.421710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:57.919840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154637+0000","last_change":"2024-05-09T01:22:39.837595+0000","last_active":"2024-05-09T01:24:21.154637+0000","last_peered":"2024-05-09T01:24:21.154637+0000","last_clean":"2024-05-09T01:24:21.154637+0000","last_became_active":"2024-05-09T01:22:39.837197+0000","last_became_peered":"2024-05-09T01:22:39.837197+0000","last_unstale":"2024-05-09T01:24:21.154637+0000","last_undegraded":"2024-05-09T01:24:21.154637+0000","last_fullsized":"2024-05-09T01:24:21.154637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:08.700741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.321286+0000","last_change":"2024-05-09T01:23:22.321286+0000","last_active":"2024-05-09T01:23:22.321286+0000","last_peered":"2024-05-09T01:23:22.321286+0000","last_clean":"2024-05-09T01:23:22.321286+0000","last_became_active":"2024-05-09T01:23:22.315810+0000","last_became_peered":"2024-05-09T01:23:22.315810+0000","last_unstale":"2024-05-09T01:23:22.321286+0000","last_undegraded":"2024-05-09T01:23:22.321286+0000","last_fullsized":"2024-05-09T01:23:22.321286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:43.100893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.165205+0000","last_change":"2024-05-09T01:22:39.827798+0000","last_active":"2024-05-09T01:24:21.165205+0000","last_peered":"2024-05-09T01:24:21.165205+0000","last_clean":"2024-05-09T01:24:21.165205+0000","last_became_active":"2024-05-09T01:22:39.827322+0000","last_became_peered":"2024-05-09T01:22:39.827322+0000","last_unstale":"2024-05-09T01:24:21.165205+0000","last_undegraded":"2024-05-09T01:24:21.165205+0000","last_fullsized":"2024-05-09T01:24:21.165205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:52.454033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.149595+0000","last_change":"2024-05-09T01:22:39.841885+0000","last_active":"2024-05-09T01:24:21.149595+0000","last_peered":"2024-05-09T01:24:21.149595+0000","last_clean":"2024-05-09T01:24:21.149595+0000","last_became_active":"2024-05-09T01:22:39.841160+0000","last_became_peered":"2024-05-09T01:22:39.841160+0000","last_unstale":"2024-05-09T01:24:21.149595+0000","last_undegraded":"2024-05-09T01:24:21.149595+0000","last_fullsized":"2024-05-09T01:24:21.149595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:57.290465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145960+0000","last_change":"2024-05-09T01:22:39.846495+0000","last_active":"2024-05-09T01:24:21.145960+0000","last_peered":"2024-05-09T01:24:21.145960+0000","last_clean":"2024-05-09T01:24:21.145960+0000","last_became_active":"2024-05-09T01:22:39.846108+0000","last_became_peered":"2024-05-09T01:22:39.846108+0000","last_unstale":"2024-05-09T01:24:21.145960+0000","last_undegraded":"2024-05-09T01:24:21.145960+0000","last_fullsized":"2024-05-09T01:24:21.145960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:39.958174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.131907+0000","last_change":"2024-05-09T01:22:39.841288+0000","last_active":"2024-05-09T01:24:21.131907+0000","last_peered":"2024-05-09T01:24:21.131907+0000","last_clean":"2024-05-09T01:24:21.131907+0000","last_became_active":"2024-05-09T01:22:39.840900+0000","last_became_peered":"2024-05-09T01:22:39.840900+0000","last_unstale":"2024-05-09T01:24:21.131907+0000","last_undegraded":"2024-05-09T01:24:21.131907+0000","last_fullsized":"2024-05-09T01:24:21.131907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:30.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141204+0000","last_change":"2024-05-09T01:23:22.313135+0000","last_active":"2024-05-09T01:24:21.141204+0000","last_peered":"2024-05-09T01:24:21.141204+0000","last_clean":"2024-05-09T01:24:21.141204+0000","last_became_active":"2024-05-09T01:23:22.312710+0000","last_became_peered":"2024-05-09T01:23:22.312710+0000","last_unstale":"2024-05-09T01:24:21.141204+0000","last_undegraded":"2024-05-09T01:24:21.141204+0000","last_fullsized":"2024-05-09T01:24:21.141204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:47.725790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153527+0000","last_change":"2024-05-09T01:22:39.838037+0000","last_active":"2024-05-09T01:24:21.153527+0000","last_peered":"2024-05-09T01:24:21.153527+0000","last_clean":"2024-05-09T01:24:21.153527+0000","last_became_active":"2024-05-09T01:22:39.837362+0000","last_became_peered":"2024-05-09T01:22:39.837362+0000","last_unstale":"2024-05-09T01:24:21.153527+0000","last_undegraded":"2024-05-09T01:24:21.153527+0000","last_fullsized":"2024-05-09T01:24:21.153527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.781318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146345+0000","last_change":"2024-05-09T01:23:22.310046+0000","last_active":"2024-05-09T01:24:21.146345+0000","last_peered":"2024-05-09T01:24:21.146345+0000","last_clean":"2024-05-09T01:24:21.146345+0000","last_became_active":"2024-05-09T01:23:22.309541+0000","last_became_peered":"2024-05-09T01:23:22.309541+0000","last_unstale":"2024-05-09T01:24:21.146345+0000","last_undegraded":"2024-05-09T01:24:21.146345+0000","last_fullsized":"2024-05-09T01:24:21.146345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:30.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.161164+0000","last_change":"2024-05-09T01:22:39.840453+0000","last_active":"2024-05-09T01:24:21.161164+0000","last_peered":"2024-05-09T01:24:21.161164+0000","last_clean":"2024-05-09T01:24:21.161164+0000","last_became_active":"2024-05-09T01:22:39.840077+0000","last_became_peered":"2024-05-09T01:22:39.840077+0000","last_unstale":"2024-05-09T01:24:21.161164+0000","last_undegraded":"2024-05-09T01:24:21.161164+0000","last_fullsized":"2024-05-09T01:24:21.161164+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:20.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.160729+0000","last_change":"2024-05-09T01:22:39.844554+0000","last_active":"2024-05-09T01:24:21.160729+0000","last_peered":"2024-05-09T01:24:21.160729+0000","last_clean":"2024-05-09T01:24:21.160729+0000","last_became_active":"2024-05-09T01:22:39.843640+0000","last_became_peered":"2024-05-09T01:22:39.843640+0000","last_unstale":"2024-05-09T01:24:21.160729+0000","last_undegraded":"2024-05-09T01:24:21.160729+0000","last_fullsized":"2024-05-09T01:24:21.160729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:27.801565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140606+0000","last_change":"2024-05-09T01:22:39.839043+0000","last_active":"2024-05-09T01:24:21.140606+0000","last_peered":"2024-05-09T01:24:21.140606+0000","last_clean":"2024-05-09T01:24:21.140606+0000","last_became_active":"2024-05-09T01:22:39.838662+0000","last_became_peered":"2024-05-09T01:22:39.838662+0000","last_unstale":"2024-05-09T01:24:21.140606+0000","last_undegraded":"2024-05-09T01:24:21.140606+0000","last_fullsized":"2024-05-09T01:24:21.140606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:16.117689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.138867+0000","last_change":"2024-05-09T01:23:22.309945+0000","last_active":"2024-05-09T01:24:21.138867+0000","last_peered":"2024-05-09T01:24:21.138867+0000","last_clean":"2024-05-09T01:24:21.138867+0000","last_became_active":"2024-05-09T01:23:22.309543+0000","last_became_peered":"2024-05-09T01:23:22.309543+0000","last_unstale":"2024-05-09T01:24:21.138867+0000","last_undegraded":"2024-05-09T01:24:21.138867+0000","last_fullsized":"2024-05-09T01:24:21.138867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:47.464524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164415+0000","last_change":"2024-05-09T01:22:39.838911+0000","last_active":"2024-05-09T01:24:21.164415+0000","last_peered":"2024-05-09T01:24:21.164415+0000","last_clean":"2024-05-09T01:24:21.164415+0000","last_became_active":"2024-05-09T01:22:39.838293+0000","last_became_peered":"2024-05-09T01:22:39.838293+0000","last_unstale":"2024-05-09T01:24:21.164415+0000","last_undegraded":"2024-05-09T01:24:21.164415+0000","last_fullsized":"2024-05-09T01:24:21.164415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:32.596306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163381+0000","last_change":"2024-05-09T01:22:39.838185+0000","last_active":"2024-05-09T01:24:21.163381+0000","last_peered":"2024-05-09T01:24:21.163381+0000","last_clean":"2024-05-09T01:24:21.163381+0000","last_became_active":"2024-05-09T01:22:39.837530+0000","last_became_peered":"2024-05-09T01:22:39.837530+0000","last_unstale":"2024-05-09T01:24:21.163381+0000","last_undegraded":"2024-05-09T01:24:21.163381+0000","last_fullsized":"2024-05-09T01:24:21.163381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:22.556710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150810+0000","last_change":"2024-05-09T01:22:39.834124+0000","last_active":"2024-05-09T01:24:21.150810+0000","last_peered":"2024-05-09T01:24:21.150810+0000","last_clean":"2024-05-09T01:24:21.150810+0000","last_became_active":"2024-05-09T01:22:39.833716+0000","last_became_peered":"2024-05-09T01:22:39.833716+0000","last_unstale":"2024-05-09T01:24:21.150810+0000","last_undegraded":"2024-05-09T01:24:21.150810+0000","last_fullsized":"2024-05-09T01:24:21.150810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:54:42.337900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70017,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1001,"num_read_kb":579,"num_write":259,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2345,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612648,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":437733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1090649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":437019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168632,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":913408,"data_stored":436613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1030588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528386,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":437268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":500852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":438607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":498698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117073,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":436561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":438831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1540096,"data_stored":1029202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T01:24:41.012 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T01:24:41.012 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T01:24:41.013 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T01:24:41.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-09T01:24:41.264 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:41.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:41 smithi059 ceph-mon[34351]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:24:41.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:41 smithi019 ceph-mon[30158]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:24:41.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:41 smithi196 ceph-mon[35041]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:24:42.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:42 smithi019 ceph-mon[30158]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:42.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:42 smithi059 ceph-mon[34351]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:42.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:42 smithi196 ceph-mon[35041]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.002+0000 7fa6f1262700 1 -- 172.21.15.19:0/2464432859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec107df0 msgr2=0x7fa6ec108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.002+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2464432859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec107df0 0x7fa6ec108200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40099c0 tx=0x7fa6d4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.002+0000 7fa6f1262700 1 -- 172.21.15.19:0/2464432859 shutdown_connections 2024-05-09T01:24:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.002+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2464432859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6ec108740 0x7fa6ec1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.002+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2464432859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6ec1034c0 0x7fa6ec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.002+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2464432859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec107df0 0x7fa6ec108200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.002+0000 7fa6f1262700 1 -- 172.21.15.19:0/2464432859 >> 172.21.15.19:0/2464432859 conn(0x7fa6ec0fedc0 msgr2=0x7fa6ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.002+0000 7fa6f1262700 1 -- 172.21.15.19:0/2464432859 shutdown_connections 2024-05-09T01:24:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.002+0000 7fa6f1262700 1 -- 172.21.15.19:0/2464432859 wait complete. 2024-05-09T01:24:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.003+0000 7fa6f1262700 1 Processor -- start 2024-05-09T01:24:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.003+0000 7fa6f1262700 1 -- start start 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.003+0000 7fa6f1262700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec1034c0 0x7fa6ec1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.003+0000 7fa6f1262700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6ec107df0 0x7fa6ec1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.003+0000 7fa6f1262700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6ec108740 0x7fa6ec1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.003+0000 7fa6f1262700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec113370 con 0x7fa6ec1034c0 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.003+0000 7fa6f1262700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1131f0 con 0x7fa6ec107df0 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.003+0000 7fa6f1262700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1134f0 con 0x7fa6ec108740 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6eaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec1034c0 0x7fa6ec1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6eaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec1034c0 0x7fa6ec1a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46834/0 (socket says 172.21.15.19:46834) 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6eaffd700 1 -- 172.21.15.19:0/2546713595 learned_addr learned my addr 172.21.15.19:0/2546713595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6e27fc700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6ec107df0 0x7fa6ec1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6eb7fe700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6ec108740 0x7fa6ec1a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6e27fc700 1 -- 172.21.15.19:0/2546713595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6ec108740 msgr2=0x7fa6ec1a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6e27fc700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6ec108740 0x7fa6ec1a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6e27fc700 1 -- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec1034c0 msgr2=0x7fa6ec1a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6e27fc700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec1034c0 0x7fa6ec1a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6e27fc700 1 -- 172.21.15.19:0/2546713595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec1a62c0 con 0x7fa6ec107df0 2024-05-09T01:24:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6eaffd700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec1034c0 0x7fa6ec1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:24:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6e27fc700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6ec107df0 0x7fa6ec1a1990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40098e0 tx=0x7fa6d4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6e8ff9700 1 -- 172.21.15.19:0/2546713595 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4026070 con 0x7fa6ec107df0 2024-05-09T01:24:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec1a6550 con 0x7fa6ec107df0 2024-05-09T01:24:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.004+0000 7fa6e8ff9700 1 -- 172.21.15.19:0/2546713595 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6d4003c70 con 0x7fa6ec107df0 2024-05-09T01:24:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.005+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec1aded0 con 0x7fa6ec107df0 2024-05-09T01:24:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.005+0000 7fa6e8ff9700 1 -- 172.21.15.19:0/2546713595 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d400f9c0 con 0x7fa6ec107df0 2024-05-09T01:24:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.006+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec104970 con 0x7fa6ec107df0 2024-05-09T01:24:43.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.007+0000 7fa6e8ff9700 1 -- 172.21.15.19:0/2546713595 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa6d4029070 con 0x7fa6ec107df0 2024-05-09T01:24:43.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.008+0000 7fa6e8ff9700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa6d80b2900 0x7fa6d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:43.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.008+0000 7fa6eaffd700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa6d80b2900 0x7fa6d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:43.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.008+0000 7fa6e8ff9700 1 -- 172.21.15.19:0/2546713595 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa6d40e2b30 con 0x7fa6ec107df0 2024-05-09T01:24:43.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.008+0000 7fa6eaffd700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa6d80b2900 0x7fa6d80b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc011630 tx=0x7fa6dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.011+0000 7fa6e8ff9700 1 -- 172.21.15.19:0/2546713595 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6d4013070 con 0x7fa6ec107df0 2024-05-09T01:24:43.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.303+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa6ec0512c0 con 0x7fa6ec107df0 2024-05-09T01:24:43.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.304+0000 7fa6e8ff9700 1 -- 172.21.15.19:0/2546713595 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa6d40aa860 con 0x7fa6ec107df0 2024-05-09T01:24:43.302 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:24:43.302 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T01:24:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.306+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa6d80b2900 msgr2=0x7fa6d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.306+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa6d80b2900 0x7fa6d80b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc011630 tx=0x7fa6dc009260 comp rx=0 tx=0).stop 2024-05-09T01:24:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.306+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6ec107df0 msgr2=0x7fa6ec1a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.306+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6ec107df0 0x7fa6ec1a1990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40098e0 tx=0x7fa6d4004270 comp rx=0 tx=0).stop 2024-05-09T01:24:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.307+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 shutdown_connections 2024-05-09T01:24:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.307+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6ec108740 0x7fa6ec1a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.307+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa6d80b2900 0x7fa6d80b4e10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.307+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ec1034c0 0x7fa6ec1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.307+0000 7fa6f1262700 1 --2- 172.21.15.19:0/2546713595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6ec107df0 0x7fa6ec1a1990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.307+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 >> 172.21.15.19:0/2546713595 conn(0x7fa6ec0fedc0 msgr2=0x7fa6ec10ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.307+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 shutdown_connections 2024-05-09T01:24:43.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:43.307+0000 7fa6f1262700 1 -- 172.21.15.19:0/2546713595 wait complete. 2024-05-09T01:24:43.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:43 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:24:43.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:43 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:24:43.607 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:43 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:24:44.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:44 smithi059 ceph-mon[34351]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:44.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:44 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2546713595' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T01:24:44.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:44 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:44.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:44 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:44.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:44 smithi019 ceph-mon[30158]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:44.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:44 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2546713595' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T01:24:44.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:44.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:44 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:44.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:44 smithi196 ceph-mon[35041]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:44.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:44 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2546713595' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T01:24:44.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:44 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:44.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:44 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:44.738 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T01:24:44.738 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:24:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:45.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:45 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:45 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:45.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:45 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:46.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:46 smithi019 ceph-mon[30158]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:24:46.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:46 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:24:46.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:46 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:24:46.435 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:46 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:46.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:46 smithi196 ceph-mon[35041]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:24:46.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:46 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:24:46.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:46 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:24:46.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:46 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.712+0000 7f803a865700 1 -- 172.21.15.19:0/3026715213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8034108740 msgr2=0x7f80341100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.712+0000 7f803a865700 1 --2- 172.21.15.19:0/3026715213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8034108740 0x7f80341100a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f802c009fc0 tx=0x7f802c00b330 comp rx=0 tx=0).stop 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.712+0000 7f803a865700 1 -- 172.21.15.19:0/3026715213 shutdown_connections 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.712+0000 7f803a865700 1 --2- 172.21.15.19:0/3026715213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8034108740 0x7f80341100a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.712+0000 7f803a865700 1 --2- 172.21.15.19:0/3026715213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80341034c0 0x7f80341038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.712+0000 7f803a865700 1 --2- 172.21.15.19:0/3026715213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8034107df0 0x7f8034108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.712+0000 7f803a865700 1 -- 172.21.15.19:0/3026715213 >> 172.21.15.19:0/3026715213 conn(0x7f80340fedf0 msgr2=0x7f8034101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.712+0000 7f803a865700 1 -- 172.21.15.19:0/3026715213 shutdown_connections 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.712+0000 7f803a865700 1 -- 172.21.15.19:0/3026715213 wait complete. 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.713+0000 7f803a865700 1 Processor -- start 2024-05-09T01:24:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.713+0000 7f803a865700 1 -- start start 2024-05-09T01:24:46.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.713+0000 7f803a865700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80341034c0 0x7f80340ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:46.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.713+0000 7f803a865700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8034107df0 0x7f80340ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:46.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.713+0000 7f803a865700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8034108740 0x7f80340ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:46.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.713+0000 7f803a865700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8034113370 con 0x7f80341034c0 2024-05-09T01:24:46.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.713+0000 7f803a865700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80341131f0 con 0x7f8034107df0 2024-05-09T01:24:46.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.713+0000 7f803a865700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80341134f0 con 0x7f8034108740 2024-05-09T01:24:46.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80341034c0 0x7f80340ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:46.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80341034c0 0x7f80340ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46848/0 (socket says 172.21.15.19:46848) 2024-05-09T01:24:46.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8033fff700 1 -- 172.21.15.19:0/430061470 learned_addr learned my addr 172.21.15.19:0/430061470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:46.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8038e02700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8034108740 0x7f80340ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:46.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f802bfff700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8034107df0 0x7f80340ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:46.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8033fff700 1 -- 172.21.15.19:0/430061470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8034108740 msgr2=0x7f80340ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:46.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8033fff700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8034108740 0x7f80340ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:46.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8033fff700 1 -- 172.21.15.19:0/430061470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8034107df0 msgr2=0x7f80340ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:46.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8033fff700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8034107df0 0x7f80340ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:46.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8033fff700 1 -- 172.21.15.19:0/430061470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80341a9140 con 0x7f80341034c0 2024-05-09T01:24:46.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8033fff700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80341034c0 0x7f80340ff2d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f802400d980 tx=0x7f802400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:46.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8031ffb700 1 -- 172.21.15.19:0/430061470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024016070 con 0x7f80341034c0 2024-05-09T01:24:46.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80341a9430 con 0x7f80341034c0 2024-05-09T01:24:46.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8031ffb700 1 -- 172.21.15.19:0/430061470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8024004830 con 0x7f80341034c0 2024-05-09T01:24:46.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f8031ffb700 1 -- 172.21.15.19:0/430061470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024005420 con 0x7f80341034c0 2024-05-09T01:24:46.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.714+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80341a9970 con 0x7f80341034c0 2024-05-09T01:24:46.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.716+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80340512c0 con 0x7f80341034c0 2024-05-09T01:24:46.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.717+0000 7f8031ffb700 1 -- 172.21.15.19:0/430061470 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f80240055c0 con 0x7f80341034c0 2024-05-09T01:24:46.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.718+0000 7f8031ffb700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f80200b2880 0x7f80200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:46.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.718+0000 7f802bfff700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f80200b2880 0x7f80200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:46.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.718+0000 7f8031ffb700 1 -- 172.21.15.19:0/430061470 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f80240e2430 con 0x7f80341034c0 2024-05-09T01:24:46.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.719+0000 7f802bfff700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f80200b2880 0x7f80200b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8034109a00 tx=0x7f801c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:46.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.722+0000 7f8031ffb700 1 -- 172.21.15.19:0/430061470 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f802401f020 con 0x7f80341034c0 2024-05-09T01:24:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:46 smithi059 ceph-mon[34351]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:24:46.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:46 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:24:46.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:46 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:24:46.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:46 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:24:46.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.985+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f803402e5f0 con 0x7f80341034c0 2024-05-09T01:24:46.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.986+0000 7f8031ffb700 1 -- 172.21.15.19:0/430061470 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11924 (secure 0 0 0) 0x7f80240aa130 con 0x7f80341034c0 2024-05-09T01:24:46.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:24:46.984 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:23:01.311002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24451,24497,24473,24452,24466]},"id":1}]} 2024-05-09T01:24:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f80200b2880 msgr2=0x7f80200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f80200b2880 0x7f80200b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8034109a00 tx=0x7f801c009310 comp rx=0 tx=0).stop 2024-05-09T01:24:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80341034c0 msgr2=0x7f80340ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80341034c0 0x7f80340ff2d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f802400d980 tx=0x7f802400dc90 comp rx=0 tx=0).stop 2024-05-09T01:24:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 shutdown_connections 2024-05-09T01:24:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8034108740 0x7f80340ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f80200b2880 0x7f80200b4d90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80341034c0 0x7f80340ff2d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 --2- 172.21.15.19:0/430061470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8034107df0 0x7f80340ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 >> 172.21.15.19:0/430061470 conn(0x7f80340fedf0 msgr2=0x7f8034075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 shutdown_connections 2024-05-09T01:24:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:46.989+0000 7f803a865700 1 -- 172.21.15.19:0/430061470 wait complete. 2024-05-09T01:24:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-09T01:24:47.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:47 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/430061470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:24:47.662 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:24:47.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:47 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/430061470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:24:47.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:47 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/430061470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:24:47.919 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:48.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:48 smithi019 ceph-mon[30158]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:24:48.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:48 smithi196 ceph-mon[35041]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:24:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:48 smithi059 ceph-mon[34351]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:24:49.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.576+0000 7f7997374700 1 -- 172.21.15.19:0/2037933646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7990109850 msgr2=0x7f79901111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:49.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.576+0000 7f7997374700 1 --2- 172.21.15.19:0/2037933646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7990109850 0x7f79901111b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f798800b0d0 tx=0x7f798800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.576+0000 7f7997374700 1 -- 172.21.15.19:0/2037933646 shutdown_connections 2024-05-09T01:24:49.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.576+0000 7f7997374700 1 --2- 172.21.15.19:0/2037933646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7990109850 0x7f79901111b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.576+0000 7f7997374700 1 --2- 172.21.15.19:0/2037933646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79901045d0 0x7f79901049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.576+0000 7f7997374700 1 --2- 172.21.15.19:0/2037933646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7990108f00 0x7f7990109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.576+0000 7f7997374700 1 -- 172.21.15.19:0/2037933646 >> 172.21.15.19:0/2037933646 conn(0x7f79900ffea0 msgr2=0x7f79901022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.576+0000 7f7997374700 1 -- 172.21.15.19:0/2037933646 shutdown_connections 2024-05-09T01:24:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.576+0000 7f7997374700 1 -- 172.21.15.19:0/2037933646 wait complete. 2024-05-09T01:24:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.577+0000 7f7997374700 1 Processor -- start 2024-05-09T01:24:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.577+0000 7f7997374700 1 -- start start 2024-05-09T01:24:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.577+0000 7f7997374700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79901045d0 0x7f799019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.577+0000 7f7997374700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7990108f00 0x7f799019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.577+0000 7f7997374700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7990109850 0x7f799019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.577+0000 7f7997374700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7990114480 con 0x7f79901045d0 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.577+0000 7f7997374700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7990114300 con 0x7f7990108f00 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.577+0000 7f7997374700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7990114600 con 0x7f7990109850 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7995110700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79901045d0 0x7f799019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f799490f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7990108f00 0x7f799019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f799490f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7990108f00 0x7f799019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:42714/0 (socket says 172.21.15.19:42714) 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f799490f700 1 -- 172.21.15.19:0/4147033211 learned_addr learned my addr 172.21.15.19:0/4147033211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7995911700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7990109850 0x7f799019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7995110700 1 -- 172.21.15.19:0/4147033211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7990109850 msgr2=0x7f799019f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7995110700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7990109850 0x7f799019f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7995110700 1 -- 172.21.15.19:0/4147033211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7990108f00 msgr2=0x7f799019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7995110700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7990108f00 0x7f799019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7995110700 1 -- 172.21.15.19:0/4147033211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799019fb70 con 0x7f79901045d0 2024-05-09T01:24:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7995110700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79901045d0 0x7f799019ad50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c970 tx=0x7f798c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f79867fc700 1 -- 172.21.15.19:0/4147033211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c014070 con 0x7f79901045d0 2024-05-09T01:24:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799019fe60 con 0x7f79901045d0 2024-05-09T01:24:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f79867fc700 1 -- 172.21.15.19:0/4147033211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c004440 con 0x7f79901045d0 2024-05-09T01:24:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f79867fc700 1 -- 172.21.15.19:0/4147033211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c007a30 con 0x7f79901045d0 2024-05-09T01:24:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.578+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79901a03a0 con 0x7f79901045d0 2024-05-09T01:24:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.580+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79900512c0 con 0x7f79901045d0 2024-05-09T01:24:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.581+0000 7f79867fc700 1 -- 172.21.15.19:0/4147033211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f798c017070 con 0x7f79901045d0 2024-05-09T01:24:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.582+0000 7f79867fc700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f797c0b2880 0x7f797c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.582+0000 7f799490f700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f797c0b2880 0x7f797c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.582+0000 7f79867fc700 1 -- 172.21.15.19:0/4147033211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f798c0e1a80 con 0x7f79901045d0 2024-05-09T01:24:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.583+0000 7f799490f700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f797c0b2880 0x7f797c0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7990107a60 tx=0x7f798000a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.587+0000 7f79867fc700 1 -- 172.21.15.19:0/4147033211 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f798c01e030 con 0x7f79901045d0 2024-05-09T01:24:49.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.854+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f799019c0e0 con 0x7f79901045d0 2024-05-09T01:24:49.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.855+0000 7f79867fc700 1 -- 172.21.15.19:0/4147033211 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11924 (secure 0 0 0) 0x7f798c054eb0 con 0x7f79901045d0 2024-05-09T01:24:49.853 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:24:49.853 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:23:01.311002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24451,24497,24473,24452,24466]},"id":1}]} 2024-05-09T01:24:49.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f797c0b2880 msgr2=0x7f797c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f797c0b2880 0x7f797c0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7990107a60 tx=0x7f798000a400 comp rx=0 tx=0).stop 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79901045d0 msgr2=0x7f799019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79901045d0 0x7f799019ad50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c970 tx=0x7f798c00cc80 comp rx=0 tx=0).stop 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 shutdown_connections 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7990109850 0x7f799019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f797c0b2880 0x7f797c0b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79901045d0 0x7f799019ad50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 --2- 172.21.15.19:0/4147033211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7990108f00 0x7f799019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.857+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 >> 172.21.15.19:0/4147033211 conn(0x7f79900ffea0 msgr2=0x7f7990108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.858+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 shutdown_connections 2024-05-09T01:24:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:49.858+0000 7f7997374700 1 -- 172.21.15.19:0/4147033211 wait complete. 2024-05-09T01:24:49.857 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-09T01:24:50.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:50 smithi019 ceph-mon[30158]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:50.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:50 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/4147033211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:24:50.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:24:50.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:50 smithi196 ceph-mon[35041]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:50.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:50 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/4147033211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:24:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:50 smithi059 ceph-mon[34351]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:24:50.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:50 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/4147033211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:24:52.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.338+0000 7f10014c5700 1 -- 172.21.15.19:0/3958582091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc107df0 msgr2=0x7f0ffc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:52.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.338+0000 7f10014c5700 1 --2- 172.21.15.19:0/3958582091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc107df0 0x7f0ffc108200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40099c0 tx=0x7f0fe4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.338+0000 7f10014c5700 1 -- 172.21.15.19:0/3958582091 shutdown_connections 2024-05-09T01:24:52.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.338+0000 7f10014c5700 1 --2- 172.21.15.19:0/3958582091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ffc108740 0x7f0ffc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.338+0000 7f10014c5700 1 --2- 172.21.15.19:0/3958582091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ffc1034c0 0x7f0ffc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.338+0000 7f10014c5700 1 --2- 172.21.15.19:0/3958582091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc107df0 0x7f0ffc108200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.338+0000 7f10014c5700 1 -- 172.21.15.19:0/3958582091 >> 172.21.15.19:0/3958582091 conn(0x7f0ffc0fedf0 msgr2=0x7f0ffc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:52.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.338+0000 7f10014c5700 1 -- 172.21.15.19:0/3958582091 shutdown_connections 2024-05-09T01:24:52.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.339+0000 7f10014c5700 1 -- 172.21.15.19:0/3958582091 wait complete. 2024-05-09T01:24:52.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.339+0000 7f10014c5700 1 Processor -- start 2024-05-09T01:24:52.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.339+0000 7f10014c5700 1 -- start start 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f10014c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ffc1034c0 0x7f0ffc19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f10014c5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ffc107df0 0x7f0ffc19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f10014c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc108740 0x7f0ffc19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f10014c5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc113350 con 0x7f0ffc108740 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f10014c5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1131d0 con 0x7f0ffc1034c0 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f10014c5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1134d0 con 0x7f0ffc107df0 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc108740 0x7f0ffc19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc108740 0x7f0ffc19c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54566/0 (socket says 172.21.15.19:54566) 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffb7fe700 1 -- 172.21.15.19:0/1784384224 learned_addr learned my addr 172.21.15.19:0/1784384224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:52.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffaffd700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ffc1034c0 0x7f0ffc19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ff27fc700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ffc107df0 0x7f0ffc19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffb7fe700 1 -- 172.21.15.19:0/1784384224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ffc107df0 msgr2=0x7f0ffc19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffb7fe700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ffc107df0 0x7f0ffc19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffb7fe700 1 -- 172.21.15.19:0/1784384224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ffc1034c0 msgr2=0x7f0ffc19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffb7fe700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ffc1034c0 0x7f0ffc19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffb7fe700 1 -- 172.21.15.19:0/1784384224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc1a7000 con 0x7f0ffc108740 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.340+0000 7f0ffaffd700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ffc1034c0 0x7f0ffc19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.341+0000 7f0ffb7fe700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc108740 0x7f0ffc19c750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400ef30 tx=0x7f0ff400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.341+0000 7f0ff8ff9700 1 -- 172.21.15.19:0/1784384224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40092e0 con 0x7f0ffc108740 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.341+0000 7f0ff8ff9700 1 -- 172.21.15.19:0/1784384224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff4004bb0 con 0x7f0ffc108740 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.341+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc1a7290 con 0x7f0ffc108740 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.341+0000 7f0ff8ff9700 1 -- 172.21.15.19:0/1784384224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4004280 con 0x7f0ffc108740 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.341+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a7750 con 0x7f0ffc108740 2024-05-09T01:24:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.342+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc0512c0 con 0x7f0ffc108740 2024-05-09T01:24:52.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.345+0000 7f0ff8ff9700 1 -- 172.21.15.19:0/1784384224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0ff401b030 con 0x7f0ffc108740 2024-05-09T01:24:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.345+0000 7f0ff8ff9700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0fe80b2880 0x7f0fe80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.346+0000 7f0ffaffd700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0fe80b2880 0x7f0fe80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.346+0000 7f0ff8ff9700 1 -- 172.21.15.19:0/1784384224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0ff40e2860 con 0x7f0ffc108740 2024-05-09T01:24:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.346+0000 7f0ffaffd700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0fe80b2880 0x7f0fe80b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0fec005950 tx=0x7f0fec0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.349+0000 7f0ff8ff9700 1 -- 172.21.15.19:0/1784384224 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ff4022020 con 0x7f0ffc108740 2024-05-09T01:24:52.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.567+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ffc02e5f0 con 0x7f0ffc108740 2024-05-09T01:24:52.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.569+0000 7f0ff8ff9700 1 -- 172.21.15.19:0/1784384224 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22716 (secure 0 0 0) 0x7f0ff4010040 con 0x7f0ffc108740 2024-05-09T01:24:52.567 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:24:52.567 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:23:22.298486+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:24:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0fe80b2880 msgr2=0x7f0fe80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0fe80b2880 0x7f0fe80b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0fec005950 tx=0x7f0fec0058e0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc108740 msgr2=0x7f0ffc19c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc108740 0x7f0ffc19c750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400ef30 tx=0x7f0ff400c450 comp rx=0 tx=0).stop 2024-05-09T01:24:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 shutdown_connections 2024-05-09T01:24:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0fe80b2880 0x7f0fe80b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ffc108740 0x7f0ffc19c750 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ffc1034c0 0x7f0ffc19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 --2- 172.21.15.19:0/1784384224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ffc107df0 0x7f0ffc19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:52.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 >> 172.21.15.19:0/1784384224 conn(0x7f0ffc0fedf0 msgr2=0x7f0ffc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:52.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.571+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 shutdown_connections 2024-05-09T01:24:52.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:52.572+0000 7f10014c5700 1 -- 172.21.15.19:0/1784384224 wait complete. 2024-05-09T01:24:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:52 smithi019 ceph-mon[30158]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:52.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:52 smithi196 ceph-mon[35041]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:52.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:52 smithi059 ceph-mon[34351]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:24:53.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:24:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:53.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:53 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1784384224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:24:53.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:53 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1784384224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:24:53.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:53 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1784384224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:24:54.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:54 smithi019 ceph-mon[30158]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:54.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:54 smithi196 ceph-mon[35041]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:54 smithi059 ceph-mon[34351]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:24:55.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.147+0000 7f4db8ab7700 1 -- 172.21.15.19:0/77794879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 msgr2=0x7f4db4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:55.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.147+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/77794879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 0x7f4db4102ef0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4da40099c0 tx=0x7f4da4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.147+0000 7f4db8ab7700 1 -- 172.21.15.19:0/77794879 shutdown_connections 2024-05-09T01:24:55.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.147+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/77794879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db4109850 0x7f4db41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.147+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/77794879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 0x7f4db4102ef0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.147+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/77794879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4108f00 0x7f4db4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.147+0000 7f4db8ab7700 1 -- 172.21.15.19:0/77794879 >> 172.21.15.19:0/77794879 conn(0x7f4db40fe540 msgr2=0x7f4db4100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.147+0000 7f4db8ab7700 1 -- 172.21.15.19:0/77794879 shutdown_connections 2024-05-09T01:24:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.147+0000 7f4db8ab7700 1 -- 172.21.15.19:0/77794879 wait complete. 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.148+0000 7f4db8ab7700 1 Processor -- start 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.148+0000 7f4db8ab7700 1 -- start start 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.148+0000 7f4db8ab7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 0x7f4db4103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.148+0000 7f4db8ab7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4108f00 0x7f4db4103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.148+0000 7f4db8ab7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db4109850 0x7f4db41044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.148+0000 7f4db8ab7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4114480 con 0x7f4db4102ae0 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.148+0000 7f4db8ab7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4114300 con 0x7f4db4108f00 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.148+0000 7f4db8ab7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4114600 con 0x7f4db4109850 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 0x7f4db4103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 0x7f4db4103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54578/0 (socket says 172.21.15.19:54578) 2024-05-09T01:24:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db259c700 1 -- 172.21.15.19:0/2267453687 learned_addr learned my addr 172.21.15.19:0/2267453687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db1d9b700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4108f00 0x7f4db4103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db2d9d700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db4109850 0x7f4db41044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db259c700 1 -- 172.21.15.19:0/2267453687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db4109850 msgr2=0x7f4db41044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db259c700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db4109850 0x7f4db41044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db259c700 1 -- 172.21.15.19:0/2267453687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4108f00 msgr2=0x7f4db4103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db259c700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4108f00 0x7f4db4103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db259c700 1 -- 172.21.15.19:0/2267453687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db41a91d0 con 0x7f4db4102ae0 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db259c700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 0x7f4db4103a30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4da400b580 tx=0x7f4da40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4da37fe700 1 -- 172.21.15.19:0/2267453687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4027070 con 0x7f4db4102ae0 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db41a9460 con 0x7f4db4102ae0 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4da37fe700 1 -- 172.21.15.19:0/2267453687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da4003f20 con 0x7f4db4102ae0 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4da37fe700 1 -- 172.21.15.19:0/2267453687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4022d70 con 0x7f4db4102ae0 2024-05-09T01:24:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.149+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db41a9910 con 0x7f4db4102ae0 2024-05-09T01:24:55.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.151+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db40512c0 con 0x7f4db4102ae0 2024-05-09T01:24:55.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.152+0000 7f4da37fe700 1 -- 172.21.15.19:0/2267453687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4da4003a70 con 0x7f4db4102ae0 2024-05-09T01:24:55.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.153+0000 7f4da37fe700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4d9c0b2880 0x7f4d9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.153+0000 7f4db1d9b700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4d9c0b2880 0x7f4d9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.153+0000 7f4da37fe700 1 -- 172.21.15.19:0/2267453687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4da40e2fc0 con 0x7f4db4102ae0 2024-05-09T01:24:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.153+0000 7f4db1d9b700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4d9c0b2880 0x7f4d9c0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4db4101c10 tx=0x7f4da800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:55.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.158+0000 7f4da37fe700 1 -- 172.21.15.19:0/2267453687 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4da4013070 con 0x7f4db4102ae0 2024-05-09T01:24:55.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.420+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4db402e5f0 con 0x7f4db4102ae0 2024-05-09T01:24:55.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.421+0000 7f4da37fe700 1 -- 172.21.15.19:0/2267453687 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11924 (secure 0 0 0) 0x7f4da40aacf0 con 0x7f4db4102ae0 2024-05-09T01:24:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:24:55.419 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:23:01.311002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24451,24497,24473,24452,24466]},"id":1}]} 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4d9c0b2880 msgr2=0x7f4d9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4d9c0b2880 0x7f4d9c0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4db4101c10 tx=0x7f4da800b410 comp rx=0 tx=0).stop 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 msgr2=0x7f4db4103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 0x7f4db4103a30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4da400b580 tx=0x7f4da40047b0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 shutdown_connections 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db4109850 0x7f4db41044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4d9c0b2880 0x7f4d9c0b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4102ae0 0x7f4db4103a30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 --2- 172.21.15.19:0/2267453687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4108f00 0x7f4db4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 >> 172.21.15.19:0/2267453687 conn(0x7f4db40fe540 msgr2=0x7f4db41070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 shutdown_connections 2024-05-09T01:24:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:55.424+0000 7f4db8ab7700 1 -- 172.21.15.19:0/2267453687 wait complete. 2024-05-09T01:24:55.423 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-09T01:24:56.098 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T01:22:41.711277+0000', 'modified': '2024-05-09T01:23:01.311002+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24497, 'mds_1': 24473, 'mds_2': 24466, 'mds_3': 24452, 'mds_4': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/909874272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 909874272}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 909874272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/555235417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 555235417}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 555235417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/2736793414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 2736793414}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 2736793414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6833/3715662430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6832', 'nonce': 3715662430}, {'type': 'v1', 'addr': '172.21.15.59:6833', 'nonce': 3715662430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/1184717757', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1184717757}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1184717757}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.196:6837/3887501643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3887501643}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3887501643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.59:6835/3872629659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 3872629659}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 3872629659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.196:6839/165915317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 165915317}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 165915317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.59:6837/2120139910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 2120139910}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2120139910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.59:6839/1954332745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 1954332745}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 1954332745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24451, 24497, 24473, 24452, 24466]} 2024-05-09T01:24:56.098 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T01:24:56.098 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T01:24:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:24:56.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:56 smithi019 ceph-mon[30158]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:24:56.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:56 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2267453687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:24:56.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:56 smithi196 ceph-mon[35041]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:24:56.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:56 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2267453687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:24:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:56 smithi059 ceph-mon[34351]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:24:56.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:56 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2267453687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.976+0000 7ff049311700 1 -- 172.21.15.19:0/1743406861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 msgr2=0x7ff0441111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.976+0000 7ff049311700 1 --2- 172.21.15.19:0/1743406861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 0x7ff0441111b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00b0d0 tx=0x7ff03c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.976+0000 7ff049311700 1 -- 172.21.15.19:0/1743406861 shutdown_connections 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.976+0000 7ff049311700 1 --2- 172.21.15.19:0/1743406861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 0x7ff0441111b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.976+0000 7ff049311700 1 --2- 172.21.15.19:0/1743406861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0441045d0 0x7ff0441049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.976+0000 7ff049311700 1 --2- 172.21.15.19:0/1743406861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff044108f00 0x7ff044109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.976+0000 7ff049311700 1 -- 172.21.15.19:0/1743406861 >> 172.21.15.19:0/1743406861 conn(0x7ff0440ffea0 msgr2=0x7ff0441022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.976+0000 7ff049311700 1 -- 172.21.15.19:0/1743406861 shutdown_connections 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.976+0000 7ff049311700 1 -- 172.21.15.19:0/1743406861 wait complete. 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.977+0000 7ff049311700 1 Processor -- start 2024-05-09T01:24:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.977+0000 7ff049311700 1 -- start start 2024-05-09T01:24:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff049311700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0441045d0 0x7ff04419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff049311700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff044108f00 0x7ff04419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff049311700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 0x7ff04419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff049311700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff044114500 con 0x7ff044109850 2024-05-09T01:24:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff049311700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff044114380 con 0x7ff044108f00 2024-05-09T01:24:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff049311700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff044114680 con 0x7ff0441045d0 2024-05-09T01:24:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 0x7ff04419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 0x7ff04419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54606/0 (socket says 172.21.15.19:54606) 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0437fe700 1 -- 172.21.15.19:0/1264560096 learned_addr learned my addr 172.21.15.19:0/1264560096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0427fc700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff044108f00 0x7ff04419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff042ffd700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0441045d0 0x7ff04419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0437fe700 1 -- 172.21.15.19:0/1264560096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0441045d0 msgr2=0x7ff04419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0437fe700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0441045d0 0x7ff04419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0437fe700 1 -- 172.21.15.19:0/1264560096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff044108f00 msgr2=0x7ff04419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0437fe700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff044108f00 0x7ff04419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0437fe700 1 -- 172.21.15.19:0/1264560096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04419fcc0 con 0x7ff044109850 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0427fc700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff044108f00 0x7ff04419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff0437fe700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 0x7ff04419f620 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0095c0 tx=0x7ff03c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff02bfff700 1 -- 172.21.15.19:0/1264560096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c01f070 con 0x7ff044109850 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff049311700 1 -- 172.21.15.19:0/1264560096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04419ff50 con 0x7ff044109850 2024-05-09T01:24:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff02bfff700 1 -- 172.21.15.19:0/1264560096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03c003c00 con 0x7ff044109850 2024-05-09T01:24:57.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.978+0000 7ff02bfff700 1 -- 172.21.15.19:0/1264560096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c01b6a0 con 0x7ff044109850 2024-05-09T01:24:57.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.979+0000 7ff049311700 1 -- 172.21.15.19:0/1264560096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0441a0430 con 0x7ff044109850 2024-05-09T01:24:57.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.981+0000 7ff02bfff700 1 -- 172.21.15.19:0/1264560096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff03c022070 con 0x7ff044109850 2024-05-09T01:24:57.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.982+0000 7ff02bfff700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff02c0b2880 0x7ff02c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.982+0000 7ff042ffd700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff02c0b2880 0x7ff02c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.982+0000 7ff02bfff700 1 -- 172.21.15.19:0/1264560096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff03c00f020 con 0x7ff044109850 2024-05-09T01:24:57.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.982+0000 7ff057d27b80 1 Processor -- start 2024-05-09T01:24:57.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.982+0000 7ff057d27b80 1 -- start start 2024-05-09T01:24:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.982+0000 7ff042ffd700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff02c0b2880 0x7ff02c0b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff034006fd0 tx=0x7ff034008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff057d27b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636573c3fa0 con 0x5636574a1650 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff057d27b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5636573627a0 con 0x56365740e110 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff057d27b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636573bff60 con 0x563657324450 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54614/0 (socket says 172.21.15.19:54614) 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0437fe700 1 -- 172.21.15.19:0/2667537379 learned_addr learned my addr 172.21.15.19:0/2667537379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0427fc700 1 --2- 172.21.15.19:0/2667537379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff042ffd700 1 --2- 172.21.15.19:0/2667537379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0437fe700 1 -- 172.21.15.19:0/2667537379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 msgr2=0x5636574a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:24:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0437fe700 1 --2- 172.21.15.19:0/2667537379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0437fe700 1 -- 172.21.15.19:0/2667537379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 msgr2=0x5636574a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:24:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0437fe700 1 --2- 172.21.15.19:0/2667537379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0437fe700 1 -- 172.21.15.19:0/2667537379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365735a320 con 0x5636574a1650 2024-05-09T01:24:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0437fe700 1 --2- 172.21.15.19:0/2667537379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a1a80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0aadb0 tx=0x7ff03c00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e4653a09d3a080 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.983+0000 7ff0297fa700 1 -- 172.21.15.19:0/2667537379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0b3460 con 0x5636574a1650 2024-05-09T01:24:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff0297fa700 1 -- 172.21.15.19:0/2667537379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03c0b3b10 con 0x5636574a1650 2024-05-09T01:24:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff0297fa700 1 -- 172.21.15.19:0/2667537379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0aa2e0 con 0x5636574a1650 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff057d27b80 1 -- 172.21.15.19:0/2667537379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 msgr2=0x5636574a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff057d27b80 1 --2- 172.21.15.19:0/2667537379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a1a80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0aadb0 tx=0x7ff03c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff057d27b80 1 -- 172.21.15.19:0/2667537379 shutdown_connections 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff057d27b80 1 --2- 172.21.15.19:0/2667537379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a1a80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff057d27b80 1 --2- 172.21.15.19:0/2667537379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff057d27b80 1 --2- 172.21.15.19:0/2667537379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff057d27b80 1 -- 172.21.15.19:0/2667537379 >> 172.21.15.19:0/2667537379 conn(0x563657384650 msgr2=0x56365749c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.984+0000 7ff057d27b80 1 -- 172.21.15.19:0/2667537379 shutdown_connections 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 1 -- 172.21.15.19:0/2667537379 wait complete. 2024-05-09T01:24:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 1 -- start start 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 1 Processor -- start 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563657324450 0x5636574b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56365740e110 0x5636574b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574ab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56365723fb50 con 0x5636574a1650 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56365737e100 con 0x563657324450 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636574a2310 con 0x56365740e110 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff057d27b80 10 client.0 _ll_register_callbacks cb 0x563657235670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.985+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54628/0 (socket says 172.21.15.19:54628) 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff0437fe700 1 -- 172.21.15.19:0/3134839717 learned_addr learned my addr 172.21.15.19:0/3134839717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff042ffd700 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563657324450 0x5636574b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff0427fc700 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56365740e110 0x5636574b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff0437fe700 1 -- 172.21.15.19:0/3134839717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56365740e110 msgr2=0x5636574b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff0437fe700 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56365740e110 0x5636574b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff0437fe700 1 -- 172.21.15.19:0/3134839717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563657324450 msgr2=0x5636574b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff0437fe700 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563657324450 0x5636574b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff0437fe700 1 -- 172.21.15.19:0/3134839717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365723f440 con 0x5636574a1650 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff0437fe700 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574ab070 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00d010 tx=0x7ff03c0b3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e426f989e4f93a68 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff026ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff026ff5700 1 -- 172.21.15.19:0/3134839717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c06f020 con 0x5636574a1650 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff026ff5700 1 -- 172.21.15.19:0/3134839717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff03c0aa290 con 0x5636574a1650 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff057d27b80 1 -- 172.21.15.19:0/3134839717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03c0b3b10 con 0x5636574a1650 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff026ff5700 1 -- 172.21.15.19:0/3134839717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff03c0cb070 con 0x5636574a1650 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff057d27b80 10 client.14994 fetch_fsmap learned FSMap version 27 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.986+0000 7ff057d27b80 1 -- 172.21.15.19:0/3134839717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5636574b8890 con 0x5636574a1650 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.987+0000 7ff026ff5700 1 -- 172.21.15.19:0/3134839717 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03c0b3b10 con 0x5636574a1650 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.987+0000 7ff026ff5700 1 -- 172.21.15.19:0/3134839717 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff03c011070 con 0x5636574a1650 2024-05-09T01:24:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.987+0000 7ff057d27b80 10 client.14994 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:24:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.987+0000 7ff057d27b80 10 client.14994 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T01:24:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.987+0000 7ff057d27b80 20 client.14994 populate_metadata read hostname 'smithi019' 2024-05-09T01:24:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.987+0000 7ff057d27b80 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5636574baab0 0x5636574bbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.987+0000 7ff057d27b80 4 client.14994 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:24:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.987+0000 7ff057d27b80 1 -- 172.21.15.19:0/3134839717 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636573d1980 con 0x5636574baab0 2024-05-09T01:24:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.987+0000 7ff042ffd700 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5636574baab0 0x5636574bbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.988+0000 7ff042ffd700 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5636574baab0 0x5636574bbf20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b80eb7fe3c6b350e server_cookie=e7d0392d8f768f92 in_seq=0 out_seq=0 2024-05-09T01:24:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.988+0000 7ff026ff5700 10 client.14994 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:24:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.988+0000 7ff026ff5700 1 -- 172.21.15.19:0/3134839717 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636573d1980 con 0x5636574baab0 2024-05-09T01:24:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff026ff5700 10 client.14994 handle_command_reply: tid=0 2024-05-09T01:24:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 client.14994 shutdown 2024-05-09T01:24:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 -- 172.21.15.19:0/3134839717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 msgr2=0x5636574ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574ab070 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00d010 tx=0x7ff03c0b3a40 comp rx=0 tx=0).stop 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 -- 172.21.15.19:0/3134839717 shutdown_connections 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5636574baab0 0x5636574bbf20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574ab070 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563657324450 0x5636574b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 --2- 172.21.15.19:0/3134839717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56365740e110 0x5636574b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 -- 172.21.15.19:0/3134839717 >> 172.21.15.19:0/3134839717 conn(0x563657384650 msgr2=0x56365749c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff026ff5700 0 client.14994 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 -- 172.21.15.19:0/3134839717 shutdown_connections 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 1 -- 172.21.15.19:0/3134839717 wait complete. 2024-05-09T01:24:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:57.989+0000 7ff057d27b80 20 client.14994 trim_cache size 0 max 16384 2024-05-09T01:24:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.010+0000 7ff057d27b80 1 Processor -- start 2024-05-09T01:24:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.010+0000 7ff057d27b80 1 -- start start 2024-05-09T01:24:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.010+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574a9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.010+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.010+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.010+0000 7ff057d27b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636574ba260 con 0x5636574a1650 2024-05-09T01:24:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.010+0000 7ff057d27b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5636574ba930 con 0x56365740e110 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.010+0000 7ff057d27b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636574bb000 con 0x563657324450 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54636/0 (socket says 172.21.15.19:54636) 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0437fe700 1 -- 172.21.15.19:0/2896373919 learned_addr learned my addr 172.21.15.19:0/2896373919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff042ffd700 1 --2- 172.21.15.19:0/2896373919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0427fc700 1 --2- 172.21.15.19:0/2896373919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0437fe700 1 -- 172.21.15.19:0/2896373919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 msgr2=0x5636574a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0437fe700 1 --2- 172.21.15.19:0/2896373919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0437fe700 1 -- 172.21.15.19:0/2896373919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 msgr2=0x5636574a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0437fe700 1 --2- 172.21.15.19:0/2896373919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0437fe700 1 -- 172.21.15.19:0/2896373919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636574bb4f0 con 0x5636574a1650 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff0437fe700 1 --2- 172.21.15.19:0/2896373919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574c5380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0aab20 tx=0x7ff03c00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de457c134e974c9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff028ff9700 1 -- 172.21.15.19:0/2896373919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c06f040 con 0x5636574a1650 2024-05-09T01:24:58.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff028ff9700 1 -- 172.21.15.19:0/2896373919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03c0a6790 con 0x5636574a1650 2024-05-09T01:24:58.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.011+0000 7ff028ff9700 1 -- 172.21.15.19:0/2896373919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c005c60 con 0x5636574a1650 2024-05-09T01:24:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.012+0000 7ff057d27b80 1 -- 172.21.15.19:0/2896373919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 msgr2=0x5636574c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.012+0000 7ff057d27b80 1 --2- 172.21.15.19:0/2896373919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574c5380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0aab20 tx=0x7ff03c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.012+0000 7ff057d27b80 1 -- 172.21.15.19:0/2896373919 shutdown_connections 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.012+0000 7ff057d27b80 1 --2- 172.21.15.19:0/2896373919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574c5380 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.012+0000 7ff057d27b80 1 --2- 172.21.15.19:0/2896373919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.012+0000 7ff057d27b80 1 --2- 172.21.15.19:0/2896373919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.012+0000 7ff057d27b80 1 -- 172.21.15.19:0/2896373919 >> 172.21.15.19:0/2896373919 conn(0x563657384650 msgr2=0x5636574b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 -- 172.21.15.19:0/2896373919 shutdown_connections 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 -- 172.21.15.19:0/2896373919 wait complete. 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 -- start start 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 Processor -- start 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636574bbda0 con 0x5636574a1650 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5636574bb6d0 con 0x56365740e110 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.013+0000 7ff057d27b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636574bc470 con 0x563657324450 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff057d27b80 10 client.0 _ll_register_callbacks cb 0x563657235670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff042ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54648/0 (socket says 172.21.15.19:54648) 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff042ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55466/0 (socket says 172.21.15.19:55466) 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0437fe700 1 -- 172.21.15.19:0/4191088214 learned_addr learned my addr 172.21.15.19:0/4191088214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0427fc700 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0437fe700 1 -- 172.21.15.19:0/4191088214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 msgr2=0x5636574b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0437fe700 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0437fe700 1 -- 172.21.15.19:0/4191088214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 msgr2=0x5636574a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0437fe700 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0437fe700 1 -- 172.21.15.19:0/4191088214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636574a8f40 con 0x5636574a1650 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff0437fe700 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a87c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009e40 tx=0x7ff03c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1933ea7459fbbeae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff026ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff026ff5700 1 -- 172.21.15.19:0/4191088214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c06f070 con 0x5636574a1650 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff026ff5700 1 -- 172.21.15.19:0/4191088214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff03c0a9d80 con 0x5636574a1650 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.014+0000 7ff057d27b80 1 -- 172.21.15.19:0/4191088214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5636573cba20 con 0x5636574a1650 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff026ff5700 1 -- 172.21.15.19:0/4191088214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff03c0a6ac0 con 0x5636574a1650 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff057d27b80 10 client.15000 fetch_fsmap learned FSMap version 27 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff057d27b80 1 -- 172.21.15.19:0/4191088214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5636574a8f40 con 0x5636574a1650 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff026ff5700 1 -- 172.21.15.19:0/4191088214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5636573cba20 con 0x5636574a1650 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff026ff5700 1 -- 172.21.15.19:0/4191088214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff03c011040 con 0x5636574a1650 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff057d27b80 10 client.15000 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff057d27b80 10 client.15000 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T01:24:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff057d27b80 20 client.15000 populate_metadata read hostname 'smithi019' 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff057d27b80 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563657527ba0 0x563657529010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff057d27b80 4 client.15000 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff057d27b80 1 -- 172.21.15.19:0/4191088214 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "status"}) -- 0x56365723f920 con 0x563657527ba0 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.015+0000 7ff042ffd700 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563657527ba0 0x563657529010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff042ffd700 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563657527ba0 0x563657529010 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=128df67d822bcd76 server_cookie=188c8fdc863fc612 in_seq=0 out_seq=0 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff026ff5700 10 client.15000 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff026ff5700 1 -- 172.21.15.19:0/4191088214 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56365723f920 con 0x563657527ba0 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff026ff5700 10 client.15000 handle_command_reply: tid=0 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff057d27b80 1 client.15000 shutdown 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff057d27b80 1 -- 172.21.15.19:0/4191088214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 msgr2=0x5636574a87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff057d27b80 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a87c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009e40 tx=0x7ff03c0099e0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff057d27b80 1 -- 172.21.15.19:0/4191088214 shutdown_connections 2024-05-09T01:24:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff057d27b80 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563657527ba0 0x563657529010 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff057d27b80 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636574a1650 0x5636574a87c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff057d27b80 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563657324450 0x5636574b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff057d27b80 1 --2- 172.21.15.19:0/4191088214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56365740e110 0x5636574a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff057d27b80 1 -- 172.21.15.19:0/4191088214 >> 172.21.15.19:0/4191088214 conn(0x563657384650 msgr2=0x5636574aa920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.016+0000 7ff026ff5700 0 client.15000 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.017+0000 7ff057d27b80 1 -- 172.21.15.19:0/4191088214 shutdown_connections 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.017+0000 7ff057d27b80 1 -- 172.21.15.19:0/4191088214 wait complete. 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.017+0000 7ff057d27b80 20 client.15000 trim_cache size 0 max 16384 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-05-09T01:24:58.019 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24451, 2024-05-09T01:24:58.020 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:24:58.020 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:24:58.020 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:24:58.020 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 119.781190826, 2024-05-09T01:24:58.020 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:24:58.020 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:24:58.020 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:24:58.020 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 167.445416623 2024-05-09T01:24:58.020 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.017+0000 7ff049311700 1 -- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff02c0b2880 msgr2=0x7ff02c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.017+0000 7ff049311700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff02c0b2880 0x7ff02c0b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff034006fd0 tx=0x7ff034008040 comp rx=0 tx=0).stop 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.017+0000 7ff049311700 1 -- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 msgr2=0x7ff04419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.017+0000 7ff049311700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 0x7ff04419f620 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0095c0 tx=0x7ff03c007ad0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.017+0000 7ff049311700 1 -- 172.21.15.19:0/1264560096 shutdown_connections 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.018+0000 7ff049311700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff02c0b2880 0x7ff02c0b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.018+0000 7ff049311700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff044109850 0x7ff04419f620 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.018+0000 7ff049311700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0441045d0 0x7ff04419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.018+0000 7ff049311700 1 --2- 172.21.15.19:0/1264560096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff044108f00 0x7ff04419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.018+0000 7ff049311700 1 -- 172.21.15.19:0/1264560096 >> 172.21.15.19:0/1264560096 conn(0x7ff0440ffea0 msgr2=0x7ff044108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.018+0000 7ff049311700 1 -- 172.21.15.19:0/1264560096 shutdown_connections 2024-05-09T01:24:58.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:24:58.018+0000 7ff049311700 1 -- 172.21.15.19:0/1264560096 wait complete. 2024-05-09T01:24:58.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:24:58 smithi019 ceph-mon[30158]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:24:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:24:58 smithi196 ceph-mon[35041]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:24:58.697 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.e status 2024-05-09T01:24:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:24:58 smithi059 ceph-mon[34351]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:24:58.953 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.510+0000 7f6a4c995700 1 -- 172.21.15.19:0/1417247002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108740 msgr2=0x7f6a481100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.510+0000 7f6a4c995700 1 --2- 172.21.15.19:0/1417247002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108740 0x7f6a481100a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6a400099c0 tx=0x7f6a40009cd0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.510+0000 7f6a4c995700 1 -- 172.21.15.19:0/1417247002 shutdown_connections 2024-05-09T01:25:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.510+0000 7f6a4c995700 1 --2- 172.21.15.19:0/1417247002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108740 0x7f6a481100a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.510+0000 7f6a4c995700 1 --2- 172.21.15.19:0/1417247002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a481034c0 0x7f6a481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.510+0000 7f6a4c995700 1 --2- 172.21.15.19:0/1417247002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a48107df0 0x7f6a48108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.510+0000 7f6a4c995700 1 -- 172.21.15.19:0/1417247002 >> 172.21.15.19:0/1417247002 conn(0x7f6a480fedc0 msgr2=0x7f6a481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.510+0000 7f6a4c995700 1 -- 172.21.15.19:0/1417247002 shutdown_connections 2024-05-09T01:25:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.511+0000 7f6a4c995700 1 -- 172.21.15.19:0/1417247002 wait complete. 2024-05-09T01:25:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a4c995700 1 Processor -- start 2024-05-09T01:25:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a4c995700 1 -- start start 2024-05-09T01:25:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a4c995700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a481034c0 0x7f6a4819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a4c995700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a48107df0 0x7f6a4819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a4c995700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108740 0x7f6a48199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a4c995700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48113370 con 0x7f6a48108740 2024-05-09T01:25:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a4c995700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481131f0 con 0x7f6a48107df0 2024-05-09T01:25:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a4c995700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481134f0 con 0x7f6a481034c0 2024-05-09T01:25:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a46d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108740 0x7f6a48199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a46d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108740 0x7f6a48199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40178/0 (socket says 172.21.15.19:40178) 2024-05-09T01:25:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a46d9d700 1 -- 172.21.15.19:0/2568673535 learned_addr learned my addr 172.21.15.19:0/2568673535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a481034c0 0x7f6a4819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.512+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a48107df0 0x7f6a4819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a4659c700 1 -- 172.21.15.19:0/2568673535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a48107df0 msgr2=0x7f6a4819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a48107df0 0x7f6a4819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a4659c700 1 -- 172.21.15.19:0/2568673535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108740 msgr2=0x7f6a48199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108740 0x7f6a48199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a4659c700 1 -- 172.21.15.19:0/2568673535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4819a160 con 0x7f6a481034c0 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a481034c0 0x7f6a4819f890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800ef10 tx=0x7f6a3800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a3ffff700 1 -- 172.21.15.19:0/2568673535 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800c9c0 con 0x7f6a481034c0 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a3ffff700 1 -- 172.21.15.19:0/2568673535 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a38019070 con 0x7f6a481034c0 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a4c995700 1 -- 172.21.15.19:0/2568673535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4819a3f0 con 0x7f6a481034c0 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a3ffff700 1 -- 172.21.15.19:0/2568673535 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38014480 con 0x7f6a481034c0 2024-05-09T01:25:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.513+0000 7f6a4c995700 1 -- 172.21.15.19:0/2568673535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4806b610 con 0x7f6a481034c0 2024-05-09T01:25:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.516+0000 7f6a3ffff700 1 -- 172.21.15.19:0/2568673535 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a38012050 con 0x7f6a481034c0 2024-05-09T01:25:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.517+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a340b2880 0x7f6a340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.517+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a340b2880 0x7f6a340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.517+0000 7f6a3ffff700 1 -- 172.21.15.19:0/2568673535 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6a380e2240 con 0x7f6a481034c0 2024-05-09T01:25:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.517+0000 7f6a5b3abb80 1 Processor -- start 2024-05-09T01:25:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a340b2880 0x7f6a340b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a481028c0 tx=0x7f6a30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a5b3abb80 1 -- start start 2024-05-09T01:25:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac554258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 0x55ac55425e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac5541fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ac5533dfa0 con 0x55ac55388110 2024-05-09T01:25:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac552dc7a0 con 0x55ac5541fa60 2024-05-09T01:25:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac55339f60 con 0x55ac5529e450 2024-05-09T01:25:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac554258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac554258f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40194/0 (socket says 172.21.15.19:40194) 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a4659c700 1 -- 172.21.15.19:0/1994430894 learned_addr learned my addr 172.21.15.19:0/1994430894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/1994430894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 0x55ac55425e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.518+0000 7f6a46d9d700 1 --2- 172.21.15.19:0/1994430894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac5541fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.519+0000 7f6a4659c700 1 -- 172.21.15.19:0/1994430894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 msgr2=0x55ac55425e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.519+0000 7f6a4659c700 1 --2- 172.21.15.19:0/1994430894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 0x55ac55425e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.519+0000 7f6a4659c700 1 -- 172.21.15.19:0/1994430894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 msgr2=0x55ac5541fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.519+0000 7f6a4659c700 1 --2- 172.21.15.19:0/1994430894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac5541fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.519+0000 7f6a4659c700 1 -- 172.21.15.19:0/1994430894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac552d4320 con 0x55ac55388110 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.519+0000 7f6a4659c700 1 --2- 172.21.15.19:0/1994430894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac554258f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6a380a5860 tx=0x7f6a380a9130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb5a8a235cefc29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.519+0000 7f6a3cd99700 1 -- 172.21.15.19:0/1994430894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a92f0 con 0x55ac55388110 2024-05-09T01:25:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.519+0000 7f6a3cd99700 1 -- 172.21.15.19:0/1994430894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a380e1070 con 0x55ac55388110 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.519+0000 7f6a3cd99700 1 -- 172.21.15.19:0/1994430894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a380b21b0 con 0x55ac55388110 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/1994430894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 msgr2=0x55ac554258f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/1994430894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac554258f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6a380a5860 tx=0x7f6a380a9130 comp rx=0 tx=0).stop 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/1994430894 shutdown_connections 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/1994430894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac5541fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/1994430894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac554258f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/1994430894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 0x55ac55425e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/1994430894 >> 172.21.15.19:0/1994430894 conn(0x55ac552fe650 msgr2=0x55ac554163b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/1994430894 shutdown_connections 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/1994430894 wait complete. 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.520+0000 7f6a5b3abb80 1 -- start start 2024-05-09T01:25:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a5b3abb80 1 Processor -- start 2024-05-09T01:25:00.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 0x55ac55429660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac55429ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac5542a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ac551b9b50 con 0x55ac55388110 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac552f8100 con 0x55ac5541fa60 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac554207d0 con 0x55ac5529e450 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a5b3abb80 10 client.0 _ll_register_callbacks cb 0x55ac551af670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a3dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac55429ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a3dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac55429ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40202/0 (socket says 172.21.15.19:40202) 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a3dd9b700 1 -- 172.21.15.19:0/2163721195 learned_addr learned my addr 172.21.15.19:0/2163721195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a46d9d700 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac5542a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 0x55ac55429660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a3dd9b700 1 -- 172.21.15.19:0/2163721195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 msgr2=0x55ac55429660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 0x55ac55429660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a3dd9b700 1 -- 172.21.15.19:0/2163721195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 msgr2=0x55ac5542a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac5542a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.521+0000 7f6a3dd9b700 1 -- 172.21.15.19:0/2163721195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac551b9440 con 0x55ac55388110 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.522+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac55429ba0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6a48106750 tx=0x7f6a300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e158256b3f5999 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.522+0000 7f6a2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.522+0000 7f6a2dffb700 1 -- 172.21.15.19:0/2163721195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3002c060 con 0x55ac55388110 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.522+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2163721195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a380e1070 con 0x55ac55388110 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.522+0000 7f6a2dffb700 1 -- 172.21.15.19:0/2163721195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6a30009040 con 0x55ac55388110 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.522+0000 7f6a2dffb700 1 -- 172.21.15.19:0/2163721195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a3001d6c0 con 0x55ac55388110 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.522+0000 7f6a5b3abb80 10 client.15012 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.522+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2163721195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac55436f60 con 0x55ac55388110 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.522+0000 7f6a2dffb700 1 -- 172.21.15.19:0/2163721195 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a380e1070 con 0x55ac55388110 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.523+0000 7f6a2dffb700 1 -- 172.21.15.19:0/2163721195 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a30027ad0 con 0x55ac55388110 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.523+0000 7f6a5b3abb80 10 client.15012 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.523+0000 7f6a5b3abb80 10 client.15012 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-09T01:25:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.523+0000 7f6a5b3abb80 20 client.15012 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.523+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac55439500 0x55ac5543a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.523+0000 7f6a5b3abb80 4 client.15012 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.523+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2163721195 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac5534b980 con 0x55ac55439500 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.523+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac55439500 0x55ac5543a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.524+0000 7f6a2dffb700 10 client.15012 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.524+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac55439500 0x55ac5543a970 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dff537fb388b15d server_cookie=db98c51ba709bfd6 in_seq=0 out_seq=0 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a2dffb700 1 -- 172.21.15.19:0/2163721195 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac5534b980 con 0x55ac55439500 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a2dffb700 10 client.15012 handle_command_reply: tid=0 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 client.15012 shutdown 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2163721195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 msgr2=0x55ac55429ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac55429ba0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6a48106750 tx=0x7f6a300047c0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2163721195 shutdown_connections 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac55439500 0x55ac5543a970 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a2dffb700 0 client.15012 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:25:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac5542a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5529e450 0x55ac55429660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2163721195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac55388110 0x55ac55429ba0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2163721195 >> 172.21.15.19:0/2163721195 conn(0x55ac552fe650 msgr2=0x55ac55416170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2163721195 shutdown_connections 2024-05-09T01:25:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2163721195 wait complete. 2024-05-09T01:25:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.525+0000 7f6a5b3abb80 20 client.15012 trim_cache size 0 max 16384 2024-05-09T01:25:00.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:00 smithi019 ceph-mon[30158]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a5b3abb80 1 Processor -- start 2024-05-09T01:25:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a5b3abb80 1 -- start start 2024-05-09T01:25:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac55429f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac55388110 0x55ac55428660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac55423250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ac55438cb0 con 0x55ac5529e450 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac55439380 con 0x55ac5541fa60 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac55439a50 con 0x55ac55388110 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac55429f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac55429f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40208/0 (socket says 172.21.15.19:40208) 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a4659c700 1 -- 172.21.15.19:0/2431260933 learned_addr learned my addr 172.21.15.19:0/2431260933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a46d9d700 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac55423250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.548+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac55388110 0x55ac55428660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a4659c700 1 -- 172.21.15.19:0/2431260933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac55388110 msgr2=0x55ac55428660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac55388110 0x55ac55428660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a4659c700 1 -- 172.21.15.19:0/2431260933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 msgr2=0x55ac55423250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac55423250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a4659c700 1 -- 172.21.15.19:0/2431260933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac554391a0 con 0x55ac5529e450 2024-05-09T01:25:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a46d9d700 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac55423250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:25:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac55388110 0x55ac55428660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a4659c700 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac55429f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a380a9770 tx=0x7f6a380b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a4d623d13cb66c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a2ffff700 1 -- 172.21.15.19:0/2431260933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a9ee0 con 0x55ac5529e450 2024-05-09T01:25:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a2ffff700 1 -- 172.21.15.19:0/2431260933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a380e1070 con 0x55ac5529e450 2024-05-09T01:25:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.549+0000 7f6a2ffff700 1 -- 172.21.15.19:0/2431260933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a8a40 con 0x55ac5529e450 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.550+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2431260933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 msgr2=0x55ac55429f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.550+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac55429f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a380a9770 tx=0x7f6a380b2040 comp rx=0 tx=0).stop 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.550+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2431260933 shutdown_connections 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.550+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac5541fa60 0x55ac55423250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.550+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac55429f20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.550+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/2431260933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac55388110 0x55ac55428660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.550+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2431260933 >> 172.21.15.19:0/2431260933 conn(0x55ac552fe650 msgr2=0x55ac55427f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.550+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2431260933 shutdown_connections 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.550+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/2431260933 wait complete. 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a5b3abb80 1 -- start start 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a5b3abb80 1 Processor -- start 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac554267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac55388110 0x55ac55426d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a5b3abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5541fa60 0x55ac55427250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ac5543a7f0 con 0x55ac5529e450 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac5543a120 con 0x55ac55388110 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a5b3abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac5543aec0 con 0x55ac5541fa60 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a5b3abb80 10 client.0 _ll_register_callbacks cb 0x55ac551af670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac554267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac554267d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40222/0 (socket says 172.21.15.19:40222) 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a4659c700 1 -- 172.21.15.19:0/232897716 learned_addr learned my addr 172.21.15.19:0/232897716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.551+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac55388110 0x55ac55426d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a4659c700 1 -- 172.21.15.19:0/232897716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5541fa60 msgr2=0x55ac55427250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a4659c700 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5541fa60 0x55ac55427250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a4659c700 1 -- 172.21.15.19:0/232897716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac55388110 msgr2=0x55ac55426d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a4659c700 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac55388110 0x55ac55426d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a4659c700 1 -- 172.21.15.19:0/232897716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac55442460 con 0x55ac5529e450 2024-05-09T01:25:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a4659c700 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac554267d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009ef0 tx=0x7f6a380a9dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69df67a9eccb76b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a2dffb700 1 -- 172.21.15.19:0/232897716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a9ee0 con 0x55ac5529e450 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a2dffb700 1 -- 172.21.15.19:0/232897716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6a380a5dd0 con 0x55ac5529e450 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/232897716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a380e1070 con 0x55ac5529e450 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a2dffb700 1 -- 172.21.15.19:0/232897716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a3808a070 con 0x55ac5529e450 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a5b3abb80 10 client.15018 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.552+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/232897716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac554a4d20 con 0x55ac5529e450 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.553+0000 7f6a2dffb700 1 -- 172.21.15.19:0/232897716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a380e1070 con 0x55ac5529e450 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.553+0000 7f6a2dffb700 1 -- 172.21.15.19:0/232897716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a380a8a40 con 0x55ac5529e450 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.553+0000 7f6a5b3abb80 10 client.15018 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.553+0000 7f6a5b3abb80 10 client.15018 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.553+0000 7f6a5b3abb80 20 client.15018 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.553+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac554a6a80 0x55ac554a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.553+0000 7f6a5b3abb80 4 client.15018 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.553+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/232897716 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "status"}) -- 0x55ac55345a20 con 0x55ac554a6a80 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.553+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac554a6a80 0x55ac554a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a3dd9b700 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac554a6a80 0x55ac554a7ef0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d303080b8c0784b server_cookie=7975f5c2f8dd9833 in_seq=0 out_seq=0 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a2dffb700 10 client.15018 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a2dffb700 1 -- 172.21.15.19:0/232897716 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ac55345a20 con 0x55ac554a6a80 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a2dffb700 10 client.15018 handle_command_reply: tid=0 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 client.15018 shutdown 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/232897716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 msgr2=0x55ac554267d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac554267d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009ef0 tx=0x7f6a380a9dc0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/232897716 shutdown_connections 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac554a6a80 0x55ac554a7ef0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac5541fa60 0x55ac55427250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a2dffb700 0 client.15018 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac5529e450 0x55ac554267d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 --2- 172.21.15.19:0/232897716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac55388110 0x55ac55426d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/232897716 >> 172.21.15.19:0/232897716 conn(0x55ac552fe650 msgr2=0x55ac55424e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/232897716 shutdown_connections 2024-05-09T01:25:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.554+0000 7f6a5b3abb80 1 -- 172.21.15.19:0/232897716 wait complete. 2024-05-09T01:25:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a5b3abb80 20 client.15018 trim_cache size 0 max 16384 2024-05-09T01:25:00.557 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:25:00.557 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:25:00.557 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-05-09T01:25:00.557 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24452, 2024-05-09T01:25:00.557 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:25:00.558 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:25:00.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:25:00.558 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 123.330324176, 2024-05-09T01:25:00.558 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:25:00.558 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:25:00.558 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:25:00.558 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 175.36533110799999 2024-05-09T01:25:00.558 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 -- 172.21.15.19:0/2568673535 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a340b2880 msgr2=0x7f6a340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a340b2880 0x7f6a340b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a481028c0 tx=0x7f6a30009260 comp rx=0 tx=0).stop 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 -- 172.21.15.19:0/2568673535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a481034c0 msgr2=0x7f6a4819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a481034c0 0x7f6a4819f890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800ef10 tx=0x7f6a3800c3a0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 -- 172.21.15.19:0/2568673535 shutdown_connections 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a340b2880 0x7f6a340b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a48108740 0x7f6a48199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a481034c0 0x7f6a4819f890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 --2- 172.21.15.19:0/2568673535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a48107df0 0x7f6a4819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.555+0000 7f6a4c995700 1 -- 172.21.15.19:0/2568673535 >> 172.21.15.19:0/2568673535 conn(0x7f6a480fedc0 msgr2=0x7f6a48107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.556+0000 7f6a4c995700 1 -- 172.21.15.19:0/2568673535 shutdown_connections 2024-05-09T01:25:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:00.556+0000 7f6a4c995700 1 -- 172.21.15.19:0/2568673535 wait complete. 2024-05-09T01:25:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:00 smithi196 ceph-mon[35041]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:00 smithi059 ceph-mon[34351]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:01.211 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T01:25:01.464 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:02.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:02 smithi196 ceph-mon[35041]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:02.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:02 smithi019 ceph-mon[30158]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:02 smithi059 ceph-mon[34351]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:02.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.972+0000 7fdcaa14d700 1 -- 172.21.15.19:0/1003761424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4107df0 msgr2=0x7fdca4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:02.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.972+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/1003761424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4107df0 0x7fdca4108200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdc980099c0 tx=0x7fdc98009cd0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.972+0000 7fdcaa14d700 1 -- 172.21.15.19:0/1003761424 shutdown_connections 2024-05-09T01:25:02.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.972+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/1003761424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4108740 0x7fdca41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.972+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/1003761424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdca41034c0 0x7fdca41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.972+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/1003761424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4107df0 0x7fdca4108200 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.972+0000 7fdcaa14d700 1 -- 172.21.15.19:0/1003761424 >> 172.21.15.19:0/1003761424 conn(0x7fdca40fedf0 msgr2=0x7fdca4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:02.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.972+0000 7fdcaa14d700 1 -- 172.21.15.19:0/1003761424 shutdown_connections 2024-05-09T01:25:02.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.972+0000 7fdcaa14d700 1 -- 172.21.15.19:0/1003761424 wait complete. 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.973+0000 7fdcaa14d700 1 Processor -- start 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.973+0000 7fdcaa14d700 1 -- start start 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.973+0000 7fdcaa14d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdca41034c0 0x7fdca40ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.973+0000 7fdcaa14d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4107df0 0x7fdca40ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.973+0000 7fdcaa14d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4108740 0x7fdca40ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.973+0000 7fdcaa14d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4113370 con 0x7fdca4108740 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.973+0000 7fdcaa14d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41131f0 con 0x7fdca41034c0 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.973+0000 7fdcaa14d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdca41134f0 con 0x7fdca4107df0 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4108740 0x7fdca40ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4108740 0x7fdca40ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40246/0 (socket says 172.21.15.19:40246) 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca3fff700 1 -- 172.21.15.19:0/3982262594 learned_addr learned my addr 172.21.15.19:0/3982262594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca37fe700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdca41034c0 0x7fdca40ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4107df0 0x7fdca40ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca3fff700 1 -- 172.21.15.19:0/3982262594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4107df0 msgr2=0x7fdca40ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca3fff700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4107df0 0x7fdca40ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca3fff700 1 -- 172.21.15.19:0/3982262594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdca41034c0 msgr2=0x7fdca40ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca3fff700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdca41034c0 0x7fdca40ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca3fff700 1 -- 172.21.15.19:0/3982262594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41a9170 con 0x7fdca4108740 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca3fff700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4108740 0x7fdca40ffd50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c007ee0 tx=0x7fdc9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca0ff9700 1 -- 172.21.15.19:0/3982262594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00dbd0 con 0x7fdca4108740 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca0ff9700 1 -- 172.21.15.19:0/3982262594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc9c00dd70 con 0x7fdca4108740 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdcaa14d700 1 -- 172.21.15.19:0/3982262594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca41a9460 con 0x7fdca4108740 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdca0ff9700 1 -- 172.21.15.19:0/3982262594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c005990 con 0x7fdca4108740 2024-05-09T01:25:02.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.974+0000 7fdcaa14d700 1 -- 172.21.15.19:0/3982262594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca41a99c0 con 0x7fdca4108740 2024-05-09T01:25:02.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.977+0000 7fdca0ff9700 1 -- 172.21.15.19:0/3982262594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdc9c01a020 con 0x7fdca4108740 2024-05-09T01:25:02.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.978+0000 7fdca0ff9700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc8c0b2900 0x7fdc8c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.978+0000 7fdca37fe700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc8c0b2900 0x7fdc8c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.978+0000 7fdca0ff9700 1 -- 172.21.15.19:0/3982262594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdc9c0e18f0 con 0x7fdca4108740 2024-05-09T01:25:02.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.978+0000 7fdcb8b63b80 1 Processor -- start 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.978+0000 7fdcb8b63b80 1 -- start start 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca37fe700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc8c0b2900 0x7fdc8c0b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc94007be0 tx=0x7fdc940073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 0x55a88c6514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c6519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c64ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a88c56dfa0 con 0x55a88c64b650 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a88c50c7a0 con 0x55a88c5b8110 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a88c569f60 con 0x55a88c4ce450 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c64ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c64ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40262/0 (socket says 172.21.15.19:40262) 2024-05-09T01:25:02.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca3fff700 1 -- 172.21.15.19:0/2211267583 learned_addr learned my addr 172.21.15.19:0/2211267583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca37fe700 1 --2- 172.21.15.19:0/2211267583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 0x55a88c6514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/2211267583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c6519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca3fff700 1 -- 172.21.15.19:0/2211267583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 msgr2=0x55a88c6519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca3fff700 1 --2- 172.21.15.19:0/2211267583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c6519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca3fff700 1 -- 172.21.15.19:0/2211267583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 msgr2=0x55a88c6514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca3fff700 1 --2- 172.21.15.19:0/2211267583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 0x55a88c6514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca3fff700 1 -- 172.21.15.19:0/2211267583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a88c504320 con 0x55a88c64b650 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.979+0000 7fdca3fff700 1 --2- 172.21.15.19:0/2211267583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c64ba80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0a8e20 tx=0x7fdc9c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c1b83ecb49924f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.980+0000 7fdc89ffb700 1 -- 172.21.15.19:0/2211267583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0a9590 con 0x55a88c64b650 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.980+0000 7fdc89ffb700 1 -- 172.21.15.19:0/2211267583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc9c0a8e90 con 0x55a88c64b650 2024-05-09T01:25:02.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.980+0000 7fdc89ffb700 1 -- 172.21.15.19:0/2211267583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0a9360 con 0x55a88c64b650 2024-05-09T01:25:02.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2211267583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 msgr2=0x55a88c64ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:02.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2211267583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c64ba80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0a8e20 tx=0x7fdc9c005bb0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2211267583 shutdown_connections 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2211267583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c64ba80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2211267583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 0x55a88c6514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2211267583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c6519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2211267583 >> 172.21.15.19:0/2211267583 conn(0x55a88c52e650 msgr2=0x55a88c6463b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2211267583 shutdown_connections 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2211267583 wait complete. 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 -- start start 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.981+0000 7fdcb8b63b80 1 Processor -- start 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c4ce450 0x55a88c655120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c5b8110 0x55a88c655660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 0x55a88c655ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a88c3e9b50 con 0x55a88c4ce450 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a88c528100 con 0x55a88c64b650 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a88c658180 con 0x55a88c5b8110 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdcb8b63b80 10 client.0 _ll_register_callbacks cb 0x55a88c3df670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c4ce450 0x55a88c655120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c4ce450 0x55a88c655120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40264/0 (socket says 172.21.15.19:40264) 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca37fe700 1 -- 172.21.15.19:0/2528428313 learned_addr learned my addr 172.21.15.19:0/2528428313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c5b8110 0x55a88c655660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca3fff700 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 0x55a88c655ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca37fe700 1 -- 172.21.15.19:0/2528428313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c5b8110 msgr2=0x55a88c655660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca37fe700 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c5b8110 0x55a88c655660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca37fe700 1 -- 172.21.15.19:0/2528428313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 msgr2=0x55a88c655ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca37fe700 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 0x55a88c655ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca37fe700 1 -- 172.21.15.19:0/2528428313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a88c3e9440 con 0x55a88c4ce450 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c5b8110 0x55a88c655660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.982+0000 7fdca3fff700 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 0x55a88c655ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdca37fe700 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c4ce450 0x55a88c655120 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc940042e0 tx=0x7fdc940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cedad879c8237a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdc82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdc82ffd700 1 -- 172.21.15.19:0/2528428313 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94024070 con 0x55a88c4ce450 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdc82ffd700 1 -- 172.21.15.19:0/2528428313 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdc94003d10 con 0x55a88c4ce450 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2528428313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9c0a8e90 con 0x55a88c4ce450 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdc82ffd700 1 -- 172.21.15.19:0/2528428313 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc94030050 con 0x55a88c4ce450 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdcb8b63b80 10 client.15030 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2528428313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a88c662b50 con 0x55a88c4ce450 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdc82ffd700 1 -- 172.21.15.19:0/2528428313 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc9c0a8e90 con 0x55a88c4ce450 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdc82ffd700 1 -- 172.21.15.19:0/2528428313 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc94020410 con 0x55a88c4ce450 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdcb8b63b80 10 client.15030 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdcb8b63b80 10 client.15030 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdcb8b63b80 20 client.15030 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55a88c6651b0 0x55a88c666620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdcb8b63b80 4 client.15030 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.983+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2528428313 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a88c57b980 con 0x55a88c6651b0 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.984+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55a88c6651b0 0x55a88c666620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.984+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55a88c6651b0 0x55a88c666620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c4eb3f074d680e5 server_cookie=854530adedeb708b in_seq=0 out_seq=0 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.984+0000 7fdc82ffd700 10 client.15030 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdc82ffd700 1 -- 172.21.15.19:0/2528428313 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a88c57b980 con 0x55a88c6651b0 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdc82ffd700 10 client.15030 handle_command_reply: tid=0 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 client.15030 shutdown 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2528428313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c4ce450 msgr2=0x55a88c655120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c4ce450 0x55a88c655120 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc940042e0 tx=0x7fdc940047c0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2528428313 shutdown_connections 2024-05-09T01:25:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55a88c6651b0 0x55a88c666620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 0x55a88c655ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c4ce450 0x55a88c655120 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/2528428313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c5b8110 0x55a88c655660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:02.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2528428313 >> 172.21.15.19:0/2528428313 conn(0x55a88c52e650 msgr2=0x55a88c646170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:02.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdc82ffd700 0 client.15030 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:25:02.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2528428313 shutdown_connections 2024-05-09T01:25:02.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.985+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/2528428313 wait complete. 2024-05-09T01:25:02.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:02.986+0000 7fdcb8b63b80 20 client.15030 trim_cache size 0 max 16384 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdcb8b63b80 1 Processor -- start 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdcb8b63b80 1 -- start start 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c650830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 0x55a88c661150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c66fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a88c664960 con 0x55a88c64b650 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a88c665030 con 0x55a88c5b8110 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a88c665700 con 0x55a88c4ce450 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c66fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c66fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40276/0 (socket says 172.21.15.19:40276) 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdca3fff700 1 -- 172.21.15.19:0/528682633 learned_addr learned my addr 172.21.15.19:0/528682633 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.007+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 0x55a88c661150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdca37fe700 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c650830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdca3fff700 1 -- 172.21.15.19:0/528682633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 msgr2=0x55a88c650830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdca3fff700 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c650830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdca3fff700 1 -- 172.21.15.19:0/528682633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 msgr2=0x55a88c661150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdca3fff700 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 0x55a88c661150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdca3fff700 1 -- 172.21.15.19:0/528682633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a88c6579c0 con 0x55a88c64b650 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdca37fe700 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c650830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdca3fff700 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c66fa80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0a8b90 tx=0x7fdc9c0a4e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3772f1d958fcf18c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdc897fa700 1 -- 172.21.15.19:0/528682633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0c8070 con 0x55a88c64b650 2024-05-09T01:25:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdc897fa700 1 -- 172.21.15.19:0/528682633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc9c0ad240 con 0x55a88c64b650 2024-05-09T01:25:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.008+0000 7fdc897fa700 1 -- 172.21.15.19:0/528682633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0b1860 con 0x55a88c64b650 2024-05-09T01:25:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.009+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/528682633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 msgr2=0x55a88c66fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.009+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c66fa80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0a8b90 tx=0x7fdc9c0a4e80 comp rx=0 tx=0).stop 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.009+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/528682633 shutdown_connections 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.009+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c64b650 0x55a88c66fa80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.009+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c650830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.009+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/528682633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c5b8110 0x55a88c661150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.009+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/528682633 >> 172.21.15.19:0/528682633 conn(0x55a88c52e650 msgr2=0x55a88c650120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.009+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/528682633 shutdown_connections 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.009+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/528682633 wait complete. 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.010+0000 7fdcb8b63b80 1 -- start start 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.010+0000 7fdcb8b63b80 1 Processor -- start 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.010+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c652000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.010+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c5b8110 0x55a88c652540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.010+0000 7fdcb8b63b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 0x55a88c652a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.010+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a88c6664a0 con 0x55a88c5b8110 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.010+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a88c665dd0 con 0x55a88c64b650 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.010+0000 7fdcb8b63b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a88c666b70 con 0x55a88c4ce450 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdcb8b63b80 10 client.0 _ll_register_callbacks cb 0x55a88c3df670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c5b8110 0x55a88c652540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c5b8110 0x55a88c652540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40284/0 (socket says 172.21.15.19:40284) 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca2ffd700 1 -- 172.21.15.19:0/1006862525 learned_addr learned my addr 172.21.15.19:0/1006862525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca2ffd700 1 -- 172.21.15.19:0/1006862525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 msgr2=0x55a88c652000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca3fff700 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 0x55a88c652a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c652000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca2ffd700 1 -- 172.21.15.19:0/1006862525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 msgr2=0x55a88c652a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 0x55a88c652a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca2ffd700 1 -- 172.21.15.19:0/1006862525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a88c66de30 con 0x55a88c5b8110 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdca2ffd700 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c5b8110 0x55a88c652540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009750 tx=0x7fdc9800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307d71a0b568aeee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdc82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:03.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdc82ffd700 1 -- 172.21.15.19:0/1006862525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9802a070 con 0x55a88c5b8110 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdc82ffd700 1 -- 172.21.15.19:0/1006862525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdc98003860 con 0x55a88c5b8110 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.011+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/1006862525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a88c575a20 con 0x55a88c5b8110 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdc82ffd700 1 -- 172.21.15.19:0/1006862525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc98036050 con 0x55a88c5b8110 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdcb8b63b80 10 client.15036 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/1006862525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a88c6d0850 con 0x55a88c5b8110 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdc82ffd700 1 -- 172.21.15.19:0/1006862525 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a88c575a20 con 0x55a88c5b8110 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdc82ffd700 1 -- 172.21.15.19:0/1006862525 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc9800f460 con 0x55a88c5b8110 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdcb8b63b80 10 client.15036 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdcb8b63b80 10 client.15036 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdcb8b63b80 20 client.15036 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55a88c6d2d90 0x55a88c6d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdcb8b63b80 4 client.15036 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/1006862525 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "status"}) -- 0x55a88c3e9920 con 0x55a88c6d2d90 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.012+0000 7fdca37fe700 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55a88c6d2d90 0x55a88c6d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.013+0000 7fdc82ffd700 10 client.15036 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.013+0000 7fdca37fe700 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55a88c6d2d90 0x55a88c6d4200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a60563033f3bdf5a server_cookie=37f329dc9ab6abed in_seq=0 out_seq=0 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.013+0000 7fdc82ffd700 1 -- 172.21.15.19:0/1006862525 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a88c3e9920 con 0x55a88c6d2d90 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.013+0000 7fdc82ffd700 10 client.15036 handle_command_reply: tid=0 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.013+0000 7fdcb8b63b80 1 client.15036 shutdown 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.013+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/1006862525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c5b8110 msgr2=0x55a88c652540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.013+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c5b8110 0x55a88c652540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009750 tx=0x7fdc9800b840 comp rx=0 tx=0).stop 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/1006862525 shutdown_connections 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55a88c6d2d90 0x55a88c6d4200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a88c64b650 0x55a88c652a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a88c4ce450 0x55a88c652000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdc82ffd700 0 client.15036 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:25:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdcb8b63b80 1 --2- 172.21.15.19:0/1006862525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a88c5b8110 0x55a88c652540 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/1006862525 >> 172.21.15.19:0/1006862525 conn(0x55a88c52e650 msgr2=0x55a88c64f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/1006862525 shutdown_connections 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdcb8b63b80 1 -- 172.21.15.19:0/1006862525 wait complete. 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.014+0000 7fdcb8b63b80 20 client.15036 trim_cache size 0 max 16384 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24466, 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:25:03.016 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:25:03.017 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:25:03.017 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 126.797903531, 2024-05-09T01:25:03.017 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:25:03.017 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:25:03.017 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:25:03.017 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 167.01784910000001 2024-05-09T01:25:03.017 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 -- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc8c0b2900 msgr2=0x7fdc8c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc8c0b2900 0x7fdc8c0b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc94007be0 tx=0x7fdc940073d0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 -- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4108740 msgr2=0x7fdca40ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4108740 0x7fdca40ffd50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c007ee0 tx=0x7fdc9c00d3b0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 -- 172.21.15.19:0/3982262594 shutdown_connections 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc8c0b2900 0x7fdc8c0b4e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca4108740 0x7fdca40ffd50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdca41034c0 0x7fdca40ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 --2- 172.21.15.19:0/3982262594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca4107df0 0x7fdca40ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 -- 172.21.15.19:0/3982262594 >> 172.21.15.19:0/3982262594 conn(0x7fdca40fedf0 msgr2=0x7fdca4075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 -- 172.21.15.19:0/3982262594 shutdown_connections 2024-05-09T01:25:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:03.015+0000 7fdcaa14d700 1 -- 172.21.15.19:0/3982262594 wait complete. 2024-05-09T01:25:03.714 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T01:25:03.975 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:04 smithi019 ceph-mon[30158]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:04 smithi196 ceph-mon[35041]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:04 smithi059 ceph-mon[34351]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:05.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.667+0000 7f5f4ed19700 1 -- 172.21.15.19:0/4228938581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f48107df0 msgr2=0x7f5f48108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:05.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.667+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/4228938581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f48107df0 0x7f5f48108200 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800b0d0 tx=0x7f5f3800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.667+0000 7f5f4ed19700 1 -- 172.21.15.19:0/4228938581 shutdown_connections 2024-05-09T01:25:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.667+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/4228938581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f48108740 0x7f5f481100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.667+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/4228938581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f481034c0 0x7f5f481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.667+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/4228938581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f48107df0 0x7f5f48108200 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.667+0000 7f5f4ed19700 1 -- 172.21.15.19:0/4228938581 >> 172.21.15.19:0/4228938581 conn(0x7f5f480fedf0 msgr2=0x7f5f48101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.667+0000 7f5f4ed19700 1 -- 172.21.15.19:0/4228938581 shutdown_connections 2024-05-09T01:25:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.667+0000 7f5f4ed19700 1 -- 172.21.15.19:0/4228938581 wait complete. 2024-05-09T01:25:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4ed19700 1 Processor -- start 2024-05-09T01:25:05.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4ed19700 1 -- start start 2024-05-09T01:25:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4ed19700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f481034c0 0x7f5f481a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4ed19700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f48107df0 0x7f5f481a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4ed19700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f48108740 0x7f5f481a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4ed19700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481133c0 con 0x7f5f481034c0 2024-05-09T01:25:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4ed19700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48113240 con 0x7f5f48108740 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4ed19700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48113540 con 0x7f5f48107df0 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4cab5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f481034c0 0x7f5f481a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4cab5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f481034c0 0x7f5f481a70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40308/0 (socket says 172.21.15.19:40308) 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4cab5700 1 -- 172.21.15.19:0/2136035519 learned_addr learned my addr 172.21.15.19:0/2136035519 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f48108740 0x7f5f481a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.668+0000 7f5f47fff700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f48107df0 0x7f5f481a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f4cab5700 1 -- 172.21.15.19:0/2136035519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f48107df0 msgr2=0x7f5f481a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f48107df0 0x7f5f481a7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f4cab5700 1 -- 172.21.15.19:0/2136035519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f48108740 msgr2=0x7f5f481a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f48108740 0x7f5f481a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f4cab5700 1 -- 172.21.15.19:0/2136035519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f481a1920 con 0x7f5f481034c0 2024-05-09T01:25:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f481034c0 0x7f5f481a70e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b530 tx=0x7f5f3000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:05.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f45ffb700 1 -- 172.21.15.19:0/2136035519 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30015020 con 0x7f5f481034c0 2024-05-09T01:25:05.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f4ed19700 1 -- 172.21.15.19:0/2136035519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f481a1c10 con 0x7f5f481034c0 2024-05-09T01:25:05.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f45ffb700 1 -- 172.21.15.19:0/2136035519 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f30011ce0 con 0x7f5f481034c0 2024-05-09T01:25:05.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f45ffb700 1 -- 172.21.15.19:0/2136035519 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30010dd0 con 0x7f5f481034c0 2024-05-09T01:25:05.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.669+0000 7f5f4ed19700 1 -- 172.21.15.19:0/2136035519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f481a2120 con 0x7f5f481034c0 2024-05-09T01:25:05.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.672+0000 7f5f45ffb700 1 -- 172.21.15.19:0/2136035519 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5f30018020 con 0x7f5f481034c0 2024-05-09T01:25:05.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.673+0000 7f5f45ffb700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f340b2880 0x7f5f340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.673+0000 7f5f47fff700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f340b2880 0x7f5f340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.673+0000 7f5f45ffb700 1 -- 172.21.15.19:0/2136035519 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5f300e1950 con 0x7f5f481034c0 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.673+0000 7f5f5d72fb80 1 Processor -- start 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.673+0000 7f5f5d72fb80 1 -- start start 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f47fff700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f340b2880 0x7f5f340b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5f48101260 tx=0x7f5f3800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a0504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a0509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a04aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e39f6cfa0 con 0x557e3a04a650 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557e39f0b7a0 con 0x557e39ecd450 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e39f68f60 con 0x557e39fb7110 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4d2b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a04aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4d2b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a04aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40312/0 (socket says 172.21.15.19:40312) 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4d2b6700 1 -- 172.21.15.19:0/3739128091 learned_addr learned my addr 172.21.15.19:0/3739128091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/3739128091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a0504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f47fff700 1 --2- 172.21.15.19:0/3739128091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a0509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4d2b6700 1 -- 172.21.15.19:0/3739128091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 msgr2=0x557e3a0504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/3739128091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a0504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4d2b6700 1 -- 172.21.15.19:0/3739128091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 msgr2=0x557e3a0509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/3739128091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a0509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4d2b6700 1 -- 172.21.15.19:0/3739128091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e39f03320 con 0x557e3a04a650 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/3739128091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a04aa80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000acb0 tx=0x7f5f4000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456783190802fe04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.674+0000 7f5f3effd700 1 -- 172.21.15.19:0/3739128091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40004830 con 0x557e3a04a650 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.675+0000 7f5f3effd700 1 -- 172.21.15.19:0/3739128091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f40005020 con 0x557e3a04a650 2024-05-09T01:25:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.675+0000 7f5f3effd700 1 -- 172.21.15.19:0/3739128091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f400186a0 con 0x557e3a04a650 2024-05-09T01:25:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.675+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3739128091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 msgr2=0x557e3a04aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.675+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/3739128091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a04aa80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000acb0 tx=0x7f5f4000f3c0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.676+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3739128091 shutdown_connections 2024-05-09T01:25:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.676+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/3739128091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a04aa80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.676+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/3739128091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a0504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.676+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/3739128091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a0509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.676+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3739128091 >> 172.21.15.19:0/3739128091 conn(0x557e39f2d650 msgr2=0x557e3a0453b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.676+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3739128091 shutdown_connections 2024-05-09T01:25:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.676+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3739128091 wait complete. 2024-05-09T01:25:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.676+0000 7f5f5d72fb80 1 -- start start 2024-05-09T01:25:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.676+0000 7f5f5d72fb80 1 Processor -- start 2024-05-09T01:25:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a0540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a054630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a054b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e39de8b50 con 0x557e3a04a650 2024-05-09T01:25:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557e39f27100 con 0x557e39ecd450 2024-05-09T01:25:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e3a057180 con 0x557e39fb7110 2024-05-09T01:25:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f5d72fb80 10 client.0 _ll_register_callbacks cb 0x557e39dde670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f47fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a054630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f4d2b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a054b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f47fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a054630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:57692/0 (socket says 172.21.15.19:57692) 2024-05-09T01:25:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f47fff700 1 -- 172.21.15.19:0/1276624681 learned_addr learned my addr 172.21.15.19:0/1276624681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a0540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f4d2b6700 1 -- 172.21.15.19:0/1276624681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 msgr2=0x557e3a054630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a054630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f4d2b6700 1 -- 172.21.15.19:0/1276624681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 msgr2=0x557e3a0540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a0540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f4d2b6700 1 -- 172.21.15.19:0/1276624681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e39de8440 con 0x557e3a04a650 2024-05-09T01:25:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f47fff700 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a054630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:25:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a054b70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x557e3a051070 tx=0x7f5f400054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f457c7398c55cfe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.677+0000 7f5f27fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f27fff700 1 -- 172.21.15.19:0/1276624681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40026020 con 0x557e3a04a650 2024-05-09T01:25:05.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f27fff700 1 -- 172.21.15.19:0/1276624681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5f40004450 con 0x557e3a04a650 2024-05-09T01:25:05.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/1276624681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f40005020 con 0x557e3a04a650 2024-05-09T01:25:05.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f27fff700 1 -- 172.21.15.19:0/1276624681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f40031050 con 0x557e3a04a650 2024-05-09T01:25:05.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f5d72fb80 10 client.15048 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:05.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/1276624681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e3a061ac0 con 0x557e3a04a650 2024-05-09T01:25:05.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f27fff700 1 -- 172.21.15.19:0/1276624681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f40005020 con 0x557e3a04a650 2024-05-09T01:25:05.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f27fff700 1 -- 172.21.15.19:0/1276624681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f40003a60 con 0x557e3a04a650 2024-05-09T01:25:05.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f5d72fb80 10 client.15048 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:05.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f5d72fb80 10 client.15048 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T01:25:05.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f5d72fb80 20 client.15048 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:05.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x557e3a063da0 0x557e3a065210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.678+0000 7f5f5d72fb80 4 client.15048 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:25:05.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.679+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/1276624681 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e39f7a980 con 0x557e3a063da0 2024-05-09T01:25:05.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.679+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x557e3a063da0 0x557e3a065210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.679+0000 7f5f27fff700 10 client.15048 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:25:05.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.679+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x557e3a063da0 0x557e3a065210 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97ee358fd0469303 server_cookie=f41e883230adf1 in_seq=0 out_seq=0 2024-05-09T01:25:05.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f27fff700 1 -- 172.21.15.19:0/1276624681 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e39f7a980 con 0x557e3a063da0 2024-05-09T01:25:05.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f27fff700 10 client.15048 handle_command_reply: tid=0 2024-05-09T01:25:05.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 client.15048 shutdown 2024-05-09T01:25:05.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/1276624681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 msgr2=0x557e3a054b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:05.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a054b70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x557e3a051070 tx=0x7f5f400054d0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/1276624681 shutdown_connections 2024-05-09T01:25:05.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x557e3a063da0 0x557e3a065210 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f27fff700 0 client.15048 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:25:05.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e3a04a650 0x557e3a054b70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a0540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/1276624681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a054630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/1276624681 >> 172.21.15.19:0/1276624681 conn(0x557e39f2d650 msgr2=0x557e3a045170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:05.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/1276624681 shutdown_connections 2024-05-09T01:25:05.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.680+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/1276624681 wait complete. 2024-05-09T01:25:05.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.681+0000 7f5f5d72fb80 20 client.15048 trim_cache size 0 max 16384 2024-05-09T01:25:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f5d72fb80 1 Processor -- start 2024-05-09T01:25:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f5d72fb80 1 -- start start 2024-05-09T01:25:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39ecd450 0x557e3a04e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a060120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e3a04a650 0x557e3a044890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e3a063550 con 0x557e39ecd450 2024-05-09T01:25:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557e3a063c20 con 0x557e3a04a650 2024-05-09T01:25:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e3a0642f0 con 0x557e39fb7110 2024-05-09T01:25:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f4cab5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39ecd450 0x557e3a04e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f4cab5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39ecd450 0x557e3a04e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40344/0 (socket says 172.21.15.19:40344) 2024-05-09T01:25:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f4cab5700 1 -- 172.21.15.19:0/3539315457 learned_addr learned my addr 172.21.15.19:0/3539315457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f47fff700 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a060120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f4cab5700 1 -- 172.21.15.19:0/3539315457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 msgr2=0x557e3a060120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e3a04a650 0x557e3a044890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a060120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.702+0000 7f5f4cab5700 1 -- 172.21.15.19:0/3539315457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e3a04a650 msgr2=0x557e3a044890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.703+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e3a04a650 0x557e3a044890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.703+0000 7f5f4cab5700 1 -- 172.21.15.19:0/3539315457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e3a04b790 con 0x557e39ecd450 2024-05-09T01:25:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.703+0000 7f5f47fff700 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a060120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.703+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e3a04a650 0x557e3a044890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.703+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39ecd450 0x557e3a04e710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f300df3d0 tx=0x7f5f30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d072b4c773adbe79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.703+0000 7f5f3e7fc700 1 -- 172.21.15.19:0/3539315457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3006c020 con 0x557e39ecd450 2024-05-09T01:25:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.703+0000 7f5f3e7fc700 1 -- 172.21.15.19:0/3539315457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f300ad2a0 con 0x557e39ecd450 2024-05-09T01:25:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.703+0000 7f5f3e7fc700 1 -- 172.21.15.19:0/3539315457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300a8150 con 0x557e39ecd450 2024-05-09T01:25:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3539315457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39ecd450 msgr2=0x557e3a04e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39ecd450 0x557e3a04e710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f300df3d0 tx=0x7f5f30000bc0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3539315457 shutdown_connections 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e3a04a650 0x557e3a044890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39ecd450 0x557e3a04e710 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/3539315457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e39fb7110 0x557e3a060120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3539315457 >> 172.21.15.19:0/3539315457 conn(0x557e39f2d650 msgr2=0x557e3a0529e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3539315457 shutdown_connections 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/3539315457 wait complete. 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 -- start start 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.704+0000 7f5f5d72fb80 1 Processor -- start 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a053e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39fb7110 0x557e3a0543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f5d72fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e3a04a650 0x557e3a054900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e3a065090 con 0x557e39fb7110 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557e3a0649c0 con 0x557e39ecd450 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f5d72fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e3a065760 con 0x557e3a04a650 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f5d72fb80 10 client.0 _ll_register_callbacks cb 0x557e39dde670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39fb7110 0x557e3a0543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39fb7110 0x557e3a0543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40352/0 (socket says 172.21.15.19:40352) 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f47fff700 1 -- 172.21.15.19:0/314952359 learned_addr learned my addr 172.21.15.19:0/314952359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a053e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f4d2b6700 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e3a04a650 0x557e3a054900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f47fff700 1 -- 172.21.15.19:0/314952359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e3a04a650 msgr2=0x557e3a054900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f47fff700 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e3a04a650 0x557e3a054900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f47fff700 1 -- 172.21.15.19:0/314952359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 msgr2=0x557e3a053e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f47fff700 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a053e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f47fff700 1 -- 172.21.15.19:0/314952359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e3a06ac50 con 0x557e39fb7110 2024-05-09T01:25:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f47fff700 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39fb7110 0x557e3a0543c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f48076b40 tx=0x7f5f380099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d666113c36d9c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f27fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.705+0000 7f5f27fff700 1 -- 172.21.15.19:0/314952359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38005250 con 0x557e39fb7110 2024-05-09T01:25:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/314952359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f300ad2a0 con 0x557e39fb7110 2024-05-09T01:25:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f27fff700 1 -- 172.21.15.19:0/314952359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5f380053f0 con 0x557e39fb7110 2024-05-09T01:25:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f27fff700 1 -- 172.21.15.19:0/314952359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f380055f0 con 0x557e39fb7110 2024-05-09T01:25:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f5d72fb80 10 client.15054 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/314952359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e3a0cf4c0 con 0x557e39fb7110 2024-05-09T01:25:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f27fff700 1 -- 172.21.15.19:0/314952359 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f300ad2a0 con 0x557e39fb7110 2024-05-09T01:25:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f27fff700 1 -- 172.21.15.19:0/314952359 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f38031070 con 0x557e39fb7110 2024-05-09T01:25:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f5d72fb80 10 client.15054 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f5d72fb80 10 client.15054 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T01:25:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f5d72fb80 20 client.15054 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x557e3a063da0 0x557e3a0d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f5d72fb80 4 client.15054 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:25:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.706+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/314952359 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "status"}) -- 0x557e39f74a20 con 0x557e3a063da0 2024-05-09T01:25:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x557e3a063da0 0x557e3a0d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f4cab5700 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x557e3a063da0 0x557e3a0d2db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=959b225a216e6104 server_cookie=17bd8ce75be58b6c in_seq=0 out_seq=0 2024-05-09T01:25:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f27fff700 10 client.15054 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:25:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f27fff700 1 -- 172.21.15.19:0/314952359 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557e39f74a20 con 0x557e3a063da0 2024-05-09T01:25:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f27fff700 10 client.15054 handle_command_reply: tid=0 2024-05-09T01:25:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f5d72fb80 1 client.15054 shutdown 2024-05-09T01:25:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/314952359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39fb7110 msgr2=0x557e3a0543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39fb7110 0x557e3a0543c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f48076b40 tx=0x7f5f380099e0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/314952359 shutdown_connections 2024-05-09T01:25:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x557e3a063da0 0x557e3a0d2db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e3a04a650 0x557e3a054900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e39ecd450 0x557e3a053e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f5d72fb80 1 --2- 172.21.15.19:0/314952359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e39fb7110 0x557e3a0543c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/314952359 >> 172.21.15.19:0/314952359 conn(0x557e39f2d650 msgr2=0x557e3a04ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.707+0000 7f5f27fff700 0 client.15054 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:25:05.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/314952359 shutdown_connections 2024-05-09T01:25:05.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f5d72fb80 1 -- 172.21.15.19:0/314952359 wait complete. 2024-05-09T01:25:05.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f5d72fb80 20 client.15054 trim_cache size 0 max 16384 2024-05-09T01:25:05.717 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:25:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:25:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-05-09T01:25:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24473, 2024-05-09T01:25:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:25:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:25:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:25:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 130.49873433600001, 2024-05-09T01:25:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:25:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:25:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:25:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 172.23096798200001 2024-05-09T01:25:05.720 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:25:05.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 -- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f340b2880 msgr2=0x7f5f340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:05.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f340b2880 0x7f5f340b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5f48101260 tx=0x7f5f3800b040 comp rx=0 tx=0).stop 2024-05-09T01:25:05.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 -- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f481034c0 msgr2=0x7f5f481a70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:05.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f481034c0 0x7f5f481a70e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b530 tx=0x7f5f3000b840 comp rx=0 tx=0).stop 2024-05-09T01:25:05.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 -- 172.21.15.19:0/2136035519 shutdown_connections 2024-05-09T01:25:05.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f48108740 0x7f5f481a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f340b2880 0x7f5f340b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f481034c0 0x7f5f481a70e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 --2- 172.21.15.19:0/2136035519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f48107df0 0x7f5f481a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:05.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 -- 172.21.15.19:0/2136035519 >> 172.21.15.19:0/2136035519 conn(0x7f5f480fedf0 msgr2=0x7f5f48075b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:05.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 -- 172.21.15.19:0/2136035519 shutdown_connections 2024-05-09T01:25:05.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:05.708+0000 7f5f4ed19700 1 -- 172.21.15.19:0/2136035519 wait complete. 2024-05-09T01:25:06.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T01:25:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:06.698 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:06 smithi196 ceph-mon[35041]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:06.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:06 smithi019 ceph-mon[30158]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:06 smithi059 ceph-mon[34351]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.319+0000 7f16645eb700 1 -- 172.21.15.19:0/4101472611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c10e8f0 msgr2=0x7f165c110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.319+0000 7f16645eb700 1 --2- 172.21.15.19:0/4101472611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c10e8f0 0x7f165c110dd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f165400b0a0 tx=0x7f165400b3b0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.319+0000 7f16645eb700 1 -- 172.21.15.19:0/4101472611 shutdown_connections 2024-05-09T01:25:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.319+0000 7f16645eb700 1 --2- 172.21.15.19:0/4101472611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c10e8f0 0x7f165c110dd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.319+0000 7f16645eb700 1 --2- 172.21.15.19:0/4101472611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f165c105480 0x7f165c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.319+0000 7f16645eb700 1 --2- 172.21.15.19:0/4101472611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f165c105dd0 0x7f165c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.319+0000 7f16645eb700 1 -- 172.21.15.19:0/4101472611 >> 172.21.15.19:0/4101472611 conn(0x7f165c0fef30 msgr2=0x7f165c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.319+0000 7f16645eb700 1 -- 172.21.15.19:0/4101472611 shutdown_connections 2024-05-09T01:25:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.319+0000 7f16645eb700 1 -- 172.21.15.19:0/4101472611 wait complete. 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.320+0000 7f16645eb700 1 Processor -- start 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.320+0000 7f16645eb700 1 -- start start 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.320+0000 7f16645eb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f165c105480 0x7f165c103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f16645eb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c105dd0 0x7f165c1036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f16645eb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f165c10e8f0 0x7f165c103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f16645eb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f165c114480 con 0x7f165c105dd0 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f16645eb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f165c114300 con 0x7f165c105480 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f16645eb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f165c114600 con 0x7f165c10e8f0 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1661b86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c105dd0 0x7f165c1036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1661b86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c105dd0 0x7f165c1036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40368/0 (socket says 172.21.15.19:40368) 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1661b86700 1 -- 172.21.15.19:0/2476215355 learned_addr learned my addr 172.21.15.19:0/2476215355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1662387700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f165c105480 0x7f165c103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1662b88700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f165c10e8f0 0x7f165c103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1661b86700 1 -- 172.21.15.19:0/2476215355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f165c10e8f0 msgr2=0x7f165c103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1661b86700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f165c10e8f0 0x7f165c103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1661b86700 1 -- 172.21.15.19:0/2476215355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f165c105480 msgr2=0x7f165c103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1661b86700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f165c105480 0x7f165c103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1661b86700 1 -- 172.21.15.19:0/2476215355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c1a9170 con 0x7f165c105dd0 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1662b88700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f165c10e8f0 0x7f165c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f1661b86700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c105dd0 0x7f165c1036b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f165000e9c0 tx=0x7f165000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.321+0000 7f164f7fe700 1 -- 172.21.15.19:0/2476215355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165000cbd0 con 0x7f165c105dd0 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.322+0000 7f164f7fe700 1 -- 172.21.15.19:0/2476215355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f165000cd70 con 0x7f165c105dd0 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.322+0000 7f16645eb700 1 -- 172.21.15.19:0/2476215355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165c1a9460 con 0x7f165c105dd0 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.322+0000 7f164f7fe700 1 -- 172.21.15.19:0/2476215355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650005610 con 0x7f165c105dd0 2024-05-09T01:25:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.322+0000 7f16645eb700 1 -- 172.21.15.19:0/2476215355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f165c1a9950 con 0x7f165c105dd0 2024-05-09T01:25:08.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.324+0000 7f164f7fe700 1 -- 172.21.15.19:0/2476215355 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1650004750 con 0x7f165c105dd0 2024-05-09T01:25:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.325+0000 7f164f7fe700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16480b2880 0x7f16480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.325+0000 7f1662387700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16480b2880 0x7f16480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.325+0000 7f164f7fe700 1 -- 172.21.15.19:0/2476215355 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f16500e24e0 con 0x7f165c105dd0 2024-05-09T01:25:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.325+0000 7f1673001b80 1 Processor -- start 2024-05-09T01:25:08.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1673001b80 1 -- start start 2024-05-09T01:25:08.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662387700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16480b2880 0x7f16480b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1658009c60 tx=0x7f1658009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:08.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1673001b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x5650947154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1673001b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 0x5650947159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1673001b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 0x56509470fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1673001b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565094631fa0 con 0x56509467c110 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1673001b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650945d07a0 con 0x565094592450 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1673001b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56509462df60 con 0x56509470f650 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662387700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x5650947154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662387700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x5650947154a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40384/0 (socket says 172.21.15.19:40384) 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662387700 1 -- 172.21.15.19:0/2598263066 learned_addr learned my addr 172.21.15.19:0/2598263066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662b88700 1 --2- 172.21.15.19:0/2598263066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 0x56509470fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1661b86700 1 --2- 172.21.15.19:0/2598263066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 0x5650947159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662387700 1 -- 172.21.15.19:0/2598263066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 msgr2=0x56509470fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662387700 1 --2- 172.21.15.19:0/2598263066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 0x56509470fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662387700 1 -- 172.21.15.19:0/2598263066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 msgr2=0x5650947159e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662387700 1 --2- 172.21.15.19:0/2598263066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 0x5650947159e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.326+0000 7f1662387700 1 -- 172.21.15.19:0/2598263066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650945c8320 con 0x56509467c110 2024-05-09T01:25:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.327+0000 7f1662387700 1 --2- 172.21.15.19:0/2598263066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x5650947154a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f165800e4e0 tx=0x7f1658014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d34c62e12ac8b8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.327+0000 7f164cff9700 1 -- 172.21.15.19:0/2598263066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165800ee00 con 0x56509467c110 2024-05-09T01:25:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.327+0000 7f164cff9700 1 -- 172.21.15.19:0/2598263066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f165801c8c0 con 0x56509467c110 2024-05-09T01:25:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.327+0000 7f164cff9700 1 -- 172.21.15.19:0/2598263066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165801da20 con 0x56509467c110 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 -- 172.21.15.19:0/2598263066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 msgr2=0x5650947154a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 --2- 172.21.15.19:0/2598263066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x5650947154a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f165800e4e0 tx=0x7f1658014000 comp rx=0 tx=0).stop 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 -- 172.21.15.19:0/2598263066 shutdown_connections 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 --2- 172.21.15.19:0/2598263066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 0x56509470fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 --2- 172.21.15.19:0/2598263066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x5650947154a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 --2- 172.21.15.19:0/2598263066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 0x5650947159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 -- 172.21.15.19:0/2598263066 >> 172.21.15.19:0/2598263066 conn(0x5650945f2650 msgr2=0x56509470a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 -- 172.21.15.19:0/2598263066 shutdown_connections 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 -- 172.21.15.19:0/2598263066 wait complete. 2024-05-09T01:25:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 -- start start 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.328+0000 7f1673001b80 1 Processor -- start 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1673001b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565094592450 0x5650947191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1673001b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509467c110 0x5650947196f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1673001b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509470f650 0x565094719c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1673001b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5650944adb50 con 0x56509470f650 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1673001b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650945ec100 con 0x56509467c110 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1673001b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56509471c180 con 0x565094592450 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1673001b80 10 client.0 _ll_register_callbacks cb 0x5650944a3670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1662b88700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509470f650 0x565094719c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1662b88700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509470f650 0x565094719c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40400/0 (socket says 172.21.15.19:40400) 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1662b88700 1 -- 172.21.15.19:0/2335479322 learned_addr learned my addr 172.21.15.19:0/2335479322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1662387700 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565094592450 0x5650947191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1661b86700 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509467c110 0x5650947196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1662b88700 1 -- 172.21.15.19:0/2335479322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565094592450 msgr2=0x5650947191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1662b88700 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565094592450 0x5650947191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1662b88700 1 -- 172.21.15.19:0/2335479322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509467c110 msgr2=0x5650947196f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1662b88700 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509467c110 0x5650947196f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.329+0000 7f1662b88700 1 -- 172.21.15.19:0/2335479322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650944ad440 con 0x56509470f650 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1662b88700 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509470f650 0x565094719c30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1654015000 tx=0x7f165400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773c18a6b7fe57d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1645ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1645ffb700 1 -- 172.21.15.19:0/2335479322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654003e10 con 0x56509470f650 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1645ffb700 1 -- 172.21.15.19:0/2335479322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f16540075c0 con 0x56509470f650 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1673001b80 1 -- 172.21.15.19:0/2335479322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f165801c8c0 con 0x56509470f650 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1645ffb700 1 -- 172.21.15.19:0/2335479322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16540077c0 con 0x56509470f650 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1673001b80 10 client.15066 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1673001b80 1 -- 172.21.15.19:0/2335479322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565094726b90 con 0x56509470f650 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1645ffb700 1 -- 172.21.15.19:0/2335479322 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f165801c8c0 con 0x56509470f650 2024-05-09T01:25:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1645ffb700 1 -- 172.21.15.19:0/2335479322 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1654023070 con 0x56509470f650 2024-05-09T01:25:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.330+0000 7f1673001b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.331+0000 7f1673001b80 10 client.15066 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T01:25:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.331+0000 7f1673001b80 20 client.15066 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.331+0000 7f1673001b80 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5650947291c0 0x56509472a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.331+0000 7f1673001b80 4 client.15066 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:25:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.331+0000 7f1673001b80 1 -- 172.21.15.19:0/2335479322 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56509463f980 con 0x5650947291c0 2024-05-09T01:25:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.331+0000 7f1662387700 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5650947291c0 0x56509472a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.331+0000 7f1645ffb700 10 client.15066 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:25:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.331+0000 7f1662387700 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5650947291c0 0x56509472a630 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d9b6b2a56f90e4 server_cookie=d0cc20facfe56038 in_seq=0 out_seq=0 2024-05-09T01:25:08.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1645ffb700 1 -- 172.21.15.19:0/2335479322 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56509463f980 con 0x5650947291c0 2024-05-09T01:25:08.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1645ffb700 10 client.15066 handle_command_reply: tid=0 2024-05-09T01:25:08.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1673001b80 1 client.15066 shutdown 2024-05-09T01:25:08.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1673001b80 1 -- 172.21.15.19:0/2335479322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509470f650 msgr2=0x565094719c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1673001b80 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509470f650 0x565094719c30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1654015000 tx=0x7f165400bce0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1673001b80 1 -- 172.21.15.19:0/2335479322 shutdown_connections 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1673001b80 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5650947291c0 0x56509472a630 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1673001b80 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509470f650 0x565094719c30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1673001b80 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565094592450 0x5650947191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1673001b80 1 --2- 172.21.15.19:0/2335479322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509467c110 0x5650947196f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1673001b80 1 -- 172.21.15.19:0/2335479322 >> 172.21.15.19:0/2335479322 conn(0x5650945f2650 msgr2=0x56509470a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.332+0000 7f1645ffb700 0 client.15066 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.333+0000 7f1673001b80 1 -- 172.21.15.19:0/2335479322 shutdown_connections 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.333+0000 7f1673001b80 1 -- 172.21.15.19:0/2335479322 wait complete. 2024-05-09T01:25:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.333+0000 7f1673001b80 20 client.15066 trim_cache size 0 max 16384 2024-05-09T01:25:08.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1673001b80 1 Processor -- start 2024-05-09T01:25:08.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1673001b80 1 -- start start 2024-05-09T01:25:08.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1673001b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565094592450 0x5650947137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1673001b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509467c110 0x565094711db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1673001b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509470f650 0x565094718710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1673001b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565094728970 con 0x565094592450 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1673001b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x565094729040 con 0x56509470f650 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1673001b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565094729710 con 0x56509467c110 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1662387700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565094592450 0x5650947137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1662387700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565094592450 0x5650947137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40410/0 (socket says 172.21.15.19:40410) 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.354+0000 7f1662387700 1 -- 172.21.15.19:0/2246301878 learned_addr learned my addr 172.21.15.19:0/2246301878 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1662b88700 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509470f650 0x565094718710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1661b86700 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509467c110 0x565094711db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1662387700 1 -- 172.21.15.19:0/2246301878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509467c110 msgr2=0x565094711db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1662387700 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509467c110 0x565094711db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1662387700 1 -- 172.21.15.19:0/2246301878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509470f650 msgr2=0x565094718710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1662387700 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509470f650 0x565094718710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1662387700 1 -- 172.21.15.19:0/2246301878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565094715000 con 0x565094592450 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1661b86700 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509467c110 0x565094711db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1662b88700 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509470f650 0x565094718710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1662387700 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565094592450 0x5650947137d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1658009ec0 tx=0x7f1658011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05efa9d95b5cc21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1647fff700 1 -- 172.21.15.19:0/2246301878 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165800ec00 con 0x565094592450 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1647fff700 1 -- 172.21.15.19:0/2246301878 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16580054f0 con 0x565094592450 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.355+0000 7f1647fff700 1 -- 172.21.15.19:0/2246301878 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165801d540 con 0x565094592450 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 -- 172.21.15.19:0/2246301878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565094592450 msgr2=0x5650947137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565094592450 0x5650947137d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1658009ec0 tx=0x7f1658011b60 comp rx=0 tx=0).stop 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 -- 172.21.15.19:0/2246301878 shutdown_connections 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56509470f650 0x565094718710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565094592450 0x5650947137d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 --2- 172.21.15.19:0/2246301878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509467c110 0x565094711db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 -- 172.21.15.19:0/2246301878 >> 172.21.15.19:0/2246301878 conn(0x5650945f2650 msgr2=0x56509470a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 -- 172.21.15.19:0/2246301878 shutdown_connections 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 -- 172.21.15.19:0/2246301878 wait complete. 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.356+0000 7f1673001b80 1 -- start start 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1673001b80 1 Processor -- start 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1673001b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 0x565094733fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1673001b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x56509472fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1673001b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 0x56509472ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1673001b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56509472a4b0 con 0x56509467c110 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1673001b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x565094729de0 con 0x565094592450 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1673001b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56509472ab80 con 0x56509470f650 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1673001b80 10 client.0 _ll_register_callbacks cb 0x5650944a3670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1661b86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x56509472fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1661b86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x56509472fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40418/0 (socket says 172.21.15.19:40418) 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1661b86700 1 -- 172.21.15.19:0/3688185052 learned_addr learned my addr 172.21.15.19:0/3688185052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1662b88700 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 0x56509472ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1662387700 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 0x565094733fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1661b86700 1 -- 172.21.15.19:0/3688185052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 msgr2=0x56509472ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1661b86700 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 0x56509472ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1661b86700 1 -- 172.21.15.19:0/3688185052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 msgr2=0x565094733fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1661b86700 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 0x565094733fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1661b86700 1 -- 172.21.15.19:0/3688185052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565094730730 con 0x56509467c110 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1662b88700 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 0x56509472ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1661b86700 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x56509472fa70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16500a97e0 tx=0x7f16500aa070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ed94c791cf746a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.357+0000 7f1645ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1645ffb700 1 -- 172.21.15.19:0/3688185052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165006e060 con 0x56509467c110 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1645ffb700 1 -- 172.21.15.19:0/3688185052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f16500b26c0 con 0x56509467c110 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1673001b80 1 -- 172.21.15.19:0/3688185052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16580054f0 con 0x56509467c110 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1645ffb700 1 -- 172.21.15.19:0/3688185052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16500c2070 con 0x56509467c110 2024-05-09T01:25:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1673001b80 10 client.15072 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1673001b80 1 -- 172.21.15.19:0/3688185052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565094724d20 con 0x56509467c110 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1645ffb700 1 -- 172.21.15.19:0/3688185052 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16580054f0 con 0x56509467c110 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1645ffb700 1 -- 172.21.15.19:0/3688185052 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16500a8d40 con 0x56509467c110 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1673001b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1673001b80 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1673001b80 20 client.15072 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1673001b80 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5650947968c0 0x565094797d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1673001b80 4 client.15072 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.358+0000 7f1673001b80 1 -- 172.21.15.19:0/3688185052 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "status"}) -- 0x565094639a20 con 0x5650947968c0 2024-05-09T01:25:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1662387700 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5650947968c0 0x565094797d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1645ffb700 10 client.15072 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1662387700 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5650947968c0 0x565094797d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9dc6d4405a46f3 server_cookie=17f8af8bafb7bcb6 in_seq=0 out_seq=0 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1645ffb700 1 -- 172.21.15.19:0/3688185052 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x565094639a20 con 0x5650947968c0 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1645ffb700 10 client.15072 handle_command_reply: tid=0 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1673001b80 1 client.15072 shutdown 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1673001b80 1 -- 172.21.15.19:0/3688185052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 msgr2=0x56509472fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1673001b80 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x56509472fa70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16500a97e0 tx=0x7f16500aa070 comp rx=0 tx=0).stop 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1673001b80 1 -- 172.21.15.19:0/3688185052 shutdown_connections 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1673001b80 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5650947968c0 0x565094797d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.359+0000 7f1673001b80 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56509470f650 0x56509472ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f1673001b80 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565094592450 0x565094733fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f1673001b80 1 --2- 172.21.15.19:0/3688185052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56509467c110 0x56509472fa70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f1673001b80 1 -- 172.21.15.19:0/3688185052 >> 172.21.15.19:0/3688185052 conn(0x5650945f2650 msgr2=0x565094718710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:08.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f1645ffb700 0 client.15072 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:25:08.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f1673001b80 1 -- 172.21.15.19:0/3688185052 shutdown_connections 2024-05-09T01:25:08.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f1673001b80 1 -- 172.21.15.19:0/3688185052 wait complete. 2024-05-09T01:25:08.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f1673001b80 20 client.15072 trim_cache size 0 max 16384 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24497, 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 146.535288909, 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 161.579689221 2024-05-09T01:25:08.365 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:25:08.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 -- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16480b2880 msgr2=0x7f16480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:08.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16480b2880 0x7f16480b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1658009c60 tx=0x7f1658009260 comp rx=0 tx=0).stop 2024-05-09T01:25:08.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 -- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c105dd0 msgr2=0x7f165c1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c105dd0 0x7f165c1036b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f165000e9c0 tx=0x7f165000ecd0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 -- 172.21.15.19:0/2476215355 shutdown_connections 2024-05-09T01:25:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f165c10e8f0 0x7f165c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f165c105480 0x7f165c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16480b2880 0x7f16480b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 --2- 172.21.15.19:0/2476215355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f165c105dd0 0x7f165c1036b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.360+0000 7f16645eb700 1 -- 172.21.15.19:0/2476215355 >> 172.21.15.19:0/2476215355 conn(0x7f165c0fef30 msgr2=0x7f165c0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.361+0000 7f16645eb700 1 -- 172.21.15.19:0/2476215355 shutdown_connections 2024-05-09T01:25:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:08.361+0000 7f16645eb700 1 -- 172.21.15.19:0/2476215355 wait complete. 2024-05-09T01:25:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:08 smithi019 ceph-mon[30158]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:08 smithi196 ceph-mon[35041]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:08.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:08 smithi059 ceph-mon[34351]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:09.115 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T01:25:09.116 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T01:25:09.126 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T01:25:09.126 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T01:25:09.380 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:10.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:10 smithi019 ceph-mon[30158]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:10.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:10 smithi196 ceph-mon[35041]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:10.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:10 smithi059 ceph-mon[34351]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.921+0000 7f73a60bd700 1 -- 172.21.15.19:0/1061633033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 msgr2=0x7f73a010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.921+0000 7f73a60bd700 1 --2- 172.21.15.19:0/1061633033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 0x7f73a010b5d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f739800b0a0 tx=0x7f739800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:25:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.921+0000 7f73a60bd700 1 -- 172.21.15.19:0/1061633033 shutdown_connections 2024-05-09T01:25:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.921+0000 7f73a60bd700 1 --2- 172.21.15.19:0/1061633033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 0x7f73a010b5d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.921+0000 7f73a60bd700 1 --2- 172.21.15.19:0/1061633033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a0103250 0x7f73a0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.921+0000 7f73a60bd700 1 --2- 172.21.15.19:0/1061633033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a0104380 0x7f73a0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.921+0000 7f73a60bd700 1 -- 172.21.15.19:0/1061633033 >> 172.21.15.19:0/1061633033 conn(0x7f73a00fe890 msgr2=0x7f73a0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.921+0000 7f73a60bd700 1 -- 172.21.15.19:0/1061633033 shutdown_connections 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.921+0000 7f73a60bd700 1 -- 172.21.15.19:0/1061633033 wait complete. 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f73a60bd700 1 Processor -- start 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f73a60bd700 1 -- start start 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f73a60bd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a0103250 0x7f73a019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f73a60bd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a0104380 0x7f73a019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f73a60bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 0x7f73a01a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f73a60bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73a010ecb0 con 0x7f73a0104d20 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f73a60bd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73a010eb30 con 0x7f73a0103250 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f73a60bd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73a010ee30 con 0x7f73a0104380 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f739ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 0x7f73a01a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f739ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 0x7f73a01a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42054/0 (socket says 172.21.15.19:42054) 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f739ffff700 1 -- 172.21.15.19:0/948424770 learned_addr learned my addr 172.21.15.19:0/948424770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.922+0000 7f739f7fe700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a0103250 0x7f73a019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f7396ffd700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a0104380 0x7f73a019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f739ffff700 1 -- 172.21.15.19:0/948424770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a0104380 msgr2=0x7f73a019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f739ffff700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a0104380 0x7f73a019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f739ffff700 1 -- 172.21.15.19:0/948424770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a0103250 msgr2=0x7f73a019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f739ffff700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a0103250 0x7f73a019a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f739ffff700 1 -- 172.21.15.19:0/948424770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a01a23e0 con 0x7f73a0104d20 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f739ffff700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 0x7f73a01a1d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f73980062a0 tx=0x7f7398004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f739d7fa700 1 -- 172.21.15.19:0/948424770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398021070 con 0x7f73a0104d20 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a01a2670 con 0x7f73a0104d20 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f739d7fa700 1 -- 172.21.15.19:0/948424770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7398007b80 con 0x7f73a0104d20 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f739d7fa700 1 -- 172.21.15.19:0/948424770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398004670 con 0x7f73a0104d20 2024-05-09T01:25:10.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.923+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a01a2c00 con 0x7f73a0104d20 2024-05-09T01:25:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.924+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a00512c0 con 0x7f73a0104d20 2024-05-09T01:25:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.926+0000 7f739d7fa700 1 -- 172.21.15.19:0/948424770 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7398003a00 con 0x7f73a0104d20 2024-05-09T01:25:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.927+0000 7f739d7fa700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f738c0b2880 0x7f738c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.927+0000 7f739f7fe700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f738c0b2880 0x7f738c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.927+0000 7f739d7fa700 1 -- 172.21.15.19:0/948424770 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f739800f020 con 0x7f73a0104d20 2024-05-09T01:25:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.927+0000 7f739f7fe700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f738c0b2880 0x7f738c0b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73a0104280 tx=0x7f739000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:10.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:10.933+0000 7f739d7fa700 1 -- 172.21.15.19:0/948424770 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f739802a020 con 0x7f73a0104d20 2024-05-09T01:25:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.165+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f73a0100890 con 0x7f738c0b2880 2024-05-09T01:25:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.166+0000 7f739d7fa700 1 -- 172.21.15.19:0/948424770 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f73a0100890 con 0x7f738c0b2880 2024-05-09T01:25:11.164 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-05-09T01:25:11.164 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-05-09T01:25:11.164 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-05-09T01:25:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f738c0b2880 msgr2=0x7f738c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f738c0b2880 0x7f738c0b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73a0104280 tx=0x7f739000a380 comp rx=0 tx=0).stop 2024-05-09T01:25:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 msgr2=0x7f73a01a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 0x7f73a01a1d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f73980062a0 tx=0x7f7398004e40 comp rx=0 tx=0).stop 2024-05-09T01:25:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 shutdown_connections 2024-05-09T01:25:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f738c0b2880 0x7f738c0b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a0104d20 0x7f73a01a1d60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a0103250 0x7f73a019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 --2- 172.21.15.19:0/948424770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a0104380 0x7f73a019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 >> 172.21.15.19:0/948424770 conn(0x7f73a00fe890 msgr2=0x7f73a0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 shutdown_connections 2024-05-09T01:25:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:11.169+0000 7f73a60bd700 1 -- 172.21.15.19:0/948424770 wait complete. 2024-05-09T01:25:11.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T01:25:12.054 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:12 smithi196 ceph-mon[35041]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:12.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:12 smithi196 ceph-mon[35041]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:12 smithi059 ceph-mon[34351]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:12.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:12 smithi059 ceph-mon[34351]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:12.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:12 smithi019 ceph-mon[30158]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:12.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:12 smithi019 ceph-mon[30158]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.701+0000 7f9ff92c1700 1 -- 172.21.15.19:0/611141818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 msgr2=0x7f9ff4105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.701+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/611141818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 0x7f9ff4105dc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009990 tx=0x7f9fe4009ca0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.701+0000 7f9ff92c1700 1 -- 172.21.15.19:0/611141818 shutdown_connections 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.701+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/611141818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ff41049e0 0x7f9ff410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.701+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/611141818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 0x7f9ff4105dc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.701+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/611141818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff4103f60 0x7f9ff4104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.701+0000 7f9ff92c1700 1 -- 172.21.15.19:0/611141818 >> 172.21.15.19:0/611141818 conn(0x7f9ff40ff9d0 msgr2=0x7f9ff4101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.701+0000 7f9ff92c1700 1 -- 172.21.15.19:0/611141818 shutdown_connections 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.701+0000 7f9ff92c1700 1 -- 172.21.15.19:0/611141818 wait complete. 2024-05-09T01:25:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.702+0000 7f9ff92c1700 1 Processor -- start 2024-05-09T01:25:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.702+0000 7f9ff92c1700 1 -- start start 2024-05-09T01:25:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.702+0000 7f9ff92c1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ff4103f60 0x7f9ff419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.702+0000 7f9ff92c1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff41049e0 0x7f9ff419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff92c1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 0x7f9ff41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff92c1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff410fed0 con 0x7f9ff41059b0 2024-05-09T01:25:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff92c1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff410fd50 con 0x7f9ff4103f60 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff92c1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff4110050 con 0x7f9ff41049e0 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 0x7f9ff41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ff4103f60 0x7f9ff419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 0x7f9ff41a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42070/0 (socket says 172.21.15.19:42070) 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff27fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff41049e0 0x7f9ff419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff37fe700 1 -- 172.21.15.19:0/3786263696 learned_addr learned my addr 172.21.15.19:0/3786263696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ff4103f60 0x7f9ff419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:48994/0 (socket says 172.21.15.19:48994) 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff37fe700 1 -- 172.21.15.19:0/3786263696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff41049e0 msgr2=0x7f9ff419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff37fe700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff41049e0 0x7f9ff419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff37fe700 1 -- 172.21.15.19:0/3786263696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ff4103f60 msgr2=0x7f9ff419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff37fe700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ff4103f60 0x7f9ff419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff37fe700 1 -- 172.21.15.19:0/3786263696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff41a2580 con 0x7f9ff41059b0 2024-05-09T01:25:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff2ffd700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ff4103f60 0x7f9ff419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:25:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.703+0000 7f9ff37fe700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 0x7f9ff41a1f00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00ef30 tx=0x7f9fec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.704+0000 7f9febfff700 1 -- 172.21.15.19:0/3786263696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0092e0 con 0x7f9ff41059b0 2024-05-09T01:25:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.704+0000 7f9febfff700 1 -- 172.21.15.19:0/3786263696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fec004bb0 con 0x7f9ff41059b0 2024-05-09T01:25:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.704+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff41a2810 con 0x7f9ff41059b0 2024-05-09T01:25:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.704+0000 7f9febfff700 1 -- 172.21.15.19:0/3786263696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec004020 con 0x7f9ff41059b0 2024-05-09T01:25:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.704+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff41a2d00 con 0x7f9ff41059b0 2024-05-09T01:25:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.705+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff40512c0 con 0x7f9ff41059b0 2024-05-09T01:25:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.707+0000 7f9febfff700 1 -- 172.21.15.19:0/3786263696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9fec01b030 con 0x7f9ff41059b0 2024-05-09T01:25:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.707+0000 7f9febfff700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fe00b2880 0x7f9fe00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.707+0000 7f9febfff700 1 -- 172.21.15.19:0/3786263696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9fec0e1d40 con 0x7f9ff41059b0 2024-05-09T01:25:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.708+0000 7f9ff2ffd700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fe00b2880 0x7f9fe00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:13.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.708+0000 7f9ff2ffd700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fe00b2880 0x7f9fe00b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400b6c0 tx=0x7f9fe400b720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:13.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.713+0000 7f9febfff700 1 -- 172.21.15.19:0/3786263696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9fec022020 con 0x7f9ff41059b0 2024-05-09T01:25:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.942+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9ff402e9d0 con 0x7f9fe00b2880 2024-05-09T01:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.956+0000 7f9febfff700 1 -- 172.21.15.19:0/3786263696 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f9ff402e9d0 con 0x7f9fe00b2880 2024-05-09T01:25:13.953 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T01:25:13.953 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (2m) 28s ago 2m 86.9M - 19.0.0-3594-gc9da4049 db794d706f49 60e1d1c7825b 2024-05-09T01:25:13.953 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi059 running (2m) 29s ago 2m 94.9M - 19.0.0-3594-gc9da4049 db794d706f49 b3d7d32a0ba7 2024-05-09T01:25:13.953 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi196 running (3m) 29s ago 3m 87.3M - 19.0.0-3594-gc9da4049 db794d706f49 e5a570786a3e 2024-05-09T01:25:13.953 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (3m) 28s ago 3m 83.8M - 19.0.0-3594-gc9da4049 db794d706f49 5805afed24c7 2024-05-09T01:25:13.953 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi059 running (3m) 29s ago 3m 87.3M - 19.0.0-3594-gc9da4049 db794d706f49 0d40aa63be46 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi196 running (3m) 29s ago 3m 88.2M - 19.0.0-3594-gc9da4049 db794d706f49 7a5bab63e85e 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (2m) 28s ago 2m 86.4M - 19.0.0-3594-gc9da4049 db794d706f49 6750fe505658 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi059 running (3m) 29s ago 3m 87.6M - 19.0.0-3594-gc9da4049 db794d706f49 1e7f4e161b91 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi196 running (2m) 29s ago 2m 86.9M - 19.0.0-3594-gc9da4049 db794d706f49 ae57a6613523 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (3m) 28s ago 3m 81.4M - 19.0.0-3594-gc9da4049 db794d706f49 ed0d63ac1776 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi059 running (3m) 29s ago 3m 86.2M - 19.0.0-3594-gc9da4049 db794d706f49 5cd8671a4aa2 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi196 running (3m) 29s ago 3m 89.7M - 19.0.0-3594-gc9da4049 db794d706f49 e2168d199905 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (10m) 28s ago 10m 598M - 19.0.0-3594-gc9da4049 db794d706f49 d07233fe87ec 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi059 *:8443,8765 running (8m) 29s ago 8m 537M - 19.0.0-3594-gc9da4049 db794d706f49 ccad773eb2af 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi196 *:8443,8765 running (8m) 29s ago 8m 534M - 19.0.0-3594-gc9da4049 db794d706f49 9b182b9c4880 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (10m) 28s ago 10m 120M 2048M 19.0.0-3594-gc9da4049 db794d706f49 308e13fd3f8c 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi059 running (8m) 29s ago 8m 109M 2048M 19.0.0-3594-gc9da4049 db794d706f49 9a88f306158a 2024-05-09T01:25:13.954 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi196 running (9m) 29s ago 9m 110M 2048M 19.0.0-3594-gc9da4049 db794d706f49 98589ef238fd 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (8m) 28s ago 8m 125M 4273M 19.0.0-3594-gc9da4049 db794d706f49 b28402698792 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi059 running (7m) 29s ago 7m 121M 4273M 19.0.0-3594-gc9da4049 db794d706f49 c96899f4b541 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi196 running (7m) 29s ago 7m 121M 4272M 19.0.0-3594-gc9da4049 db794d706f49 4841ccf6b924 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (7m) 28s ago 7m 125M 4273M 19.0.0-3594-gc9da4049 db794d706f49 a701beef54eb 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi059 running (6m) 29s ago 6m 122M 4273M 19.0.0-3594-gc9da4049 db794d706f49 f4aa789beefa 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi196 running (6m) 29s ago 6m 123M 4272M 19.0.0-3594-gc9da4049 db794d706f49 262bb24ed4f1 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (5m) 28s ago 5m 120M 4273M 19.0.0-3594-gc9da4049 db794d706f49 2039531a3d37 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi059 running (5m) 29s ago 5m 115M 4273M 19.0.0-3594-gc9da4049 db794d706f49 7f926f41f4df 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi196 running (5m) 29s ago 5m 123M 4272M 19.0.0-3594-gc9da4049 db794d706f49 35c16b4ebe35 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (4m) 28s ago 4m 114M 4273M 19.0.0-3594-gc9da4049 db794d706f49 004b242765f9 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi059 running (4m) 29s ago 4m 116M 4273M 19.0.0-3594-gc9da4049 db794d706f49 fc8a1ca79e13 2024-05-09T01:25:13.955 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi196 running (3m) 29s ago 3m 121M 4272M 19.0.0-3594-gc9da4049 db794d706f49 0efac5a8c5f4 2024-05-09T01:25:13.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fe00b2880 msgr2=0x7f9fe00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:13.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fe00b2880 0x7f9fe00b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400b6c0 tx=0x7f9fe400b720 comp rx=0 tx=0).stop 2024-05-09T01:25:13.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 msgr2=0x7f9ff41a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:13.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 0x7f9ff41a1f00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00ef30 tx=0x7f9fec00c3a0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 shutdown_connections 2024-05-09T01:25:13.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fe00b2880 0x7f9fe00b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ff41059b0 0x7f9ff41a1f00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ff4103f60 0x7f9ff419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 --2- 172.21.15.19:0/3786263696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff41049e0 0x7f9ff419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:13.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 >> 172.21.15.19:0/3786263696 conn(0x7f9ff40ff9d0 msgr2=0x7f9ff41086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:13.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 shutdown_connections 2024-05-09T01:25:13.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:13.959+0000 7f9ff92c1700 1 -- 172.21.15.19:0/3786263696 wait complete. 2024-05-09T01:25:14.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:14 smithi019 ceph-mon[30158]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:14.570 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T01:25:14.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:14 smithi196 ceph-mon[35041]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:14.823 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:14 smithi059 ceph-mon[34351]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:15.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:15 smithi019 ceph-mon[30158]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:15.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:15 smithi196 ceph-mon[35041]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:15.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:15 smithi059 ceph-mon[34351]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:16.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.372+0000 7f0376228700 1 -- 172.21.15.19:0/457512444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 msgr2=0x7f0370109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:16.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.372+0000 7f0376228700 1 --2- 172.21.15.19:0/457512444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 0x7f0370109bd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0358009990 tx=0x7f0358009ca0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.372+0000 7f0376228700 1 -- 172.21.15.19:0/457512444 shutdown_connections 2024-05-09T01:25:16.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.372+0000 7f0376228700 1 --2- 172.21.15.19:0/457512444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f037010a110 0x7f037010c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.372+0000 7f0376228700 1 --2- 172.21.15.19:0/457512444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370104e50 0x7f0370107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.372+0000 7f0376228700 1 --2- 172.21.15.19:0/457512444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 0x7f0370109bd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.372+0000 7f0376228700 1 -- 172.21.15.19:0/457512444 >> 172.21.15.19:0/457512444 conn(0x7f03700fe890 msgr2=0x7f0370100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:16.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.372+0000 7f0376228700 1 -- 172.21.15.19:0/457512444 shutdown_connections 2024-05-09T01:25:16.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.372+0000 7f0376228700 1 -- 172.21.15.19:0/457512444 wait complete. 2024-05-09T01:25:16.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.373+0000 7f0376228700 1 Processor -- start 2024-05-09T01:25:16.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.373+0000 7f0376228700 1 -- start start 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.373+0000 7f0376228700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370104e50 0x7f037019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.373+0000 7f0376228700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 0x7f037019af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.373+0000 7f0376228700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f037010a110 0x7f03701a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.373+0000 7f0376228700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f037010fea0 con 0x7f03701077b0 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.373+0000 7f0376228700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f037010fd20 con 0x7f0370104e50 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.373+0000 7f0376228700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0370110020 con 0x7f037010a110 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0366ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 0x7f037019af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0366ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 0x7f037019af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42080/0 (socket says 172.21.15.19:42080) 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0366ffd700 1 -- 172.21.15.19:0/2888791907 learned_addr learned my addr 172.21.15.19:0/2888791907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f036ffff700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f037010a110 0x7f03701a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f036f7fe700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370104e50 0x7f037019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0366ffd700 1 -- 172.21.15.19:0/2888791907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f037010a110 msgr2=0x7f03701a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0366ffd700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f037010a110 0x7f03701a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0366ffd700 1 -- 172.21.15.19:0/2888791907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370104e50 msgr2=0x7f037019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:16.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0366ffd700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370104e50 0x7f037019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0366ffd700 1 -- 172.21.15.19:0/2888791907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03701a25a0 con 0x7f03701077b0 2024-05-09T01:25:16.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0366ffd700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 0x7f037019af80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0358003800 tx=0x7f0358009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:16.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f036d7fa700 1 -- 172.21.15.19:0/2888791907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358027070 con 0x7f03701077b0 2024-05-09T01:25:16.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f036d7fa700 1 -- 172.21.15.19:0/2888791907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0358022470 con 0x7f03701077b0 2024-05-09T01:25:16.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03701a2830 con 0x7f03701077b0 2024-05-09T01:25:16.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f036d7fa700 1 -- 172.21.15.19:0/2888791907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035800f6e0 con 0x7f03701077b0 2024-05-09T01:25:16.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.374+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03701a2d00 con 0x7f03701077b0 2024-05-09T01:25:16.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.375+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03700512c0 con 0x7f03701077b0 2024-05-09T01:25:16.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.377+0000 7f036d7fa700 1 -- 172.21.15.19:0/2888791907 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f035802a070 con 0x7f03701077b0 2024-05-09T01:25:16.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.378+0000 7f036d7fa700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f035c0b2880 0x7f035c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:16.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.378+0000 7f036f7fe700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f035c0b2880 0x7f035c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:16.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.378+0000 7f036d7fa700 1 -- 172.21.15.19:0/2888791907 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03580e35f0 con 0x7f03701077b0 2024-05-09T01:25:16.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.378+0000 7f036f7fe700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f035c0b2880 0x7f035c0b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:16.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.381+0000 7f036d7fa700 1 -- 172.21.15.19:0/2888791907 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0358031080 con 0x7f03701077b0 2024-05-09T01:25:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.604+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f0370063cc0 con 0x7f035c0b2880 2024-05-09T01:25:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.611+0000 7f036d7fa700 1 -- 172.21.15.19:0/2888791907 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f0370063cc0 con 0x7f035c0b2880 2024-05-09T01:25:16.607 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T01:25:16.608 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 32s ago 3m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-05-09T01:25:16.608 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 32s ago 8m smithi019=x;smithi059=y;smithi196=z;count:3 2024-05-09T01:25:16.608 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 32s ago 9m smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c;count:3 2024-05-09T01:25:16.608 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 32s ago - 2024-05-09T01:25:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f035c0b2880 msgr2=0x7f035c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f035c0b2880 0x7f035c0b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).stop 2024-05-09T01:25:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 msgr2=0x7f037019af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 0x7f037019af80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0358003800 tx=0x7f0358009f90 comp rx=0 tx=0).stop 2024-05-09T01:25:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 shutdown_connections 2024-05-09T01:25:16.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f037010a110 0x7f03701a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370104e50 0x7f037019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f035c0b2880 0x7f035c0b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 --2- 172.21.15.19:0/2888791907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03701077b0 0x7f037019af80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:16.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 >> 172.21.15.19:0/2888791907 conn(0x7f03700fe890 msgr2=0x7f0370105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:16.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 shutdown_connections 2024-05-09T01:25:16.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:16.613+0000 7f0376228700 1 -- 172.21.15.19:0/2888791907 wait complete. 2024-05-09T01:25:16.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:16 smithi019 ceph-mon[30158]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:16 smithi196 ceph-mon[35041]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:16.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:16 smithi059 ceph-mon[34351]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:17.322 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T01:25:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:18.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:18 smithi019 ceph-mon[30158]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:18.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:18 smithi019 ceph-mon[30158]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:18 smithi059 ceph-mon[34351]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:18.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:18 smithi059 ceph-mon[34351]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:18.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:18 smithi196 ceph-mon[35041]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:18.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:18 smithi196 ceph-mon[35041]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.364+0000 7f04d4dcb700 1 -- 172.21.15.19:0/3997181307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 msgr2=0x7f04d010c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.364+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/3997181307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 0x7f04d010c6d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009990 tx=0x7f04c8009ca0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.364+0000 7f04d4dcb700 1 -- 172.21.15.19:0/3997181307 shutdown_connections 2024-05-09T01:25:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.364+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/3997181307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 0x7f04d010c6d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.364+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/3997181307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d0104350 0x7f04d0104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.364+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/3997181307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d0105480 0x7f04d0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.364+0000 7f04d4dcb700 1 -- 172.21.15.19:0/3997181307 >> 172.21.15.19:0/3997181307 conn(0x7f04d00ff990 msgr2=0x7f04d0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.364+0000 7f04d4dcb700 1 -- 172.21.15.19:0/3997181307 shutdown_connections 2024-05-09T01:25:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.364+0000 7f04d4dcb700 1 -- 172.21.15.19:0/3997181307 wait complete. 2024-05-09T01:25:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.365+0000 7f04d4dcb700 1 Processor -- start 2024-05-09T01:25:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.365+0000 7f04d4dcb700 1 -- start start 2024-05-09T01:25:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.365+0000 7f04d4dcb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d0104350 0x7f04d0077460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.365+0000 7f04d4dcb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d0105480 0x7f04d0075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04d4dcb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 0x7f04d0076050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04d4dcb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04d010fde0 con 0x7f04d0105e20 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04d4dcb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04d010fc60 con 0x7f04d0105480 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04d4dcb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04d010ff60 con 0x7f04d0104350 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 0x7f04d0076050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 0x7f04d0076050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42092/0 (socket says 172.21.15.19:42092) 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ced9d700 1 -- 172.21.15.19:0/2176730643 learned_addr learned my addr 172.21.15.19:0/2176730643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04cdd9b700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d0105480 0x7f04d0075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ce59c700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d0104350 0x7f04d0077460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ced9d700 1 -- 172.21.15.19:0/2176730643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d0104350 msgr2=0x7f04d0077460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ced9d700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d0104350 0x7f04d0077460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ced9d700 1 -- 172.21.15.19:0/2176730643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d0105480 msgr2=0x7f04d0075b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ced9d700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d0105480 0x7f04d0075b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ced9d700 1 -- 172.21.15.19:0/2176730643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d00767b0 con 0x7f04d0105e20 2024-05-09T01:25:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04ced9d700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 0x7f04d0076050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f04c80038a0 tx=0x7f04c8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04c77fe700 1 -- 172.21.15.19:0/2176730643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8027070 con 0x7f04d0105e20 2024-05-09T01:25:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d01a4970 con 0x7f04d0105e20 2024-05-09T01:25:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04c77fe700 1 -- 172.21.15.19:0/2176730643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c8004000 con 0x7f04d0105e20 2024-05-09T01:25:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.366+0000 7f04c77fe700 1 -- 172.21.15.19:0/2176730643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8022da0 con 0x7f04d0105e20 2024-05-09T01:25:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.367+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d01a4d60 con 0x7f04d0105e20 2024-05-09T01:25:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.367+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d00512c0 con 0x7f04d0105e20 2024-05-09T01:25:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.370+0000 7f04c77fe700 1 -- 172.21.15.19:0/2176730643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f04c802a070 con 0x7f04d0105e20 2024-05-09T01:25:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.371+0000 7f04c77fe700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04bc0b2880 0x7f04bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:19.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.371+0000 7f04ce59c700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04bc0b2880 0x7f04bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.371+0000 7f04c77fe700 1 -- 172.21.15.19:0/2176730643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f04c80e35d0 con 0x7f04d0105e20 2024-05-09T01:25:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.371+0000 7f04ce59c700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04bc0b2880 0x7f04bc0b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04d0105380 tx=0x7f04c0009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.375+0000 7f04c77fe700 1 -- 172.21.15.19:0/2176730643 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04c8031020 con 0x7f04d0105e20 2024-05-09T01:25:19.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.597+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f04d002e9f0 con 0x7f04bc0b2880 2024-05-09T01:25:19.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.599+0000 7f04c77fe700 1 -- 172.21.15.19:0/2176730643 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f04d002e9f0 con 0x7f04bc0b2880 2024-05-09T01:25:19.596 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-05-09T01:25:19.597 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-05-09T01:25:19.597 INFO:teuthology.orchestra.run.smithi019.stdout:smithi059 172.21.15.59 2024-05-09T01:25:19.597 INFO:teuthology.orchestra.run.smithi019.stdout:smithi196 172.21.15.196 2024-05-09T01:25:19.597 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-05-09T01:25:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04bc0b2880 msgr2=0x7f04bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04bc0b2880 0x7f04bc0b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04d0105380 tx=0x7f04c0009040 comp rx=0 tx=0).stop 2024-05-09T01:25:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 msgr2=0x7f04d0076050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 0x7f04d0076050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f04c80038a0 tx=0x7f04c8005b00 comp rx=0 tx=0).stop 2024-05-09T01:25:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 shutdown_connections 2024-05-09T01:25:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04bc0b2880 0x7f04bc0b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d0105e20 0x7f04d0076050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d0104350 0x7f04d0077460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 --2- 172.21.15.19:0/2176730643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d0105480 0x7f04d0075b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:19.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 >> 172.21.15.19:0/2176730643 conn(0x7f04d00ff990 msgr2=0x7f04d01086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:19.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 shutdown_connections 2024-05-09T01:25:19.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:19.601+0000 7f04d4dcb700 1 -- 172.21.15.19:0/2176730643 wait complete. 2024-05-09T01:25:20.384 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T01:25:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:20 smithi019 ceph-mon[30158]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:20.645 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:20.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:20 smithi059 ceph-mon[34351]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:20.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:20 smithi196 ceph-mon[35041]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:21.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:21 smithi019 ceph-mon[30158]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:21.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:21 smithi059 ceph-mon[34351]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:21.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:21 smithi196 ceph-mon[35041]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:22.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.330+0000 7f68ecdbb700 1 -- 172.21.15.19:0/388538118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 msgr2=0x7f68e8109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:22.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.330+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/388538118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 0x7f68e8109bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009990 tx=0x7f68dc009ca0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.330+0000 7f68ecdbb700 1 -- 172.21.15.19:0/388538118 shutdown_connections 2024-05-09T01:25:22.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.330+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/388538118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e810a110 0x7f68e810c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.330+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/388538118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e8104e50 0x7f68e8107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.330+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/388538118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 0x7f68e8109bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.330+0000 7f68ecdbb700 1 -- 172.21.15.19:0/388538118 >> 172.21.15.19:0/388538118 conn(0x7f68e80fe890 msgr2=0x7f68e8100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:22.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.330+0000 7f68ecdbb700 1 -- 172.21.15.19:0/388538118 shutdown_connections 2024-05-09T01:25:22.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.330+0000 7f68ecdbb700 1 -- 172.21.15.19:0/388538118 wait complete. 2024-05-09T01:25:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.331+0000 7f68ecdbb700 1 Processor -- start 2024-05-09T01:25:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.331+0000 7f68ecdbb700 1 -- start start 2024-05-09T01:25:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68ecdbb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e8104e50 0x7f68e819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68ecdbb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 0x7f68e819d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68ecdbb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e810a110 0x7f68e81a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68ecdbb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68e810fdc0 con 0x7f68e81077b0 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68ecdbb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f68e810fc40 con 0x7f68e8104e50 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68ecdbb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68e810ff40 con 0x7f68e810a110 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 0x7f68e819d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 0x7f68e819d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41164/0 (socket says 172.21.15.19:41164) 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e5d9b700 1 -- 172.21.15.19:0/29147055 learned_addr learned my addr 172.21.15.19:0/29147055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e659c700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e8104e50 0x7f68e819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e6d9d700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e810a110 0x7f68e81a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e5d9b700 1 -- 172.21.15.19:0/29147055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e810a110 msgr2=0x7f68e81a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e5d9b700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e810a110 0x7f68e81a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e5d9b700 1 -- 172.21.15.19:0/29147055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e8104e50 msgr2=0x7f68e819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e5d9b700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e8104e50 0x7f68e819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e5d9b700 1 -- 172.21.15.19:0/29147055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e81a4700 con 0x7f68e81077b0 2024-05-09T01:25:22.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.332+0000 7f68e5d9b700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 0x7f68e819d0e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0098e0 tx=0x7f68dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.333+0000 7f68d77fe700 1 -- 172.21.15.19:0/29147055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc026070 con 0x7f68e81077b0 2024-05-09T01:25:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.333+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e81a4990 con 0x7f68e81077b0 2024-05-09T01:25:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.333+0000 7f68d77fe700 1 -- 172.21.15.19:0/29147055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68dc021410 con 0x7f68e81077b0 2024-05-09T01:25:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.333+0000 7f68d77fe700 1 -- 172.21.15.19:0/29147055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc017650 con 0x7f68e81077b0 2024-05-09T01:25:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.333+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e81a4e40 con 0x7f68e81077b0 2024-05-09T01:25:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.334+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e80512c0 con 0x7f68e81077b0 2024-05-09T01:25:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.335+0000 7f68d77fe700 1 -- 172.21.15.19:0/29147055 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f68dc029070 con 0x7f68e81077b0 2024-05-09T01:25:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.336+0000 7f68d77fe700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68d00b2880 0x7f68d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.336+0000 7f68e659c700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68d00b2880 0x7f68d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.336+0000 7f68d77fe700 1 -- 172.21.15.19:0/29147055 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f68dc0e3380 con 0x7f68e81077b0 2024-05-09T01:25:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.337+0000 7f68e659c700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68d00b2880 0x7f68d00b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f68e8076410 tx=0x7f68d800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:22.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.340+0000 7f68d77fe700 1 -- 172.21.15.19:0/29147055 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f68dc013070 con 0x7f68e81077b0 2024-05-09T01:25:22.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.563+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f68e81033b0 con 0x7f68d00b2880 2024-05-09T01:25:22.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.565+0000 7f68d77fe700 1 -- 172.21.15.19:0/29147055 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f68e81033b0 con 0x7f68d00b2880 2024-05-09T01:25:22.562 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T01:25:22.562 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-09T01:25:22.563 INFO:teuthology.orchestra.run.smithi019.stdout:smithi059 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009L400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-09T01:25:22.563 INFO:teuthology.orchestra.run.smithi019.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68d00b2880 msgr2=0x7f68d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68d00b2880 0x7f68d00b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f68e8076410 tx=0x7f68d800b3c0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 msgr2=0x7f68e819d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 0x7f68e819d0e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0098e0 tx=0x7f68dc0047b0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 shutdown_connections 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e810a110 0x7f68e81a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e8104e50 0x7f68e819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68d00b2880 0x7f68d00b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 --2- 172.21.15.19:0/29147055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e81077b0 0x7f68e819d0e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 >> 172.21.15.19:0/29147055 conn(0x7f68e80fe890 msgr2=0x7f68e8102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 shutdown_connections 2024-05-09T01:25:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:22.567+0000 7f68ecdbb700 1 -- 172.21.15.19:0/29147055 wait complete. 2024-05-09T01:25:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:22 smithi019 ceph-mon[30158]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:22 smithi059 ceph-mon[34351]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:22.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:22 smithi196 ceph-mon[35041]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:23.236 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T01:25:23.245 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T01:25:23.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T01:25:23.499 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:24.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:24 smithi019 ceph-mon[30158]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:24.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:24 smithi019 ceph-mon[30158]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:24 smithi059 ceph-mon[34351]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:24 smithi059 ceph-mon[34351]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:24 smithi196 ceph-mon[35041]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:24.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:24 smithi196 ceph-mon[35041]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:25.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.062+0000 7f827db78700 1 -- 172.21.15.19:0/3945933289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 msgr2=0x7f8278104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:25.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.062+0000 7f827db78700 1 --2- 172.21.15.19:0/3945933289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 0x7f8278104770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f826c009990 tx=0x7f826c009ca0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.062+0000 7f827db78700 1 -- 172.21.15.19:0/3945933289 shutdown_connections 2024-05-09T01:25:25.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.062+0000 7f827db78700 1 --2- 172.21.15.19:0/3945933289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8278104d00 0x7f827810b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.062+0000 7f827db78700 1 --2- 172.21.15.19:0/3945933289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8278103230 0x7f8278103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.062+0000 7f827db78700 1 --2- 172.21.15.19:0/3945933289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 0x7f8278104770 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.062+0000 7f827db78700 1 -- 172.21.15.19:0/3945933289 >> 172.21.15.19:0/3945933289 conn(0x7f82780fe870 msgr2=0x7f8278100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.062+0000 7f827db78700 1 -- 172.21.15.19:0/3945933289 shutdown_connections 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.062+0000 7f827db78700 1 -- 172.21.15.19:0/3945933289 wait complete. 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f827db78700 1 Processor -- start 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f827db78700 1 -- start start 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f827db78700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8278103230 0x7f827806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f827db78700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 0x7f827819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f827db78700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8278104d00 0x7f82781a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f827db78700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f827810ec90 con 0x7f8278104360 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f827db78700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f827810eb10 con 0x7f8278104d00 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f827db78700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f827810ee10 con 0x7f8278103230 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f8276ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 0x7f827819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f8276ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 0x7f827819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41184/0 (socket says 172.21.15.19:41184) 2024-05-09T01:25:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.063+0000 7f8276ffd700 1 -- 172.21.15.19:0/1993157316 learned_addr learned my addr 172.21.15.19:0/1993157316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8277fff700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8278104d00 0x7f82781a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f82777fe700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8278103230 0x7f827806caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8276ffd700 1 -- 172.21.15.19:0/1993157316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8278103230 msgr2=0x7f827806caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8276ffd700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8278103230 0x7f827806caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8276ffd700 1 -- 172.21.15.19:0/1993157316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8278104d00 msgr2=0x7f82781a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8276ffd700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8278104d00 0x7f82781a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8276ffd700 1 -- 172.21.15.19:0/1993157316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82781a2440 con 0x7f8278104360 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8276ffd700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 0x7f827819f6a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f826c0098e0 tx=0x7f826c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8274ff9700 1 -- 172.21.15.19:0/1993157316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c026070 con 0x7f8278104360 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8274ff9700 1 -- 172.21.15.19:0/1993157316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f826c00fbc0 con 0x7f8278104360 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82781a26d0 con 0x7f8278104360 2024-05-09T01:25:25.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f8274ff9700 1 -- 172.21.15.19:0/1993157316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c017650 con 0x7f8278104360 2024-05-09T01:25:25.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.064+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82781a2b20 con 0x7f8278104360 2024-05-09T01:25:25.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.065+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82780512c0 con 0x7f8278104360 2024-05-09T01:25:25.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.067+0000 7f8274ff9700 1 -- 172.21.15.19:0/1993157316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f826c029070 con 0x7f8278104360 2024-05-09T01:25:25.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.068+0000 7f8274ff9700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f82600b2880 0x7f82600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:25.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.068+0000 7f82777fe700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f82600b2880 0x7f82600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:25.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.068+0000 7f8274ff9700 1 -- 172.21.15.19:0/1993157316 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f826c0e3690 con 0x7f8278104360 2024-05-09T01:25:25.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.068+0000 7f82777fe700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f82600b2880 0x7f82600b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8268005950 tx=0x7f82680058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:25.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.072+0000 7f8274ff9700 1 -- 172.21.15.19:0/1993157316 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f826c013070 con 0x7f8278104360 2024-05-09T01:25:25.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.335+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f827802e5f0 con 0x7f8278104360 2024-05-09T01:25:25.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.335+0000 7f8274ff9700 1 -- 172.21.15.19:0/1993157316 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3086 (secure 0 0 0) 0x7f826c0ab3c0 con 0x7f8278104360 2024-05-09T01:25:25.333 INFO:teuthology.orchestra.run.smithi019.stdout:e27 2024-05-09T01:25:25.333 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T01:25:25.333 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 27 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-05-09T01:22:41.711277+0000 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-05-09T01:23:01.311002+0000 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-05-09T01:25:25.334 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 5 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1,2,3,4 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24497,1=24473,2=24466,3=24452,4=24451} 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-05-09T01:25:25.335 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24497 members: 24451,24497,24473,24452,24466 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{0:24497} state up:active seq 5 addr [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{0:24445} state up:standby-replay seq 1 addr [v2:172.21.15.196:6834/1184717757,v1:172.21.15.196:6835/1184717757] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{1:24473} state up:active seq 12 addr [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{1:24428} state up:standby-replay seq 1 addr [v2:172.21.15.59:6832/3715662430,v1:172.21.15.59:6833/3715662430] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{2:24466} state up:active seq 12 addr [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.336 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{2:24418} state up:standby-replay seq 1 addr [v2:172.21.15.196:6832/2736793414,v1:172.21.15.196:6833/2736793414] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{3:24452} state up:active seq 15 addr [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{3:14640} state up:standby-replay seq 1 addr [v2:172.21.15.19:6840/555235417,v1:172.21.15.19:6841/555235417] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{4:24451} state up:active seq 13 addr [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{4:14628} state up:standby-replay seq 1 addr [v2:172.21.15.19:6838/909874272,v1:172.21.15.19:6839/909874272] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{-1:14565} state up:standby seq 1 addr [v2:172.21.15.19:6834/523355371,v1:172.21.15.19:6835/523355371] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.337 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{-1:14571} state up:standby seq 1 addr [v2:172.21.15.19:6836/3245904831,v1:172.21.15.19:6837/3245904831] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T01:25:25.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f82600b2880 msgr2=0x7f82600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:25.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f82600b2880 0x7f82600b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8268005950 tx=0x7f82680058e0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 msgr2=0x7f827819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 0x7f827819f6a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f826c0098e0 tx=0x7f826c0047b0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 shutdown_connections 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8278104d00 0x7f82781a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8278103230 0x7f827806caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f82600b2880 0x7f82600b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 --2- 172.21.15.19:0/1993157316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8278104360 0x7f827819f6a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 >> 172.21.15.19:0/1993157316 conn(0x7f82780fe870 msgr2=0x7f8278107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 shutdown_connections 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:25.338+0000 7f827db78700 1 -- 172.21.15.19:0/1993157316 wait complete. 2024-05-09T01:25:25.339 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-09T01:25:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:25 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1993157316' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T01:25:25.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:25 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1993157316' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T01:25:25.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:25 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1993157316' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T01:25:26.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T01:25:26.262 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:26 smithi019 ceph-mon[30158]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:26 smithi059 ceph-mon[34351]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:26.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:26 smithi196 ceph-mon[35041]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:28.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.033+0000 7fc5fad06700 1 -- 172.21.15.19:0/1396801203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 msgr2=0x7fc5f41058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:28.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.033+0000 7fc5fad06700 1 --2- 172.21.15.19:0/1396801203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 0x7fc5f41058b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009990 tx=0x7fc5e0009ca0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.033+0000 7fc5fad06700 1 -- 172.21.15.19:0/1396801203 shutdown_connections 2024-05-09T01:25:28.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.033+0000 7fc5fad06700 1 --2- 172.21.15.19:0/1396801203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4105e40 0x7fc5f410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.033+0000 7fc5fad06700 1 --2- 172.21.15.19:0/1396801203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f4104370 0x7fc5f4104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.033+0000 7fc5fad06700 1 --2- 172.21.15.19:0/1396801203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 0x7fc5f41058b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.033+0000 7fc5fad06700 1 -- 172.21.15.19:0/1396801203 >> 172.21.15.19:0/1396801203 conn(0x7fc5f40ff9d0 msgr2=0x7fc5f4101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:28.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.033+0000 7fc5fad06700 1 -- 172.21.15.19:0/1396801203 shutdown_connections 2024-05-09T01:25:28.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.033+0000 7fc5fad06700 1 -- 172.21.15.19:0/1396801203 wait complete. 2024-05-09T01:25:28.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5fad06700 1 Processor -- start 2024-05-09T01:25:28.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5fad06700 1 -- start start 2024-05-09T01:25:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5fad06700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4104370 0x7fc5f419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5fad06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 0x7fc5f419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5fad06700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f4105e40 0x7fc5f41a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5fad06700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f410fdd0 con 0x7fc5f41054a0 2024-05-09T01:25:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5fad06700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f410fc50 con 0x7fc5f4105e40 2024-05-09T01:25:28.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5fad06700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f410ff50 con 0x7fc5f4104370 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 0x7fc5f419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 0x7fc5f419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41196/0 (socket says 172.21.15.19:41196) 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5f3fff700 1 -- 172.21.15.19:0/3653181458 learned_addr learned my addr 172.21.15.19:0/3653181458 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5f8aa2700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4104370 0x7fc5f419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.034+0000 7fc5f92a3700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f4105e40 0x7fc5f41a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5f3fff700 1 -- 172.21.15.19:0/3653181458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4104370 msgr2=0x7fc5f419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5f3fff700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4104370 0x7fc5f419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5f3fff700 1 -- 172.21.15.19:0/3653181458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f4105e40 msgr2=0x7fc5f41a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5f3fff700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f4105e40 0x7fc5f41a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5f3fff700 1 -- 172.21.15.19:0/3653181458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f41a2410 con 0x7fc5f41054a0 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5f3fff700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 0x7fc5f419adf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0006010 tx=0x7fc5e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5f1ffb700 1 -- 172.21.15.19:0/3653181458 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0027070 con 0x7fc5f41054a0 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5f1ffb700 1 -- 172.21.15.19:0/3653181458 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5e0004000 con 0x7fc5f41054a0 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5f1ffb700 1 -- 172.21.15.19:0/3653181458 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0022d30 con 0x7fc5f41054a0 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f41a26a0 con 0x7fc5f41054a0 2024-05-09T01:25:28.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.035+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f41a2af0 con 0x7fc5f41054a0 2024-05-09T01:25:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.036+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f40512c0 con 0x7fc5f41054a0 2024-05-09T01:25:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.038+0000 7fc5f1ffb700 1 -- 172.21.15.19:0/3653181458 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc5e002a070 con 0x7fc5f41054a0 2024-05-09T01:25:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.039+0000 7fc5f1ffb700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc5e40b2880 0x7fc5e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.039+0000 7fc5f8aa2700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc5e40b2880 0x7fc5e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:28.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.039+0000 7fc5f1ffb700 1 -- 172.21.15.19:0/3653181458 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc5e00e3350 con 0x7fc5f41054a0 2024-05-09T01:25:28.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.039+0000 7fc5f8aa2700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc5e40b2880 0x7fc5e40b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5f41053a0 tx=0x7fc5e800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:28.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.042+0000 7fc5f1ffb700 1 -- 172.21.15.19:0/3653181458 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5e0013070 con 0x7fc5f41054a0 2024-05-09T01:25:28.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.257+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc5f40643d0 con 0x7fc5f41054a0 2024-05-09T01:25:28.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.258+0000 7fc5f1ffb700 1 -- 172.21.15.19:0/3653181458 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5321 (secure 0 0 0) 0x7fc5e00041b0 con 0x7fc5f41054a0 2024-05-09T01:25:28.256 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 83 2024-05-09T01:25:28.256 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:25:28.256 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-05-09T01:14:18.495533+0000 2024-05-09T01:25:28.256 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-05-09T01:23:22.298486+0000 2024-05-09T01:25:28.256 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T01:25:28.256 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 26 2024-05-09T01:25:28.256 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/2513726191,v1:172.21.15.19:6803/2513726191] [v2:172.21.15.19:6804/2513726191,v1:172.21.15.19:6805/2513726191] exists,up 3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6800/2880894912,v1:172.21.15.59:6801/2880894912] [v2:172.21.15.59:6802/2880894912,v1:172.21.15.59:6803/2880894912] exists,up d6305a55-e657-43de-83fd-8e9568e91363 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/1599368048,v1:172.21.15.196:6801/1599368048] [v2:172.21.15.196:6802/1599368048,v1:172.21.15.196:6803/1599368048] exists,up 632fb397-639e-4ccf-b0a2-31e1f8c7aa65 2024-05-09T01:25:28.257 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 27 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/713020110,v1:172.21.15.19:6811/713020110] [v2:172.21.15.19:6812/713020110,v1:172.21.15.19:6813/713020110] exists,up 067c1dbe-5d98-4603-bc43-64343344993e 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 33 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6808/2419703664,v1:172.21.15.59:6809/2419703664] [v2:172.21.15.59:6810/2419703664,v1:172.21.15.59:6811/2419703664] exists,up 322cb4e2-84e1-494f-93d6-bd5b09ce3a82 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 39 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/1450867392,v1:172.21.15.196:6809/1450867392] [v2:172.21.15.196:6810/1450867392,v1:172.21.15.196:6811/1450867392] exists,up c53b8d7f-d032-46d4-98c0-99a6b77d9fd8 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 45 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/3027498508,v1:172.21.15.19:6819/3027498508] [v2:172.21.15.19:6820/3027498508,v1:172.21.15.19:6821/3027498508] exists,up 95d655ea-93ee-4077-96be-3bbddc722520 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 50 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6816/2903314697,v1:172.21.15.59:6817/2903314697] [v2:172.21.15.59:6818/2903314697,v1:172.21.15.59:6819/2903314697] exists,up 792a321f-5e7a-4ec4-b6bd-020b1fb8bf07 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 56 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/2419396882,v1:172.21.15.196:6817/2419396882] [v2:172.21.15.196:6818/2419396882,v1:172.21.15.196:6819/2419396882] exists,up 6a9e69b9-50a5-4379-be85-ef221d1b6cf9 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 62 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/3240481605,v1:172.21.15.19:6827/3240481605] [v2:172.21.15.19:6828/3240481605,v1:172.21.15.19:6829/3240481605] exists,up a7437a16-b125-4a63-80f3-987d87d3365d 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 67 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6824/1290773390,v1:172.21.15.59:6825/1290773390] [v2:172.21.15.59:6826/1290773390,v1:172.21.15.59:6827/1290773390] exists,up 56aa036a-6771-45e1-80a9-eb0fbee10f5e 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 73 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/1918691791,v1:172.21.15.196:6825/1918691791] [v2:172.21.15.196:6826/1918691791,v1:172.21.15.196:6827/1918691791] exists,up 3b4010a5-eb86-4f92-93de-4d7733ff5ec2 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.a [2,11] 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.c [2,4] 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.e [2,4] 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.10 [8,11] 2024-05-09T01:25:28.258 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1f [8,1] 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.3 [4,9] 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.6 [4,7] 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.23 [4,8] 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.25 [5,9] 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.29 [3,9] 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3911883227 expires 2024-05-10T01:15:21.027129+0000 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/272437452 expires 2024-05-10T01:15:21.027129+0000 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/582848527 expires 2024-05-10T01:14:53.392608+0000 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/455701507 expires 2024-05-10T01:14:53.392608+0000 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/112627556 expires 2024-05-10T01:15:21.027129+0000 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/292723758 expires 2024-05-10T01:15:21.027129+0000 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/112627556 expires 2024-05-10T01:15:21.027129+0000 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/3332485717 expires 2024-05-10T01:14:53.392608+0000 2024-05-09T01:25:28.259 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/3332485717 expires 2024-05-10T01:14:53.392608+0000 2024-05-09T01:25:28.260 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3768545435 expires 2024-05-10T01:14:53.392608+0000 2024-05-09T01:25:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc5e40b2880 msgr2=0x7fc5e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc5e40b2880 0x7fc5e40b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5f41053a0 tx=0x7fc5e800a380 comp rx=0 tx=0).stop 2024-05-09T01:25:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 msgr2=0x7fc5f419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 0x7fc5f419adf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0006010 tx=0x7fc5e00047b0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 shutdown_connections 2024-05-09T01:25:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f4105e40 0x7fc5f41a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4104370 0x7fc5f419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc5e40b2880 0x7fc5e40b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 --2- 172.21.15.19:0/3653181458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f41054a0 0x7fc5f419adf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 >> 172.21.15.19:0/3653181458 conn(0x7fc5f40ff9d0 msgr2=0x7fc5f41086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 shutdown_connections 2024-05-09T01:25:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:28.262+0000 7fc5fad06700 1 -- 172.21.15.19:0/3653181458 wait complete. 2024-05-09T01:25:28.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:28 smithi019 ceph-mon[30158]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:28.578 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:28 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3653181458' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T01:25:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:28 smithi059 ceph-mon[34351]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:28.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:28 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3653181458' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T01:25:28.891 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T01:25:28.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:25:28.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:28 smithi196 ceph-mon[35041]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:28.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:28 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3653181458' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T01:25:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:30.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:30 smithi019 ceph-mon[30158]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:30 smithi059 ceph-mon[34351]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:30 smithi196 ceph-mon[35041]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:25:31.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.026+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/2397208838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 msgr2=0x7ff9c8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.026+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/2397208838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 0x7ff9c8110640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009990 tx=0x7ff9b8009ca0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.026+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/2397208838 shutdown_connections 2024-05-09T01:25:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.026+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/2397208838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9c8110b80 0x7ff9c8112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.026+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/2397208838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c8076e40 0x7ff9c8077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.026+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/2397208838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 0x7ff9c8110640 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.026+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/2397208838 >> 172.21.15.19:0/2397208838 conn(0x7ff9c80ffea0 msgr2=0x7ff9c81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:31.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.026+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/2397208838 shutdown_connections 2024-05-09T01:25:31.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.026+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/2397208838 wait complete. 2024-05-09T01:25:31.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.027+0000 7ff9cfbc9700 1 Processor -- start 2024-05-09T01:25:31.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.027+0000 7ff9cfbc9700 1 -- start start 2024-05-09T01:25:31.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.027+0000 7ff9cfbc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9c8076e40 0x7ff9c819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:31.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.027+0000 7ff9cfbc9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 0x7ff9c819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:31.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.027+0000 7ff9cfbc9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c8110b80 0x7ff9c81a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:31.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.027+0000 7ff9cfbc9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8116680 con 0x7ff9c8077790 2024-05-09T01:25:31.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.027+0000 7ff9cfbc9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8116500 con 0x7ff9c8076e40 2024-05-09T01:25:31.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.027+0000 7ff9cfbc9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8116800 con 0x7ff9c8110b80 2024-05-09T01:25:31.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd164700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 0x7ff9c819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:31.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd164700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 0x7ff9c819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59514/0 (socket says 172.21.15.19:59514) 2024-05-09T01:25:31.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd164700 1 -- 172.21.15.19:0/3972239142 learned_addr learned my addr 172.21.15.19:0/3972239142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:31.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd965700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9c8076e40 0x7ff9c819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:31.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9ce166700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c8110b80 0x7ff9c81a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:31.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd164700 1 -- 172.21.15.19:0/3972239142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c8110b80 msgr2=0x7ff9c81a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:31.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd164700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c8110b80 0x7ff9c81a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd164700 1 -- 172.21.15.19:0/3972239142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9c8076e40 msgr2=0x7ff9c819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:31.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd164700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9c8076e40 0x7ff9c819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd164700 1 -- 172.21.15.19:0/3972239142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c81a1ea0 con 0x7ff9c8077790 2024-05-09T01:25:31.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cd164700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 0x7ff9c819d570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8000c00 tx=0x7ff9b80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:31.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9beffd700 1 -- 172.21.15.19:0/3972239142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8027070 con 0x7ff9c8077790 2024-05-09T01:25:31.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9beffd700 1 -- 172.21.15.19:0/3972239142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b8003f20 con 0x7ff9c8077790 2024-05-09T01:25:31.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c81a2130 con 0x7ff9c8077790 2024-05-09T01:25:31.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.028+0000 7ff9beffd700 1 -- 172.21.15.19:0/3972239142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8022cf0 con 0x7ff9c8077790 2024-05-09T01:25:31.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.029+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c81a2610 con 0x7ff9c8077790 2024-05-09T01:25:31.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.030+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c80512c0 con 0x7ff9c8077790 2024-05-09T01:25:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.031+0000 7ff9beffd700 1 -- 172.21.15.19:0/3972239142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff9b802a070 con 0x7ff9c8077790 2024-05-09T01:25:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.032+0000 7ff9beffd700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff9b40b2880 0x7ff9b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.032+0000 7ff9cd965700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff9b40b2880 0x7ff9b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.032+0000 7ff9beffd700 1 -- 172.21.15.19:0/3972239142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff9b80e3370 con 0x7ff9c8077790 2024-05-09T01:25:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.032+0000 7ff9cd965700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff9b40b2880 0x7ff9b40b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005950 tx=0x7ff9c400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:31.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.035+0000 7ff9beffd700 1 -- 172.21.15.19:0/3972239142 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9b8013070 con 0x7ff9c8077790 2024-05-09T01:25:31.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.297+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9c802e5f0 con 0x7ff9c8077790 2024-05-09T01:25:31.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.297+0000 7ff9beffd700 1 -- 172.21.15.19:0/3972239142 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11924 (secure 0 0 0) 0x7ff9b80ab0a0 con 0x7ff9c8077790 2024-05-09T01:25:31.296 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:25:31.296 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:23:01.311002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24451,24497,24473,24452,24466]},"id":1}]} 2024-05-09T01:25:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff9b40b2880 msgr2=0x7ff9b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff9b40b2880 0x7ff9b40b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005950 tx=0x7ff9c400b410 comp rx=0 tx=0).stop 2024-05-09T01:25:31.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 msgr2=0x7ff9c819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 0x7ff9c819d570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8000c00 tx=0x7ff9b80047b0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 shutdown_connections 2024-05-09T01:25:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c8110b80 0x7ff9c81a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9c8076e40 0x7ff9c819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff9b40b2880 0x7ff9b40b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 --2- 172.21.15.19:0/3972239142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9c8077790 0x7ff9c819d570 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 >> 172.21.15.19:0/3972239142 conn(0x7ff9c80ffea0 msgr2=0x7ff9c8101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 shutdown_connections 2024-05-09T01:25:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:31.300+0000 7ff9cfbc9700 1 -- 172.21.15.19:0/3972239142 wait complete. 2024-05-09T01:25:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-09T01:25:31.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:31 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3972239142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:25:31.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:31 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3972239142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:25:31.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:31 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3972239142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:25:31.952 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:25:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:32 smithi019 ceph-mon[30158]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:32.748 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:32 smithi059 ceph-mon[34351]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:32.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:32 smithi196 ceph-mon[35041]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.993+0000 7f37ed676700 1 -- 172.21.15.19:0/2359386211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e8108740 msgr2=0x7f37e81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.993+0000 7f37ed676700 1 --2- 172.21.15.19:0/2359386211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e8108740 0x7f37e81100a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f37e000b0d0 tx=0x7f37e000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:25:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.993+0000 7f37ed676700 1 -- 172.21.15.19:0/2359386211 shutdown_connections 2024-05-09T01:25:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.993+0000 7f37ed676700 1 --2- 172.21.15.19:0/2359386211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e8108740 0x7f37e81100a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.993+0000 7f37ed676700 1 --2- 172.21.15.19:0/2359386211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e81034c0 0x7f37e81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.993+0000 7f37ed676700 1 --2- 172.21.15.19:0/2359386211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e8107df0 0x7f37e8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.993+0000 7f37ed676700 1 -- 172.21.15.19:0/2359386211 >> 172.21.15.19:0/2359386211 conn(0x7f37e80fedc0 msgr2=0x7f37e81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.993+0000 7f37ed676700 1 -- 172.21.15.19:0/2359386211 shutdown_connections 2024-05-09T01:25:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.993+0000 7f37ed676700 1 -- 172.21.15.19:0/2359386211 wait complete. 2024-05-09T01:25:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.994+0000 7f37ed676700 1 Processor -- start 2024-05-09T01:25:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.994+0000 7f37ed676700 1 -- start start 2024-05-09T01:25:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37ed676700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e81034c0 0x7f37e819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37ed676700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e8107df0 0x7f37e819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37ed676700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e8108740 0x7f37e819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37ed676700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8113370 con 0x7f37e81034c0 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37ed676700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81131f0 con 0x7f37e8108740 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37ed676700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81134f0 con 0x7f37e8107df0 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e81034c0 0x7f37e819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e81034c0 0x7f37e819ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59530/0 (socket says 172.21.15.19:59530) 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e6ffd700 1 -- 172.21.15.19:0/3937443715 learned_addr learned my addr 172.21.15.19:0/3937443715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e67fc700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e8107df0 0x7f37e819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e77fe700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e8108740 0x7f37e819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e6ffd700 1 -- 172.21.15.19:0/3937443715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e8107df0 msgr2=0x7f37e819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e6ffd700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e8107df0 0x7f37e819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e6ffd700 1 -- 172.21.15.19:0/3937443715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e8108740 msgr2=0x7f37e819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e6ffd700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e8108740 0x7f37e819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e6ffd700 1 -- 172.21.15.19:0/3937443715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e819fc40 con 0x7f37e81034c0 2024-05-09T01:25:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37e6ffd700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e81034c0 0x7f37e819ad70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f37d800d980 tx=0x7f37d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.995+0000 7f37cffff700 1 -- 172.21.15.19:0/3937443715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8016070 con 0x7f37e81034c0 2024-05-09T01:25:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.996+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e819ff30 con 0x7f37e81034c0 2024-05-09T01:25:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.996+0000 7f37cffff700 1 -- 172.21.15.19:0/3937443715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d8004830 con 0x7f37e81034c0 2024-05-09T01:25:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.996+0000 7f37cffff700 1 -- 172.21.15.19:0/3937443715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8005540 con 0x7f37e81034c0 2024-05-09T01:25:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.996+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a7850 con 0x7f37e81034c0 2024-05-09T01:25:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.997+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e80512c0 con 0x7f37e81034c0 2024-05-09T01:25:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.998+0000 7f37cffff700 1 -- 172.21.15.19:0/3937443715 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f37d8004220 con 0x7f37e81034c0 2024-05-09T01:25:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.999+0000 7f37cffff700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37d00b2880 0x7f37d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.999+0000 7f37e67fc700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37d00b2880 0x7f37d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.999+0000 7f37cffff700 1 -- 172.21.15.19:0/3937443715 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f37d80e1ff0 con 0x7f37e81034c0 2024-05-09T01:25:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:33.999+0000 7f37e67fc700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37d00b2880 0x7f37d00b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37e8109890 tx=0x7f37dc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.004+0000 7f37cffff700 1 -- 172.21.15.19:0/3937443715 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37d801f020 con 0x7f37e81034c0 2024-05-09T01:25:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.270+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37e819c0d0 con 0x7f37e81034c0 2024-05-09T01:25:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.271+0000 7f37cffff700 1 -- 172.21.15.19:0/3937443715 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11924 (secure 0 0 0) 0x7f37d80a9cf0 con 0x7f37e81034c0 2024-05-09T01:25:34.269 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:25:34.269 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:23:01.311002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24451,24497,24473,24452,24466]},"id":1}]} 2024-05-09T01:25:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37d00b2880 msgr2=0x7f37d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37d00b2880 0x7f37d00b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37e8109890 tx=0x7f37dc00b410 comp rx=0 tx=0).stop 2024-05-09T01:25:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e81034c0 msgr2=0x7f37e819ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e81034c0 0x7f37e819ad70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f37d800d980 tx=0x7f37d800dc90 comp rx=0 tx=0).stop 2024-05-09T01:25:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 shutdown_connections 2024-05-09T01:25:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e8108740 0x7f37e819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f37d00b2880 0x7f37d00b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e81034c0 0x7f37e819ad70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 --2- 172.21.15.19:0/3937443715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e8107df0 0x7f37e819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.273+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 >> 172.21.15.19:0/3937443715 conn(0x7f37e80fedc0 msgr2=0x7f37e806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.274+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 shutdown_connections 2024-05-09T01:25:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:34.274+0000 7f37ed676700 1 -- 172.21.15.19:0/3937443715 wait complete. 2024-05-09T01:25:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-05-09T01:25:34.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:34 smithi019 ceph-mon[30158]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:25:34.635 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:34 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3937443715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:25:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:34 smithi059 ceph-mon[34351]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:25:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:34 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3937443715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:25:34.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:34 smithi196 ceph-mon[35041]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:25:34.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:34 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3937443715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:25:35.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:25:35.276 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:36.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:36 smithi019 ceph-mon[30158]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:25:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:36 smithi059 ceph-mon[34351]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:25:36.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:36 smithi196 ceph-mon[35041]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:25:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.965+0000 7f70e5404700 1 -- 172.21.15.19:0/1699647151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 msgr2=0x7f70e0110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.965+0000 7f70e5404700 1 --2- 172.21.15.19:0/1699647151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 0x7f70e0110c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b0a0 tx=0x7f70d800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.965+0000 7f70e5404700 1 -- 172.21.15.19:0/1699647151 shutdown_connections 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.965+0000 7f70e5404700 1 --2- 172.21.15.19:0/1699647151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 0x7f70e0110c10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.965+0000 7f70e5404700 1 --2- 172.21.15.19:0/1699647151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70e01052c0 0x7f70e01056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.965+0000 7f70e5404700 1 --2- 172.21.15.19:0/1699647151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e0105c10 0x7f70e010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.965+0000 7f70e5404700 1 -- 172.21.15.19:0/1699647151 >> 172.21.15.19:0/1699647151 conn(0x7f70e00fed00 msgr2=0x7f70e0101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.965+0000 7f70e5404700 1 -- 172.21.15.19:0/1699647151 shutdown_connections 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.965+0000 7f70e5404700 1 -- 172.21.15.19:0/1699647151 wait complete. 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.966+0000 7f70e5404700 1 Processor -- start 2024-05-09T01:25:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.966+0000 7f70e5404700 1 -- start start 2024-05-09T01:25:36.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70e5404700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e01052c0 0x7f70e01a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:36.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70e5404700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70e0105c10 0x7f70e01a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:36.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70e5404700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 0x7f70e019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70e5404700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70e0114290 con 0x7f70e010e730 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70e5404700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70e0114110 con 0x7f70e0105c10 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70e5404700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70e0114410 con 0x7f70e01052c0 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 0x7f70e019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 0x7f70e019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59550/0 (socket says 172.21.15.19:59550) 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70df7fe700 1 -- 172.21.15.19:0/3373161629 learned_addr learned my addr 172.21.15.19:0/3373161629 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70deffd700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e01052c0 0x7f70e01a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70de7fc700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70e0105c10 0x7f70e01a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70df7fe700 1 -- 172.21.15.19:0/3373161629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e01052c0 msgr2=0x7f70e01a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70df7fe700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e01052c0 0x7f70e01a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70df7fe700 1 -- 172.21.15.19:0/3373161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70e0105c10 msgr2=0x7f70e01a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70df7fe700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70e0105c10 0x7f70e01a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70df7fe700 1 -- 172.21.15.19:0/3373161629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e019b040 con 0x7f70e010e730 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70de7fc700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70e0105c10 0x7f70e01a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.967+0000 7f70df7fe700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 0x7f70e019a990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b830 tx=0x7f70d8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:36.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.968+0000 7f70c7fff700 1 -- 172.21.15.19:0/3373161629 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8021070 con 0x7f70e010e730 2024-05-09T01:25:36.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.968+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e019b2d0 con 0x7f70e010e730 2024-05-09T01:25:36.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.968+0000 7f70c7fff700 1 -- 172.21.15.19:0/3373161629 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70d8007e30 con 0x7f70e010e730 2024-05-09T01:25:36.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.968+0000 7f70c7fff700 1 -- 172.21.15.19:0/3373161629 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d80047d0 con 0x7f70e010e730 2024-05-09T01:25:36.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.968+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e01a7600 con 0x7f70e010e730 2024-05-09T01:25:36.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.969+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e006c950 con 0x7f70e010e730 2024-05-09T01:25:36.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.970+0000 7f70c7fff700 1 -- 172.21.15.19:0/3373161629 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f70d8007980 con 0x7f70e010e730 2024-05-09T01:25:36.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.971+0000 7f70c7fff700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70c80b2880 0x7f70c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:36.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.971+0000 7f70deffd700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70c80b2880 0x7f70c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:36.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.971+0000 7f70c7fff700 1 -- 172.21.15.19:0/3373161629 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f70d800f020 con 0x7f70e010e730 2024-05-09T01:25:36.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.972+0000 7f70deffd700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70c80b2880 0x7f70c80b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f70e019bad0 tx=0x7f70d00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:36.975+0000 7f70c7fff700 1 -- 172.21.15.19:0/3373161629 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70d802a020 con 0x7f70e010e730 2024-05-09T01:25:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.194+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70e019bec0 con 0x7f70e010e730 2024-05-09T01:25:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.196+0000 7f70c7fff700 1 -- 172.21.15.19:0/3373161629 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22716 (secure 0 0 0) 0x7f70d80ab980 con 0x7f70e010e730 2024-05-09T01:25:37.194 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:25:37.194 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:23:22.298486+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:25:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70c80b2880 msgr2=0x7f70c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70c80b2880 0x7f70c80b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f70e019bad0 tx=0x7f70d00073d0 comp rx=0 tx=0).stop 2024-05-09T01:25:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 msgr2=0x7f70e019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 0x7f70e019a990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b830 tx=0x7f70d8004ce0 comp rx=0 tx=0).stop 2024-05-09T01:25:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 shutdown_connections 2024-05-09T01:25:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70c80b2880 0x7f70c80b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e010e730 0x7f70e019a990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e01052c0 0x7f70e01a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 --2- 172.21.15.19:0/3373161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70e0105c10 0x7f70e01a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 >> 172.21.15.19:0/3373161629 conn(0x7f70e00fed00 msgr2=0x7f70e00ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 shutdown_connections 2024-05-09T01:25:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:37.198+0000 7f70e5404700 1 -- 172.21.15.19:0/3373161629 wait complete. 2024-05-09T01:25:37.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:37 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3373161629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:25:37.798 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T01:22:41.711277+0000', 'modified': '2024-05-09T01:23:01.311002+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24497, 'mds_1': 24473, 'mds_2': 24466, 'mds_3': 24452, 'mds_4': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/909874272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 909874272}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 909874272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/555235417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 555235417}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 555235417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/2736793414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 2736793414}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 2736793414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6833/3715662430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6832', 'nonce': 3715662430}, {'type': 'v1', 'addr': '172.21.15.59:6833', 'nonce': 3715662430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/1184717757', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1184717757}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1184717757}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.196:6837/3887501643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3887501643}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3887501643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.59:6835/3872629659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 3872629659}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 3872629659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.196:6839/165915317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 165915317}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 165915317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.59:6837/2120139910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 2120139910}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2120139910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.59:6839/1954332745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 1954332745}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 1954332745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24451, 24497, 24473, 24452, 24466]} 2024-05-09T01:25:37.798 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T01:25:37.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T01:25:37.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:37 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3373161629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:25:37.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:37 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3373161629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:25:38.048 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:38 smithi059 ceph-mon[34351]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T01:25:38.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:38 smithi019 ceph-mon[30158]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T01:25:38.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:38 smithi196 ceph-mon[35041]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T01:25:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.786+0000 7f41e80f9700 1 -- 172.21.15.19:0/3032492483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 msgr2=0x7f41e01049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.786+0000 7f41e80f9700 1 --2- 172.21.15.19:0/3032492483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 0x7f41e01049e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00b0d0 tx=0x7f41dc00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.787+0000 7f41e80f9700 1 -- 172.21.15.19:0/3032492483 shutdown_connections 2024-05-09T01:25:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.787+0000 7f41e80f9700 1 --2- 172.21.15.19:0/3032492483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e0109850 0x7f41e01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.787+0000 7f41e80f9700 1 --2- 172.21.15.19:0/3032492483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 0x7f41e01049e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.787+0000 7f41e80f9700 1 --2- 172.21.15.19:0/3032492483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e0108f00 0x7f41e0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.787+0000 7f41e80f9700 1 -- 172.21.15.19:0/3032492483 >> 172.21.15.19:0/3032492483 conn(0x7f41e00ffe80 msgr2=0x7f41e01022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.787+0000 7f41e80f9700 1 -- 172.21.15.19:0/3032492483 shutdown_connections 2024-05-09T01:25:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.787+0000 7f41e80f9700 1 -- 172.21.15.19:0/3032492483 wait complete. 2024-05-09T01:25:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.787+0000 7f41e80f9700 1 Processor -- start 2024-05-09T01:25:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e80f9700 1 -- start start 2024-05-09T01:25:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e80f9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 0x7f41e0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e80f9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e0108f00 0x7f41e0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e80f9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e0109850 0x7f41e0077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e80f9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41e0114480 con 0x7f41e01045d0 2024-05-09T01:25:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e80f9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41e0114300 con 0x7f41e0108f00 2024-05-09T01:25:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e80f9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41e0114600 con 0x7f41e0109850 2024-05-09T01:25:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e5e95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 0x7f41e0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e5e95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 0x7f41e0076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60260/0 (socket says 172.21.15.19:60260) 2024-05-09T01:25:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e5e95700 1 -- 172.21.15.19:0/295492442 learned_addr learned my addr 172.21.15.19:0/295492442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e6696700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e0109850 0x7f41e0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e5694700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e0108f00 0x7f41e0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e5e95700 1 -- 172.21.15.19:0/295492442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e0109850 msgr2=0x7f41e0077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e5e95700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e0109850 0x7f41e0077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e5e95700 1 -- 172.21.15.19:0/295492442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e0108f00 msgr2=0x7f41e0077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e5e95700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e0108f00 0x7f41e0077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e5e95700 1 -- 172.21.15.19:0/295492442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e01a9170 con 0x7f41e01045d0 2024-05-09T01:25:39.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.788+0000 7f41e6696700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e0109850 0x7f41e0077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:39.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.789+0000 7f41e5e95700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 0x7f41e0076bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00d010 tx=0x7f41dc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:39.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.789+0000 7f41d6ffd700 1 -- 172.21.15.19:0/295492442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc01f070 con 0x7f41e01045d0 2024-05-09T01:25:39.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.789+0000 7f41d6ffd700 1 -- 172.21.15.19:0/295492442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41dc003c00 con 0x7f41e01045d0 2024-05-09T01:25:39.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.789+0000 7f41e80f9700 1 -- 172.21.15.19:0/295492442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e01a9400 con 0x7f41e01045d0 2024-05-09T01:25:39.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.789+0000 7f41d6ffd700 1 -- 172.21.15.19:0/295492442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc01b6a0 con 0x7f41e01045d0 2024-05-09T01:25:39.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.789+0000 7f41e80f9700 1 -- 172.21.15.19:0/295492442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e01a98b0 con 0x7f41e01045d0 2024-05-09T01:25:39.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.792+0000 7f41d6ffd700 1 -- 172.21.15.19:0/295492442 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f41dc022070 con 0x7f41e01045d0 2024-05-09T01:25:39.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41d6ffd700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41cc0b2900 0x7f41cc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41e5694700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41cc0b2900 0x7f41cc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41d6ffd700 1 -- 172.21.15.19:0/295492442 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f41dc00f020 con 0x7f41e01045d0 2024-05-09T01:25:39.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41f6b0fb80 1 Processor -- start 2024-05-09T01:25:39.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41f6b0fb80 1 -- start start 2024-05-09T01:25:39.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41e5694700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41cc0b2900 0x7f41cc0b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f41e0108990 tx=0x7f41d000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:39.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd3e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd3e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 0x563f2fd38a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f2fc5afa0 con 0x563f2fca5110 2024-05-09T01:25:39.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f2fbf97a0 con 0x563f2fd38650 2024-05-09T01:25:39.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.793+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563f2fc56f60 con 0x563f2fbbb450 2024-05-09T01:25:39.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5e95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd3e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5e95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd3e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60276/0 (socket says 172.21.15.19:60276) 2024-05-09T01:25:39.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5e95700 1 -- 172.21.15.19:0/583759476 learned_addr learned my addr 172.21.15.19:0/583759476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:39.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5694700 1 --2- 172.21.15.19:0/583759476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd3e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e6696700 1 --2- 172.21.15.19:0/583759476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 0x563f2fd38a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5e95700 1 -- 172.21.15.19:0/583759476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 msgr2=0x563f2fd3e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:39.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5e95700 1 --2- 172.21.15.19:0/583759476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd3e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5e95700 1 -- 172.21.15.19:0/583759476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 msgr2=0x563f2fd38a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:39.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5e95700 1 --2- 172.21.15.19:0/583759476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 0x563f2fd38a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5e95700 1 -- 172.21.15.19:0/583759476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f2fbf1320 con 0x563f2fca5110 2024-05-09T01:25:39.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41e5e95700 1 --2- 172.21.15.19:0/583759476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd3e4a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0a6b10 tx=0x7f41dc0aa4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af080e39379e0834 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:39.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41cbfff700 1 -- 172.21.15.19:0/583759476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0aa5a0 con 0x563f2fca5110 2024-05-09T01:25:39.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41cbfff700 1 -- 172.21.15.19:0/583759476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41dc0ab190 con 0x563f2fca5110 2024-05-09T01:25:39.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.794+0000 7f41cbfff700 1 -- 172.21.15.19:0/583759476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0aacf0 con 0x563f2fca5110 2024-05-09T01:25:39.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.795+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/583759476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 msgr2=0x563f2fd3e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:39.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.795+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/583759476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd3e4a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0a6b10 tx=0x7f41dc0aa4b0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.795+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/583759476 shutdown_connections 2024-05-09T01:25:39.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.795+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/583759476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 0x563f2fd38a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.795+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/583759476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd3e4a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.795+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/583759476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd3e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.795+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/583759476 >> 172.21.15.19:0/583759476 conn(0x563f2fc1b650 msgr2=0x563f2fd333b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:39.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.795+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/583759476 shutdown_connections 2024-05-09T01:25:39.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.795+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/583759476 wait complete. 2024-05-09T01:25:39.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.796+0000 7f41f6b0fb80 1 -- start start 2024-05-09T01:25:39.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.796+0000 7f41f6b0fb80 1 Processor -- start 2024-05-09T01:25:39.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd47c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 0x563f2fd42070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f2fad6b50 con 0x563f2fca5110 2024-05-09T01:25:39.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f2fc15100 con 0x563f2fd38650 2024-05-09T01:25:39.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563f2fd39310 con 0x563f2fbbb450 2024-05-09T01:25:39.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41f6b0fb80 10 client.0 _ll_register_callbacks cb 0x563f2facc670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:39.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5694700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5e95700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd47c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5694700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60286/0 (socket says 172.21.15.19:60286) 2024-05-09T01:25:39.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5e95700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd47c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:45320/0 (socket says 172.21.15.19:45320) 2024-05-09T01:25:39.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5694700 1 -- 172.21.15.19:0/363691724 learned_addr learned my addr 172.21.15.19:0/363691724 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:39.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e6696700 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 0x563f2fd42070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5694700 1 -- 172.21.15.19:0/363691724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 msgr2=0x563f2fd47c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:39.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5694700 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd47c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5694700 1 -- 172.21.15.19:0/363691724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 msgr2=0x563f2fd42070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:39.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5694700 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 0x563f2fd42070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5694700 1 -- 172.21.15.19:0/363691724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f2fad6440 con 0x563f2fca5110 2024-05-09T01:25:39.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5e95700 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd47c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:25:39.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41e5694700 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd481b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f41e006c130 tx=0x7f41d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385376997dd191f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:39.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.797+0000 7f41c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:39.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41c97fa700 1 -- 172.21.15.19:0/363691724 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0028470 con 0x563f2fca5110 2024-05-09T01:25:39.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41c97fa700 1 -- 172.21.15.19:0/363691724 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f41d0028610 con 0x563f2fca5110 2024-05-09T01:25:39.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/363691724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41dc0ab190 con 0x563f2fca5110 2024-05-09T01:25:39.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41c97fa700 1 -- 172.21.15.19:0/363691724 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41d002e070 con 0x563f2fca5110 2024-05-09T01:25:39.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41f6b0fb80 10 client.15144 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:39.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/363691724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f2fd4f8f0 con 0x563f2fca5110 2024-05-09T01:25:39.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41c97fa700 1 -- 172.21.15.19:0/363691724 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41dc0ab190 con 0x563f2fca5110 2024-05-09T01:25:39.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41c97fa700 1 -- 172.21.15.19:0/363691724 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f41d001f810 con 0x563f2fca5110 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41f6b0fb80 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41f6b0fb80 10 client.15144 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41f6b0fb80 20 client.15144 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563f2fd51f10 0x563f2fd53380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41f6b0fb80 4 client.15144 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.798+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/363691724 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f2fc68980 con 0x563f2fd51f10 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.799+0000 7f41e5e95700 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563f2fd51f10 0x563f2fd53380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.799+0000 7f41e5e95700 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563f2fd51f10 0x563f2fd53380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c22c344b3f8cd657 server_cookie=9561456bf4a6fbb in_seq=0 out_seq=0 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.799+0000 7f41c97fa700 10 client.15144 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41c97fa700 1 -- 172.21.15.19:0/363691724 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f2fc68980 con 0x563f2fd51f10 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41c97fa700 10 client.15144 handle_command_reply: tid=0 2024-05-09T01:25:39.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 client.15144 shutdown 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/363691724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 msgr2=0x563f2fd481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd481b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f41e006c130 tx=0x7f41d00047c0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/363691724 shutdown_connections 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563f2fd51f10 0x563f2fd53380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fd38650 0x563f2fd42070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd47c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/363691724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fca5110 0x563f2fd481b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41c97fa700 0 client.15144 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/363691724 >> 172.21.15.19:0/363691724 conn(0x563f2fc1b650 msgr2=0x563f2fd33170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/363691724 shutdown_connections 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/363691724 wait complete. 2024-05-09T01:25:39.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.800+0000 7f41f6b0fb80 20 client.15144 trim_cache size 0 max 16384 2024-05-09T01:25:39.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.821+0000 7f41f6b0fb80 1 Processor -- start 2024-05-09T01:25:39.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.821+0000 7f41f6b0fb80 1 -- start start 2024-05-09T01:25:39.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd3c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fca5110 0x563f2fd3adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd40980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f2fd516c0 con 0x563f2fd38650 2024-05-09T01:25:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f2fd51d90 con 0x563f2fca5110 2024-05-09T01:25:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563f2fd52460 con 0x563f2fbbb450 2024-05-09T01:25:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e6696700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd40980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e6696700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd40980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60294/0 (socket says 172.21.15.19:60294) 2024-05-09T01:25:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e6696700 1 -- 172.21.15.19:0/1678207917 learned_addr learned my addr 172.21.15.19:0/1678207917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e5694700 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fca5110 0x563f2fd3adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e5e95700 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd3c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e6696700 1 -- 172.21.15.19:0/1678207917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 msgr2=0x563f2fd3c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e6696700 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd3c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e6696700 1 -- 172.21.15.19:0/1678207917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fca5110 msgr2=0x563f2fd3adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e6696700 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fca5110 0x563f2fd3adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e6696700 1 -- 172.21.15.19:0/1678207917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f2fd40600 con 0x563f2fd38650 2024-05-09T01:25:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e5694700 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fca5110 0x563f2fd3adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.822+0000 7f41e6696700 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd40980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f41d800df20 tx=0x7f41d8012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cd87e8bbfa042 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:39.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41cb7fe700 1 -- 172.21.15.19:0/1678207917 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8004830 con 0x563f2fd38650 2024-05-09T01:25:39.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41cb7fe700 1 -- 172.21.15.19:0/1678207917 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41d8005020 con 0x563f2fd38650 2024-05-09T01:25:39.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41cb7fe700 1 -- 172.21.15.19:0/1678207917 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d801b650 con 0x563f2fd38650 2024-05-09T01:25:39.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1678207917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 msgr2=0x563f2fd40980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:39.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd40980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f41d800df20 tx=0x7f41d8012730 comp rx=0 tx=0).stop 2024-05-09T01:25:39.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1678207917 shutdown_connections 2024-05-09T01:25:39.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd40980 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fbbb450 0x563f2fd3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1678207917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fca5110 0x563f2fd3adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1678207917 >> 172.21.15.19:0/1678207917 conn(0x563f2fc1b650 msgr2=0x563f2fd3e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1678207917 shutdown_connections 2024-05-09T01:25:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.823+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1678207917 wait complete. 2024-05-09T01:25:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.824+0000 7f41f6b0fb80 1 -- start start 2024-05-09T01:25:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.824+0000 7f41f6b0fb80 1 Processor -- start 2024-05-09T01:25:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fbbb450 0x563f2fd48b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fca5110 0x563f2fd490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41f6b0fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd4d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f2fd53200 con 0x563f2fd38650 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f2fd52b30 con 0x563f2fbbb450 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41f6b0fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563f2fd538d0 con 0x563f2fca5110 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41f6b0fb80 10 client.0 _ll_register_callbacks cb 0x563f2facc670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e6696700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd4d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e5e95700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fbbb450 0x563f2fd48b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e6696700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd4d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60304/0 (socket says 172.21.15.19:60304) 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e6696700 1 -- 172.21.15.19:0/1895973307 learned_addr learned my addr 172.21.15.19:0/1895973307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e5694700 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fca5110 0x563f2fd490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e6696700 1 -- 172.21.15.19:0/1895973307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fca5110 msgr2=0x563f2fd490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e6696700 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fca5110 0x563f2fd490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e6696700 1 -- 172.21.15.19:0/1895973307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fbbb450 msgr2=0x563f2fd48b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e6696700 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fbbb450 0x563f2fd48b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e6696700 1 -- 172.21.15.19:0/1895973307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f2fd4df00 con 0x563f2fd38650 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e5e95700 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fbbb450 0x563f2fd48b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41e6696700 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd4d9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f41d801d040 tx=0x7f41d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155c3cdc5fd04ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.825+0000 7f41c97fa700 1 -- 172.21.15.19:0/1895973307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8029020 con 0x563f2fd38650 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41c97fa700 1 -- 172.21.15.19:0/1895973307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f41d8004b80 con 0x563f2fd38650 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1895973307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41d8005020 con 0x563f2fd38650 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41c97fa700 1 -- 172.21.15.19:0/1895973307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41d8034050 con 0x563f2fd38650 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41f6b0fb80 10 client.15150 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1895973307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f2fdbd910 con 0x563f2fd38650 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41c97fa700 1 -- 172.21.15.19:0/1895973307 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41d8005020 con 0x563f2fd38650 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41c97fa700 1 -- 172.21.15.19:0/1895973307 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f41d801b430 con 0x563f2fd38650 2024-05-09T01:25:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41f6b0fb80 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41f6b0fb80 10 client.15150 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41f6b0fb80 20 client.15150 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563f2fdbfac0 0x563f2fdc0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41f6b0fb80 4 client.15150 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.826+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1895973307 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "status"}) -- 0x563f2fc62a20 con 0x563f2fdbfac0 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41e5e95700 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563f2fdbfac0 0x563f2fdc0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41e5e95700 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563f2fdbfac0 0x563f2fdc0f30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb790fe12421c92d server_cookie=cbe0e04c01cb1b96 in_seq=0 out_seq=0 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41c97fa700 10 client.15150 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41c97fa700 1 -- 172.21.15.19:0/1895973307 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563f2fc62a20 con 0x563f2fdbfac0 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41c97fa700 10 client.15150 handle_command_reply: tid=0 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41f6b0fb80 1 client.15150 shutdown 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1895973307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 msgr2=0x563f2fd4d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd4d9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f41d801d040 tx=0x7f41d8004750 comp rx=0 tx=0).stop 2024-05-09T01:25:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1895973307 shutdown_connections 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563f2fdbfac0 0x563f2fdc0f30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f2fd38650 0x563f2fd4d9c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f2fbbb450 0x563f2fd48b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41f6b0fb80 1 --2- 172.21.15.19:0/1895973307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563f2fca5110 0x563f2fd490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41c97fa700 0 client.15150 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.827+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1895973307 >> 172.21.15.19:0/1895973307 conn(0x563f2fc1b650 msgr2=0x563f2fd40810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1895973307 shutdown_connections 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41f6b0fb80 1 -- 172.21.15.19:0/1895973307 wait complete. 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41f6b0fb80 20 client.15150 trim_cache size 0 max 16384 2024-05-09T01:25:39.834 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:25:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:25:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-05-09T01:25:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24451, 2024-05-09T01:25:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:25:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:25:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:25:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 161.59224210900001, 2024-05-09T01:25:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:25:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:25:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:25:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 209.25647079500001 2024-05-09T01:25:39.836 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 -- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41cc0b2900 msgr2=0x7f41cc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41cc0b2900 0x7f41cc0b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f41e0108990 tx=0x7f41d000b410 comp rx=0 tx=0).stop 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 -- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 msgr2=0x7f41e0076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 0x7f41e0076bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00d010 tx=0x7f41dc007a20 comp rx=0 tx=0).stop 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 -- 172.21.15.19:0/295492442 shutdown_connections 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e0109850 0x7f41e0077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41cc0b2900 0x7f41cc0b4e10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41e01045d0 0x7f41e0076bd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 --2- 172.21.15.19:0/295492442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e0108f00 0x7f41e0077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 -- 172.21.15.19:0/295492442 >> 172.21.15.19:0/295492442 conn(0x7f41e00ffe80 msgr2=0x7f41e01022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:39.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 -- 172.21.15.19:0/295492442 shutdown_connections 2024-05-09T01:25:39.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:39.828+0000 7f41e80f9700 1 -- 172.21.15.19:0/295492442 wait complete. 2024-05-09T01:25:40.630 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.e status 2024-05-09T01:25:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:40 smithi059 ceph-mon[34351]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-09T01:25:40.886 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:40.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:40 smithi019 ceph-mon[30158]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-09T01:25:40.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:40 smithi196 ceph-mon[35041]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-09T01:25:42.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 -- 172.21.15.19:0/1943694208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99341034c0 msgr2=0x7f99341038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 --2- 172.21.15.19:0/1943694208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99341034c0 0x7f99341038d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f99240099c0 tx=0x7f9924009cd0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 -- 172.21.15.19:0/1943694208 shutdown_connections 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 --2- 172.21.15.19:0/1943694208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9934108740 0x7f99341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 --2- 172.21.15.19:0/1943694208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99341034c0 0x7f99341038d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 --2- 172.21.15.19:0/1943694208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9934107df0 0x7f9934108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 -- 172.21.15.19:0/1943694208 >> 172.21.15.19:0/1943694208 conn(0x7f99340fedf0 msgr2=0x7f9934101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 -- 172.21.15.19:0/1943694208 shutdown_connections 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 -- 172.21.15.19:0/1943694208 wait complete. 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.610+0000 7f9939bc0700 1 Processor -- start 2024-05-09T01:25:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9939bc0700 1 -- start start 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9939bc0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99341034c0 0x7f993419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9939bc0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9934107df0 0x7f993419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9939bc0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9934108740 0x7f993419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9939bc0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9934113370 con 0x7f9934108740 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9939bc0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f99341131f0 con 0x7f99341034c0 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9939bc0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99341134f0 con 0x7f9934107df0 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9934108740 0x7f993419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9934108740 0x7f993419f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60318/0 (socket says 172.21.15.19:60318) 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9933fff700 1 -- 172.21.15.19:0/37890137 learned_addr learned my addr 172.21.15.19:0/37890137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f99337fe700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99341034c0 0x7f993419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9932ffd700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9934107df0 0x7f993419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9933fff700 1 -- 172.21.15.19:0/37890137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9934107df0 msgr2=0x7f993419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9933fff700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9934107df0 0x7f993419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9933fff700 1 -- 172.21.15.19:0/37890137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99341034c0 msgr2=0x7f993419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9933fff700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99341034c0 0x7f993419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.611+0000 7f9933fff700 1 -- 172.21.15.19:0/37890137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993419fd20 con 0x7f9934108740 2024-05-09T01:25:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.612+0000 7f9933fff700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9934108740 0x7f993419f680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f992c007ee0 tx=0x7f992c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.612+0000 7f9930ff9700 1 -- 172.21.15.19:0/37890137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c00dbd0 con 0x7f9934108740 2024-05-09T01:25:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.612+0000 7f9930ff9700 1 -- 172.21.15.19:0/37890137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c00dd70 con 0x7f9934108740 2024-05-09T01:25:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.612+0000 7f9939bc0700 1 -- 172.21.15.19:0/37890137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99341a0010 con 0x7f9934108740 2024-05-09T01:25:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.612+0000 7f9930ff9700 1 -- 172.21.15.19:0/37890137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c005990 con 0x7f9934108740 2024-05-09T01:25:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.612+0000 7f9939bc0700 1 -- 172.21.15.19:0/37890137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99341a7930 con 0x7f9934108740 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.615+0000 7f9930ff9700 1 -- 172.21.15.19:0/37890137 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f992c01a020 con 0x7f9934108740 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.615+0000 7f9930ff9700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f991c0b2880 0x7f991c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.615+0000 7f99337fe700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f991c0b2880 0x7f991c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f9930ff9700 1 -- 172.21.15.19:0/37890137 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f992c0e1990 con 0x7f9934108740 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99485d6b80 1 Processor -- start 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99485d6b80 1 -- start start 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99337fe700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f991c0b2880 0x7f991c0b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9924009990 tx=0x7f99240036e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf30961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 0x557bf3096730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf30907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99485d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bf2fb3050 con 0x557bf2f8dcb0 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99485d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bf2f58950 con 0x557bf30903b0 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99485d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bf2faf010 con 0x557bf2f13450 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf30961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf30961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60334/0 (socket says 172.21.15.19:60334) 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99337fe700 1 -- 172.21.15.19:0/1482303569 learned_addr learned my addr 172.21.15.19:0/1482303569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f9932ffd700 1 --2- 172.21.15.19:0/1482303569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 0x557bf3096730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f9933fff700 1 --2- 172.21.15.19:0/1482303569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf30907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99337fe700 1 -- 172.21.15.19:0/1482303569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 msgr2=0x557bf3096730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99337fe700 1 --2- 172.21.15.19:0/1482303569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 0x557bf3096730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99337fe700 1 -- 172.21.15.19:0/1482303569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 msgr2=0x557bf30907c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.616+0000 7f99337fe700 1 --2- 172.21.15.19:0/1482303569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf30907c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.617+0000 7f99337fe700 1 -- 172.21.15.19:0/1482303569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf2f494b0 con 0x557bf2f8dcb0 2024-05-09T01:25:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.617+0000 7f99337fe700 1 --2- 172.21.15.19:0/1482303569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf30961f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99240099c0 tx=0x7f992401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e747ab45875fd1f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.617+0000 7f9919ffb700 1 -- 172.21.15.19:0/1482303569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924005430 con 0x557bf2f8dcb0 2024-05-09T01:25:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.617+0000 7f9919ffb700 1 -- 172.21.15.19:0/1482303569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99240055d0 con 0x557bf2f8dcb0 2024-05-09T01:25:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.617+0000 7f9919ffb700 1 -- 172.21.15.19:0/1482303569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992400fa20 con 0x557bf2f8dcb0 2024-05-09T01:25:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 -- 172.21.15.19:0/1482303569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 msgr2=0x557bf30961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1482303569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf30961f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99240099c0 tx=0x7f992401e000 comp rx=0 tx=0).stop 2024-05-09T01:25:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 -- 172.21.15.19:0/1482303569 shutdown_connections 2024-05-09T01:25:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1482303569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf30907c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1482303569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf30961f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1482303569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 0x557bf3096730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 -- 172.21.15.19:0/1482303569 >> 172.21.15.19:0/1482303569 conn(0x557bf2f16f20 msgr2=0x557bf308b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 -- 172.21.15.19:0/1482303569 shutdown_connections 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 -- 172.21.15.19:0/1482303569 wait complete. 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.618+0000 7f99485d6b80 1 -- start start 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f99485d6b80 1 Processor -- start 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 0x557bf309a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf309a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf309aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f99485d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bf3091050 con 0x557bf2f8dcb0 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f99485d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bf2f4dad0 con 0x557bf30903b0 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f99485d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bf30911d0 con 0x557bf2f13450 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f9932ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf309a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f9932ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf309a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60336/0 (socket says 172.21.15.19:60336) 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f9932ffd700 1 -- 172.21.15.19:0/1241839772 learned_addr learned my addr 172.21.15.19:0/1241839772 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f99337fe700 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 0x557bf309a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f99485d6b80 10 client.0 _ll_register_callbacks cb 0x557bf2e24670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.619+0000 7f9933fff700 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf309aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9932ffd700 1 -- 172.21.15.19:0/1241839772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 msgr2=0x557bf309a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9932ffd700 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 0x557bf309a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9932ffd700 1 -- 172.21.15.19:0/1241839772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 msgr2=0x557bf309aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9932ffd700 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf309aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9932ffd700 1 -- 172.21.15.19:0/1241839772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf2e2e440 con 0x557bf2f8dcb0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9933fff700 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf309aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9932ffd700 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf309a540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9928011820 tx=0x7f9928011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89eea78efebcf55d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9912ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9912ffd700 1 -- 172.21.15.19:0/1241839772 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928004220 con 0x557bf2f8dcb0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f99485d6b80 1 -- 172.21.15.19:0/1241839772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99240055d0 con 0x557bf2f8dcb0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9912ffd700 1 -- 172.21.15.19:0/1241839772 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f99280043c0 con 0x557bf2f8dcb0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9912ffd700 1 -- 172.21.15.19:0/1241839772 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9928004830 con 0x557bf2f8dcb0 2024-05-09T01:25:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f99485d6b80 10 client.15162 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f99485d6b80 1 -- 172.21.15.19:0/1241839772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf30a79c0 con 0x557bf2f8dcb0 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.620+0000 7f9912ffd700 1 -- 172.21.15.19:0/1241839772 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99240055d0 con 0x557bf2f8dcb0 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.621+0000 7f9912ffd700 1 -- 172.21.15.19:0/1241839772 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9928005020 con 0x557bf2f8dcb0 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.621+0000 7f99485d6b80 10 client.15162 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.621+0000 7f99485d6b80 10 client.15162 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.621+0000 7f99485d6b80 20 client.15162 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.621+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x557bf30aa050 0x557bf30ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.621+0000 7f99485d6b80 4 client.15162 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.621+0000 7f99485d6b80 1 -- 172.21.15.19:0/1241839772 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bf2fc0a30 con 0x557bf30aa050 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.621+0000 7f99337fe700 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x557bf30aa050 0x557bf30ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.622+0000 7f9912ffd700 10 client.15162 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.622+0000 7f99337fe700 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x557bf30aa050 0x557bf30ab4c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73a96c79217f53c server_cookie=d2134961982382a8 in_seq=0 out_seq=0 2024-05-09T01:25:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.622+0000 7f9912ffd700 1 -- 172.21.15.19:0/1241839772 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557bf2fc0a30 con 0x557bf30aa050 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.622+0000 7f9912ffd700 10 client.15162 handle_command_reply: tid=0 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.622+0000 7f99485d6b80 1 client.15162 shutdown 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 -- 172.21.15.19:0/1241839772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 msgr2=0x557bf309a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf309a540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9928011820 tx=0x7f9928011b30 comp rx=0 tx=0).stop 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 -- 172.21.15.19:0/1241839772 shutdown_connections 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x557bf30aa050 0x557bf30ab4c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf309aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f13450 0x557bf309a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1241839772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f8dcb0 0x557bf309a540 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f9912ffd700 0 client.15162 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 -- 172.21.15.19:0/1241839772 >> 172.21.15.19:0/1241839772 conn(0x557bf2f16f20 msgr2=0x557bf3084fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 -- 172.21.15.19:0/1241839772 shutdown_connections 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 1 -- 172.21.15.19:0/1241839772 wait complete. 2024-05-09T01:25:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.623+0000 7f99485d6b80 20 client.15162 trim_cache size 0 max 16384 2024-05-09T01:25:42.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.644+0000 7f99485d6b80 1 Processor -- start 2024-05-09T01:25:42.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.644+0000 7f99485d6b80 1 -- start start 2024-05-09T01:25:42.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:42 smithi019 ceph-mon[30158]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-05-09T01:25:42.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.644+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf3095800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.644+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 0x557bf3097200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.644+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf3097740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99485d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bf30a9800 con 0x557bf2f13450 2024-05-09T01:25:42.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99485d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bf30a9ed0 con 0x557bf30903b0 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99485d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bf30aa5a0 con 0x557bf2f8dcb0 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf3095800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf3095800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60350/0 (socket says 172.21.15.19:60350) 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99337fe700 1 -- 172.21.15.19:0/3656277247 learned_addr learned my addr 172.21.15.19:0/3656277247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f9932ffd700 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 0x557bf3097200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f9933fff700 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf3097740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99337fe700 1 -- 172.21.15.19:0/3656277247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 msgr2=0x557bf3097200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99337fe700 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 0x557bf3097200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99337fe700 1 -- 172.21.15.19:0/3656277247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 msgr2=0x557bf3097740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99337fe700 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf3097740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99337fe700 1 -- 172.21.15.19:0/3656277247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf3087980 con 0x557bf2f13450 2024-05-09T01:25:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f9932ffd700 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 0x557bf3097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99337fe700 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf3095800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9924004010 tx=0x7f992401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74715052a4dd2f08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.645+0000 7f99197fa700 1 -- 172.21.15.19:0/3656277247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992403c070 con 0x557bf2f13450 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.646+0000 7f99197fa700 1 -- 172.21.15.19:0/3656277247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992400b790 con 0x557bf2f13450 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.646+0000 7f99197fa700 1 -- 172.21.15.19:0/3656277247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992403c210 con 0x557bf2f13450 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.646+0000 7f99485d6b80 1 -- 172.21.15.19:0/3656277247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 msgr2=0x557bf3095800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.646+0000 7f99485d6b80 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf3095800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9924004010 tx=0x7f992401a040 comp rx=0 tx=0).stop 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.646+0000 7f99485d6b80 1 -- 172.21.15.19:0/3656277247 shutdown_connections 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.646+0000 7f99485d6b80 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf3097740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.646+0000 7f99485d6b80 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf3095800 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.646+0000 7f99485d6b80 1 --2- 172.21.15.19:0/3656277247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 0x557bf3097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.646+0000 7f99485d6b80 1 -- 172.21.15.19:0/3656277247 >> 172.21.15.19:0/3656277247 conn(0x557bf2f16f20 msgr2=0x557bf30950f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:42.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.647+0000 7f99485d6b80 1 -- 172.21.15.19:0/3656277247 shutdown_connections 2024-05-09T01:25:42.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.647+0000 7f99485d6b80 1 -- 172.21.15.19:0/3656277247 wait complete. 2024-05-09T01:25:42.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.647+0000 7f99485d6b80 1 -- start start 2024-05-09T01:25:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.647+0000 7f99485d6b80 1 Processor -- start 2024-05-09T01:25:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.647+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf30aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.647+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 0x557bf30af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99485d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf30af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99485d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bf30ab340 con 0x557bf2f13450 2024-05-09T01:25:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99485d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bf30aac70 con 0x557bf30903b0 2024-05-09T01:25:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99485d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bf30aba10 con 0x557bf2f8dcb0 2024-05-09T01:25:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf30aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99485d6b80 10 client.0 _ll_register_callbacks cb 0x557bf2e24670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf30aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60352/0 (socket says 172.21.15.19:60352) 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99337fe700 1 -- 172.21.15.19:0/1594707275 learned_addr learned my addr 172.21.15.19:0/1594707275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f9933fff700 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf30af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f9932ffd700 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 0x557bf30af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99337fe700 1 -- 172.21.15.19:0/1594707275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 msgr2=0x557bf30af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99337fe700 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 0x557bf30af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99337fe700 1 -- 172.21.15.19:0/1594707275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 msgr2=0x557bf30af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99337fe700 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf30af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99337fe700 1 -- 172.21.15.19:0/1594707275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf30b40a0 con 0x557bf2f13450 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f99337fe700 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf30aeec0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f99240092d0 tx=0x7f992400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6c2a4de4ebee78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f9912ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.648+0000 7f9912ffd700 1 -- 172.21.15.19:0/1594707275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992403c040 con 0x557bf2f13450 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f99485d6b80 1 -- 172.21.15.19:0/1594707275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f992400b790 con 0x557bf2f13450 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f9912ffd700 1 -- 172.21.15.19:0/1594707275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f992400f940 con 0x557bf2f13450 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f9912ffd700 1 -- 172.21.15.19:0/1594707275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f992403c1e0 con 0x557bf2f13450 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f99485d6b80 10 client.15168 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f99485d6b80 1 -- 172.21.15.19:0/1594707275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf3119c10 con 0x557bf2f13450 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f9912ffd700 1 -- 172.21.15.19:0/1594707275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992400b790 con 0x557bf2f13450 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f9912ffd700 1 -- 172.21.15.19:0/1594707275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9924022070 con 0x557bf2f13450 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f99485d6b80 10 client.15168 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f99485d6b80 10 client.15168 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f99485d6b80 20 client.15168 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x557bf311b930 0x557bf311cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f99485d6b80 4 client.15168 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.649+0000 7f99485d6b80 1 -- 172.21.15.19:0/1594707275 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "status"}) -- 0x557bf2fbaad0 con 0x557bf311b930 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.650+0000 7f9932ffd700 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x557bf311b930 0x557bf311cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.650+0000 7f9912ffd700 10 client.15168 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.650+0000 7f9932ffd700 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x557bf311b930 0x557bf311cda0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6bad4f41cfa92420 server_cookie=eab8031002c71dd in_seq=0 out_seq=0 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.650+0000 7f9912ffd700 1 -- 172.21.15.19:0/1594707275 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557bf2fbaad0 con 0x557bf311b930 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.650+0000 7f9912ffd700 10 client.15168 handle_command_reply: tid=0 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.650+0000 7f99485d6b80 1 client.15168 shutdown 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 -- 172.21.15.19:0/1594707275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 msgr2=0x557bf30aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf30aeec0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f99240092d0 tx=0x7f992400bd30 comp rx=0 tx=0).stop 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 -- 172.21.15.19:0/1594707275 shutdown_connections 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x557bf311b930 0x557bf311cda0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f9912ffd700 0 client.15168 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:25:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf30903b0 0x557bf30af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bf2f13450 0x557bf30aeec0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 --2- 172.21.15.19:0/1594707275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bf2f8dcb0 0x557bf30af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 -- 172.21.15.19:0/1594707275 >> 172.21.15.19:0/1594707275 conn(0x557bf2f16f20 msgr2=0x557bf30984e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 -- 172.21.15.19:0/1594707275 shutdown_connections 2024-05-09T01:25:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 1 -- 172.21.15.19:0/1594707275 wait complete. 2024-05-09T01:25:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.651+0000 7f99485d6b80 20 client.15168 trim_cache size 0 max 16384 2024-05-09T01:25:42.654 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:25:42.655 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:25:42.655 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-05-09T01:25:42.655 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24452, 2024-05-09T01:25:42.655 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:25:42.655 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:25:42.655 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:25:42.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 165.42657443600001, 2024-05-09T01:25:42.655 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:25:42.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:25:42.656 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:25:42.656 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 217.461586716 2024-05-09T01:25:42.656 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:25:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 -- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f991c0b2880 msgr2=0x7f991c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f991c0b2880 0x7f991c0b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9924009990 tx=0x7f99240036e0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 -- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9934108740 msgr2=0x7f993419f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9934108740 0x7f993419f680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f992c007ee0 tx=0x7f992c00d3b0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 -- 172.21.15.19:0/37890137 shutdown_connections 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f991c0b2880 0x7f991c0b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9934108740 0x7f993419f680 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99341034c0 0x7f993419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 --2- 172.21.15.19:0/37890137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9934107df0 0x7f993419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 -- 172.21.15.19:0/37890137 >> 172.21.15.19:0/37890137 conn(0x7f99340fedf0 msgr2=0x7f9934075db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 -- 172.21.15.19:0/37890137 shutdown_connections 2024-05-09T01:25:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:42.652+0000 7f9939bc0700 1 -- 172.21.15.19:0/37890137 wait complete. 2024-05-09T01:25:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:42 smithi059 ceph-mon[34351]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-05-09T01:25:42.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:42 smithi196 ceph-mon[35041]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-05-09T01:25:43.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T01:25:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:44 smithi059 ceph-mon[34351]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:25:44.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:44 smithi019 ceph-mon[30158]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:25:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:44 smithi196 ceph-mon[35041]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:25:45.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.448+0000 7f24658b5700 1 -- 172.21.15.19:0/2130660211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2460109850 msgr2=0x7f24601111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:45.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.448+0000 7f24658b5700 1 --2- 172.21.15.19:0/2130660211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2460109850 0x7f24601111b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f245800b0d0 tx=0x7f245800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.448+0000 7f24658b5700 1 -- 172.21.15.19:0/2130660211 shutdown_connections 2024-05-09T01:25:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.448+0000 7f24658b5700 1 --2- 172.21.15.19:0/2130660211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2460109850 0x7f24601111b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.448+0000 7f24658b5700 1 --2- 172.21.15.19:0/2130660211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24601045d0 0x7f24601049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.448+0000 7f24658b5700 1 --2- 172.21.15.19:0/2130660211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2460108f00 0x7f2460109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.448+0000 7f24658b5700 1 -- 172.21.15.19:0/2130660211 >> 172.21.15.19:0/2130660211 conn(0x7f24600ffea0 msgr2=0x7f24601022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.448+0000 7f24658b5700 1 -- 172.21.15.19:0/2130660211 shutdown_connections 2024-05-09T01:25:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.448+0000 7f24658b5700 1 -- 172.21.15.19:0/2130660211 wait complete. 2024-05-09T01:25:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f24658b5700 1 Processor -- start 2024-05-09T01:25:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f24658b5700 1 -- start start 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f24658b5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24601045d0 0x7f246019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f24658b5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2460108f00 0x7f246019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f24658b5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2460109850 0x7f246019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f24658b5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2460114500 con 0x7f24601045d0 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f24658b5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2460114380 con 0x7f2460108f00 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f24658b5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2460114680 con 0x7f2460109850 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f245effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24601045d0 0x7f246019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f245effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24601045d0 0x7f246019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60364/0 (socket says 172.21.15.19:60364) 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f245effd700 1 -- 172.21.15.19:0/3575000334 learned_addr learned my addr 172.21.15.19:0/3575000334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.449+0000 7f245e7fc700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2460108f00 0x7f246019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f245f7fe700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2460109850 0x7f246019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f245effd700 1 -- 172.21.15.19:0/3575000334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2460109850 msgr2=0x7f246019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f245effd700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2460109850 0x7f246019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f245effd700 1 -- 172.21.15.19:0/3575000334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2460108f00 msgr2=0x7f246019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f245effd700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2460108f00 0x7f246019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f245effd700 1 -- 172.21.15.19:0/3575000334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246019fce0 con 0x7f24601045d0 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f245f7fe700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2460109850 0x7f246019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f245effd700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24601045d0 0x7f246019ae10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f245000b530 tx=0x7f245000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f24648b3700 1 -- 172.21.15.19:0/3575000334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450015020 con 0x7f24601045d0 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f24658b5700 1 -- 172.21.15.19:0/3575000334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246019ffd0 con 0x7f24601045d0 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f24648b3700 1 -- 172.21.15.19:0/3575000334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2450011ce0 con 0x7f24601045d0 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.450+0000 7f24658b5700 1 -- 172.21.15.19:0/3575000334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24601a78f0 con 0x7f24601045d0 2024-05-09T01:25:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.451+0000 7f24648b3700 1 -- 172.21.15.19:0/3575000334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450010e40 con 0x7f24601045d0 2024-05-09T01:25:45.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.453+0000 7f24648b3700 1 -- 172.21.15.19:0/3575000334 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2450018020 con 0x7f24601045d0 2024-05-09T01:25:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.454+0000 7f24648b3700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24480b2900 0x7f24480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.454+0000 7f245e7fc700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24480b2900 0x7f24480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.454+0000 7f24648b3700 1 -- 172.21.15.19:0/3575000334 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f24500e1930 con 0x7f24601045d0 2024-05-09T01:25:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.454+0000 7f24742cbb80 1 Processor -- start 2024-05-09T01:25:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.454+0000 7f24742cbb80 1 -- start start 2024-05-09T01:25:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24480b2900 0x7f24480b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2460103440 tx=0x7f245400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:45.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc69730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f24742cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bb86050 con 0x55ec6bae6450 2024-05-09T01:25:45.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f24742cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bb2b950 con 0x55ec6bc633b0 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f24742cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bb82010 con 0x55ec6bb60cb0 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc69730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc69730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60370/0 (socket says 172.21.15.19:60370) 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 -- 172.21.15.19:0/2930266475 learned_addr learned my addr 172.21.15.19:0/2930266475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245effd700 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245f7fe700 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 -- 172.21.15.19:0/2930266475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 msgr2=0x55ec6bc691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 -- 172.21.15.19:0/2930266475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 msgr2=0x55ec6bc637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 -- 172.21.15.19:0/2930266475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6bb1c4b0 con 0x55ec6bae6450 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245f7fe700 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f245e7fc700 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc69730 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f245400fd70 tx=0x7f2454012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d970899ef0f7947d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.455+0000 7f2445ffb700 1 -- 172.21.15.19:0/2930266475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454016070 con 0x55ec6bae6450 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.456+0000 7f2445ffb700 1 -- 172.21.15.19:0/2930266475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245400fe50 con 0x55ec6bae6450 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.456+0000 7f2445ffb700 1 -- 172.21.15.19:0/2930266475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245401b420 con 0x55ec6bae6450 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.456+0000 7f24742cbb80 1 -- 172.21.15.19:0/2930266475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 msgr2=0x55ec6bc69730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.456+0000 7f24742cbb80 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc69730 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f245400fd70 tx=0x7f2454012000 comp rx=0 tx=0).stop 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.456+0000 7f24742cbb80 1 -- 172.21.15.19:0/2930266475 shutdown_connections 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.456+0000 7f24742cbb80 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.456+0000 7f24742cbb80 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.456+0000 7f24742cbb80 1 --2- 172.21.15.19:0/2930266475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc69730 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.456+0000 7f24742cbb80 1 -- 172.21.15.19:0/2930266475 >> 172.21.15.19:0/2930266475 conn(0x55ec6bae9f20 msgr2=0x55ec6bc5e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.457+0000 7f24742cbb80 1 -- 172.21.15.19:0/2930266475 shutdown_connections 2024-05-09T01:25:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.457+0000 7f24742cbb80 1 -- 172.21.15.19:0/2930266475 wait complete. 2024-05-09T01:25:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.457+0000 7f24742cbb80 1 -- start start 2024-05-09T01:25:45.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.457+0000 7f24742cbb80 1 Processor -- start 2024-05-09T01:25:45.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.457+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bae6450 0x55ec6bc6cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f24742cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bc64110 con 0x55ec6bc633b0 2024-05-09T01:25:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f24742cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bb20ad0 con 0x55ec6bae6450 2024-05-09T01:25:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f24742cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bc6ffd0 con 0x55ec6bb60cb0 2024-05-09T01:25:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f24742cbb80 10 client.0 _ll_register_callbacks cb 0x55ec6b9f7670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60376/0 (socket says 172.21.15.19:60376) 2024-05-09T01:25:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245f7fe700 1 -- 172.21.15.19:0/1964596866 learned_addr learned my addr 172.21.15.19:0/1964596866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245e7fc700 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245effd700 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bae6450 0x55ec6bc6cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245f7fe700 1 -- 172.21.15.19:0/1964596866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 msgr2=0x55ec6bc6d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245f7fe700 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245f7fe700 1 -- 172.21.15.19:0/1964596866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bae6450 msgr2=0x55ec6bc6cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245f7fe700 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bae6450 0x55ec6bc6cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245f7fe700 1 -- 172.21.15.19:0/1964596866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec6ba01440 con 0x55ec6bc633b0 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245e7fc700 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f245f7fe700 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2458004d00 tx=0x7f2458004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11df66927c10f1d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f243effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.458+0000 7f243effd700 1 -- 172.21.15.19:0/1964596866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458020070 con 0x55ec6bc633b0 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f243effd700 1 -- 172.21.15.19:0/1964596866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f24580079d0 con 0x55ec6bc633b0 2024-05-09T01:25:45.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f24742cbb80 1 -- 172.21.15.19:0/1964596866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f245400fe50 con 0x55ec6bc633b0 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f243effd700 1 -- 172.21.15.19:0/1964596866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2458007bd0 con 0x55ec6bc633b0 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f24742cbb80 10 client.15180 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f24742cbb80 1 -- 172.21.15.19:0/1964596866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec6bc7a8e0 con 0x55ec6bc633b0 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f243effd700 1 -- 172.21.15.19:0/1964596866 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245400fe50 con 0x55ec6bc633b0 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f243effd700 1 -- 172.21.15.19:0/1964596866 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2458011070 con 0x55ec6bc633b0 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f24742cbb80 10 client.15180 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f24742cbb80 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f24742cbb80 20 client.15180 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f24742cbb80 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ec6bc7cf90 0x55ec6bc7e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f24742cbb80 4 client.15180 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f24742cbb80 1 -- 172.21.15.19:0/1964596866 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec6bb93a30 con 0x55ec6bc7cf90 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.459+0000 7f245effd700 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ec6bc7cf90 0x55ec6bc7e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.460+0000 7f243effd700 10 client.15180 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.460+0000 7f245effd700 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ec6bc7cf90 0x55ec6bc7e400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e98a141faa39b14b server_cookie=b4dc775618ce4791 in_seq=0 out_seq=0 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f243effd700 1 -- 172.21.15.19:0/1964596866 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec6bb93a30 con 0x55ec6bc7cf90 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f243effd700 10 client.15180 handle_command_reply: tid=0 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 client.15180 shutdown 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 -- 172.21.15.19:0/1964596866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bc633b0 msgr2=0x55ec6bc6d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2458004d00 tx=0x7f2458004de0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 -- 172.21.15.19:0/1964596866 shutdown_connections 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ec6bc7cf90 0x55ec6bc7e400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bae6450 0x55ec6bc6cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 --2- 172.21.15.19:0/1964596866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 -- 172.21.15.19:0/1964596866 >> 172.21.15.19:0/1964596866 conn(0x55ec6bae9f20 msgr2=0x55ec6bc57fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:45.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f243effd700 0 client.15180 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:25:45.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 -- 172.21.15.19:0/1964596866 shutdown_connections 2024-05-09T01:25:45.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 1 -- 172.21.15.19:0/1964596866 wait complete. 2024-05-09T01:25:45.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.461+0000 7f24742cbb80 20 client.15180 trim_cache size 0 max 16384 2024-05-09T01:25:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f24742cbb80 1 Processor -- start 2024-05-09T01:25:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f24742cbb80 1 -- start start 2024-05-09T01:25:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc79230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc87860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f24742cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bc7c740 con 0x55ec6bae6450 2024-05-09T01:25:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f24742cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bc7ce10 con 0x55ec6bc633b0 2024-05-09T01:25:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f24742cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bc7d4e0 con 0x55ec6bb60cb0 2024-05-09T01:25:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc79230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc79230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60384/0 (socket says 172.21.15.19:60384) 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245effd700 1 -- 172.21.15.19:0/727584659 learned_addr learned my addr 172.21.15.19:0/727584659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245f7fe700 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc87860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245e7fc700 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245effd700 1 -- 172.21.15.19:0/727584659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 msgr2=0x55ec6bc6d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245effd700 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245effd700 1 -- 172.21.15.19:0/727584659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 msgr2=0x55ec6bc87860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245effd700 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc87860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245effd700 1 -- 172.21.15.19:0/727584659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6bc6e370 con 0x55ec6bae6450 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.483+0000 7f245f7fe700 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc87860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.484+0000 7f245effd700 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc79230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f245000bc40 tx=0x7f2450004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e358f27c3bcf472e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.484+0000 7f24457fa700 1 -- 172.21.15.19:0/727584659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500b1d90 con 0x55ec6bae6450 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.484+0000 7f24457fa700 1 -- 172.21.15.19:0/727584659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24500df520 con 0x55ec6bae6450 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.484+0000 7f24457fa700 1 -- 172.21.15.19:0/727584659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500a48a0 con 0x55ec6bae6450 2024-05-09T01:25:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 -- 172.21.15.19:0/727584659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 msgr2=0x55ec6bc79230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc79230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f245000bc40 tx=0x7f2450004860 comp rx=0 tx=0).stop 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 -- 172.21.15.19:0/727584659 shutdown_connections 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc87860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc79230 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 --2- 172.21.15.19:0/727584659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 -- 172.21.15.19:0/727584659 >> 172.21.15.19:0/727584659 conn(0x55ec6bae9f20 msgr2=0x55ec6bc5dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 -- 172.21.15.19:0/727584659 shutdown_connections 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 -- 172.21.15.19:0/727584659 wait complete. 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 -- start start 2024-05-09T01:25:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.485+0000 7f24742cbb80 1 Processor -- start 2024-05-09T01:25:45.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc87f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f24742cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f24742cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bc7e280 con 0x55ec6bae6450 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f24742cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bc7dbb0 con 0x55ec6bc633b0 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc87f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f24742cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bc7e950 con 0x55ec6bb60cb0 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc87f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60398/0 (socket says 172.21.15.19:60398) 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245effd700 1 -- 172.21.15.19:0/379348142 learned_addr learned my addr 172.21.15.19:0/379348142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f24742cbb80 10 client.0 _ll_register_callbacks cb 0x55ec6b9f7670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245e7fc700 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245f7fe700 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245effd700 1 -- 172.21.15.19:0/379348142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 msgr2=0x55ec6bc6c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245effd700 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245effd700 1 -- 172.21.15.19:0/379348142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 msgr2=0x55ec6bc6ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245effd700 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245effd700 1 -- 172.21.15.19:0/379348142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec6bc6d5d0 con 0x55ec6bae6450 2024-05-09T01:25:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245f7fe700 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f245effd700 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc87f20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24500a8c30 tx=0x7f24500a97a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d05df7d85281bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.486+0000 7f243effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f243effd700 1 -- 172.21.15.19:0/379348142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500b1b10 con 0x55ec6bae6450 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f24742cbb80 1 -- 172.21.15.19:0/379348142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24500df520 con 0x55ec6bae6450 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f243effd700 1 -- 172.21.15.19:0/379348142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2450011830 con 0x55ec6bae6450 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f243effd700 1 -- 172.21.15.19:0/379348142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2450011a30 con 0x55ec6bae6450 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f24742cbb80 10 client.15186 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f24742cbb80 1 -- 172.21.15.19:0/379348142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec6bce8400 con 0x55ec6bae6450 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f243effd700 1 -- 172.21.15.19:0/379348142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24500df520 con 0x55ec6bae6450 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f243effd700 1 -- 172.21.15.19:0/379348142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f24500ae470 con 0x55ec6bae6450 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f24742cbb80 10 client.15186 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f24742cbb80 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f24742cbb80 20 client.15186 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f24742cbb80 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ec6bcea350 0x55ec6bceb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f24742cbb80 4 client.15186 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f24742cbb80 1 -- 172.21.15.19:0/379348142 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "status"}) -- 0x55ec6bb8dad0 con 0x55ec6bcea350 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.487+0000 7f245e7fc700 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ec6bcea350 0x55ec6bceb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f245e7fc700 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ec6bcea350 0x55ec6bceb7c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d4d35c5fad84a21 server_cookie=91db50c863be549d in_seq=0 out_seq=0 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f243effd700 10 client.15186 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f243effd700 1 -- 172.21.15.19:0/379348142 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ec6bb8dad0 con 0x55ec6bcea350 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f243effd700 10 client.15186 handle_command_reply: tid=0 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f24742cbb80 1 client.15186 shutdown 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f24742cbb80 1 -- 172.21.15.19:0/379348142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 msgr2=0x55ec6bc87f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f24742cbb80 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc87f20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24500a8c30 tx=0x7f24500a97a0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f24742cbb80 1 -- 172.21.15.19:0/379348142 shutdown_connections 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f24742cbb80 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ec6bcea350 0x55ec6bceb7c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f24742cbb80 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec6bc633b0 0x55ec6bc6ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f24742cbb80 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bae6450 0x55ec6bc87f20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f24742cbb80 1 --2- 172.21.15.19:0/379348142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec6bb60cb0 0x55ec6bc6c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f24742cbb80 1 -- 172.21.15.19:0/379348142 >> 172.21.15.19:0/379348142 conn(0x55ec6bae9f20 msgr2=0x55ec6bc66c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.488+0000 7f243effd700 0 client.15186 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24742cbb80 1 -- 172.21.15.19:0/379348142 shutdown_connections 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24742cbb80 1 -- 172.21.15.19:0/379348142 wait complete. 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24742cbb80 20 client.15186 trim_cache size 0 max 16384 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24466, 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:25:45.492 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:25:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 169.27281764899999, 2024-05-09T01:25:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:25:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:25:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:25:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 209.492763642 2024-05-09T01:25:45.493 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:25:45.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 -- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24480b2900 msgr2=0x7f24480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24480b2900 0x7f24480b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2460103440 tx=0x7f245400a300 comp rx=0 tx=0).stop 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 -- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24601045d0 msgr2=0x7f246019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24601045d0 0x7f246019ae10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f245000b530 tx=0x7f245000b840 comp rx=0 tx=0).stop 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 -- 172.21.15.19:0/3575000334 shutdown_connections 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2460109850 0x7f246019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24480b2900 0x7f24480b4e10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24601045d0 0x7f246019ae10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 --2- 172.21.15.19:0/3575000334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2460108f00 0x7f246019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.489+0000 7f24658b5700 1 -- 172.21.15.19:0/3575000334 >> 172.21.15.19:0/3575000334 conn(0x7f24600ffea0 msgr2=0x7f2460108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.490+0000 7f24658b5700 1 -- 172.21.15.19:0/3575000334 shutdown_connections 2024-05-09T01:25:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:45.490+0000 7f24658b5700 1 -- 172.21.15.19:0/3575000334 wait complete. 2024-05-09T01:25:46.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T01:25:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:46.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:46 smithi019 ceph-mon[30158]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:25:46.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:46 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:25:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:46 smithi059 ceph-mon[34351]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:25:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:46 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:25:46.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:46 smithi196 ceph-mon[35041]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:25:46.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:46 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:25:47.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:47.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:47.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:47 smithi196 ceph-mon[35041]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T01:25:47.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:47.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:47 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:47.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:47.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:47.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:47 smithi019 ceph-mon[30158]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T01:25:47.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:47.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:47 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:48.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:48.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:48.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:47 smithi059 ceph-mon[34351]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T01:25:48.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:48.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:47 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:48.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.234+0000 7f6034f83700 1 -- 172.21.15.19:0/2977405682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301054a0 msgr2=0x7f60301058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:48.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.234+0000 7f6034f83700 1 --2- 172.21.15.19:0/2977405682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301054a0 0x7f60301058b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6020009990 tx=0x7f6020009ca0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.234+0000 7f6034f83700 1 -- 172.21.15.19:0/2977405682 shutdown_connections 2024-05-09T01:25:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.234+0000 7f6034f83700 1 --2- 172.21.15.19:0/2977405682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f603010e910 0x7f6030110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.234+0000 7f6034f83700 1 --2- 172.21.15.19:0/2977405682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301054a0 0x7f60301058b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.234+0000 7f6034f83700 1 --2- 172.21.15.19:0/2977405682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6030105df0 0x7f603010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.234+0000 7f6034f83700 1 -- 172.21.15.19:0/2977405682 >> 172.21.15.19:0/2977405682 conn(0x7f6030078e50 msgr2=0x7f6030079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.234+0000 7f6034f83700 1 -- 172.21.15.19:0/2977405682 shutdown_connections 2024-05-09T01:25:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.234+0000 7f6034f83700 1 -- 172.21.15.19:0/2977405682 wait complete. 2024-05-09T01:25:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.235+0000 7f6034f83700 1 Processor -- start 2024-05-09T01:25:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.235+0000 7f6034f83700 1 -- start start 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.235+0000 7f6034f83700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60301054a0 0x7f603019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f6034f83700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6030105df0 0x7f603019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f6034f83700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603010e910 0x7f603019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f6034f83700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6030114530 con 0x7f603010e910 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f6034f83700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60301143b0 con 0x7f60301054a0 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f6034f83700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60301146b0 con 0x7f6030105df0 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603010e910 0x7f603019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603010e910 0x7f603019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60422/0 (socket says 172.21.15.19:60422) 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602ed9d700 1 -- 172.21.15.19:0/3419433318 learned_addr learned my addr 172.21.15.19:0/3419433318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602e59c700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60301054a0 0x7f603019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602dd9b700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6030105df0 0x7f603019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602ed9d700 1 -- 172.21.15.19:0/3419433318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6030105df0 msgr2=0x7f603019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602ed9d700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6030105df0 0x7f603019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602ed9d700 1 -- 172.21.15.19:0/3419433318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60301054a0 msgr2=0x7f603019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602ed9d700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60301054a0 0x7f603019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602ed9d700 1 -- 172.21.15.19:0/3419433318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603019fca0 con 0x7f603010e910 2024-05-09T01:25:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602dd9b700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6030105df0 0x7f603019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f602ed9d700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603010e910 0x7f603019f6b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f602800b970 tx=0x7f602800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.236+0000 7f601f7fe700 1 -- 172.21.15.19:0/3419433318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028019070 con 0x7f603010e910 2024-05-09T01:25:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.237+0000 7f6034f83700 1 -- 172.21.15.19:0/3419433318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603019ff90 con 0x7f603010e910 2024-05-09T01:25:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.237+0000 7f601f7fe700 1 -- 172.21.15.19:0/3419433318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602800cde0 con 0x7f603010e910 2024-05-09T01:25:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.237+0000 7f601f7fe700 1 -- 172.21.15.19:0/3419433318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028007370 con 0x7f603010e910 2024-05-09T01:25:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.237+0000 7f6034f83700 1 -- 172.21.15.19:0/3419433318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301a04d0 con 0x7f603010e910 2024-05-09T01:25:48.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.239+0000 7f601f7fe700 1 -- 172.21.15.19:0/3419433318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f602801c070 con 0x7f603010e910 2024-05-09T01:25:48.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.240+0000 7f601f7fe700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60180b2880 0x7f60180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.241+0000 7f602e59c700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60180b2880 0x7f60180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.241+0000 7f601f7fe700 1 -- 172.21.15.19:0/3419433318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f60280e17a0 con 0x7f603010e910 2024-05-09T01:25:48.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.241+0000 7f6043999b80 1 Processor -- start 2024-05-09T01:25:48.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.241+0000 7f6043999b80 1 -- start start 2024-05-09T01:25:48.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.241+0000 7f602e59c700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60180b2880 0x7f60180b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f602000b5c0 tx=0x7f6020005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:48.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.241+0000 7f6043999b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 0x55a5ff77cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.241+0000 7f6043999b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff5fb450 0x55a5ff77d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.241+0000 7f6043999b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7817c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f6043999b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff69afa0 con 0x55a5ff7813b0 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f6043999b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff6397a0 con 0x55a5ff6e5110 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f6043999b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff696f60 con 0x55a5ff5fb450 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7817c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7817c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60436/0 (socket says 172.21.15.19:60436) 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602dd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff5fb450 0x55a5ff77d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602ed9d700 1 -- 172.21.15.19:0/1707221904 learned_addr learned my addr 172.21.15.19:0/1707221904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602e59c700 1 --2- 172.21.15.19:0/1707221904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 0x55a5ff77cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602ed9d700 1 -- 172.21.15.19:0/1707221904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff5fb450 msgr2=0x55a5ff77d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602ed9d700 1 --2- 172.21.15.19:0/1707221904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff5fb450 0x55a5ff77d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602ed9d700 1 -- 172.21.15.19:0/1707221904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 msgr2=0x55a5ff77cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:48.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602ed9d700 1 --2- 172.21.15.19:0/1707221904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 0x55a5ff77cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602ed9d700 1 -- 172.21.15.19:0/1707221904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ff631320 con 0x55a5ff7813b0 2024-05-09T01:25:48.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f602ed9d700 1 --2- 172.21.15.19:0/1707221904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7817c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f60280a5fb0 tx=0x7f6028004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3907a18d640db9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:48.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f601cff9700 1 -- 172.21.15.19:0/1707221904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280ad0f0 con 0x55a5ff7813b0 2024-05-09T01:25:48.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.242+0000 7f601cff9700 1 -- 172.21.15.19:0/1707221904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60280b1ba0 con 0x55a5ff7813b0 2024-05-09T01:25:48.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.243+0000 7f601cff9700 1 -- 172.21.15.19:0/1707221904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280b1910 con 0x55a5ff7813b0 2024-05-09T01:25:48.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.243+0000 7f6043999b80 1 -- 172.21.15.19:0/1707221904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 msgr2=0x55a5ff7817c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:48.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.243+0000 7f6043999b80 1 --2- 172.21.15.19:0/1707221904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7817c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f60280a5fb0 tx=0x7f6028004ad0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.243+0000 7f6043999b80 1 -- 172.21.15.19:0/1707221904 shutdown_connections 2024-05-09T01:25:48.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.243+0000 7f6043999b80 1 --2- 172.21.15.19:0/1707221904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7817c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.243+0000 7f6043999b80 1 --2- 172.21.15.19:0/1707221904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 0x55a5ff77cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.243+0000 7f6043999b80 1 --2- 172.21.15.19:0/1707221904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff5fb450 0x55a5ff77d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.243+0000 7f6043999b80 1 -- 172.21.15.19:0/1707221904 >> 172.21.15.19:0/1707221904 conn(0x55a5ff65b650 msgr2=0x55a5ff7733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 -- 172.21.15.19:0/1707221904 shutdown_connections 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 -- 172.21.15.19:0/1707221904 wait complete. 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 -- start start 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 Processor -- start 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 0x55a5ff786660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff6e5110 0x55a5ff786ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7870e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff516b50 con 0x55a5ff7813b0 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff655100 con 0x55a5ff5fb450 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.244+0000 7f6043999b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff782160 con 0x55a5ff6e5110 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f6043999b80 10 client.0 _ll_register_callbacks cb 0x55a5ff50c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7870e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7870e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60438/0 (socket says 172.21.15.19:60438) 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602ed9d700 1 -- 172.21.15.19:0/3261339494 learned_addr learned my addr 172.21.15.19:0/3261339494 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602e59c700 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 0x55a5ff786660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602dd9b700 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff6e5110 0x55a5ff786ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602ed9d700 1 -- 172.21.15.19:0/3261339494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff6e5110 msgr2=0x55a5ff786ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602ed9d700 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff6e5110 0x55a5ff786ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602ed9d700 1 -- 172.21.15.19:0/3261339494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 msgr2=0x55a5ff786660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602ed9d700 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 0x55a5ff786660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602ed9d700 1 -- 172.21.15.19:0/3261339494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ff516440 con 0x55a5ff7813b0 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f602ed9d700 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7870e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f602800bfd0 tx=0x7f60280ad1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0db5108f9e01c44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f6015ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f6015ffb700 1 -- 172.21.15.19:0/3261339494 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280b1710 con 0x55a5ff7813b0 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f6015ffb700 1 -- 172.21.15.19:0/3261339494 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f60280b18b0 con 0x55a5ff7813b0 2024-05-09T01:25:48.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.245+0000 7f6043999b80 1 -- 172.21.15.19:0/3261339494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60280b1ba0 con 0x55a5ff7813b0 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6015ffb700 1 -- 172.21.15.19:0/3261339494 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f602808a050 con 0x55a5ff7813b0 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6043999b80 10 client.15198 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6043999b80 1 -- 172.21.15.19:0/3261339494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5ff793f40 con 0x55a5ff7813b0 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6015ffb700 1 -- 172.21.15.19:0/3261339494 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60280b1ba0 con 0x55a5ff7813b0 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6015ffb700 1 -- 172.21.15.19:0/3261339494 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f60280a4780 con 0x55a5ff7813b0 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6043999b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6043999b80 10 client.15198 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6043999b80 20 client.15198 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6043999b80 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55a5ff795290 0x55a5ff797460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6043999b80 4 client.15198 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f6043999b80 1 -- 172.21.15.19:0/3261339494 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5ff6a8980 con 0x55a5ff795290 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.246+0000 7f602e59c700 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55a5ff795290 0x55a5ff797460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.247+0000 7f602e59c700 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55a5ff795290 0x55a5ff797460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81fdb15ae518e48a server_cookie=46208b61ecbcbedc in_seq=0 out_seq=0 2024-05-09T01:25:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.247+0000 7f6015ffb700 10 client.15198 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6015ffb700 1 -- 172.21.15.19:0/3261339494 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a5ff6a8980 con 0x55a5ff795290 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6015ffb700 10 client.15198 handle_command_reply: tid=0 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 client.15198 shutdown 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 -- 172.21.15.19:0/3261339494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 msgr2=0x55a5ff7870e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7870e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f602800bfd0 tx=0x7f60280ad1a0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 -- 172.21.15.19:0/3261339494 shutdown_connections 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55a5ff795290 0x55a5ff797460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff7813b0 0x55a5ff7870e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 0x55a5ff786660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6015ffb700 0 client.15198 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 --2- 172.21.15.19:0/3261339494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff6e5110 0x55a5ff786ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 -- 172.21.15.19:0/3261339494 >> 172.21.15.19:0/3261339494 conn(0x55a5ff65b650 msgr2=0x55a5ff773170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 -- 172.21.15.19:0/3261339494 shutdown_connections 2024-05-09T01:25:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 1 -- 172.21.15.19:0/3261339494 wait complete. 2024-05-09T01:25:48.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.248+0000 7f6043999b80 20 client.15198 trim_cache size 0 max 16384 2024-05-09T01:25:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f6043999b80 1 Processor -- start 2024-05-09T01:25:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f6043999b80 1 -- start start 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f6043999b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff5fb450 0x55a5ff792740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f6043999b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 0x55a5ff785530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f6043999b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 0x55a5ff772890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f6043999b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff7957a0 con 0x55a5ff5fb450 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f6043999b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff795e70 con 0x55a5ff6e5110 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f6043999b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff796540 con 0x55a5ff7813b0 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f602e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff5fb450 0x55a5ff792740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f602e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff5fb450 0x55a5ff792740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60444/0 (socket says 172.21.15.19:60444) 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f602e59c700 1 -- 172.21.15.19:0/1230040247 learned_addr learned my addr 172.21.15.19:0/1230040247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.270+0000 7f602dd9b700 1 --2- 172.21.15.19:0/1230040247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 0x55a5ff785530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f602ed9d700 1 --2- 172.21.15.19:0/1230040247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 0x55a5ff772890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f602e59c700 1 -- 172.21.15.19:0/1230040247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 msgr2=0x55a5ff772890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f602e59c700 1 --2- 172.21.15.19:0/1230040247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 0x55a5ff772890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f602e59c700 1 -- 172.21.15.19:0/1230040247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 msgr2=0x55a5ff785530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f602e59c700 1 --2- 172.21.15.19:0/1230040247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 0x55a5ff785530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f602e59c700 1 -- 172.21.15.19:0/1230040247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ff796360 con 0x55a5ff5fb450 2024-05-09T01:25:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f602e59c700 1 --2- 172.21.15.19:0/1230040247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff5fb450 0x55a5ff792740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6020009990 tx=0x7f602001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e7124d1f2a5ffa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f6017fff700 1 -- 172.21.15.19:0/1230040247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60200056e0 con 0x55a5ff5fb450 2024-05-09T01:25:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f6017fff700 1 -- 172.21.15.19:0/1230040247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602003a070 con 0x55a5ff5fb450 2024-05-09T01:25:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.271+0000 7f6017fff700 1 -- 172.21.15.19:0/1230040247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602000f6c0 con 0x55a5ff5fb450 2024-05-09T01:25:48.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.272+0000 7f6043999b80 1 -- 172.21.15.19:0/1230040247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff5fb450 msgr2=0x55a5ff792740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:48.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.272+0000 7f6043999b80 1 --2- 172.21.15.19:0/1230040247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff5fb450 0x55a5ff792740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6020009990 tx=0x7f602001e000 comp rx=0 tx=0).stop 2024-05-09T01:25:48.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.272+0000 7f6043999b80 1 -- 172.21.15.19:0/1230040247 shutdown_connections 2024-05-09T01:25:48.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.272+0000 7f6043999b80 1 --2- 172.21.15.19:0/1230040247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 0x55a5ff772890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.272+0000 7f6043999b80 1 --2- 172.21.15.19:0/1230040247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff5fb450 0x55a5ff792740 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.272+0000 7f6043999b80 1 --2- 172.21.15.19:0/1230040247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff6e5110 0x55a5ff785530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.272+0000 7f6043999b80 1 -- 172.21.15.19:0/1230040247 >> 172.21.15.19:0/1230040247 conn(0x55a5ff65b650 msgr2=0x55a5ff784e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.272+0000 7f6043999b80 1 -- 172.21.15.19:0/1230040247 shutdown_connections 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.272+0000 7f6043999b80 1 -- 172.21.15.19:0/1230040247 wait complete. 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f6043999b80 1 -- start start 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f6043999b80 1 Processor -- start 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f6043999b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 0x55a5ff7a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f6043999b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff6e5110 0x55a5ff77eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f6043999b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 0x55a5ff77f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f6043999b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff7972e0 con 0x55a5ff6e5110 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f6043999b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff796c10 con 0x55a5ff5fb450 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f6043999b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a5ff7979b0 con 0x55a5ff7813b0 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f6043999b80 10 client.0 _ll_register_callbacks cb 0x55a5ff50c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f602dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff6e5110 0x55a5ff77eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f602dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff6e5110 0x55a5ff77eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60452/0 (socket says 172.21.15.19:60452) 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f602dd9b700 1 -- 172.21.15.19:0/4038294237 learned_addr learned my addr 172.21.15.19:0/4038294237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.273+0000 7f602e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 0x55a5ff7a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f602ed9d700 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 0x55a5ff77f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f602dd9b700 1 -- 172.21.15.19:0/4038294237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 msgr2=0x55a5ff77f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f602dd9b700 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 0x55a5ff77f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f602dd9b700 1 -- 172.21.15.19:0/4038294237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 msgr2=0x55a5ff7a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f602dd9b700 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 0x55a5ff7a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f602dd9b700 1 -- 172.21.15.19:0/4038294237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ff77f580 con 0x55a5ff6e5110 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f602dd9b700 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff6e5110 0x55a5ff77eb00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6030102810 tx=0x7f6024010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9fc58f1c49f83c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f6015ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f6015ffb700 1 -- 172.21.15.19:0/4038294237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602400fd10 con 0x55a5ff6e5110 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f6015ffb700 1 -- 172.21.15.19:0/4038294237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6024004830 con 0x55a5ff6e5110 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f6043999b80 1 -- 172.21.15.19:0/4038294237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602003a070 con 0x55a5ff6e5110 2024-05-09T01:25:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f6015ffb700 1 -- 172.21.15.19:0/4038294237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6024004a30 con 0x55a5ff6e5110 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f6043999b80 10 client.15204 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.274+0000 7f6043999b80 1 -- 172.21.15.19:0/4038294237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5ff801460 con 0x55a5ff6e5110 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.275+0000 7f6015ffb700 1 -- 172.21.15.19:0/4038294237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602003a070 con 0x55a5ff6e5110 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.275+0000 7f6015ffb700 1 -- 172.21.15.19:0/4038294237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f602401a070 con 0x55a5ff6e5110 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.275+0000 7f6043999b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.275+0000 7f6043999b80 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.275+0000 7f6043999b80 20 client.15204 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.275+0000 7f6043999b80 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55a5ff795290 0x55a5ff804c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.275+0000 7f6043999b80 4 client.15204 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.275+0000 7f6043999b80 1 -- 172.21.15.19:0/4038294237 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "status"}) -- 0x55a5ff6a2a20 con 0x55a5ff795290 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.275+0000 7f602e59c700 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55a5ff795290 0x55a5ff804c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:48.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f602e59c700 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55a5ff795290 0x55a5ff804c30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5989653930bfcb1 server_cookie=e8b6618218a1766b in_seq=0 out_seq=0 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6015ffb700 10 client.15204 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6015ffb700 1 -- 172.21.15.19:0/4038294237 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a5ff6a2a20 con 0x55a5ff795290 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6015ffb700 10 client.15204 handle_command_reply: tid=0 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 client.15204 shutdown 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 -- 172.21.15.19:0/4038294237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff6e5110 msgr2=0x55a5ff77eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff6e5110 0x55a5ff77eb00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6030102810 tx=0x7f6024010dd0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 -- 172.21.15.19:0/4038294237 shutdown_connections 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55a5ff795290 0x55a5ff804c30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5ff7813b0 0x55a5ff77f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5ff5fb450 0x55a5ff7a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 --2- 172.21.15.19:0/4038294237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5ff6e5110 0x55a5ff77eb00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 -- 172.21.15.19:0/4038294237 >> 172.21.15.19:0/4038294237 conn(0x55a5ff65b650 msgr2=0x55a5ff784050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6015ffb700 0 client.15204 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 -- 172.21.15.19:0/4038294237 shutdown_connections 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 1 -- 172.21.15.19:0/4038294237 wait complete. 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.276+0000 7f6043999b80 20 client.15204 trim_cache size 0 max 16384 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24473, 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:25:48.279 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:25:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:25:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 173.067539831, 2024-05-09T01:25:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:25:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:25:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:25:48.280 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 214.79977792299999 2024-05-09T01:25:48.280 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:25:48.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 -- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60180b2880 msgr2=0x7f60180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60180b2880 0x7f60180b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f602000b5c0 tx=0x7f6020005960 comp rx=0 tx=0).stop 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 -- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603010e910 msgr2=0x7f603019f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603010e910 0x7f603019f6b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f602800b970 tx=0x7f602800bc80 comp rx=0 tx=0).stop 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 -- 172.21.15.19:0/3419433318 shutdown_connections 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f60180b2880 0x7f60180b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603010e910 0x7f603019f6b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60301054a0 0x7f603019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 --2- 172.21.15.19:0/3419433318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6030105df0 0x7f603019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 -- 172.21.15.19:0/3419433318 >> 172.21.15.19:0/3419433318 conn(0x7f6030078e50 msgr2=0x7f6030100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 -- 172.21.15.19:0/3419433318 shutdown_connections 2024-05-09T01:25:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:48.277+0000 7f6034f83700 1 -- 172.21.15.19:0/3419433318 wait complete. 2024-05-09T01:25:48.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:25:48.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:25:48.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:48 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:48.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T01:25:49.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:25:49.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:25:49.097 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:48 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:49.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:25:49.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:25:49.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:48 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:25:49.226 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:50.033 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:49 smithi019 ceph-mon[30158]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-09T01:25:50.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:49 smithi059 ceph-mon[34351]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-09T01:25:50.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:49 smithi196 ceph-mon[35041]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-09T01:25:50.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.882+0000 7f0d203be700 1 -- 172.21.15.19:0/2357245029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 msgr2=0x7f0d181100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.882+0000 7f0d203be700 1 --2- 172.21.15.19:0/2357245029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 0x7f0d181100a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0d100099c0 tx=0x7f0d10009cd0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.883+0000 7f0d203be700 1 -- 172.21.15.19:0/2357245029 shutdown_connections 2024-05-09T01:25:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.883+0000 7f0d203be700 1 --2- 172.21.15.19:0/2357245029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 0x7f0d181100a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.883+0000 7f0d203be700 1 --2- 172.21.15.19:0/2357245029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d181034c0 0x7f0d181038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.883+0000 7f0d203be700 1 --2- 172.21.15.19:0/2357245029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d18107df0 0x7f0d18108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.883+0000 7f0d203be700 1 -- 172.21.15.19:0/2357245029 >> 172.21.15.19:0/2357245029 conn(0x7f0d180fedf0 msgr2=0x7f0d18101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.883+0000 7f0d203be700 1 -- 172.21.15.19:0/2357245029 shutdown_connections 2024-05-09T01:25:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.883+0000 7f0d203be700 1 -- 172.21.15.19:0/2357245029 wait complete. 2024-05-09T01:25:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.883+0000 7f0d203be700 1 Processor -- start 2024-05-09T01:25:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d203be700 1 -- start start 2024-05-09T01:25:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d203be700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d181034c0 0x7f0d18199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d203be700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d18107df0 0x7f0d18199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d203be700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 0x7f0d1819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d203be700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18113370 con 0x7f0d18108740 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d203be700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0d181131f0 con 0x7f0d18107df0 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d203be700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d181134f0 con 0x7f0d181034c0 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e95b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 0x7f0d1819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e95b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 0x7f0d1819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55444/0 (socket says 172.21.15.19:55444) 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e95b700 1 -- 172.21.15.19:0/2667956497 learned_addr learned my addr 172.21.15.19:0/2667956497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d181034c0 0x7f0d18199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1d959700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d18107df0 0x7f0d18199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e95b700 1 -- 172.21.15.19:0/2667956497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d181034c0 msgr2=0x7f0d18199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d181034c0 0x7f0d18199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e95b700 1 -- 172.21.15.19:0/2667956497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d18107df0 msgr2=0x7f0d18199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d18107df0 0x7f0d18199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e95b700 1 -- 172.21.15.19:0/2667956497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d181a9100 con 0x7f0d18108740 2024-05-09T01:25:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d181034c0 0x7f0d18199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.884+0000 7f0d1d959700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d18107df0 0x7f0d18199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.885+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 0x7f0d1819a400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0d10005910 tx=0x7f0d10009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.885+0000 7f0d0f7fe700 1 -- 172.21.15.19:0/2667956497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10027070 con 0x7f0d18108740 2024-05-09T01:25:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.885+0000 7f0d203be700 1 -- 172.21.15.19:0/2667956497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d181a9390 con 0x7f0d18108740 2024-05-09T01:25:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.885+0000 7f0d0f7fe700 1 -- 172.21.15.19:0/2667956497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d10003fa0 con 0x7f0d18108740 2024-05-09T01:25:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.885+0000 7f0d203be700 1 -- 172.21.15.19:0/2667956497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d181a9840 con 0x7f0d18108740 2024-05-09T01:25:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.885+0000 7f0d0f7fe700 1 -- 172.21.15.19:0/2667956497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10022d10 con 0x7f0d18108740 2024-05-09T01:25:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.888+0000 7f0d0f7fe700 1 -- 172.21.15.19:0/2667956497 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0d1002a070 con 0x7f0d18108740 2024-05-09T01:25:50.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.889+0000 7f0d0f7fe700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0d040b2880 0x7f0d040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.889+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0d040b2880 0x7f0d040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.889+0000 7f0d0f7fe700 1 -- 172.21.15.19:0/2667956497 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0d100e31d0 con 0x7f0d18108740 2024-05-09T01:25:50.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.889+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0d040b2880 0x7f0d040b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009dd0 tx=0x7f0d14009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:50.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d2edd4b80 1 Processor -- start 2024-05-09T01:25:50.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d2edd4b80 1 -- start start 2024-05-09T01:25:50.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e0de110 0x55c83e1774a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 0x55c83e1779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e171650 0x55c83e171a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c83e093fa0 con 0x55c83e0de110 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c83e0327a0 con 0x55c83dff4450 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c83e08ff60 con 0x55c83e171650 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d1e15a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e0de110 0x55c83e1774a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d1e15a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e0de110 0x55c83e1774a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55452/0 (socket says 172.21.15.19:55452) 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d1e15a700 1 -- 172.21.15.19:0/1180136215 learned_addr learned my addr 172.21.15.19:0/1180136215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d1d959700 1 --2- 172.21.15.19:0/1180136215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 0x55c83e1779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.890+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/1180136215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e171650 0x55c83e171a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.891+0000 7f0d1e15a700 1 -- 172.21.15.19:0/1180136215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e171650 msgr2=0x55c83e171a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.891+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/1180136215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e171650 0x55c83e171a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.891+0000 7f0d1e15a700 1 -- 172.21.15.19:0/1180136215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 msgr2=0x55c83e1779e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.891+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/1180136215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 0x55c83e1779e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.891+0000 7f0d1e15a700 1 -- 172.21.15.19:0/1180136215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83e02a320 con 0x55c83e0de110 2024-05-09T01:25:50.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.891+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/1180136215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e0de110 0x55c83e1774a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0d14016550 tx=0x7f0d14012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8015f48e42bb6a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:50.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.891+0000 7f0d0cd47700 1 -- 172.21.15.19:0/1180136215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14026070 con 0x55c83e0de110 2024-05-09T01:25:50.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.891+0000 7f0d0cd47700 1 -- 172.21.15.19:0/1180136215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d14021ab0 con 0x55c83e0de110 2024-05-09T01:25:50.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.891+0000 7f0d0cd47700 1 -- 172.21.15.19:0/1180136215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d140149e0 con 0x55c83e0de110 2024-05-09T01:25:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.892+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1180136215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e0de110 msgr2=0x55c83e1774a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.892+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1180136215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e0de110 0x55c83e1774a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0d14016550 tx=0x7f0d14012000 comp rx=0 tx=0).stop 2024-05-09T01:25:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.892+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1180136215 shutdown_connections 2024-05-09T01:25:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.892+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1180136215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e171650 0x55c83e171a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.892+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1180136215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e0de110 0x55c83e1774a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.892+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1180136215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 0x55c83e1779e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.892+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1180136215 >> 172.21.15.19:0/1180136215 conn(0x55c83e054650 msgr2=0x55c83e16c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.892+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1180136215 shutdown_connections 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.892+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1180136215 wait complete. 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d2edd4b80 1 -- start start 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d2edd4b80 1 Processor -- start 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83dff4450 0x55c83e17b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e0de110 0x55c83e17b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e171650 0x55c83e17bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c83df0fb50 con 0x55c83dff4450 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c83e04e100 con 0x55c83e171650 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c83e17e180 con 0x55c83e0de110 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d2edd4b80 10 client.0 _ll_register_callbacks cb 0x55c83df05670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d1e15a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83dff4450 0x55c83e17b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d1e15a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83dff4450 0x55c83e17b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55468/0 (socket says 172.21.15.19:55468) 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d1e15a700 1 -- 172.21.15.19:0/1309582871 learned_addr learned my addr 172.21.15.19:0/1309582871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d1d959700 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e0de110 0x55c83e17b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e171650 0x55c83e17bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d1e15a700 1 -- 172.21.15.19:0/1309582871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e0de110 msgr2=0x55c83e17b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.893+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e0de110 0x55c83e17b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d1e15a700 1 -- 172.21.15.19:0/1309582871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e171650 msgr2=0x55c83e17bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e171650 0x55c83e17bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d1e15a700 1 -- 172.21.15.19:0/1309582871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83df0f440 con 0x55c83dff4450 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e171650 0x55c83e17bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83dff4450 0x55c83e17b230 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009fd0 tx=0x7f0d14014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264b843cb67fd991 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d01ffb700 1 -- 172.21.15.19:0/1309582871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14030020 con 0x55c83dff4450 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d01ffb700 1 -- 172.21.15.19:0/1309582871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0d14004c90 con 0x55c83dff4450 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1309582871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d14021ab0 con 0x55c83dff4450 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d01ffb700 1 -- 172.21.15.19:0/1309582871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d1403b050 con 0x55c83dff4450 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d2edd4b80 10 client.15216 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1309582871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c83e188c10 con 0x55c83dff4450 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.894+0000 7f0d01ffb700 1 -- 172.21.15.19:0/1309582871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d14021ab0 con 0x55c83dff4450 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.895+0000 7f0d01ffb700 1 -- 172.21.15.19:0/1309582871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0d14029030 con 0x55c83dff4450 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.895+0000 7f0d2edd4b80 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.895+0000 7f0d2edd4b80 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.895+0000 7f0d2edd4b80 20 client.15216 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.895+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c83e18aee0 0x55c83e18c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.895+0000 7f0d2edd4b80 4 client.15216 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.895+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1309582871 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c83e0a1980 con 0x55c83e18aee0 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.895+0000 7f0d1d959700 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c83e18aee0 0x55c83e18c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.896+0000 7f0d1d959700 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c83e18aee0 0x55c83e18c350 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba27bcea256289e server_cookie=6a7081a956e0ecec in_seq=0 out_seq=0 2024-05-09T01:25:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.896+0000 7f0d01ffb700 10 client.15216 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:25:50.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d01ffb700 1 -- 172.21.15.19:0/1309582871 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c83e0a1980 con 0x55c83e18aee0 2024-05-09T01:25:50.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d01ffb700 10 client.15216 handle_command_reply: tid=0 2024-05-09T01:25:50.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 client.15216 shutdown 2024-05-09T01:25:50.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1309582871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83dff4450 msgr2=0x55c83e17b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:50.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83dff4450 0x55c83e17b230 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009fd0 tx=0x7f0d14014f90 comp rx=0 tx=0).stop 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1309582871 shutdown_connections 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c83e18aee0 0x55c83e18c350 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e171650 0x55c83e17bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d01ffb700 0 client.15216 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83dff4450 0x55c83e17b230 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/1309582871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e0de110 0x55c83e17b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1309582871 >> 172.21.15.19:0/1309582871 conn(0x55c83e054650 msgr2=0x55c83e16c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1309582871 shutdown_connections 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/1309582871 wait complete. 2024-05-09T01:25:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.897+0000 7f0d2edd4b80 20 client.15216 trim_cache size 0 max 16384 2024-05-09T01:25:50.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.918+0000 7f0d2edd4b80 1 Processor -- start 2024-05-09T01:25:50.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.918+0000 7f0d2edd4b80 1 -- start start 2024-05-09T01:25:50.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 0x55c83e175990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e0de110 0x55c83e17b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e187080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c83e18a690 con 0x55c83e171650 2024-05-09T01:25:50.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c83e18ad60 con 0x55c83dff4450 2024-05-09T01:25:50.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c83e18b430 con 0x55c83e0de110 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e95b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e187080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e95b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e187080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55474/0 (socket says 172.21.15.19:55474) 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e95b700 1 -- 172.21.15.19:0/821635280 learned_addr learned my addr 172.21.15.19:0/821635280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e95b700 1 -- 172.21.15.19:0/821635280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e0de110 msgr2=0x55c83e17b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/821635280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e0de110 0x55c83e17b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/821635280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 0x55c83e175990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e95b700 1 -- 172.21.15.19:0/821635280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 msgr2=0x55c83e175990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/821635280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 0x55c83e175990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e95b700 1 -- 172.21.15.19:0/821635280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83e16dcb0 con 0x55c83e171650 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/821635280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 0x55c83e175990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/821635280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e187080 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0d100e0cb0 tx=0x7f0d100b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10dbc3d154f599e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d03fff700 1 -- 172.21.15.19:0/821635280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100e2070 con 0x55c83e171650 2024-05-09T01:25:50.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d03fff700 1 -- 172.21.15.19:0/821635280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d100aa860 con 0x55c83e171650 2024-05-09T01:25:50.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.919+0000 7f0d03fff700 1 -- 172.21.15.19:0/821635280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100aaac0 con 0x55c83e171650 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.920+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/821635280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 msgr2=0x55c83e187080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.920+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/821635280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e187080 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0d100e0cb0 tx=0x7f0d100b3040 comp rx=0 tx=0).stop 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.920+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/821635280 shutdown_connections 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.920+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/821635280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e187080 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.920+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/821635280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83dff4450 0x55c83e175990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.920+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/821635280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83e0de110 0x55c83e17b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.920+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/821635280 >> 172.21.15.19:0/821635280 conn(0x55c83e054650 msgr2=0x55c83e17b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.920+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/821635280 shutdown_connections 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.920+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/821635280 wait complete. 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d2edd4b80 1 -- start start 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d2edd4b80 1 Processor -- start 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83dff4450 0x55c83e174af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e0de110 0x55c83e175030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d2edd4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e175570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c83e18c1d0 con 0x55c83e171650 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c83e18bb00 con 0x55c83e0de110 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d2edd4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c83e18c8a0 con 0x55c83dff4450 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d2edd4b80 10 client.0 _ll_register_callbacks cb 0x55c83df05670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d1e95b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e175570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d1e95b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e175570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55484/0 (socket says 172.21.15.19:55484) 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d1e95b700 1 -- 172.21.15.19:0/4064610668 learned_addr learned my addr 172.21.15.19:0/4064610668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83dff4450 0x55c83e174af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d1d959700 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e0de110 0x55c83e175030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d1e95b700 1 -- 172.21.15.19:0/4064610668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83dff4450 msgr2=0x55c83e174af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83dff4450 0x55c83e174af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d1e95b700 1 -- 172.21.15.19:0/4064610668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e0de110 msgr2=0x55c83e175030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:25:50.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.921+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e0de110 0x55c83e175030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d1e95b700 1 -- 172.21.15.19:0/4064610668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83e191bc0 con 0x55c83e171650 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d1e95b700 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e175570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0d10022750 tx=0x7f0d100aa630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e1c3717a463d77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d01ffb700 1 -- 172.21.15.19:0/4064610668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100c3070 con 0x55c83e171650 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d01ffb700 1 -- 172.21.15.19:0/4064610668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0d100a6e50 con 0x55c83e171650 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/4064610668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d100aa860 con 0x55c83e171650 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d01ffb700 1 -- 172.21.15.19:0/4064610668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d100e2070 con 0x55c83e171650 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d2edd4b80 10 client.15222 fetch_fsmap learned FSMap version 27 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/4064610668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c83e1f64f0 con 0x55c83e171650 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d01ffb700 1 -- 172.21.15.19:0/4064610668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d100aa860 con 0x55c83e171650 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d01ffb700 1 -- 172.21.15.19:0/4064610668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0d100abb90 con 0x55c83e171650 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.922+0000 7f0d2edd4b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d2edd4b80 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d2edd4b80 20 client.15222 populate_metadata read hostname 'smithi019' 2024-05-09T01:25:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c83e18aee0 0x55c83e1f9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d2edd4b80 4 client.15222 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/4064610668 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "status"}) -- 0x55c83e09ba20 con 0x55c83e18aee0 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c83e18aee0 0x55c83e1f9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d01ffb700 10 client.15222 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d1e15a700 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c83e18aee0 0x55c83e1f9ac0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56f40f1a067c5ba server_cookie=92de989ddc7359de in_seq=0 out_seq=0 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d01ffb700 1 -- 172.21.15.19:0/4064610668 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c83e09ba20 con 0x55c83e18aee0 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d01ffb700 10 client.15222 handle_command_reply: tid=0 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.923+0000 7f0d2edd4b80 1 client.15222 shutdown 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/4064610668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 msgr2=0x55c83e175570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e175570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0d10022750 tx=0x7f0d100aa630 comp rx=0 tx=0).stop 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/4064610668 shutdown_connections 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c83e18aee0 0x55c83e1f9ac0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c83e171650 0x55c83e175570 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c83dff4450 0x55c83e174af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 --2- 172.21.15.19:0/4064610668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83e0de110 0x55c83e175030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/4064610668 >> 172.21.15.19:0/4064610668 conn(0x55c83e054650 msgr2=0x55c83e170dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d01ffb700 0 client.15222 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/4064610668 shutdown_connections 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 1 -- 172.21.15.19:0/4064610668 wait complete. 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.924+0000 7f0d2edd4b80 20 client.15222 trim_cache size 0 max 16384 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "58c4b1c8-0da1-11ef-bc97-c7b262605968", 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-09T01:25:50.927 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24497, 2024-05-09T01:25:50.928 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T01:25:50.928 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T01:25:50.928 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T01:25:50.928 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 189.09946834799999, 2024-05-09T01:25:50.928 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-05-09T01:25:50.928 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 83, 2024-05-09T01:25:50.928 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-05-09T01:25:50.928 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 204.143869145 2024-05-09T01:25:50.928 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 -- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0d040b2880 msgr2=0x7f0d040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0d040b2880 0x7f0d040b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009dd0 tx=0x7f0d14009040 comp rx=0 tx=0).stop 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 -- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 msgr2=0x7f0d1819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 0x7f0d1819a400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0d10005910 tx=0x7f0d10009580 comp rx=0 tx=0).stop 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 -- 172.21.15.19:0/2667956497 shutdown_connections 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0d040b2880 0x7f0d040b4d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d18108740 0x7f0d1819a400 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d181034c0 0x7f0d18199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 --2- 172.21.15.19:0/2667956497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d18107df0 0x7f0d18199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 -- 172.21.15.19:0/2667956497 >> 172.21.15.19:0/2667956497 conn(0x7f0d180fedf0 msgr2=0x7f0d18107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 -- 172.21.15.19:0/2667956497 shutdown_connections 2024-05-09T01:25:50.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:50.925+0000 7f0d203be700 1 -- 172.21.15.19:0/2667956497 wait complete. 2024-05-09T01:25:51.573 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T01:25:51.583 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T01:25:51.583 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T01:25:51.583 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T01:25:51.583 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T01:25:51.618 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T01:25:51.619 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-09T01:25:51.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:25:51.619 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-09T01:25:51.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:25:51.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-05-09T01:25:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-05-09T01:25:51.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:25:51.711 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-09T01:25:51.711 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T01:25:51.711 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T01:25:51.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T01:25:52.006 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:52.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:52 smithi019 ceph-mon[30158]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:52.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:52 smithi059 ceph-mon[34351]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:52.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:52 smithi196 ceph-mon[35041]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-05-09T01:25:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.634+0000 7f690e0ad700 1 -- 172.21.15.19:0/3576398948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 msgr2=0x7f69081111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.634+0000 7f690e0ad700 1 --2- 172.21.15.19:0/3576398948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 0x7f69081111b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f690000b0d0 tx=0x7f690000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.634+0000 7f690e0ad700 1 -- 172.21.15.19:0/3576398948 shutdown_connections 2024-05-09T01:25:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.634+0000 7f690e0ad700 1 --2- 172.21.15.19:0/3576398948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 0x7f69081111b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.634+0000 7f690e0ad700 1 --2- 172.21.15.19:0/3576398948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69081045d0 0x7f69081049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.634+0000 7f690e0ad700 1 --2- 172.21.15.19:0/3576398948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6908108f00 0x7f6908109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.634+0000 7f690e0ad700 1 -- 172.21.15.19:0/3576398948 >> 172.21.15.19:0/3576398948 conn(0x7f69080ffea0 msgr2=0x7f69081022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.634+0000 7f690e0ad700 1 -- 172.21.15.19:0/3576398948 shutdown_connections 2024-05-09T01:25:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.634+0000 7f690e0ad700 1 -- 172.21.15.19:0/3576398948 wait complete. 2024-05-09T01:25:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.635+0000 7f690e0ad700 1 Processor -- start 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.635+0000 7f690e0ad700 1 -- start start 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.635+0000 7f690e0ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69081045d0 0x7f6908076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.635+0000 7f690e0ad700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908108f00 0x7f6908077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.635+0000 7f690e0ad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 0x7f6908077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.635+0000 7f690e0ad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6908114480 con 0x7f6908109850 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.635+0000 7f690e0ad700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6908114300 con 0x7f69081045d0 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.635+0000 7f690e0ad700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6908114600 con 0x7f6908108f00 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6907fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 0x7f6908077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6907fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 0x7f6908077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55508/0 (socket says 172.21.15.19:55508) 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6907fff700 1 -- 172.21.15.19:0/2961587398 learned_addr learned my addr 172.21.15.19:0/2961587398 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f69077fe700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69081045d0 0x7f6908076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6906ffd700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908108f00 0x7f6908077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6907fff700 1 -- 172.21.15.19:0/2961587398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908108f00 msgr2=0x7f6908077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6907fff700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908108f00 0x7f6908077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6907fff700 1 -- 172.21.15.19:0/2961587398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69081045d0 msgr2=0x7f6908076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6907fff700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69081045d0 0x7f6908076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6907fff700 1 -- 172.21.15.19:0/2961587398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69081a90a0 con 0x7f6908109850 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6907fff700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 0x7f6908077650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6900009fd0 tx=0x7f6900007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6904ff9700 1 -- 172.21.15.19:0/2961587398 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690001f070 con 0x7f6908109850 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6904ff9700 1 -- 172.21.15.19:0/2961587398 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6900007e50 con 0x7f6908109850 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f6904ff9700 1 -- 172.21.15.19:0/2961587398 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690001b6a0 con 0x7f6908109850 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.636+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69081a9330 con 0x7f6908109850 2024-05-09T01:25:53.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.637+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69081a9730 con 0x7f6908109850 2024-05-09T01:25:53.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.638+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69080512c0 con 0x7f6908109850 2024-05-09T01:25:53.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.639+0000 7f6904ff9700 1 -- 172.21.15.19:0/2961587398 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6900022070 con 0x7f6908109850 2024-05-09T01:25:53.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.640+0000 7f6904ff9700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68f00b2880 0x7f68f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:53.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.640+0000 7f69077fe700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68f00b2880 0x7f68f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:53.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.640+0000 7f6904ff9700 1 -- 172.21.15.19:0/2961587398 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f690000f020 con 0x7f6908109850 2024-05-09T01:25:53.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.640+0000 7f69077fe700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68f00b2880 0x7f68f00b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68f800ac50 tx=0x7f68f800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:53.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.645+0000 7f6904ff9700 1 -- 172.21.15.19:0/2961587398 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6900029020 con 0x7f6908109850 2024-05-09T01:25:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.907+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f690802e5f0 con 0x7f6908109850 2024-05-09T01:25:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.907+0000 7f6904ff9700 1 -- 172.21.15.19:0/2961587398 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f69000ab220 con 0x7f6908109850 2024-05-09T01:25:53.905 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T01:25:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68f00b2880 msgr2=0x7f68f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68f00b2880 0x7f68f00b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68f800ac50 tx=0x7f68f800a380 comp rx=0 tx=0).stop 2024-05-09T01:25:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 msgr2=0x7f6908077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 0x7f6908077650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6900009fd0 tx=0x7f6900007a20 comp rx=0 tx=0).stop 2024-05-09T01:25:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 shutdown_connections 2024-05-09T01:25:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68f00b2880 0x7f68f00b4d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6908109850 0x7f6908077650 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69081045d0 0x7f6908076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 --2- 172.21.15.19:0/2961587398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908108f00 0x7f6908077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.909+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 >> 172.21.15.19:0/2961587398 conn(0x7f69080ffea0 msgr2=0x7f6908108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.910+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 shutdown_connections 2024-05-09T01:25:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:53.910+0000 7f690e0ad700 1 -- 172.21.15.19:0/2961587398 wait complete. 2024-05-09T01:25:54.524 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:54 smithi019 ceph-mon[30158]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:54.524 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:54 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2961587398' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T01:25:54.554 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T01:25:54.554 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T01:25:54.554 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-05-09T01:25:54.554 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T01:25:54.554 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T01:25:54.554 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T01:25:54.554 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T01:25:54.554 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T01:25:54.554 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T01:25:54.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:25:54.555 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-05-09T01:25:54.572 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T01:25:54.572 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T01:25:54.572 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T01:25:54.572 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-09T01:25:54.572 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-05-09T01:25:54.572 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-09T01:25:54.572 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T01:25:54.572 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-05-09T01:25:54.572 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42031sec preferred_lft 42031sec 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-05-09T01:25:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-05-09T01:25:54.574 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T01:25:54.574 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T01:25:54.574 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T01:25:54.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-05-09T01:25:54.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-05-09T01:25:54.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-05-09T01:25:54.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T01:25:54.574 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T01:25:54.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:54 smithi059 ceph-mon[34351]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:54.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:54 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2961587398' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T01:25:54.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T01:25:54.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:54 smithi196 ceph-mon[35041]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:25:54.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:54 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2961587398' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T01:25:54.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T01:25:54.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:25:54.764 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T01:25:54.790 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-05-09T01:25:54.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:25:54.792 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-05-09T01:25:54.862 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-05-09T01:25:54.862 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T01:25:54.862 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T01:25:54.862 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T01:25:54.862 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T01:25:54.863 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T01:25:54.863 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T01:25:54.863 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T01:25:54.863 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T01:25:54.863 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T01:25:54.863 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T01:25:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T01:25:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T01:25:55.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:25:55.158 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-09T01:25:55.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:25:55.175 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-05-09T01:25:55.231 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T01:25:55.231 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T01:25:55.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T01:25:55.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T01:25:55.232 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T01:25:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T01:25:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T01:25:55.353 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T01:25:55.353 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T01:25:55.354 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T01:25:55.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T01:25:55.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T01:25:55.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T01:25:55.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T01:25:55.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T01:25:55.354 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T01:25:55.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T01:25:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T01:25:55.487 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T01:25:55.487 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T01:25:55.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-05-09T01:25:55.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T01:25:55.488 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T01:25:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T01:25:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T01:25:55.567 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T01:25:55.568 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T01:25:55.584 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T01:25:55.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T01:25:55.584 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T01:25:55.640 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-09T01:25:55.640 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-09T01:25:55.640 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/3c21c65b-0a8c-4ecf-8aec-2b4de05b1a10 2024-05-09T01:25:55.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:25:55.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/3c21c65b-0a8c-4ecf-8aec-2b4de05b1a10 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-09T01:25:55.751 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-09T01:25:55.751 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-09T01:25:55.870 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-09T01:25:55.871 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-09T01:25:55.871 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.59:3300,172.21.15.196:3300" 2024-05-09T01:25:55.871 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@58c4b1c8-0da1-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/3c21c65b-0a8c-4ecf-8aec-2b4de05b1a10 2024-05-09T01:25:55.871 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.19:3300/172.21.15.59:3300/172.21.15.196:3300" will pass to kernel 2024-05-09T01:25:55.872 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T01:25:55.872 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-05-09T01:25:55.872 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T01:25:55.872 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T01:25:55.872 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T01:25:55.874 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T01:25:55.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:25:55.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T01:25:55.901 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T01:25:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T01:25:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T01:25:56.000 INFO:teuthology.orchestra.run.smithi019.stdout:client24929 2024-05-09T01:25:56.001 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:25:56.001 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T01:25:56.018 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T01:25:56.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T01:25:56.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T01:25:56.117 INFO:teuthology.orchestra.run.smithi019.stdout:client24929 2024-05-09T01:25:56.117 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T01:25:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T01:25:56.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T01:25:56.178 INFO:teuthology.orchestra.run.smithi019.stdout:client24929 2024-05-09T01:25:56.178 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968.client24929/status') 2024-05-09T01:25:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T01:25:56.240 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.24929 (3)172.21.15.19:0/3598205662 2024-05-09T01:25:56.240 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-05-09T01:25:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-05-09T01:25:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-05-09T01:25:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:71 bytes copied, 2.6107e-05 s, 2.7 MB/s 2024-05-09T01:25:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T01:25:56.247 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T01:25:56.258 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T01:25:56.258 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T01:25:56.258 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-09T01:25:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.377+0000 7f1bdbb34640 1 -- 172.21.15.19:0/867706158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4104740 msgr2=0x7f1bd4106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.377+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/867706158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4104740 0x7f1bd4106b60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009ca0 tx=0x7f1bc402a8c0 comp rx=0 tx=0).stop 2024-05-09T01:25:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.377+0000 7f1bdbb34640 1 -- 172.21.15.19:0/867706158 shutdown_connections 2024-05-09T01:25:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.377+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/867706158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd4109a00 0x7f1bd410be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.377+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/867706158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd41070a0 0x7f1bd41094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.377+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/867706158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4104740 0x7f1bd4106b60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.377+0000 7f1bdbb34640 1 -- 172.21.15.19:0/867706158 >> 172.21.15.19:0/867706158 conn(0x7f1bd40fdd40 msgr2=0x7f1bd41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 -- 172.21.15.19:0/867706158 shutdown_connections 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 -- 172.21.15.19:0/867706158 wait complete. 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 Processor -- start 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 -- start start 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd4104740 0x7f1bd419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd41070a0 0x7f1bd419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4109a00 0x7f1bd41a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd410f750 con 0x7f1bd4109a00 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd410f5d0 con 0x7f1bd41070a0 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.378+0000 7f1bdbb34640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd410f8d0 con 0x7f1bd4104740 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bda0d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4109a00 0x7f1bd41a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bda0d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4109a00 0x7f1bd41a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55526/0 (socket says 172.21.15.19:55526) 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bd98d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd4104740 0x7f1bd419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bda0d1640 1 -- 172.21.15.19:0/1090341653 learned_addr learned my addr 172.21.15.19:0/1090341653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bd90cf640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd41070a0 0x7f1bd419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bda0d1640 1 -- 172.21.15.19:0/1090341653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd4104740 msgr2=0x7f1bd419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bda0d1640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd4104740 0x7f1bd419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bda0d1640 1 -- 172.21.15.19:0/1090341653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd41070a0 msgr2=0x7f1bd419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bda0d1640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd41070a0 0x7f1bd419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bda0d1640 1 -- 172.21.15.19:0/1090341653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd41a1ec0 con 0x7f1bd4109a00 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bda0d1640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4109a00 0x7f1bd41a1850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00d720 tx=0x7f1bcc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bd2ffd640 1 -- 172.21.15.19:0/1090341653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc017070 con 0x7f1bd4109a00 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bd2ffd640 1 -- 172.21.15.19:0/1090341653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc00de30 con 0x7f1bd4109a00 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd41a21b0 con 0x7f1bd4109a00 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bd2ffd640 1 -- 172.21.15.19:0/1090341653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc004d00 con 0x7f1bd4109a00 2024-05-09T01:25:56.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.379+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd41a2750 con 0x7f1bd4109a00 2024-05-09T01:25:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.381+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd40ff860 con 0x7f1bd4109a00 2024-05-09T01:25:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.382+0000 7f1bd2ffd640 1 -- 172.21.15.19:0/1090341653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1bcc004ea0 con 0x7f1bd4109a00 2024-05-09T01:25:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.383+0000 7f1bd2ffd640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1bbc0b2860 0x7f1bbc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.383+0000 7f1bd98d0640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1bbc0b2860 0x7f1bbc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.383+0000 7f1bd2ffd640 1 -- 172.21.15.19:0/1090341653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f1bcc0e1e00 con 0x7f1bd4109a00 2024-05-09T01:25:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.383+0000 7f1bd98d0640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1bbc0b2860 0x7f1bbc0b4d70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40094e0 tx=0x7f1bc4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:56.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.387+0000 7f1bd2ffd640 1 -- 172.21.15.19:0/1090341653 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1bcc0a9530 con 0x7f1bd4109a00 2024-05-09T01:25:56.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:56 smithi059 ceph-mon[34351]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:56.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:56.622+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f1bd402dc30 con 0x7f1bd4109a00 2024-05-09T01:25:56.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:56 smithi019 ceph-mon[30158]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:56.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:56 smithi196 ceph-mon[35041]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:25:57.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.241+0000 7f1bd2ffd640 1 -- 172.21.15.19:0/1090341653 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f1bcc0ad830 con 0x7f1bd4109a00 2024-05-09T01:25:57.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1bbc0b2860 msgr2=0x7f1bbc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1bbc0b2860 0x7f1bbc0b4d70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40094e0 tx=0x7f1bc4004a40 comp rx=0 tx=0).stop 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4109a00 msgr2=0x7f1bd41a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4109a00 0x7f1bd41a1850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00d720 tx=0x7f1bcc00dbe0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 shutdown_connections 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1bbc0b2860 0x7f1bbc0b4d70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd4109a00 0x7f1bd41a1850 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd41070a0 0x7f1bd419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 --2- 172.21.15.19:0/1090341653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bd4104740 0x7f1bd419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 >> 172.21.15.19:0/1090341653 conn(0x7f1bd40fdd40 msgr2=0x7f1bd4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 shutdown_connections 2024-05-09T01:25:57.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.244+0000 7f1bdbb34640 1 -- 172.21.15.19:0/1090341653 wait complete. 2024-05-09T01:25:57.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.382+0000 7fa55368b640 1 -- 172.21.15.19:0/2844645110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 msgr2=0x7fa54c104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.382+0000 7fa55368b640 1 --2- 172.21.15.19:0/2844645110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 0x7fa54c104970 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa548009900 tx=0x7fa54802a410 comp rx=0 tx=0).stop 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.382+0000 7fa55368b640 1 -- 172.21.15.19:0/2844645110 shutdown_connections 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.382+0000 7fa55368b640 1 --2- 172.21.15.19:0/2844645110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa54c109bb0 0x7fa54c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.382+0000 7fa55368b640 1 --2- 172.21.15.19:0/2844645110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c104eb0 0x7fa54c1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.382+0000 7fa55368b640 1 --2- 172.21.15.19:0/2844645110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 0x7fa54c104970 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.382+0000 7fa55368b640 1 -- 172.21.15.19:0/2844645110 >> 172.21.15.19:0/2844645110 conn(0x7fa54c0fee40 msgr2=0x7fa54c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.383+0000 7fa55368b640 1 -- 172.21.15.19:0/2844645110 shutdown_connections 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.383+0000 7fa55368b640 1 -- 172.21.15.19:0/2844645110 wait complete. 2024-05-09T01:25:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.383+0000 7fa55368b640 1 Processor -- start 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.383+0000 7fa55368b640 1 -- start start 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.383+0000 7fa55368b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 0x7fa54c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.383+0000 7fa55368b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c104eb0 0x7fa54c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa55368b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa54c109bb0 0x7fa54c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa55368b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c10f770 con 0x7fa54c103c70 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa55368b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c10f5f0 con 0x7fa54c109bb0 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa55368b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c10f8f0 con 0x7fa54c104eb0 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551427640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 0x7fa54c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551427640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 0x7fa54c19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55544/0 (socket says 172.21.15.19:55544) 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551427640 1 -- 172.21.15.19:0/3302821065 learned_addr learned my addr 172.21.15.19:0/3302821065 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551c28640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa54c109bb0 0x7fa54c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa550c26640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c104eb0 0x7fa54c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551427640 1 -- 172.21.15.19:0/3302821065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c104eb0 msgr2=0x7fa54c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551427640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c104eb0 0x7fa54c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551427640 1 -- 172.21.15.19:0/3302821065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa54c109bb0 msgr2=0x7fa54c1a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551427640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa54c109bb0 0x7fa54c1a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551427640 1 -- 172.21.15.19:0/3302821065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54c1a1ef0 con 0x7fa54c103c70 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551c28640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa54c109bb0 0x7fa54c1a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa550c26640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c104eb0 0x7fa54c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa551427640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 0x7fa54c19a3b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa548006010 tx=0x7fa548004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa5427fc640 1 -- 172.21.15.19:0/3302821065 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54803f030 con 0x7fa54c103c70 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa5427fc640 1 -- 172.21.15.19:0/3302821065 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa548003aa0 con 0x7fa54c103c70 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54c1a2180 con 0x7fa54c103c70 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.384+0000 7fa5427fc640 1 -- 172.21.15.19:0/3302821065 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548035070 con 0x7fa54c103c70 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.385+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54c1a2670 con 0x7fa54c103c70 2024-05-09T01:25:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.385+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa54c1089c0 con 0x7fa54c103c70 2024-05-09T01:25:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.387+0000 7fa5427fc640 1 -- 172.21.15.19:0/3302821065 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa548042030 con 0x7fa54c103c70 2024-05-09T01:25:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.388+0000 7fa5427fc640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5380b27e0 0x7fa5380b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.388+0000 7fa5427fc640 1 -- 172.21.15.19:0/3302821065 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa548102cf0 con 0x7fa54c103c70 2024-05-09T01:25:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.388+0000 7fa550c26640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5380b27e0 0x7fa5380b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.388+0000 7fa550c26640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5380b27e0 0x7fa5380b4cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa53c005fd0 tx=0x7fa53c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.391+0000 7fa5427fc640 1 -- 172.21.15.19:0/3302821065 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5480ca9f0 con 0x7fa54c103c70 2024-05-09T01:25:57.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:57 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1090341653' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T01:25:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.616+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa54c0631b0 con 0x7fa5380b27e0 2024-05-09T01:25:57.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:57 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1090341653' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T01:25:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.637+0000 7fa5427fc640 1 -- 172.21.15.19:0/3302821065 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fa54c0631b0 con 0x7fa5380b27e0 2024-05-09T01:25:57.635 INFO:teuthology.orchestra.run.smithi019.stdout:{} 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5380b27e0 msgr2=0x7fa5380b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5380b27e0 0x7fa5380b4cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa53c005fd0 tx=0x7fa53c005850 comp rx=0 tx=0).stop 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 msgr2=0x7fa54c19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 0x7fa54c19a3b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa548006010 tx=0x7fa548004270 comp rx=0 tx=0).stop 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 shutdown_connections 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5380b27e0 0x7fa5380b4cf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa54c109bb0 0x7fa54c1a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c104eb0 0x7fa54c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 --2- 172.21.15.19:0/3302821065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa54c103c70 0x7fa54c19a3b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 >> 172.21.15.19:0/3302821065 conn(0x7fa54c0fee40 msgr2=0x7fa54c105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 shutdown_connections 2024-05-09T01:25:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.639+0000 7fa55368b640 1 -- 172.21.15.19:0/3302821065 wait complete. 2024-05-09T01:25:57.648 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T01:25:57.660 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-09T01:25:57.678 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T01:25:57.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:57 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1090341653' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T01:25:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.795+0000 7ff2db576640 1 -- 172.21.15.19:0/70952222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d50 msgr2=0x7ff2d4104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.795+0000 7ff2db576640 1 --2- 172.21.15.19:0/70952222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d50 0x7ff2d4104160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009860 tx=0x7ff2c402a340 comp rx=0 tx=0).stop 2024-05-09T01:25:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.796+0000 7ff2db576640 1 -- 172.21.15.19:0/70952222 shutdown_connections 2024-05-09T01:25:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.796+0000 7ff2db576640 1 --2- 172.21.15.19:0/70952222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4108a50 0x7ff2d410ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.796+0000 7ff2db576640 1 --2- 172.21.15.19:0/70952222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d50 0x7ff2d4104160 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.796+0000 7ff2db576640 1 --2- 172.21.15.19:0/70952222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4102b10 0x7ff2d4103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.796+0000 7ff2db576640 1 -- 172.21.15.19:0/70952222 >> 172.21.15.19:0/70952222 conn(0x7ff2d40fdd00 msgr2=0x7ff2d41001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.796+0000 7ff2db576640 1 -- 172.21.15.19:0/70952222 shutdown_connections 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.796+0000 7ff2db576640 1 -- 172.21.15.19:0/70952222 wait complete. 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.796+0000 7ff2db576640 1 Processor -- start 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.796+0000 7ff2db576640 1 -- start start 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2db576640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4102b10 0x7ff2d419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2db576640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d50 0x7ff2d419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2db576640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4108a50 0x7ff2d41a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2db576640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d410e640 con 0x7ff2d4108a50 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2db576640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d410e4c0 con 0x7ff2d4103d50 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2db576640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d410e7c0 con 0x7ff2d4102b10 2024-05-09T01:25:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9b13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4108a50 0x7ff2d41a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9b13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4108a50 0x7ff2d41a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55598/0 (socket says 172.21.15.19:55598) 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9b13640 1 -- 172.21.15.19:0/66854598 learned_addr learned my addr 172.21.15.19:0/66854598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d8b11640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d50 0x7ff2d419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9312640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4102b10 0x7ff2d419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9b13640 1 -- 172.21.15.19:0/66854598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4102b10 msgr2=0x7ff2d419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9b13640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4102b10 0x7ff2d419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9b13640 1 -- 172.21.15.19:0/66854598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d50 msgr2=0x7ff2d419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9b13640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d50 0x7ff2d419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9b13640 1 -- 172.21.15.19:0/66854598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d41a1f10 con 0x7ff2d4108a50 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2d9b13640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4108a50 0x7ff2d41a18a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00eaa0 tx=0x7ff2cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/66854598 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc00cdb0 con 0x7ff2d4108a50 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d41a2200 con 0x7ff2d4108a50 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/66854598 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc005020 con 0x7ff2d4108a50 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d41a27a0 con 0x7ff2d4108a50 2024-05-09T01:25:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.797+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/66854598 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc01f060 con 0x7ff2d4108a50 2024-05-09T01:25:57.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.798+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d41a2a60 con 0x7ff2d4108a50 2024-05-09T01:25:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.800+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/66854598 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff2cc0054a0 con 0x7ff2d4108a50 2024-05-09T01:25:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.801+0000 7ff2ca7fc640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27f0 0x7ff2c00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.801+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/66854598 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff2cc0e2700 con 0x7ff2d4108a50 2024-05-09T01:25:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.801+0000 7ff2d9312640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27f0 0x7ff2c00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.801+0000 7ff2d9312640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27f0 0x7ff2c00b4d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00096c0 tx=0x7ff2d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.804+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/66854598 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2cc016030 con 0x7ff2d4108a50 2024-05-09T01:25:57.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:25:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:25:57.621+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:25:57.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:25:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:25:57.621+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:25:57.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:25:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:25:57.621+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:25:57.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:25:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:25:57.621+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:25:57.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:25:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:25:57.621+0000 7f9247a03700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T01:25:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:57.998+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2d41a2c30 con 0x7ff2d4108a50 2024-05-09T01:25:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.000+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/66854598 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22716 (secure 0 0 0) 0x7ff2cc0aa490 con 0x7ff2d4108a50 2024-05-09T01:25:57.998 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:25:57.998 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:23:22.298486+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:25:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27f0 msgr2=0x7ff2c00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27f0 0x7ff2c00b4d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00096c0 tx=0x7ff2d0009260 comp rx=0 tx=0).stop 2024-05-09T01:25:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4108a50 msgr2=0x7ff2d41a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4108a50 0x7ff2d41a18a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00eaa0 tx=0x7ff2cc00ef60 comp rx=0 tx=0).stop 2024-05-09T01:25:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 shutdown_connections 2024-05-09T01:25:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27f0 0x7ff2c00b4d00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4108a50 0x7ff2d41a18a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d50 0x7ff2d419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 --2- 172.21.15.19:0/66854598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4102b10 0x7ff2d419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 >> 172.21.15.19:0/66854598 conn(0x7ff2d40fdd00 msgr2=0x7ff2d410a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 shutdown_connections 2024-05-09T01:25:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.002+0000 7ff2db576640 1 -- 172.21.15.19:0/66854598 wait complete. 2024-05-09T01:25:58.011 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T01:17:49.120859+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T01:22:34.932415+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T01:22:38.555946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T01:25:58.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T01:25:58.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 -- 172.21.15.19:0/4013336624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74104e60 msgr2=0x7efe74105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 --2- 172.21.15.19:0/4013336624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74104e60 0x7efe74105270 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe6802a410 comp rx=0 tx=0).stop 2024-05-09T01:25:58.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 -- 172.21.15.19:0/4013336624 shutdown_connections 2024-05-09T01:25:58.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 --2- 172.21.15.19:0/4013336624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe74109b60 0x7efe7410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 --2- 172.21.15.19:0/4013336624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74104e60 0x7efe74105270 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 --2- 172.21.15.19:0/4013336624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe74103c20 0x7efe74104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 -- 172.21.15.19:0/4013336624 >> 172.21.15.19:0/4013336624 conn(0x7efe740fee10 msgr2=0x7efe741012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 -- 172.21.15.19:0/4013336624 shutdown_connections 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 -- 172.21.15.19:0/4013336624 wait complete. 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.132+0000 7efe79984640 1 Processor -- start 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe79984640 1 -- start start 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe79984640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe74103c20 0x7efe7419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe79984640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe74104e60 0x7efe7419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe79984640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74109b60 0x7efe741a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe79984640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe7410f740 con 0x7efe74109b60 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe79984640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe7410f5c0 con 0x7efe74104e60 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe79984640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe7410f8c0 con 0x7efe74103c20 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74109b60 0x7efe741a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74109b60 0x7efe741a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55608/0 (socket says 172.21.15.19:55608) 2024-05-09T01:25:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe737fe640 1 -- 172.21.15.19:0/1214208023 learned_addr learned my addr 172.21.15.19:0/1214208023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe727fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe74104e60 0x7efe7419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe72ffd640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe74103c20 0x7efe7419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe737fe640 1 -- 172.21.15.19:0/1214208023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe74103c20 msgr2=0x7efe7419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe737fe640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe74103c20 0x7efe7419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe737fe640 1 -- 172.21.15.19:0/1214208023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe74104e60 msgr2=0x7efe7419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe737fe640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe74104e60 0x7efe7419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe737fe640 1 -- 172.21.15.19:0/1214208023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe741a1eb0 con 0x7efe74109b60 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe727fc640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe74104e60 0x7efe7419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.133+0000 7efe737fe640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74109b60 0x7efe741a1840 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00efd0 tx=0x7efe6c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.134+0000 7efe78982640 1 -- 172.21.15.19:0/1214208023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c003b70 con 0x7efe74109b60 2024-05-09T01:25:58.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.134+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe741a21a0 con 0x7efe74109b60 2024-05-09T01:25:58.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.134+0000 7efe78982640 1 -- 172.21.15.19:0/1214208023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6c003d10 con 0x7efe74109b60 2024-05-09T01:25:58.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.134+0000 7efe78982640 1 -- 172.21.15.19:0/1214208023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0108a0 con 0x7efe74109b60 2024-05-09T01:25:58.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.134+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741a26f0 con 0x7efe74109b60 2024-05-09T01:25:58.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.135+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe74108970 con 0x7efe74109b60 2024-05-09T01:25:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.136+0000 7efe78982640 1 -- 172.21.15.19:0/1214208023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efe6c004b40 con 0x7efe74109b60 2024-05-09T01:25:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.137+0000 7efe78982640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efe5c0b27f0 0x7efe5c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.137+0000 7efe72ffd640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efe5c0b27f0 0x7efe5c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.137+0000 7efe78982640 1 -- 172.21.15.19:0/1214208023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7efe6c0e2b70 con 0x7efe74109b60 2024-05-09T01:25:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.138+0000 7efe72ffd640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efe5c0b27f0 0x7efe5c0b4d00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efe74104820 tx=0x7efe64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.141+0000 7efe78982640 1 -- 172.21.15.19:0/1214208023 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe6c0aa810 con 0x7efe74109b60 2024-05-09T01:25:58.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.333+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe740638d0 con 0x7efe74109b60 2024-05-09T01:25:58.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.333+0000 7efe78982640 1 -- 172.21.15.19:0/1214208023 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7efe6c0aa110 con 0x7efe74109b60 2024-05-09T01:25:58.331 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-09T01:25:58.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efe5c0b27f0 msgr2=0x7efe5c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efe5c0b27f0 0x7efe5c0b4d00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efe74104820 tx=0x7efe64005f90 comp rx=0 tx=0).stop 2024-05-09T01:25:58.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74109b60 msgr2=0x7efe741a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74109b60 0x7efe741a1840 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00efd0 tx=0x7efe6c00c6a0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 shutdown_connections 2024-05-09T01:25:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efe5c0b27f0 0x7efe5c0b4d00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe74109b60 0x7efe741a1840 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe74104e60 0x7efe7419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 --2- 172.21.15.19:0/1214208023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe74103c20 0x7efe7419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.335+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 >> 172.21.15.19:0/1214208023 conn(0x7efe740fee10 msgr2=0x7efe74105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.336+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 shutdown_connections 2024-05-09T01:25:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.336+0000 7efe79984640 1 -- 172.21.15.19:0/1214208023 wait complete. 2024-05-09T01:25:58.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:58 smithi019 ceph-mon[30158]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:58.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:58 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1090341653' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-09T01:25:58.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:58 smithi019 ceph-mon[30158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:25:58.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:58 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/66854598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:25:58.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T01:25:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.465+0000 7f5d98843640 1 -- 172.21.15.19:0/3695889596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 msgr2=0x7f5d941055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.465+0000 7f5d98843640 1 --2- 172.21.15.19:0/3695889596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 0x7f5d941055c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009900 tx=0x7f5d8402a410 comp rx=0 tx=0).stop 2024-05-09T01:25:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.465+0000 7f5d98843640 1 -- 172.21.15.19:0/3695889596 shutdown_connections 2024-05-09T01:25:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.465+0000 7f5d98843640 1 --2- 172.21.15.19:0/3695889596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d94102360 0x7f5d9410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.465+0000 7f5d98843640 1 --2- 172.21.15.19:0/3695889596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d94101860 0x7f5d94101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.465+0000 7f5d98843640 1 --2- 172.21.15.19:0/3695889596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 0x7f5d941055c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.465+0000 7f5d98843640 1 -- 172.21.15.19:0/3695889596 >> 172.21.15.19:0/3695889596 conn(0x7f5d940fd460 msgr2=0x7f5d940ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:58.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.465+0000 7f5d98843640 1 -- 172.21.15.19:0/3695889596 shutdown_connections 2024-05-09T01:25:58.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.465+0000 7f5d98843640 1 -- 172.21.15.19:0/3695889596 wait complete. 2024-05-09T01:25:58.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d98843640 1 Processor -- start 2024-05-09T01:25:58.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d98843640 1 -- start start 2024-05-09T01:25:58.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d98843640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d94101860 0x7f5d9419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d98843640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d94102360 0x7f5d9419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d98843640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 0x7f5d941a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d98843640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410f780 con 0x7f5d941051b0 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d98843640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410f600 con 0x7f5d94102360 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d98843640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410f900 con 0x7f5d94101860 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d92d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 0x7f5d941a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d92d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 0x7f5d941a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55618/0 (socket says 172.21.15.19:55618) 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d92d9d640 1 -- 172.21.15.19:0/4209991159 learned_addr learned my addr 172.21.15.19:0/4209991159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d91d9b640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d94102360 0x7f5d9419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d9259c640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d94101860 0x7f5d9419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d92d9d640 1 -- 172.21.15.19:0/4209991159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d94101860 msgr2=0x7f5d9419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d92d9d640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d94101860 0x7f5d9419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d92d9d640 1 -- 172.21.15.19:0/4209991159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d94102360 msgr2=0x7f5d9419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d92d9d640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d94102360 0x7f5d9419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d92d9d640 1 -- 172.21.15.19:0/4209991159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d941a1e50 con 0x7f5d941051b0 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.466+0000 7f5d9259c640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d94101860 0x7f5d9419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.467+0000 7f5d92d9d640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 0x7f5d941a1890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00b970 tx=0x7f5d8c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.467+0000 7f5d837fe640 1 -- 172.21.15.19:0/4209991159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c019070 con 0x7f5d941051b0 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.467+0000 7f5d837fe640 1 -- 172.21.15.19:0/4209991159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c014440 con 0x7f5d941051b0 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.467+0000 7f5d837fe640 1 -- 172.21.15.19:0/4209991159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0075d0 con 0x7f5d941051b0 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.467+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d941a2140 con 0x7f5d941051b0 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.467+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d941a2760 con 0x7f5d941051b0 2024-05-09T01:25:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.468+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d94108b40 con 0x7f5d941051b0 2024-05-09T01:25:58.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.469+0000 7f5d837fe640 1 -- 172.21.15.19:0/4209991159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5d8c01c070 con 0x7f5d941051b0 2024-05-09T01:25:58.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.470+0000 7f5d837fe640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d7c0b2870 0x7f5d7c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.471+0000 7f5d9259c640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d7c0b2870 0x7f5d7c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.471+0000 7f5d837fe640 1 -- 172.21.15.19:0/4209991159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5d8c0e1bf0 con 0x7f5d941051b0 2024-05-09T01:25:58.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.471+0000 7f5d9259c640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d7c0b2870 0x7f5d7c0b4d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009230 tx=0x7f5d84000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:58.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.474+0000 7f5d837fe640 1 -- 172.21.15.19:0/4209991159 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d8c0a98d0 con 0x7f5d941051b0 2024-05-09T01:25:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:58 smithi059 ceph-mon[34351]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:58 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1090341653' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-09T01:25:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:58 smithi059 ceph-mon[34351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:25:58.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:58 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/66854598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:25:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.666+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5d9402dde0 con 0x7f5d941051b0 2024-05-09T01:25:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.666+0000 7f5d837fe640 1 -- 172.21.15.19:0/4209991159 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f5d8c0a91d0 con 0x7f5d941051b0 2024-05-09T01:25:58.664 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-09T01:25:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d7c0b2870 msgr2=0x7f5d7c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d7c0b2870 0x7f5d7c0b4d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009230 tx=0x7f5d84000bc0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 msgr2=0x7f5d941a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 0x7f5d941a1890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00b970 tx=0x7f5d8c00bd70 comp rx=0 tx=0).stop 2024-05-09T01:25:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 shutdown_connections 2024-05-09T01:25:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d7c0b2870 0x7f5d7c0b4d80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d941051b0 0x7f5d941a1890 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d94102360 0x7f5d9419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 --2- 172.21.15.19:0/4209991159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d94101860 0x7f5d9419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.668+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 >> 172.21.15.19:0/4209991159 conn(0x7f5d940fd460 msgr2=0x7f5d94105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:58.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.669+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 shutdown_connections 2024-05-09T01:25:58.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.669+0000 7f5d98843640 1 -- 172.21.15.19:0/4209991159 wait complete. 2024-05-09T01:25:58.678 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T01:25:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:58 smithi196 ceph-mon[35041]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T01:25:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:58 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1090341653' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-09T01:25:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:58 smithi196 ceph-mon[35041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:25:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:58 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/66854598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:25:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 -- 172.21.15.19:0/805647622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 msgr2=0x7fca9410ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 --2- 172.21.15.19:0/805647622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 0x7fca9410ae70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009ca0 tx=0x7fca8c02a8c0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 -- 172.21.15.19:0/805647622 shutdown_connections 2024-05-09T01:25:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 --2- 172.21.15.19:0/805647622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 0x7fca9410ae70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 --2- 172.21.15.19:0/805647622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca94103d50 0x7fca94104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 --2- 172.21.15.19:0/805647622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca94102b10 0x7fca94103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 -- 172.21.15.19:0/805647622 >> 172.21.15.19:0/805647622 conn(0x7fca940fdd00 msgr2=0x7fca941001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 -- 172.21.15.19:0/805647622 shutdown_connections 2024-05-09T01:25:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 -- 172.21.15.19:0/805647622 wait complete. 2024-05-09T01:25:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.798+0000 7fca99b86640 1 Processor -- start 2024-05-09T01:25:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca99b86640 1 -- start start 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca99b86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca94102b10 0x7fca94199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca99b86640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca94103d50 0x7fca94199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca99b86640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 0x7fca941a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca99b86640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca9410e640 con 0x7fca94108a50 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca99b86640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca9410e4c0 con 0x7fca94103d50 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca99b86640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca9410e7c0 con 0x7fca94102b10 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca93fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 0x7fca941a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca93fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 0x7fca941a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55638/0 (socket says 172.21.15.19:55638) 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca93fff640 1 -- 172.21.15.19:0/2665815380 learned_addr learned my addr 172.21.15.19:0/2665815380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca92ffd640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca94103d50 0x7fca94199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca937fe640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca94102b10 0x7fca94199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca93fff640 1 -- 172.21.15.19:0/2665815380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca94102b10 msgr2=0x7fca94199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca93fff640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca94102b10 0x7fca94199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca93fff640 1 -- 172.21.15.19:0/2665815380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca94103d50 msgr2=0x7fca94199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca93fff640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca94103d50 0x7fca94199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca93fff640 1 -- 172.21.15.19:0/2665815380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca941a1080 con 0x7fca94108a50 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.799+0000 7fca92ffd640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca94103d50 0x7fca94199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.800+0000 7fca93fff640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 0x7fca941a0a10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0097b0 tx=0x7fca8c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.800+0000 7fca90ff9640 1 -- 172.21.15.19:0/2665815380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004240 con 0x7fca94108a50 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.800+0000 7fca90ff9640 1 -- 172.21.15.19:0/2665815380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca8c02f380 con 0x7fca94108a50 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.800+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca941a1310 con 0x7fca94108a50 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.800+0000 7fca90ff9640 1 -- 172.21.15.19:0/2665815380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c02fc30 con 0x7fca94108a50 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.800+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca941a18b0 con 0x7fca94108a50 2024-05-09T01:25:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.801+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca94107860 con 0x7fca94108a50 2024-05-09T01:25:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.802+0000 7fca90ff9640 1 -- 172.21.15.19:0/2665815380 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fca8c0043e0 con 0x7fca94108a50 2024-05-09T01:25:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.803+0000 7fca90ff9640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca7c0b2870 0x7fca7c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:25:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.803+0000 7fca937fe640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca7c0b2870 0x7fca7c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:25:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.803+0000 7fca90ff9640 1 -- 172.21.15.19:0/2665815380 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fca8c103b30 con 0x7fca94108a50 2024-05-09T01:25:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.804+0000 7fca937fe640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca7c0b2870 0x7fca7c0b4d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca84005fd0 tx=0x7fca84005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:25:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:58.807+0000 7fca90ff9640 1 -- 172.21.15.19:0/2665815380 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca8c0cb830 con 0x7fca94108a50 2024-05-09T01:25:58.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.000+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca9402dd50 con 0x7fca94108a50 2024-05-09T01:25:58.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.001+0000 7fca90ff9640 1 -- 172.21.15.19:0/2665815380 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7fca8c0cb130 con 0x7fca94108a50 2024-05-09T01:25:58.998 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca7c0b2870 msgr2=0x7fca7c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca7c0b2870 0x7fca7c0b4d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca84005fd0 tx=0x7fca84005850 comp rx=0 tx=0).stop 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 msgr2=0x7fca941a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 0x7fca941a0a10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0097b0 tx=0x7fca8c003c20 comp rx=0 tx=0).stop 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 shutdown_connections 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca7c0b2870 0x7fca7c0b4d80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca94108a50 0x7fca941a0a10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca94103d50 0x7fca94199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 --2- 172.21.15.19:0/2665815380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca94102b10 0x7fca94199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 >> 172.21.15.19:0/2665815380 conn(0x7fca940fdd00 msgr2=0x7fca94104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:25:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 shutdown_connections 2024-05-09T01:25:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:25:59.003+0000 7fca99b86640 1 -- 172.21.15.19:0/2665815380 wait complete. 2024-05-09T01:25:59.012 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-09T01:25:59.012 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:25:59.270 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:25:59.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:59 smithi019 ceph-mon[30158]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:59.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:59 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1214208023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:25:59.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:59 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/4209991159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T01:25:59.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:25:59 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2665815380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T01:25:59.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:59 smithi059 ceph-mon[34351]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:59.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:59 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1214208023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:25:59.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:59 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/4209991159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T01:25:59.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:25:59 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2665815380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T01:25:59.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:59 smithi196 ceph-mon[35041]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:25:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:59 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1214208023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:25:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:59 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/4209991159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T01:25:59.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:25:59 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2665815380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T01:26:00.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:00 smithi019 ceph-mon[30158]: pgmap v392: 97 pgs: 97 active+clean; 646 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:26:00.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:00 smithi059 ceph-mon[34351]: pgmap v392: 97 pgs: 97 active+clean; 646 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:26:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:00 smithi196 ceph-mon[35041]: pgmap v392: 97 pgs: 97 active+clean; 646 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:26:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.892+0000 7fd7f5723700 1 -- 172.21.15.19:0/3066427734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f01034c0 msgr2=0x7fd7f01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.892+0000 7fd7f5723700 1 --2- 172.21.15.19:0/3066427734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f01034c0 0x7fd7f01038d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00099c0 tx=0x7fd7e0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:26:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.892+0000 7fd7f5723700 1 -- 172.21.15.19:0/3066427734 shutdown_connections 2024-05-09T01:26:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.892+0000 7fd7f5723700 1 --2- 172.21.15.19:0/3066427734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f0108740 0x7fd7f01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.892+0000 7fd7f5723700 1 --2- 172.21.15.19:0/3066427734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f01034c0 0x7fd7f01038d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.892+0000 7fd7f5723700 1 --2- 172.21.15.19:0/3066427734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f0107df0 0x7fd7f0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.892+0000 7fd7f5723700 1 -- 172.21.15.19:0/3066427734 >> 172.21.15.19:0/3066427734 conn(0x7fd7f00fedf0 msgr2=0x7fd7f0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.893+0000 7fd7f5723700 1 -- 172.21.15.19:0/3066427734 shutdown_connections 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.893+0000 7fd7f5723700 1 -- 172.21.15.19:0/3066427734 wait complete. 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7f5723700 1 Processor -- start 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7f5723700 1 -- start start 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7f5723700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f01034c0 0x7fd7f019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7f5723700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f0107df0 0x7fd7f019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7f5723700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f0108740 0x7fd7f019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7f5723700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0113370 con 0x7fd7f0107df0 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7f5723700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01131f0 con 0x7fd7f01034c0 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7f5723700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01134f0 con 0x7fd7f0108740 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7ee7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f0107df0 0x7fd7f019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7ee7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f0107df0 0x7fd7f019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58364/0 (socket says 172.21.15.19:58364) 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7ee7fc700 1 -- 172.21.15.19:0/2965250631 learned_addr learned my addr 172.21.15.19:0/2965250631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7ef7fe700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f0108740 0x7fd7f019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.894+0000 7fd7eeffd700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f01034c0 0x7fd7f019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7ee7fc700 1 -- 172.21.15.19:0/2965250631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f0108740 msgr2=0x7fd7f019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7ee7fc700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f0108740 0x7fd7f019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7ee7fc700 1 -- 172.21.15.19:0/2965250631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f01034c0 msgr2=0x7fd7f019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7ee7fc700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f01034c0 0x7fd7f019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7ee7fc700 1 -- 172.21.15.19:0/2965250631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f019fcc0 con 0x7fd7f0107df0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7ee7fc700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f0107df0 0x7fd7f019b3f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400d980 tx=0x7fd7e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7d7fff700 1 -- 172.21.15.19:0/2965250631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4017070 con 0x7fd7f0107df0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7d7fff700 1 -- 172.21.15.19:0/2965250631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e4009de0 con 0x7fd7f0107df0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f019ffb0 con 0x7fd7f0107df0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7d7fff700 1 -- 172.21.15.19:0/2965250631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400f3c0 con 0x7fd7f0107df0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.895+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f01a78d0 con 0x7fd7f0107df0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.896+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f00512c0 con 0x7fd7f0107df0 2024-05-09T01:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.899+0000 7fd7d7fff700 1 -- 172.21.15.19:0/2965250631 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fd7e4009930 con 0x7fd7f0107df0 2024-05-09T01:26:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.899+0000 7fd7d7fff700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7d80b2910 0x7fd7d80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.899+0000 7fd7eeffd700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7d80b2910 0x7fd7d80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.899+0000 7fd7d7fff700 1 -- 172.21.15.19:0/2965250631 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd7e40e1e90 con 0x7fd7f0107df0 2024-05-09T01:26:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.900+0000 7fd7eeffd700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7d80b2910 0x7fd7d80b4e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0003800 tx=0x7fd7e001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:00.903+0000 7fd7d7fff700 1 -- 172.21.15.19:0/2965250631 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7e40a9a30 con 0x7fd7f0107df0 2024-05-09T01:26:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.163+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7f01a7aa0 con 0x7fd7f0107df0 2024-05-09T01:26:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.164+0000 7fd7d7fff700 1 -- 172.21.15.19:0/2965250631 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fd7e40a9330 con 0x7fd7f0107df0 2024-05-09T01:26:01.162 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:26:01.162 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7d80b2910 msgr2=0x7fd7d80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7d80b2910 0x7fd7d80b4e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0003800 tx=0x7fd7e001a040 comp rx=0 tx=0).stop 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f0107df0 msgr2=0x7fd7f019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f0107df0 0x7fd7f019b3f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400d980 tx=0x7fd7e400dc90 comp rx=0 tx=0).stop 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 shutdown_connections 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f0108740 0x7fd7f019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f01034c0 0x7fd7f019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7d80b2910 0x7fd7d80b4e20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 --2- 172.21.15.19:0/2965250631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7f0107df0 0x7fd7f019b3f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 >> 172.21.15.19:0/2965250631 conn(0x7fd7f00fedf0 msgr2=0x7fd7f0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.166+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 shutdown_connections 2024-05-09T01:26:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.167+0000 7fd7f5723700 1 -- 172.21.15.19:0/2965250631 wait complete. 2024-05-09T01:26:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:26:01.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:01 smithi019 ceph-mon[30158]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T01:26:01.483 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:01 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2965250631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:26:01.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:01 smithi059 ceph-mon[34351]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T01:26:01.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:01 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2965250631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:26:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:01 smithi196 ceph-mon[35041]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T01:26:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:01 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2965250631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:26:01.818 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-09T01:26:01.818 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-09T01:26:01.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T01:26:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.939+0000 7f574d900640 1 -- 172.21.15.19:0/891619542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748109b70 msgr2=0x7f574810bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.939+0000 7f574d900640 1 --2- 172.21.15.19:0/891619542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748109b70 0x7f574810bf90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5740009ca0 tx=0x7f574002a8c0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.939+0000 7f574d900640 1 -- 172.21.15.19:0/891619542 shutdown_connections 2024-05-09T01:26:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.939+0000 7f574d900640 1 --2- 172.21.15.19:0/891619542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748109b70 0x7f574810bf90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.939+0000 7f574d900640 1 --2- 172.21.15.19:0/891619542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5748104e70 0x7f5748105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.939+0000 7f574d900640 1 --2- 172.21.15.19:0/891619542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5748103c30 0x7f5748104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.939+0000 7f574d900640 1 -- 172.21.15.19:0/891619542 >> 172.21.15.19:0/891619542 conn(0x7f57480fee40 msgr2=0x7f57481012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.939+0000 7f574d900640 1 -- 172.21.15.19:0/891619542 shutdown_connections 2024-05-09T01:26:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.939+0000 7f574d900640 1 -- 172.21.15.19:0/891619542 wait complete. 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f574d900640 1 Processor -- start 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f574d900640 1 -- start start 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f574d900640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748103c30 0x7f5748077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f574d900640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5748104e70 0x7f5748076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f574d900640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5748109b70 0x7f5748076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f574d900640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f574810f740 con 0x7f5748103c30 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f574d900640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f574810f5c0 con 0x7f5748109b70 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f574d900640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f574810f8c0 con 0x7f5748104e70 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f5746ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748103c30 0x7f5748077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f5746ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748103c30 0x7f5748077a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58382/0 (socket says 172.21.15.19:58382) 2024-05-09T01:26:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f5746ffd640 1 -- 172.21.15.19:0/3859215632 learned_addr learned my addr 172.21.15.19:0/3859215632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f573ffff640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5748104e70 0x7f5748076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.940+0000 7f57477fe640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5748109b70 0x7f5748076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f5746ffd640 1 -- 172.21.15.19:0/3859215632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5748104e70 msgr2=0x7f5748076130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f5746ffd640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5748104e70 0x7f5748076130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f5746ffd640 1 -- 172.21.15.19:0/3859215632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5748109b70 msgr2=0x7f5748076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f5746ffd640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5748109b70 0x7f5748076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f5746ffd640 1 -- 172.21.15.19:0/3859215632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5748076da0 con 0x7f5748103c30 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f5746ffd640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748103c30 0x7f5748077a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5738005fd0 tx=0x7f573800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f574c8fe640 1 -- 172.21.15.19:0/3859215632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738017070 con 0x7f5748103c30 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f574c8fe640 1 -- 172.21.15.19:0/3859215632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5738009c30 con 0x7f5748103c30 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57481a43e0 con 0x7f5748103c30 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f574c8fe640 1 -- 172.21.15.19:0/3859215632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738003d10 con 0x7f5748103c30 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.941+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57481a48a0 con 0x7f5748103c30 2024-05-09T01:26:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.942+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5748108980 con 0x7f5748103c30 2024-05-09T01:26:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.944+0000 7f574c8fe640 1 -- 172.21.15.19:0/3859215632 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f5738003eb0 con 0x7f5748103c30 2024-05-09T01:26:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.944+0000 7f574c8fe640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f57340b2880 0x7f57340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.945+0000 7f573ffff640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f57340b2880 0x7f57340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.945+0000 7f574c8fe640 1 -- 172.21.15.19:0/3859215632 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f57380e2770 con 0x7f5748103c30 2024-05-09T01:26:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.945+0000 7f573ffff640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f57340b2880 0x7f57340b4d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5748077430 tx=0x7f5730005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:01.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:01.948+0000 7f574c8fe640 1 -- 172.21.15.19:0/3859215632 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5738020030 con 0x7f5748103c30 2024-05-09T01:26:02.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.137+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5748063110 con 0x7f57340b2880 2024-05-09T01:26:02.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.145+0000 7f574c8fe640 1 -- 172.21.15.19:0/3859215632 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7f5748063110 con 0x7f57340b2880 2024-05-09T01:26:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:26:02.147 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f57340b2880 msgr2=0x7f57340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f57340b2880 0x7f57340b4d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5748077430 tx=0x7f5730005f90 comp rx=0 tx=0).stop 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748103c30 msgr2=0x7f5748077a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748103c30 0x7f5748077a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5738005fd0 tx=0x7f573800dde0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 shutdown_connections 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f57340b2880 0x7f57340b4d90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5748109b70 0x7f5748076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5748104e70 0x7f5748076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 --2- 172.21.15.19:0/3859215632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5748103c30 0x7f5748077a90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 >> 172.21.15.19:0/3859215632 conn(0x7f57480fee40 msgr2=0x7f5748105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:02.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 shutdown_connections 2024-05-09T01:26:02.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.148+0000 7f574d900640 1 -- 172.21.15.19:0/3859215632 wait complete. 2024-05-09T01:26:02.158 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-05-09T01:26:01.178394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667473,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1910,"num_read_kb":1076,"num_write":398,"num_write_kb":2315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":2954185,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":828,"ondisk_log_size":828,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386964,"kb_used_data":12852,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554860,"statfs":{"total":1151940427776,"available":1151544176640,"internally_reserved":0,"allocated":13160448,"data_stored":7233183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6854,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":85,"num_write":14,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003416"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158424+0000","last_change":"2024-05-09T01:22:39.836756+0000","last_active":"2024-05-09T01:24:21.158424+0000","last_peered":"2024-05-09T01:24:21.158424+0000","last_clean":"2024-05-09T01:24:21.158424+0000","last_became_active":"2024-05-09T01:22:39.835505+0000","last_became_peered":"2024-05-09T01:22:39.835505+0000","last_unstale":"2024-05-09T01:24:21.158424+0000","last_undegraded":"2024-05-09T01:24:21.158424+0000","last_fullsized":"2024-05-09T01:24:21.158424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:29.109762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148168+0000","last_change":"2024-05-09T01:22:39.842897+0000","last_active":"2024-05-09T01:24:21.148168+0000","last_peered":"2024-05-09T01:24:21.148168+0000","last_clean":"2024-05-09T01:24:21.148168+0000","last_became_active":"2024-05-09T01:22:39.842537+0000","last_became_peered":"2024-05-09T01:22:39.842537+0000","last_unstale":"2024-05-09T01:24:21.148168+0000","last_undegraded":"2024-05-09T01:24:21.148168+0000","last_fullsized":"2024-05-09T01:24:21.148168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:20.429654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157917+0000","last_change":"2024-05-09T01:22:39.847176+0000","last_active":"2024-05-09T01:24:21.157917+0000","last_peered":"2024-05-09T01:24:21.157917+0000","last_clean":"2024-05-09T01:24:21.157917+0000","last_became_active":"2024-05-09T01:22:39.846782+0000","last_became_peered":"2024-05-09T01:22:39.846782+0000","last_unstale":"2024-05-09T01:24:21.157917+0000","last_undegraded":"2024-05-09T01:24:21.157917+0000","last_fullsized":"2024-05-09T01:24:21.157917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:06.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.137185+0000","last_change":"2024-05-09T01:22:39.842399+0000","last_active":"2024-05-09T01:24:21.137185+0000","last_peered":"2024-05-09T01:24:21.137185+0000","last_clean":"2024-05-09T01:24:21.137185+0000","last_became_active":"2024-05-09T01:22:39.842035+0000","last_became_peered":"2024-05-09T01:22:39.842035+0000","last_unstale":"2024-05-09T01:24:21.137185+0000","last_undegraded":"2024-05-09T01:24:21.137185+0000","last_fullsized":"2024-05-09T01:24:21.137185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:35.011920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.124665+0000","last_change":"2024-05-09T01:22:39.838574+0000","last_active":"2024-05-09T01:24:21.124665+0000","last_peered":"2024-05-09T01:24:21.124665+0000","last_clean":"2024-05-09T01:24:21.124665+0000","last_became_active":"2024-05-09T01:22:39.838059+0000","last_became_peered":"2024-05-09T01:22:39.838059+0000","last_unstale":"2024-05-09T01:24:21.124665+0000","last_undegraded":"2024-05-09T01:24:21.124665+0000","last_fullsized":"2024-05-09T01:24:21.124665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:20.968526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153364+0000","last_change":"2024-05-09T01:22:39.842407+0000","last_active":"2024-05-09T01:24:21.153364+0000","last_peered":"2024-05-09T01:24:21.153364+0000","last_clean":"2024-05-09T01:24:21.153364+0000","last_became_active":"2024-05-09T01:22:39.841982+0000","last_became_peered":"2024-05-09T01:22:39.841982+0000","last_unstale":"2024-05-09T01:24:21.153364+0000","last_undegraded":"2024-05-09T01:24:21.153364+0000","last_fullsized":"2024-05-09T01:24:21.153364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:17.086304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155290+0000","last_change":"2024-05-09T01:22:39.833800+0000","last_active":"2024-05-09T01:24:21.155290+0000","last_peered":"2024-05-09T01:24:21.155290+0000","last_clean":"2024-05-09T01:24:21.155290+0000","last_became_active":"2024-05-09T01:22:39.833388+0000","last_became_peered":"2024-05-09T01:22:39.833388+0000","last_unstale":"2024-05-09T01:24:21.155290+0000","last_undegraded":"2024-05-09T01:24:21.155290+0000","last_fullsized":"2024-05-09T01:24:21.155290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:35.341428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142200+0000","last_change":"2024-05-09T01:22:39.845395+0000","last_active":"2024-05-09T01:24:21.142200+0000","last_peered":"2024-05-09T01:24:21.142200+0000","last_clean":"2024-05-09T01:24:21.142200+0000","last_became_active":"2024-05-09T01:22:39.844946+0000","last_became_peered":"2024-05-09T01:22:39.844946+0000","last_unstale":"2024-05-09T01:24:21.142200+0000","last_undegraded":"2024-05-09T01:24:21.142200+0000","last_fullsized":"2024-05-09T01:24:21.142200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:44.750614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159768+0000","last_change":"2024-05-09T01:22:39.832894+0000","last_active":"2024-05-09T01:24:21.159768+0000","last_peered":"2024-05-09T01:24:21.159768+0000","last_clean":"2024-05-09T01:24:21.159768+0000","last_became_active":"2024-05-09T01:22:39.829743+0000","last_became_peered":"2024-05-09T01:22:39.829743+0000","last_unstale":"2024-05-09T01:24:21.159768+0000","last_undegraded":"2024-05-09T01:24:21.159768+0000","last_fullsized":"2024-05-09T01:24:21.159768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.463833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155902+0000","last_change":"2024-05-09T01:22:39.843131+0000","last_active":"2024-05-09T01:24:21.155902+0000","last_peered":"2024-05-09T01:24:21.155902+0000","last_clean":"2024-05-09T01:24:21.155902+0000","last_became_active":"2024-05-09T01:22:39.842690+0000","last_became_peered":"2024-05-09T01:22:39.842690+0000","last_unstale":"2024-05-09T01:24:21.155902+0000","last_undegraded":"2024-05-09T01:24:21.155902+0000","last_fullsized":"2024-05-09T01:24:21.155902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:14.172365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166267+0000","last_change":"2024-05-09T01:22:39.845557+0000","last_active":"2024-05-09T01:24:21.166267+0000","last_peered":"2024-05-09T01:24:21.166267+0000","last_clean":"2024-05-09T01:24:21.166267+0000","last_became_active":"2024-05-09T01:22:39.845059+0000","last_became_peered":"2024-05-09T01:22:39.845059+0000","last_unstale":"2024-05-09T01:24:21.166267+0000","last_undegraded":"2024-05-09T01:24:21.166267+0000","last_fullsized":"2024-05-09T01:24:21.166267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:47.238080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.129595+0000","last_change":"2024-05-09T01:22:39.842237+0000","last_active":"2024-05-09T01:24:21.129595+0000","last_peered":"2024-05-09T01:24:21.129595+0000","last_clean":"2024-05-09T01:24:21.129595+0000","last_became_active":"2024-05-09T01:22:39.841849+0000","last_became_peered":"2024-05-09T01:22:39.841849+0000","last_unstale":"2024-05-09T01:24:21.129595+0000","last_undegraded":"2024-05-09T01:24:21.129595+0000","last_fullsized":"2024-05-09T01:24:21.129595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:40.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150804+0000","last_change":"2024-05-09T01:22:39.839943+0000","last_active":"2024-05-09T01:24:21.150804+0000","last_peered":"2024-05-09T01:24:21.150804+0000","last_clean":"2024-05-09T01:24:21.150804+0000","last_became_active":"2024-05-09T01:22:39.839604+0000","last_became_peered":"2024-05-09T01:22:39.839604+0000","last_unstale":"2024-05-09T01:24:21.150804+0000","last_undegraded":"2024-05-09T01:24:21.150804+0000","last_fullsized":"2024-05-09T01:24:21.150804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:34.854263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145018+0000","last_change":"2024-05-09T01:22:39.846981+0000","last_active":"2024-05-09T01:24:21.145018+0000","last_peered":"2024-05-09T01:24:21.145018+0000","last_clean":"2024-05-09T01:24:21.145018+0000","last_became_active":"2024-05-09T01:22:39.846586+0000","last_became_peered":"2024-05-09T01:22:39.846586+0000","last_unstale":"2024-05-09T01:24:21.145018+0000","last_undegraded":"2024-05-09T01:24:21.145018+0000","last_fullsized":"2024-05-09T01:24:21.145018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:50.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151239+0000","last_change":"2024-05-09T01:22:39.848964+0000","last_active":"2024-05-09T01:24:21.151239+0000","last_peered":"2024-05-09T01:24:21.151239+0000","last_clean":"2024-05-09T01:24:21.151239+0000","last_became_active":"2024-05-09T01:22:39.848442+0000","last_became_peered":"2024-05-09T01:22:39.848442+0000","last_unstale":"2024-05-09T01:24:21.151239+0000","last_undegraded":"2024-05-09T01:24:21.151239+0000","last_fullsized":"2024-05-09T01:24:21.151239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:35.802739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164175+0000","last_change":"2024-05-09T01:22:39.840586+0000","last_active":"2024-05-09T01:24:21.164175+0000","last_peered":"2024-05-09T01:24:21.164175+0000","last_clean":"2024-05-09T01:24:21.164175+0000","last_became_active":"2024-05-09T01:22:39.839887+0000","last_became_peered":"2024-05-09T01:22:39.839887+0000","last_unstale":"2024-05-09T01:24:21.164175+0000","last_undegraded":"2024-05-09T01:24:21.164175+0000","last_fullsized":"2024-05-09T01:24:21.164175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:36.510039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158620+0000","last_change":"2024-05-09T01:22:39.836152+0000","last_active":"2024-05-09T01:24:21.158620+0000","last_peered":"2024-05-09T01:24:21.158620+0000","last_clean":"2024-05-09T01:24:21.158620+0000","last_became_active":"2024-05-09T01:22:39.834691+0000","last_became_peered":"2024-05-09T01:22:39.834691+0000","last_unstale":"2024-05-09T01:24:21.158620+0000","last_undegraded":"2024-05-09T01:24:21.158620+0000","last_fullsized":"2024-05-09T01:24:21.158620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:31.331096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163485+0000","last_change":"2024-05-09T01:22:39.839760+0000","last_active":"2024-05-09T01:24:21.163485+0000","last_peered":"2024-05-09T01:24:21.163485+0000","last_clean":"2024-05-09T01:24:21.163485+0000","last_became_active":"2024-05-09T01:22:39.839403+0000","last_became_peered":"2024-05-09T01:22:39.839403+0000","last_unstale":"2024-05-09T01:24:21.163485+0000","last_undegraded":"2024-05-09T01:24:21.163485+0000","last_fullsized":"2024-05-09T01:24:21.163485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:19.248368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142981+0000","last_change":"2024-05-09T01:22:39.837538+0000","last_active":"2024-05-09T01:24:21.142981+0000","last_peered":"2024-05-09T01:24:21.142981+0000","last_clean":"2024-05-09T01:24:21.142981+0000","last_became_active":"2024-05-09T01:22:39.837167+0000","last_became_peered":"2024-05-09T01:22:39.837167+0000","last_unstale":"2024-05-09T01:24:21.142981+0000","last_undegraded":"2024-05-09T01:24:21.142981+0000","last_fullsized":"2024-05-09T01:24:21.142981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:35.695077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'15","reported_seq":244,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:58.648201+0000","last_change":"2024-05-09T01:23:22.433852+0000","last_active":"2024-05-09T01:25:58.648201+0000","last_peered":"2024-05-09T01:25:58.648201+0000","last_clean":"2024-05-09T01:25:58.648201+0000","last_became_active":"2024-05-09T01:23:22.423465+0000","last_became_peered":"2024-05-09T01:23:22.423465+0000","last_unstale":"2024-05-09T01:25:58.648201+0000","last_undegraded":"2024-05-09T01:25:58.648201+0000","last_fullsized":"2024-05-09T01:25:58.648201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:58.578798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.137595+0000","last_change":"2024-05-09T01:22:39.842667+0000","last_active":"2024-05-09T01:24:21.137595+0000","last_peered":"2024-05-09T01:24:21.137595+0000","last_clean":"2024-05-09T01:24:21.137595+0000","last_became_active":"2024-05-09T01:22:39.842256+0000","last_became_peered":"2024-05-09T01:22:39.842256+0000","last_unstale":"2024-05-09T01:24:21.137595+0000","last_undegraded":"2024-05-09T01:24:21.137595+0000","last_fullsized":"2024-05-09T01:24:21.137595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:28.525790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.314581+0000","last_change":"2024-05-09T01:22:36.841515+0000","last_active":"2024-05-09T01:23:22.314581+0000","last_peered":"2024-05-09T01:23:22.314581+0000","last_clean":"2024-05-09T01:23:22.314581+0000","last_became_active":"2024-05-09T01:22:36.841128+0000","last_became_peered":"2024-05-09T01:22:36.841128+0000","last_unstale":"2024-05-09T01:23:22.314581+0000","last_undegraded":"2024-05-09T01:23:22.314581+0000","last_fullsized":"2024-05-09T01:23:22.314581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:13.598177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157725+0000","last_change":"2024-05-09T01:22:39.843491+0000","last_active":"2024-05-09T01:24:21.157725+0000","last_peered":"2024-05-09T01:24:21.157725+0000","last_clean":"2024-05-09T01:24:21.157725+0000","last_became_active":"2024-05-09T01:22:39.843098+0000","last_became_peered":"2024-05-09T01:22:39.843098+0000","last_unstale":"2024-05-09T01:24:21.157725+0000","last_undegraded":"2024-05-09T01:24:21.157725+0000","last_fullsized":"2024-05-09T01:24:21.157725+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:03:33.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:08.922898+0000","last_change":"2024-05-09T01:22:36.798217+0000","last_active":"2024-05-09T01:25:08.922898+0000","last_peered":"2024-05-09T01:25:08.922898+0000","last_clean":"2024-05-09T01:25:08.922898+0000","last_became_active":"2024-05-09T01:22:36.797847+0000","last_became_peered":"2024-05-09T01:22:36.797847+0000","last_unstale":"2024-05-09T01:25:08.922898+0000","last_undegraded":"2024-05-09T01:25:08.922898+0000","last_fullsized":"2024-05-09T01:25:08.922898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:21.116535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164715+0000","last_change":"2024-05-09T01:22:39.837568+0000","last_active":"2024-05-09T01:24:21.164715+0000","last_peered":"2024-05-09T01:24:21.164715+0000","last_clean":"2024-05-09T01:24:21.164715+0000","last_became_active":"2024-05-09T01:22:39.836159+0000","last_became_peered":"2024-05-09T01:22:39.836159+0000","last_unstale":"2024-05-09T01:24:21.164715+0000","last_undegraded":"2024-05-09T01:24:21.164715+0000","last_fullsized":"2024-05-09T01:24:21.164715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:28.089830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303449+0000","last_change":"2024-05-09T01:22:36.798185+0000","last_active":"2024-05-09T01:23:22.303449+0000","last_peered":"2024-05-09T01:23:22.303449+0000","last_clean":"2024-05-09T01:23:22.303449+0000","last_became_active":"2024-05-09T01:22:36.797522+0000","last_became_peered":"2024-05-09T01:22:36.797522+0000","last_unstale":"2024-05-09T01:23:22.303449+0000","last_undegraded":"2024-05-09T01:23:22.303449+0000","last_fullsized":"2024-05-09T01:23:22.303449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:40.960006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153901+0000","last_change":"2024-05-09T01:22:39.837436+0000","last_active":"2024-05-09T01:24:21.153901+0000","last_peered":"2024-05-09T01:24:21.153901+0000","last_clean":"2024-05-09T01:24:21.153901+0000","last_became_active":"2024-05-09T01:22:39.837050+0000","last_became_peered":"2024-05-09T01:22:39.837050+0000","last_unstale":"2024-05-09T01:24:21.153901+0000","last_undegraded":"2024-05-09T01:24:21.153901+0000","last_fullsized":"2024-05-09T01:24:21.153901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:07.149311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":214,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:56.900380+0000","last_change":"2024-05-09T01:22:36.800767+0000","last_active":"2024-05-09T01:25:56.900380+0000","last_peered":"2024-05-09T01:25:56.900380+0000","last_clean":"2024-05-09T01:25:56.900380+0000","last_became_active":"2024-05-09T01:22:36.800183+0000","last_became_peered":"2024-05-09T01:22:36.800183+0000","last_unstale":"2024-05-09T01:25:56.900380+0000","last_undegraded":"2024-05-09T01:25:56.900380+0000","last_fullsized":"2024-05-09T01:25:56.900380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:05.682889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.131431+0000","last_change":"2024-05-09T01:22:39.832798+0000","last_active":"2024-05-09T01:24:21.131431+0000","last_peered":"2024-05-09T01:24:21.131431+0000","last_clean":"2024-05-09T01:24:21.131431+0000","last_became_active":"2024-05-09T01:22:39.828803+0000","last_became_peered":"2024-05-09T01:22:39.828803+0000","last_unstale":"2024-05-09T01:24:21.131431+0000","last_undegraded":"2024-05-09T01:24:21.131431+0000","last_fullsized":"2024-05-09T01:24:21.131431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:23.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306039+0000","last_change":"2024-05-09T01:22:36.794171+0000","last_active":"2024-05-09T01:23:22.306039+0000","last_peered":"2024-05-09T01:23:22.306039+0000","last_clean":"2024-05-09T01:23:22.306039+0000","last_became_active":"2024-05-09T01:22:36.793713+0000","last_became_peered":"2024-05-09T01:22:36.793713+0000","last_unstale":"2024-05-09T01:23:22.306039+0000","last_undegraded":"2024-05-09T01:23:22.306039+0000","last_fullsized":"2024-05-09T01:23:22.306039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:06.654667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153390+0000","last_change":"2024-05-09T01:22:39.838266+0000","last_active":"2024-05-09T01:24:21.153390+0000","last_peered":"2024-05-09T01:24:21.153390+0000","last_clean":"2024-05-09T01:24:21.153390+0000","last_became_active":"2024-05-09T01:22:39.837911+0000","last_became_peered":"2024-05-09T01:22:39.837911+0000","last_unstale":"2024-05-09T01:24:21.153390+0000","last_undegraded":"2024-05-09T01:24:21.153390+0000","last_fullsized":"2024-05-09T01:24:21.153390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:39.414176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308603+0000","last_change":"2024-05-09T01:22:36.799766+0000","last_active":"2024-05-09T01:23:22.308603+0000","last_peered":"2024-05-09T01:23:22.308603+0000","last_clean":"2024-05-09T01:23:22.308603+0000","last_became_active":"2024-05-09T01:22:36.799388+0000","last_became_peered":"2024-05-09T01:22:36.799388+0000","last_unstale":"2024-05-09T01:23:22.308603+0000","last_undegraded":"2024-05-09T01:23:22.308603+0000","last_fullsized":"2024-05-09T01:23:22.308603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:13.401659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141757+0000","last_change":"2024-05-09T01:22:39.838493+0000","last_active":"2024-05-09T01:24:21.141757+0000","last_peered":"2024-05-09T01:24:21.141757+0000","last_clean":"2024-05-09T01:24:21.141757+0000","last_became_active":"2024-05-09T01:22:39.838086+0000","last_became_peered":"2024-05-09T01:22:39.838086+0000","last_unstale":"2024-05-09T01:24:21.141757+0000","last_undegraded":"2024-05-09T01:24:21.141757+0000","last_fullsized":"2024-05-09T01:24:21.141757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:06.340693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421578+0000","last_change":"2024-05-09T01:22:36.804872+0000","last_active":"2024-05-09T01:23:22.421578+0000","last_peered":"2024-05-09T01:23:22.421578+0000","last_clean":"2024-05-09T01:23:22.421578+0000","last_became_active":"2024-05-09T01:22:36.804439+0000","last_became_peered":"2024-05-09T01:22:36.804439+0000","last_unstale":"2024-05-09T01:23:22.421578+0000","last_undegraded":"2024-05-09T01:23:22.421578+0000","last_fullsized":"2024-05-09T01:23:22.421578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:08.584749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.162012+0000","last_change":"2024-05-09T01:22:39.836579+0000","last_active":"2024-05-09T01:24:21.162012+0000","last_peered":"2024-05-09T01:24:21.162012+0000","last_clean":"2024-05-09T01:24:21.162012+0000","last_became_active":"2024-05-09T01:22:39.836244+0000","last_became_peered":"2024-05-09T01:22:39.836244+0000","last_unstale":"2024-05-09T01:24:21.162012+0000","last_undegraded":"2024-05-09T01:24:21.162012+0000","last_fullsized":"2024-05-09T01:24:21.162012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:43.329448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":395,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:59.590521+0000","last_change":"2024-05-09T01:22:36.796639+0000","last_active":"2024-05-09T01:25:59.590521+0000","last_peered":"2024-05-09T01:25:59.590521+0000","last_clean":"2024-05-09T01:25:59.590521+0000","last_became_active":"2024-05-09T01:22:36.796227+0000","last_became_peered":"2024-05-09T01:22:36.796227+0000","last_unstale":"2024-05-09T01:25:59.590521+0000","last_undegraded":"2024-05-09T01:25:59.590521+0000","last_fullsized":"2024-05-09T01:25:59.590521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:56.219281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":363,"num_read_kb":363,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.162895+0000","last_change":"2024-05-09T01:22:39.841515+0000","last_active":"2024-05-09T01:24:21.162895+0000","last_peered":"2024-05-09T01:24:21.162895+0000","last_clean":"2024-05-09T01:24:21.162895+0000","last_became_active":"2024-05-09T01:22:39.840057+0000","last_became_peered":"2024-05-09T01:22:39.840057+0000","last_unstale":"2024-05-09T01:24:21.162895+0000","last_undegraded":"2024-05-09T01:24:21.162895+0000","last_fullsized":"2024-05-09T01:24:21.162895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:43.258466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309168+0000","last_change":"2024-05-09T01:22:36.801937+0000","last_active":"2024-05-09T01:23:22.309168+0000","last_peered":"2024-05-09T01:23:22.309168+0000","last_clean":"2024-05-09T01:23:22.309168+0000","last_became_active":"2024-05-09T01:22:36.801563+0000","last_became_peered":"2024-05-09T01:22:36.801563+0000","last_unstale":"2024-05-09T01:23:22.309168+0000","last_undegraded":"2024-05-09T01:23:22.309168+0000","last_fullsized":"2024-05-09T01:23:22.309168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:00:42.170156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151364+0000","last_change":"2024-05-09T01:22:39.849154+0000","last_active":"2024-05-09T01:24:21.151364+0000","last_peered":"2024-05-09T01:24:21.151364+0000","last_clean":"2024-05-09T01:24:21.151364+0000","last_became_active":"2024-05-09T01:22:39.848815+0000","last_became_peered":"2024-05-09T01:22:39.848815+0000","last_unstale":"2024-05-09T01:24:21.151364+0000","last_undegraded":"2024-05-09T01:24:21.151364+0000","last_fullsized":"2024-05-09T01:24:21.151364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:00.300208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.422043+0000","last_change":"2024-05-09T01:22:36.843201+0000","last_active":"2024-05-09T01:23:22.422043+0000","last_peered":"2024-05-09T01:23:22.422043+0000","last_clean":"2024-05-09T01:23:22.422043+0000","last_became_active":"2024-05-09T01:22:36.842500+0000","last_became_peered":"2024-05-09T01:22:36.842500+0000","last_unstale":"2024-05-09T01:23:22.422043+0000","last_undegraded":"2024-05-09T01:23:22.422043+0000","last_fullsized":"2024-05-09T01:23:22.422043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:35.092246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159850+0000","last_change":"2024-05-09T01:22:39.835120+0000","last_active":"2024-05-09T01:24:21.159850+0000","last_peered":"2024-05-09T01:24:21.159850+0000","last_clean":"2024-05-09T01:24:21.159850+0000","last_became_active":"2024-05-09T01:22:39.834613+0000","last_became_peered":"2024-05-09T01:22:39.834613+0000","last_unstale":"2024-05-09T01:24:21.159850+0000","last_undegraded":"2024-05-09T01:24:21.159850+0000","last_fullsized":"2024-05-09T01:24:21.159850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:08.380445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":443,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:59.930492+0000","last_change":"2024-05-09T01:22:36.800816+0000","last_active":"2024-05-09T01:25:59.930492+0000","last_peered":"2024-05-09T01:25:59.930492+0000","last_clean":"2024-05-09T01:25:59.930492+0000","last_became_active":"2024-05-09T01:22:36.798797+0000","last_became_peered":"2024-05-09T01:22:36.798797+0000","last_unstale":"2024-05-09T01:25:59.930492+0000","last_undegraded":"2024-05-09T01:25:59.930492+0000","last_fullsized":"2024-05-09T01:25:59.930492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:37.670163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68071,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":377,"num_read_kb":255,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134714+0000","last_change":"2024-05-09T01:22:39.831504+0000","last_active":"2024-05-09T01:24:21.134714+0000","last_peered":"2024-05-09T01:24:21.134714+0000","last_clean":"2024-05-09T01:24:21.134714+0000","last_became_active":"2024-05-09T01:22:39.831129+0000","last_became_peered":"2024-05-09T01:22:39.831129+0000","last_unstale":"2024-05-09T01:24:21.134714+0000","last_undegraded":"2024-05-09T01:24:21.134714+0000","last_fullsized":"2024-05-09T01:24:21.134714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:19.212473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304494+0000","last_change":"2024-05-09T01:22:36.803447+0000","last_active":"2024-05-09T01:23:22.304494+0000","last_peered":"2024-05-09T01:23:22.304494+0000","last_clean":"2024-05-09T01:23:22.304494+0000","last_became_active":"2024-05-09T01:22:36.802986+0000","last_became_peered":"2024-05-09T01:22:36.802986+0000","last_unstale":"2024-05-09T01:23:22.304494+0000","last_undegraded":"2024-05-09T01:23:22.304494+0000","last_fullsized":"2024-05-09T01:23:22.304494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:28.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163307+0000","last_change":"2024-05-09T01:22:39.836546+0000","last_active":"2024-05-09T01:24:21.163307+0000","last_peered":"2024-05-09T01:24:21.163307+0000","last_clean":"2024-05-09T01:24:21.163307+0000","last_became_active":"2024-05-09T01:22:39.836168+0000","last_became_peered":"2024-05-09T01:22:39.836168+0000","last_unstale":"2024-05-09T01:24:21.163307+0000","last_undegraded":"2024-05-09T01:24:21.163307+0000","last_fullsized":"2024-05-09T01:24:21.163307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:45.522279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421659+0000","last_change":"2024-05-09T01:22:36.798703+0000","last_active":"2024-05-09T01:23:22.421659+0000","last_peered":"2024-05-09T01:23:22.421659+0000","last_clean":"2024-05-09T01:23:22.421659+0000","last_became_active":"2024-05-09T01:22:36.798280+0000","last_became_peered":"2024-05-09T01:22:36.798280+0000","last_unstale":"2024-05-09T01:23:22.421659+0000","last_undegraded":"2024-05-09T01:23:22.421659+0000","last_fullsized":"2024-05-09T01:23:22.421659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:33.794488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166230+0000","last_change":"2024-05-09T01:22:39.831223+0000","last_active":"2024-05-09T01:24:21.166230+0000","last_peered":"2024-05-09T01:24:21.166230+0000","last_clean":"2024-05-09T01:24:21.166230+0000","last_became_active":"2024-05-09T01:22:39.830845+0000","last_became_peered":"2024-05-09T01:22:39.830845+0000","last_unstale":"2024-05-09T01:24:21.166230+0000","last_undegraded":"2024-05-09T01:24:21.166230+0000","last_fullsized":"2024-05-09T01:24:21.166230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:23.365950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":220,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.952967+0000","last_change":"2024-05-09T01:22:36.802327+0000","last_active":"2024-05-09T01:25:57.952967+0000","last_peered":"2024-05-09T01:25:57.952967+0000","last_clean":"2024-05-09T01:25:57.952967+0000","last_became_active":"2024-05-09T01:22:36.801951+0000","last_became_peered":"2024-05-09T01:22:36.801951+0000","last_unstale":"2024-05-09T01:25:57.952967+0000","last_undegraded":"2024-05-09T01:25:57.952967+0000","last_fullsized":"2024-05-09T01:25:57.952967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:12.282424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154280+0000","last_change":"2024-05-09T01:22:39.833626+0000","last_active":"2024-05-09T01:24:21.154280+0000","last_peered":"2024-05-09T01:24:21.154280+0000","last_clean":"2024-05-09T01:24:21.154280+0000","last_became_active":"2024-05-09T01:22:39.833176+0000","last_became_peered":"2024-05-09T01:22:39.833176+0000","last_unstale":"2024-05-09T01:24:21.154280+0000","last_undegraded":"2024-05-09T01:24:21.154280+0000","last_fullsized":"2024-05-09T01:24:21.154280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:25.082300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:26.194840+0000","last_change":"2024-05-09T01:23:22.328011+0000","last_active":"2024-05-09T01:23:26.194840+0000","last_peered":"2024-05-09T01:23:26.194840+0000","last_clean":"2024-05-09T01:23:26.194840+0000","last_became_active":"2024-05-09T01:23:22.313049+0000","last_became_peered":"2024-05-09T01:23:22.313049+0000","last_unstale":"2024-05-09T01:23:26.194840+0000","last_undegraded":"2024-05-09T01:23:26.194840+0000","last_fullsized":"2024-05-09T01:23:26.194840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:17.854491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153329+0000","last_change":"2024-05-09T01:22:39.837529+0000","last_active":"2024-05-09T01:24:21.153329+0000","last_peered":"2024-05-09T01:24:21.153329+0000","last_clean":"2024-05-09T01:24:21.153329+0000","last_became_active":"2024-05-09T01:22:39.837155+0000","last_became_peered":"2024-05-09T01:22:39.837155+0000","last_unstale":"2024-05-09T01:24:21.153329+0000","last_undegraded":"2024-05-09T01:24:21.153329+0000","last_fullsized":"2024-05-09T01:24:21.153329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:44.485629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303600+0000","last_change":"2024-05-09T01:22:36.800073+0000","last_active":"2024-05-09T01:23:22.303600+0000","last_peered":"2024-05-09T01:23:22.303600+0000","last_clean":"2024-05-09T01:23:22.303600+0000","last_became_active":"2024-05-09T01:22:36.799463+0000","last_became_peered":"2024-05-09T01:22:36.799463+0000","last_unstale":"2024-05-09T01:23:22.303600+0000","last_undegraded":"2024-05-09T01:23:22.303600+0000","last_fullsized":"2024-05-09T01:23:22.303600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.942945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150713+0000","last_change":"2024-05-09T01:22:39.833615+0000","last_active":"2024-05-09T01:24:21.150713+0000","last_peered":"2024-05-09T01:24:21.150713+0000","last_clean":"2024-05-09T01:24:21.150713+0000","last_became_active":"2024-05-09T01:22:39.833212+0000","last_became_peered":"2024-05-09T01:22:39.833212+0000","last_unstale":"2024-05-09T01:24:21.150713+0000","last_undegraded":"2024-05-09T01:24:21.150713+0000","last_fullsized":"2024-05-09T01:24:21.150713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:35.387442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305695+0000","last_change":"2024-05-09T01:22:36.843143+0000","last_active":"2024-05-09T01:23:22.305695+0000","last_peered":"2024-05-09T01:23:22.305695+0000","last_clean":"2024-05-09T01:23:22.305695+0000","last_became_active":"2024-05-09T01:22:36.842759+0000","last_became_peered":"2024-05-09T01:22:36.842759+0000","last_unstale":"2024-05-09T01:23:22.305695+0000","last_undegraded":"2024-05-09T01:23:22.305695+0000","last_fullsized":"2024-05-09T01:23:22.305695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:42.666186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.143568+0000","last_change":"2024-05-09T01:22:39.838267+0000","last_active":"2024-05-09T01:24:21.143568+0000","last_peered":"2024-05-09T01:24:21.143568+0000","last_clean":"2024-05-09T01:24:21.143568+0000","last_became_active":"2024-05-09T01:22:39.837910+0000","last_became_peered":"2024-05-09T01:22:39.837910+0000","last_unstale":"2024-05-09T01:24:21.143568+0000","last_undegraded":"2024-05-09T01:24:21.143568+0000","last_fullsized":"2024-05-09T01:24:21.143568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:34.620332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'8","reported_seq":219,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.647507+0000","last_change":"2024-05-09T01:22:36.800515+0000","last_active":"2024-05-09T01:25:57.647507+0000","last_peered":"2024-05-09T01:25:57.647507+0000","last_clean":"2024-05-09T01:25:57.647507+0000","last_became_active":"2024-05-09T01:22:36.800065+0000","last_became_peered":"2024-05-09T01:22:36.800065+0000","last_unstale":"2024-05-09T01:25:57.647507+0000","last_undegraded":"2024-05-09T01:25:57.647507+0000","last_fullsized":"2024-05-09T01:25:57.647507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:50.160463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145505+0000","last_change":"2024-05-09T01:23:22.313627+0000","last_active":"2024-05-09T01:24:21.145505+0000","last_peered":"2024-05-09T01:24:21.145505+0000","last_clean":"2024-05-09T01:24:21.145505+0000","last_became_active":"2024-05-09T01:23:22.313138+0000","last_became_peered":"2024-05-09T01:23:22.313138+0000","last_unstale":"2024-05-09T01:24:21.145505+0000","last_undegraded":"2024-05-09T01:24:21.145505+0000","last_fullsized":"2024-05-09T01:24:21.145505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:07.216086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153456+0000","last_change":"2024-05-09T01:22:39.832188+0000","last_active":"2024-05-09T01:24:21.153456+0000","last_peered":"2024-05-09T01:24:21.153456+0000","last_clean":"2024-05-09T01:24:21.153456+0000","last_became_active":"2024-05-09T01:22:39.831302+0000","last_became_peered":"2024-05-09T01:22:39.831302+0000","last_unstale":"2024-05-09T01:24:21.153456+0000","last_undegraded":"2024-05-09T01:24:21.153456+0000","last_fullsized":"2024-05-09T01:24:21.153456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:06.703950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'7","reported_seq":216,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.952377+0000","last_change":"2024-05-09T01:22:36.841446+0000","last_active":"2024-05-09T01:25:57.952377+0000","last_peered":"2024-05-09T01:25:57.952377+0000","last_clean":"2024-05-09T01:25:57.952377+0000","last_became_active":"2024-05-09T01:22:36.841071+0000","last_became_peered":"2024-05-09T01:22:36.841071+0000","last_unstale":"2024-05-09T01:25:57.952377+0000","last_undegraded":"2024-05-09T01:25:57.952377+0000","last_fullsized":"2024-05-09T01:25:57.952377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:20.777482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163865+0000","last_change":"2024-05-09T01:22:39.846711+0000","last_active":"2024-05-09T01:24:21.163865+0000","last_peered":"2024-05-09T01:24:21.163865+0000","last_clean":"2024-05-09T01:24:21.163865+0000","last_became_active":"2024-05-09T01:22:39.846293+0000","last_became_peered":"2024-05-09T01:22:39.846293+0000","last_unstale":"2024-05-09T01:24:21.163865+0000","last_undegraded":"2024-05-09T01:24:21.163865+0000","last_fullsized":"2024-05-09T01:24:21.163865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:27.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303855+0000","last_change":"2024-05-09T01:22:36.798302+0000","last_active":"2024-05-09T01:23:22.303855+0000","last_peered":"2024-05-09T01:23:22.303855+0000","last_clean":"2024-05-09T01:23:22.303855+0000","last_became_active":"2024-05-09T01:22:36.797906+0000","last_became_peered":"2024-05-09T01:22:36.797906+0000","last_unstale":"2024-05-09T01:23:22.303855+0000","last_undegraded":"2024-05-09T01:23:22.303855+0000","last_fullsized":"2024-05-09T01:23:22.303855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:13.599240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:59.123370+0000","last_change":"2024-05-09T01:21:12.781366+0000","last_active":"2024-05-09T01:23:59.123370+0000","last_peered":"2024-05-09T01:23:59.123370+0000","last_clean":"2024-05-09T01:23:59.123370+0000","last_became_active":"2024-05-09T01:21:12.774967+0000","last_became_peered":"2024-05-09T01:21:12.774967+0000","last_unstale":"2024-05-09T01:23:59.123370+0000","last_undegraded":"2024-05-09T01:23:59.123370+0000","last_fullsized":"2024-05-09T01:23:59.123370+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_clean_scrub_stamp":"2024-05-09T01:17:49.682508+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:08.075365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155221+0000","last_change":"2024-05-09T01:22:39.840952+0000","last_active":"2024-05-09T01:24:21.155221+0000","last_peered":"2024-05-09T01:24:21.155221+0000","last_clean":"2024-05-09T01:24:21.155221+0000","last_became_active":"2024-05-09T01:22:39.840323+0000","last_became_peered":"2024-05-09T01:22:39.840323+0000","last_unstale":"2024-05-09T01:24:21.155221+0000","last_undegraded":"2024-05-09T01:24:21.155221+0000","last_fullsized":"2024-05-09T01:24:21.155221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:23.837123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308860+0000","last_change":"2024-05-09T01:22:36.798631+0000","last_active":"2024-05-09T01:23:22.308860+0000","last_peered":"2024-05-09T01:23:22.308860+0000","last_clean":"2024-05-09T01:23:22.308860+0000","last_became_active":"2024-05-09T01:22:36.798242+0000","last_became_peered":"2024-05-09T01:22:36.798242+0000","last_unstale":"2024-05-09T01:23:22.308860+0000","last_undegraded":"2024-05-09T01:23:22.308860+0000","last_fullsized":"2024-05-09T01:23:22.308860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:18.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303800+0000","last_change":"2024-05-09T01:22:36.843779+0000","last_active":"2024-05-09T01:23:22.303800+0000","last_peered":"2024-05-09T01:23:22.303800+0000","last_clean":"2024-05-09T01:23:22.303800+0000","last_became_active":"2024-05-09T01:22:36.842084+0000","last_became_peered":"2024-05-09T01:22:36.842084+0000","last_unstale":"2024-05-09T01:23:22.303800+0000","last_undegraded":"2024-05-09T01:23:22.303800+0000","last_fullsized":"2024-05-09T01:23:22.303800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:34.743372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146032+0000","last_change":"2024-05-09T01:22:39.842266+0000","last_active":"2024-05-09T01:24:21.146032+0000","last_peered":"2024-05-09T01:24:21.146032+0000","last_clean":"2024-05-09T01:24:21.146032+0000","last_became_active":"2024-05-09T01:22:39.841927+0000","last_became_peered":"2024-05-09T01:22:39.841927+0000","last_unstale":"2024-05-09T01:24:21.146032+0000","last_undegraded":"2024-05-09T01:24:21.146032+0000","last_fullsized":"2024-05-09T01:24:21.146032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:26.774250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'19","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:56.860389+0000","last_change":"2024-05-09T01:22:36.796929+0000","last_active":"2024-05-09T01:25:56.860389+0000","last_peered":"2024-05-09T01:25:56.860389+0000","last_clean":"2024-05-09T01:25:56.860389+0000","last_became_active":"2024-05-09T01:22:36.796480+0000","last_became_peered":"2024-05-09T01:22:36.796480+0000","last_unstale":"2024-05-09T01:25:56.860389+0000","last_undegraded":"2024-05-09T01:25:56.860389+0000","last_fullsized":"2024-05-09T01:25:56.860389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:45.168305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140120+0000","last_change":"2024-05-09T01:22:39.843093+0000","last_active":"2024-05-09T01:24:21.140120+0000","last_peered":"2024-05-09T01:24:21.140120+0000","last_clean":"2024-05-09T01:24:21.140120+0000","last_became_active":"2024-05-09T01:22:39.842729+0000","last_became_peered":"2024-05-09T01:22:39.842729+0000","last_unstale":"2024-05-09T01:24:21.140120+0000","last_undegraded":"2024-05-09T01:24:21.140120+0000","last_fullsized":"2024-05-09T01:24:21.140120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:02.074843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303716+0000","last_change":"2024-05-09T01:22:36.798166+0000","last_active":"2024-05-09T01:23:22.303716+0000","last_peered":"2024-05-09T01:23:22.303716+0000","last_clean":"2024-05-09T01:23:22.303716+0000","last_became_active":"2024-05-09T01:22:36.797752+0000","last_became_peered":"2024-05-09T01:22:36.797752+0000","last_unstale":"2024-05-09T01:23:22.303716+0000","last_undegraded":"2024-05-09T01:23:22.303716+0000","last_fullsized":"2024-05-09T01:23:22.303716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:05.211720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142128+0000","last_change":"2024-05-09T01:23:22.316228+0000","last_active":"2024-05-09T01:24:21.142128+0000","last_peered":"2024-05-09T01:24:21.142128+0000","last_clean":"2024-05-09T01:24:21.142128+0000","last_became_active":"2024-05-09T01:23:22.315683+0000","last_became_peered":"2024-05-09T01:23:22.315683+0000","last_unstale":"2024-05-09T01:24:21.142128+0000","last_undegraded":"2024-05-09T01:24:21.142128+0000","last_fullsized":"2024-05-09T01:24:21.142128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:04.623701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421356+0000","last_change":"2024-05-09T01:22:36.841999+0000","last_active":"2024-05-09T01:23:22.421356+0000","last_peered":"2024-05-09T01:23:22.421356+0000","last_clean":"2024-05-09T01:23:22.421356+0000","last_became_active":"2024-05-09T01:22:36.841664+0000","last_became_peered":"2024-05-09T01:22:36.841664+0000","last_unstale":"2024-05-09T01:23:22.421356+0000","last_undegraded":"2024-05-09T01:23:22.421356+0000","last_fullsized":"2024-05-09T01:23:22.421356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:27.110258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134153+0000","last_change":"2024-05-09T01:22:39.839553+0000","last_active":"2024-05-09T01:24:21.134153+0000","last_peered":"2024-05-09T01:24:21.134153+0000","last_clean":"2024-05-09T01:24:21.134153+0000","last_became_active":"2024-05-09T01:22:39.839189+0000","last_became_peered":"2024-05-09T01:22:39.839189+0000","last_unstale":"2024-05-09T01:24:21.134153+0000","last_undegraded":"2024-05-09T01:24:21.134153+0000","last_fullsized":"2024-05-09T01:24:21.134153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:02.794073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309908+0000","last_change":"2024-05-09T01:22:36.792516+0000","last_active":"2024-05-09T01:23:22.309908+0000","last_peered":"2024-05-09T01:23:22.309908+0000","last_clean":"2024-05-09T01:23:22.309908+0000","last_became_active":"2024-05-09T01:22:36.792129+0000","last_became_peered":"2024-05-09T01:22:36.792129+0000","last_unstale":"2024-05-09T01:23:22.309908+0000","last_undegraded":"2024-05-09T01:23:22.309908+0000","last_fullsized":"2024-05-09T01:23:22.309908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:46.596723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148884+0000","last_change":"2024-05-09T01:22:39.841523+0000","last_active":"2024-05-09T01:24:21.148884+0000","last_peered":"2024-05-09T01:24:21.148884+0000","last_clean":"2024-05-09T01:24:21.148884+0000","last_became_active":"2024-05-09T01:22:39.840667+0000","last_became_peered":"2024-05-09T01:22:39.840667+0000","last_unstale":"2024-05-09T01:24:21.148884+0000","last_undegraded":"2024-05-09T01:24:21.148884+0000","last_fullsized":"2024-05-09T01:24:21.148884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:39.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:26.194870+0000","last_change":"2024-05-09T01:23:22.331855+0000","last_active":"2024-05-09T01:23:26.194870+0000","last_peered":"2024-05-09T01:23:26.194870+0000","last_clean":"2024-05-09T01:23:26.194870+0000","last_became_active":"2024-05-09T01:23:22.313059+0000","last_became_peered":"2024-05-09T01:23:22.313059+0000","last_unstale":"2024-05-09T01:23:26.194870+0000","last_undegraded":"2024-05-09T01:23:26.194870+0000","last_fullsized":"2024-05-09T01:23:26.194870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:16.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142481+0000","last_change":"2024-05-09T01:22:39.845031+0000","last_active":"2024-05-09T01:24:21.142481+0000","last_peered":"2024-05-09T01:24:21.142481+0000","last_clean":"2024-05-09T01:24:21.142481+0000","last_became_active":"2024-05-09T01:22:39.844653+0000","last_became_peered":"2024-05-09T01:22:39.844653+0000","last_unstale":"2024-05-09T01:24:21.142481+0000","last_undegraded":"2024-05-09T01:24:21.142481+0000","last_fullsized":"2024-05-09T01:24:21.142481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:14.382445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303240+0000","last_change":"2024-05-09T01:22:36.801395+0000","last_active":"2024-05-09T01:23:22.303240+0000","last_peered":"2024-05-09T01:23:22.303240+0000","last_clean":"2024-05-09T01:23:22.303240+0000","last_became_active":"2024-05-09T01:22:36.800992+0000","last_became_peered":"2024-05-09T01:22:36.800992+0000","last_unstale":"2024-05-09T01:23:22.303240+0000","last_undegraded":"2024-05-09T01:23:22.303240+0000","last_fullsized":"2024-05-09T01:23:22.303240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:16.089068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166011+0000","last_change":"2024-05-09T01:22:39.835842+0000","last_active":"2024-05-09T01:24:21.166011+0000","last_peered":"2024-05-09T01:24:21.166011+0000","last_clean":"2024-05-09T01:24:21.166011+0000","last_became_active":"2024-05-09T01:22:39.835372+0000","last_became_peered":"2024-05-09T01:22:39.835372+0000","last_unstale":"2024-05-09T01:24:21.166011+0000","last_undegraded":"2024-05-09T01:24:21.166011+0000","last_fullsized":"2024-05-09T01:24:21.166011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:13.197946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":179,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.588036+0000","last_change":"2024-05-09T01:23:22.332514+0000","last_active":"2024-05-09T01:25:57.588036+0000","last_peered":"2024-05-09T01:25:57.588036+0000","last_clean":"2024-05-09T01:25:57.588036+0000","last_became_active":"2024-05-09T01:23:22.312966+0000","last_became_peered":"2024-05-09T01:23:22.312966+0000","last_unstale":"2024-05-09T01:25:57.588036+0000","last_undegraded":"2024-05-09T01:25:57.588036+0000","last_fullsized":"2024-05-09T01:25:57.588036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:52.637108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":163,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166259+0000","last_change":"2024-05-09T01:22:39.845390+0000","last_active":"2024-05-09T01:24:21.166259+0000","last_peered":"2024-05-09T01:24:21.166259+0000","last_clean":"2024-05-09T01:24:21.166259+0000","last_became_active":"2024-05-09T01:22:39.844918+0000","last_became_peered":"2024-05-09T01:22:39.844918+0000","last_unstale":"2024-05-09T01:24:21.166259+0000","last_undegraded":"2024-05-09T01:24:21.166259+0000","last_fullsized":"2024-05-09T01:24:21.166259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:04.455794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421891+0000","last_change":"2024-05-09T01:22:36.803329+0000","last_active":"2024-05-09T01:23:22.421891+0000","last_peered":"2024-05-09T01:23:22.421891+0000","last_clean":"2024-05-09T01:23:22.421891+0000","last_became_active":"2024-05-09T01:22:36.802831+0000","last_became_peered":"2024-05-09T01:22:36.802831+0000","last_unstale":"2024-05-09T01:23:22.421891+0000","last_undegraded":"2024-05-09T01:23:22.421891+0000","last_fullsized":"2024-05-09T01:23:22.421891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:57.919840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154788+0000","last_change":"2024-05-09T01:22:39.837595+0000","last_active":"2024-05-09T01:24:21.154788+0000","last_peered":"2024-05-09T01:24:21.154788+0000","last_clean":"2024-05-09T01:24:21.154788+0000","last_became_active":"2024-05-09T01:22:39.837197+0000","last_became_peered":"2024-05-09T01:22:39.837197+0000","last_unstale":"2024-05-09T01:24:21.154788+0000","last_undegraded":"2024-05-09T01:24:21.154788+0000","last_fullsized":"2024-05-09T01:24:21.154788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:08.700741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.321791+0000","last_change":"2024-05-09T01:23:22.321286+0000","last_active":"2024-05-09T01:23:22.321791+0000","last_peered":"2024-05-09T01:23:22.321791+0000","last_clean":"2024-05-09T01:23:22.321791+0000","last_became_active":"2024-05-09T01:23:22.315810+0000","last_became_peered":"2024-05-09T01:23:22.315810+0000","last_unstale":"2024-05-09T01:23:22.321791+0000","last_undegraded":"2024-05-09T01:23:22.321791+0000","last_fullsized":"2024-05-09T01:23:22.321791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:43.100893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.165342+0000","last_change":"2024-05-09T01:22:39.827798+0000","last_active":"2024-05-09T01:24:21.165342+0000","last_peered":"2024-05-09T01:24:21.165342+0000","last_clean":"2024-05-09T01:24:21.165342+0000","last_became_active":"2024-05-09T01:22:39.827322+0000","last_became_peered":"2024-05-09T01:22:39.827322+0000","last_unstale":"2024-05-09T01:24:21.165342+0000","last_undegraded":"2024-05-09T01:24:21.165342+0000","last_fullsized":"2024-05-09T01:24:21.165342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:52.454033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.149749+0000","last_change":"2024-05-09T01:22:39.841885+0000","last_active":"2024-05-09T01:24:21.149749+0000","last_peered":"2024-05-09T01:24:21.149749+0000","last_clean":"2024-05-09T01:24:21.149749+0000","last_became_active":"2024-05-09T01:22:39.841160+0000","last_became_peered":"2024-05-09T01:22:39.841160+0000","last_unstale":"2024-05-09T01:24:21.149749+0000","last_undegraded":"2024-05-09T01:24:21.149749+0000","last_fullsized":"2024-05-09T01:24:21.149749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:57.290465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146092+0000","last_change":"2024-05-09T01:22:39.846495+0000","last_active":"2024-05-09T01:24:21.146092+0000","last_peered":"2024-05-09T01:24:21.146092+0000","last_clean":"2024-05-09T01:24:21.146092+0000","last_became_active":"2024-05-09T01:22:39.846108+0000","last_became_peered":"2024-05-09T01:22:39.846108+0000","last_unstale":"2024-05-09T01:24:21.146092+0000","last_undegraded":"2024-05-09T01:24:21.146092+0000","last_fullsized":"2024-05-09T01:24:21.146092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:39.958174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.132056+0000","last_change":"2024-05-09T01:22:39.841288+0000","last_active":"2024-05-09T01:24:21.132056+0000","last_peered":"2024-05-09T01:24:21.132056+0000","last_clean":"2024-05-09T01:24:21.132056+0000","last_became_active":"2024-05-09T01:22:39.840900+0000","last_became_peered":"2024-05-09T01:22:39.840900+0000","last_unstale":"2024-05-09T01:24:21.132056+0000","last_undegraded":"2024-05-09T01:24:21.132056+0000","last_fullsized":"2024-05-09T01:24:21.132056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:30.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141526+0000","last_change":"2024-05-09T01:23:22.313135+0000","last_active":"2024-05-09T01:24:21.141526+0000","last_peered":"2024-05-09T01:24:21.141526+0000","last_clean":"2024-05-09T01:24:21.141526+0000","last_became_active":"2024-05-09T01:23:22.312710+0000","last_became_peered":"2024-05-09T01:23:22.312710+0000","last_unstale":"2024-05-09T01:24:21.141526+0000","last_undegraded":"2024-05-09T01:24:21.141526+0000","last_fullsized":"2024-05-09T01:24:21.141526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:47.725790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.156722+0000","last_change":"2024-05-09T01:22:39.838037+0000","last_active":"2024-05-09T01:24:21.156722+0000","last_peered":"2024-05-09T01:24:21.156722+0000","last_clean":"2024-05-09T01:24:21.156722+0000","last_became_active":"2024-05-09T01:22:39.837362+0000","last_became_peered":"2024-05-09T01:22:39.837362+0000","last_unstale":"2024-05-09T01:24:21.156722+0000","last_undegraded":"2024-05-09T01:24:21.156722+0000","last_fullsized":"2024-05-09T01:24:21.156722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.781318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146468+0000","last_change":"2024-05-09T01:23:22.310046+0000","last_active":"2024-05-09T01:24:21.146468+0000","last_peered":"2024-05-09T01:24:21.146468+0000","last_clean":"2024-05-09T01:24:21.146468+0000","last_became_active":"2024-05-09T01:23:22.309541+0000","last_became_peered":"2024-05-09T01:23:22.309541+0000","last_unstale":"2024-05-09T01:24:21.146468+0000","last_undegraded":"2024-05-09T01:24:21.146468+0000","last_fullsized":"2024-05-09T01:24:21.146468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:30.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.161756+0000","last_change":"2024-05-09T01:22:39.840453+0000","last_active":"2024-05-09T01:24:21.161756+0000","last_peered":"2024-05-09T01:24:21.161756+0000","last_clean":"2024-05-09T01:24:21.161756+0000","last_became_active":"2024-05-09T01:22:39.840077+0000","last_became_peered":"2024-05-09T01:22:39.840077+0000","last_unstale":"2024-05-09T01:24:21.161756+0000","last_undegraded":"2024-05-09T01:24:21.161756+0000","last_fullsized":"2024-05-09T01:24:21.161756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:20.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.160942+0000","last_change":"2024-05-09T01:22:39.844554+0000","last_active":"2024-05-09T01:24:21.160942+0000","last_peered":"2024-05-09T01:24:21.160942+0000","last_clean":"2024-05-09T01:24:21.160942+0000","last_became_active":"2024-05-09T01:22:39.843640+0000","last_became_peered":"2024-05-09T01:22:39.843640+0000","last_unstale":"2024-05-09T01:24:21.160942+0000","last_undegraded":"2024-05-09T01:24:21.160942+0000","last_fullsized":"2024-05-09T01:24:21.160942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:27.801565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140795+0000","last_change":"2024-05-09T01:22:39.839043+0000","last_active":"2024-05-09T01:24:21.140795+0000","last_peered":"2024-05-09T01:24:21.140795+0000","last_clean":"2024-05-09T01:24:21.140795+0000","last_became_active":"2024-05-09T01:22:39.838662+0000","last_became_peered":"2024-05-09T01:22:39.838662+0000","last_unstale":"2024-05-09T01:24:21.140795+0000","last_undegraded":"2024-05-09T01:24:21.140795+0000","last_fullsized":"2024-05-09T01:24:21.140795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:16.117689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.139048+0000","last_change":"2024-05-09T01:23:22.309945+0000","last_active":"2024-05-09T01:24:21.139048+0000","last_peered":"2024-05-09T01:24:21.139048+0000","last_clean":"2024-05-09T01:24:21.139048+0000","last_became_active":"2024-05-09T01:23:22.309543+0000","last_became_peered":"2024-05-09T01:23:22.309543+0000","last_unstale":"2024-05-09T01:24:21.139048+0000","last_undegraded":"2024-05-09T01:24:21.139048+0000","last_fullsized":"2024-05-09T01:24:21.139048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:47.464524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164552+0000","last_change":"2024-05-09T01:22:39.838911+0000","last_active":"2024-05-09T01:24:21.164552+0000","last_peered":"2024-05-09T01:24:21.164552+0000","last_clean":"2024-05-09T01:24:21.164552+0000","last_became_active":"2024-05-09T01:22:39.838293+0000","last_became_peered":"2024-05-09T01:22:39.838293+0000","last_unstale":"2024-05-09T01:24:21.164552+0000","last_undegraded":"2024-05-09T01:24:21.164552+0000","last_fullsized":"2024-05-09T01:24:21.164552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:32.596306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163721+0000","last_change":"2024-05-09T01:22:39.838185+0000","last_active":"2024-05-09T01:24:21.163721+0000","last_peered":"2024-05-09T01:24:21.163721+0000","last_clean":"2024-05-09T01:24:21.163721+0000","last_became_active":"2024-05-09T01:22:39.837530+0000","last_became_peered":"2024-05-09T01:22:39.837530+0000","last_unstale":"2024-05-09T01:24:21.163721+0000","last_undegraded":"2024-05-09T01:24:21.163721+0000","last_fullsized":"2024-05-09T01:24:21.163721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:22.556710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150895+0000","last_change":"2024-05-09T01:22:39.834124+0000","last_active":"2024-05-09T01:24:21.150895+0000","last_peered":"2024-05-09T01:24:21.150895+0000","last_clean":"2024-05-09T01:24:21.150895+0000","last_became_active":"2024-05-09T01:22:39.833716+0000","last_became_peered":"2024-05-09T01:22:39.833716+0000","last_unstale":"2024-05-09T01:24:21.150895+0000","last_undegraded":"2024-05-09T01:24:21.150895+0000","last_fullsized":"2024-05-09T01:24:21.150895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:54:42.337900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76871,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1834,"num_read_kb":1012,"num_write":273,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2345,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":230613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612664,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":437733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808893,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1097112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972418,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":437019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168648,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":913408,"data_stored":436613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1031370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528403,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":437659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":507315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":438607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":504770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117089,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":436561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":439222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724631,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1540096,"data_stored":1029202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T01:26:02.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T01:26:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 -- 172.21.15.19:0/1342966504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc108a50 msgr2=0x7f32fc10ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 --2- 172.21.15.19:0/1342966504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc108a50 0x7f32fc10ae70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009ca0 tx=0x7f32f402a8c0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 -- 172.21.15.19:0/1342966504 shutdown_connections 2024-05-09T01:26:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 --2- 172.21.15.19:0/1342966504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc108a50 0x7f32fc10ae70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 --2- 172.21.15.19:0/1342966504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32fc103d50 0x7f32fc104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 --2- 172.21.15.19:0/1342966504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc102b10 0x7f32fc103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 -- 172.21.15.19:0/1342966504 >> 172.21.15.19:0/1342966504 conn(0x7f32fc0fdd00 msgr2=0x7f32fc1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 -- 172.21.15.19:0/1342966504 shutdown_connections 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 -- 172.21.15.19:0/1342966504 wait complete. 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.317+0000 7f33043ee640 1 Processor -- start 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f33043ee640 1 -- start start 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f33043ee640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32fc102b10 0x7f32fc075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f33043ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc103d50 0x7f32fc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f33043ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc108a50 0x7f32fc1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f33043ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc10e690 con 0x7f32fc103d50 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f33043ee640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc10e510 con 0x7f32fc102b10 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f33043ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc10e810 con 0x7f32fc108a50 2024-05-09T01:26:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f3301989640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc103d50 0x7f32fc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f3301989640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc103d50 0x7f32fc19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58396/0 (socket says 172.21.15.19:58396) 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f3301989640 1 -- 172.21.15.19:0/3474580333 learned_addr learned my addr 172.21.15.19:0/3474580333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f330218a640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32fc102b10 0x7f32fc075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f330298b640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc108a50 0x7f32fc1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f3301989640 1 -- 172.21.15.19:0/3474580333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc108a50 msgr2=0x7f32fc1a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f3301989640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc108a50 0x7f32fc1a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f3301989640 1 -- 172.21.15.19:0/3474580333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32fc102b10 msgr2=0x7f32fc075970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f3301989640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32fc102b10 0x7f32fc075970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f3301989640 1 -- 172.21.15.19:0/3474580333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc1a1e80 con 0x7f32fc103d50 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.318+0000 7f330298b640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc108a50 0x7f32fc1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.319+0000 7f3301989640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc103d50 0x7f32fc19f110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00e9f0 tx=0x7f32ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.319+0000 7f32f37fe640 1 -- 172.21.15.19:0/3474580333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00cd80 con 0x7f32fc103d50 2024-05-09T01:26:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.319+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc1a2170 con 0x7f32fc103d50 2024-05-09T01:26:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.319+0000 7f32f37fe640 1 -- 172.21.15.19:0/3474580333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ec0108e0 con 0x7f32fc103d50 2024-05-09T01:26:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.319+0000 7f32f37fe640 1 -- 172.21.15.19:0/3474580333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec018980 con 0x7f32fc103d50 2024-05-09T01:26:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.319+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc1a2710 con 0x7f32fc103d50 2024-05-09T01:26:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.320+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32fc107860 con 0x7f32fc103d50 2024-05-09T01:26:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.321+0000 7f32f37fe640 1 -- 172.21.15.19:0/3474580333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f32ec010430 con 0x7f32fc103d50 2024-05-09T01:26:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.322+0000 7f32f37fe640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32e80b2880 0x7f32e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.322+0000 7f330218a640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32e80b2880 0x7f32e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.322+0000 7f32f37fe640 1 -- 172.21.15.19:0/3474580333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f32ec0e2d70 con 0x7f32fc103d50 2024-05-09T01:26:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.323+0000 7f330218a640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32e80b2880 0x7f32e80b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f32fc103710 tx=0x7f32f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.326+0000 7f32f37fe640 1 -- 172.21.15.19:0/3474580333 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32ec0aa8b0 con 0x7f32fc103d50 2024-05-09T01:26:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.515+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32fc063110 con 0x7f32e80b2880 2024-05-09T01:26:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.523+0000 7f32f37fe640 1 -- 172.21.15.19:0/3474580333 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7f32fc063110 con 0x7f32e80b2880 2024-05-09T01:26:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:26:02.524 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32e80b2880 msgr2=0x7f32e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32e80b2880 0x7f32e80b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f32fc103710 tx=0x7f32f8005f90 comp rx=0 tx=0).stop 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc103d50 msgr2=0x7f32fc19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc103d50 0x7f32fc19f110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00e9f0 tx=0x7f32ec00eeb0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 shutdown_connections 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32e80b2880 0x7f32e80b4d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc108a50 0x7f32fc1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc103d50 0x7f32fc19f110 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 --2- 172.21.15.19:0/3474580333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32fc102b10 0x7f32fc075970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 >> 172.21.15.19:0/3474580333 conn(0x7f32fc0fdd00 msgr2=0x7f32fc104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 shutdown_connections 2024-05-09T01:26:02.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.526+0000 7f33043ee640 1 -- 172.21.15.19:0/3474580333 wait complete. 2024-05-09T01:26:02.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:02 smithi019 ceph-mon[30158]: pgmap v393: 97 pgs: 97 active+clean; 652 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-05-09T01:26:02.535 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-05-09T01:26:01.178394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667473,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1910,"num_read_kb":1076,"num_write":398,"num_write_kb":2315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":2954185,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":828,"ondisk_log_size":828,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386964,"kb_used_data":12852,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554860,"statfs":{"total":1151940427776,"available":1151544176640,"internally_reserved":0,"allocated":13160448,"data_stored":7233183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6854,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":85,"num_write":14,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003416"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158424+0000","last_change":"2024-05-09T01:22:39.836756+0000","last_active":"2024-05-09T01:24:21.158424+0000","last_peered":"2024-05-09T01:24:21.158424+0000","last_clean":"2024-05-09T01:24:21.158424+0000","last_became_active":"2024-05-09T01:22:39.835505+0000","last_became_peered":"2024-05-09T01:22:39.835505+0000","last_unstale":"2024-05-09T01:24:21.158424+0000","last_undegraded":"2024-05-09T01:24:21.158424+0000","last_fullsized":"2024-05-09T01:24:21.158424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:29.109762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148168+0000","last_change":"2024-05-09T01:22:39.842897+0000","last_active":"2024-05-09T01:24:21.148168+0000","last_peered":"2024-05-09T01:24:21.148168+0000","last_clean":"2024-05-09T01:24:21.148168+0000","last_became_active":"2024-05-09T01:22:39.842537+0000","last_became_peered":"2024-05-09T01:22:39.842537+0000","last_unstale":"2024-05-09T01:24:21.148168+0000","last_undegraded":"2024-05-09T01:24:21.148168+0000","last_fullsized":"2024-05-09T01:24:21.148168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:20.429654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157917+0000","last_change":"2024-05-09T01:22:39.847176+0000","last_active":"2024-05-09T01:24:21.157917+0000","last_peered":"2024-05-09T01:24:21.157917+0000","last_clean":"2024-05-09T01:24:21.157917+0000","last_became_active":"2024-05-09T01:22:39.846782+0000","last_became_peered":"2024-05-09T01:22:39.846782+0000","last_unstale":"2024-05-09T01:24:21.157917+0000","last_undegraded":"2024-05-09T01:24:21.157917+0000","last_fullsized":"2024-05-09T01:24:21.157917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:06.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.137185+0000","last_change":"2024-05-09T01:22:39.842399+0000","last_active":"2024-05-09T01:24:21.137185+0000","last_peered":"2024-05-09T01:24:21.137185+0000","last_clean":"2024-05-09T01:24:21.137185+0000","last_became_active":"2024-05-09T01:22:39.842035+0000","last_became_peered":"2024-05-09T01:22:39.842035+0000","last_unstale":"2024-05-09T01:24:21.137185+0000","last_undegraded":"2024-05-09T01:24:21.137185+0000","last_fullsized":"2024-05-09T01:24:21.137185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:35.011920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.124665+0000","last_change":"2024-05-09T01:22:39.838574+0000","last_active":"2024-05-09T01:24:21.124665+0000","last_peered":"2024-05-09T01:24:21.124665+0000","last_clean":"2024-05-09T01:24:21.124665+0000","last_became_active":"2024-05-09T01:22:39.838059+0000","last_became_peered":"2024-05-09T01:22:39.838059+0000","last_unstale":"2024-05-09T01:24:21.124665+0000","last_undegraded":"2024-05-09T01:24:21.124665+0000","last_fullsized":"2024-05-09T01:24:21.124665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:20.968526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153364+0000","last_change":"2024-05-09T01:22:39.842407+0000","last_active":"2024-05-09T01:24:21.153364+0000","last_peered":"2024-05-09T01:24:21.153364+0000","last_clean":"2024-05-09T01:24:21.153364+0000","last_became_active":"2024-05-09T01:22:39.841982+0000","last_became_peered":"2024-05-09T01:22:39.841982+0000","last_unstale":"2024-05-09T01:24:21.153364+0000","last_undegraded":"2024-05-09T01:24:21.153364+0000","last_fullsized":"2024-05-09T01:24:21.153364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:17.086304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155290+0000","last_change":"2024-05-09T01:22:39.833800+0000","last_active":"2024-05-09T01:24:21.155290+0000","last_peered":"2024-05-09T01:24:21.155290+0000","last_clean":"2024-05-09T01:24:21.155290+0000","last_became_active":"2024-05-09T01:22:39.833388+0000","last_became_peered":"2024-05-09T01:22:39.833388+0000","last_unstale":"2024-05-09T01:24:21.155290+0000","last_undegraded":"2024-05-09T01:24:21.155290+0000","last_fullsized":"2024-05-09T01:24:21.155290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:35.341428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142200+0000","last_change":"2024-05-09T01:22:39.845395+0000","last_active":"2024-05-09T01:24:21.142200+0000","last_peered":"2024-05-09T01:24:21.142200+0000","last_clean":"2024-05-09T01:24:21.142200+0000","last_became_active":"2024-05-09T01:22:39.844946+0000","last_became_peered":"2024-05-09T01:22:39.844946+0000","last_unstale":"2024-05-09T01:24:21.142200+0000","last_undegraded":"2024-05-09T01:24:21.142200+0000","last_fullsized":"2024-05-09T01:24:21.142200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:44.750614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159768+0000","last_change":"2024-05-09T01:22:39.832894+0000","last_active":"2024-05-09T01:24:21.159768+0000","last_peered":"2024-05-09T01:24:21.159768+0000","last_clean":"2024-05-09T01:24:21.159768+0000","last_became_active":"2024-05-09T01:22:39.829743+0000","last_became_peered":"2024-05-09T01:22:39.829743+0000","last_unstale":"2024-05-09T01:24:21.159768+0000","last_undegraded":"2024-05-09T01:24:21.159768+0000","last_fullsized":"2024-05-09T01:24:21.159768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.463833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155902+0000","last_change":"2024-05-09T01:22:39.843131+0000","last_active":"2024-05-09T01:24:21.155902+0000","last_peered":"2024-05-09T01:24:21.155902+0000","last_clean":"2024-05-09T01:24:21.155902+0000","last_became_active":"2024-05-09T01:22:39.842690+0000","last_became_peered":"2024-05-09T01:22:39.842690+0000","last_unstale":"2024-05-09T01:24:21.155902+0000","last_undegraded":"2024-05-09T01:24:21.155902+0000","last_fullsized":"2024-05-09T01:24:21.155902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:14.172365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166267+0000","last_change":"2024-05-09T01:22:39.845557+0000","last_active":"2024-05-09T01:24:21.166267+0000","last_peered":"2024-05-09T01:24:21.166267+0000","last_clean":"2024-05-09T01:24:21.166267+0000","last_became_active":"2024-05-09T01:22:39.845059+0000","last_became_peered":"2024-05-09T01:22:39.845059+0000","last_unstale":"2024-05-09T01:24:21.166267+0000","last_undegraded":"2024-05-09T01:24:21.166267+0000","last_fullsized":"2024-05-09T01:24:21.166267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:47.238080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.129595+0000","last_change":"2024-05-09T01:22:39.842237+0000","last_active":"2024-05-09T01:24:21.129595+0000","last_peered":"2024-05-09T01:24:21.129595+0000","last_clean":"2024-05-09T01:24:21.129595+0000","last_became_active":"2024-05-09T01:22:39.841849+0000","last_became_peered":"2024-05-09T01:22:39.841849+0000","last_unstale":"2024-05-09T01:24:21.129595+0000","last_undegraded":"2024-05-09T01:24:21.129595+0000","last_fullsized":"2024-05-09T01:24:21.129595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:40.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150804+0000","last_change":"2024-05-09T01:22:39.839943+0000","last_active":"2024-05-09T01:24:21.150804+0000","last_peered":"2024-05-09T01:24:21.150804+0000","last_clean":"2024-05-09T01:24:21.150804+0000","last_became_active":"2024-05-09T01:22:39.839604+0000","last_became_peered":"2024-05-09T01:22:39.839604+0000","last_unstale":"2024-05-09T01:24:21.150804+0000","last_undegraded":"2024-05-09T01:24:21.150804+0000","last_fullsized":"2024-05-09T01:24:21.150804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:34.854263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145018+0000","last_change":"2024-05-09T01:22:39.846981+0000","last_active":"2024-05-09T01:24:21.145018+0000","last_peered":"2024-05-09T01:24:21.145018+0000","last_clean":"2024-05-09T01:24:21.145018+0000","last_became_active":"2024-05-09T01:22:39.846586+0000","last_became_peered":"2024-05-09T01:22:39.846586+0000","last_unstale":"2024-05-09T01:24:21.145018+0000","last_undegraded":"2024-05-09T01:24:21.145018+0000","last_fullsized":"2024-05-09T01:24:21.145018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:50.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151239+0000","last_change":"2024-05-09T01:22:39.848964+0000","last_active":"2024-05-09T01:24:21.151239+0000","last_peered":"2024-05-09T01:24:21.151239+0000","last_clean":"2024-05-09T01:24:21.151239+0000","last_became_active":"2024-05-09T01:22:39.848442+0000","last_became_peered":"2024-05-09T01:22:39.848442+0000","last_unstale":"2024-05-09T01:24:21.151239+0000","last_undegraded":"2024-05-09T01:24:21.151239+0000","last_fullsized":"2024-05-09T01:24:21.151239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:35.802739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164175+0000","last_change":"2024-05-09T01:22:39.840586+0000","last_active":"2024-05-09T01:24:21.164175+0000","last_peered":"2024-05-09T01:24:21.164175+0000","last_clean":"2024-05-09T01:24:21.164175+0000","last_became_active":"2024-05-09T01:22:39.839887+0000","last_became_peered":"2024-05-09T01:22:39.839887+0000","last_unstale":"2024-05-09T01:24:21.164175+0000","last_undegraded":"2024-05-09T01:24:21.164175+0000","last_fullsized":"2024-05-09T01:24:21.164175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:36.510039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158620+0000","last_change":"2024-05-09T01:22:39.836152+0000","last_active":"2024-05-09T01:24:21.158620+0000","last_peered":"2024-05-09T01:24:21.158620+0000","last_clean":"2024-05-09T01:24:21.158620+0000","last_became_active":"2024-05-09T01:22:39.834691+0000","last_became_peered":"2024-05-09T01:22:39.834691+0000","last_unstale":"2024-05-09T01:24:21.158620+0000","last_undegraded":"2024-05-09T01:24:21.158620+0000","last_fullsized":"2024-05-09T01:24:21.158620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:31.331096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163485+0000","last_change":"2024-05-09T01:22:39.839760+0000","last_active":"2024-05-09T01:24:21.163485+0000","last_peered":"2024-05-09T01:24:21.163485+0000","last_clean":"2024-05-09T01:24:21.163485+0000","last_became_active":"2024-05-09T01:22:39.839403+0000","last_became_peered":"2024-05-09T01:22:39.839403+0000","last_unstale":"2024-05-09T01:24:21.163485+0000","last_undegraded":"2024-05-09T01:24:21.163485+0000","last_fullsized":"2024-05-09T01:24:21.163485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:19.248368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142981+0000","last_change":"2024-05-09T01:22:39.837538+0000","last_active":"2024-05-09T01:24:21.142981+0000","last_peered":"2024-05-09T01:24:21.142981+0000","last_clean":"2024-05-09T01:24:21.142981+0000","last_became_active":"2024-05-09T01:22:39.837167+0000","last_became_peered":"2024-05-09T01:22:39.837167+0000","last_unstale":"2024-05-09T01:24:21.142981+0000","last_undegraded":"2024-05-09T01:24:21.142981+0000","last_fullsized":"2024-05-09T01:24:21.142981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:35.695077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'15","reported_seq":244,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:58.648201+0000","last_change":"2024-05-09T01:23:22.433852+0000","last_active":"2024-05-09T01:25:58.648201+0000","last_peered":"2024-05-09T01:25:58.648201+0000","last_clean":"2024-05-09T01:25:58.648201+0000","last_became_active":"2024-05-09T01:23:22.423465+0000","last_became_peered":"2024-05-09T01:23:22.423465+0000","last_unstale":"2024-05-09T01:25:58.648201+0000","last_undegraded":"2024-05-09T01:25:58.648201+0000","last_fullsized":"2024-05-09T01:25:58.648201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:58.578798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.137595+0000","last_change":"2024-05-09T01:22:39.842667+0000","last_active":"2024-05-09T01:24:21.137595+0000","last_peered":"2024-05-09T01:24:21.137595+0000","last_clean":"2024-05-09T01:24:21.137595+0000","last_became_active":"2024-05-09T01:22:39.842256+0000","last_became_peered":"2024-05-09T01:22:39.842256+0000","last_unstale":"2024-05-09T01:24:21.137595+0000","last_undegraded":"2024-05-09T01:24:21.137595+0000","last_fullsized":"2024-05-09T01:24:21.137595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:28.525790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.314581+0000","last_change":"2024-05-09T01:22:36.841515+0000","last_active":"2024-05-09T01:23:22.314581+0000","last_peered":"2024-05-09T01:23:22.314581+0000","last_clean":"2024-05-09T01:23:22.314581+0000","last_became_active":"2024-05-09T01:22:36.841128+0000","last_became_peered":"2024-05-09T01:22:36.841128+0000","last_unstale":"2024-05-09T01:23:22.314581+0000","last_undegraded":"2024-05-09T01:23:22.314581+0000","last_fullsized":"2024-05-09T01:23:22.314581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:13.598177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157725+0000","last_change":"2024-05-09T01:22:39.843491+0000","last_active":"2024-05-09T01:24:21.157725+0000","last_peered":"2024-05-09T01:24:21.157725+0000","last_clean":"2024-05-09T01:24:21.157725+0000","last_became_active":"2024-05-09T01:22:39.843098+0000","last_became_peered":"2024-05-09T01:22:39.843098+0000","last_unstale":"2024-05-09T01:24:21.157725+0000","last_undegraded":"2024-05-09T01:24:21.157725+0000","last_fullsized":"2024-05-09T01:24:21.157725+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:03:33.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:08.922898+0000","last_change":"2024-05-09T01:22:36.798217+0000","last_active":"2024-05-09T01:25:08.922898+0000","last_peered":"2024-05-09T01:25:08.922898+0000","last_clean":"2024-05-09T01:25:08.922898+0000","last_became_active":"2024-05-09T01:22:36.797847+0000","last_became_peered":"2024-05-09T01:22:36.797847+0000","last_unstale":"2024-05-09T01:25:08.922898+0000","last_undegraded":"2024-05-09T01:25:08.922898+0000","last_fullsized":"2024-05-09T01:25:08.922898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:21.116535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164715+0000","last_change":"2024-05-09T01:22:39.837568+0000","last_active":"2024-05-09T01:24:21.164715+0000","last_peered":"2024-05-09T01:24:21.164715+0000","last_clean":"2024-05-09T01:24:21.164715+0000","last_became_active":"2024-05-09T01:22:39.836159+0000","last_became_peered":"2024-05-09T01:22:39.836159+0000","last_unstale":"2024-05-09T01:24:21.164715+0000","last_undegraded":"2024-05-09T01:24:21.164715+0000","last_fullsized":"2024-05-09T01:24:21.164715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:28.089830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303449+0000","last_change":"2024-05-09T01:22:36.798185+0000","last_active":"2024-05-09T01:23:22.303449+0000","last_peered":"2024-05-09T01:23:22.303449+0000","last_clean":"2024-05-09T01:23:22.303449+0000","last_became_active":"2024-05-09T01:22:36.797522+0000","last_became_peered":"2024-05-09T01:22:36.797522+0000","last_unstale":"2024-05-09T01:23:22.303449+0000","last_undegraded":"2024-05-09T01:23:22.303449+0000","last_fullsized":"2024-05-09T01:23:22.303449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:40.960006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153901+0000","last_change":"2024-05-09T01:22:39.837436+0000","last_active":"2024-05-09T01:24:21.153901+0000","last_peered":"2024-05-09T01:24:21.153901+0000","last_clean":"2024-05-09T01:24:21.153901+0000","last_became_active":"2024-05-09T01:22:39.837050+0000","last_became_peered":"2024-05-09T01:22:39.837050+0000","last_unstale":"2024-05-09T01:24:21.153901+0000","last_undegraded":"2024-05-09T01:24:21.153901+0000","last_fullsized":"2024-05-09T01:24:21.153901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:07.149311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":214,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:56.900380+0000","last_change":"2024-05-09T01:22:36.800767+0000","last_active":"2024-05-09T01:25:56.900380+0000","last_peered":"2024-05-09T01:25:56.900380+0000","last_clean":"2024-05-09T01:25:56.900380+0000","last_became_active":"2024-05-09T01:22:36.800183+0000","last_became_peered":"2024-05-09T01:22:36.800183+0000","last_unstale":"2024-05-09T01:25:56.900380+0000","last_undegraded":"2024-05-09T01:25:56.900380+0000","last_fullsized":"2024-05-09T01:25:56.900380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:05.682889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.131431+0000","last_change":"2024-05-09T01:22:39.832798+0000","last_active":"2024-05-09T01:24:21.131431+0000","last_peered":"2024-05-09T01:24:21.131431+0000","last_clean":"2024-05-09T01:24:21.131431+0000","last_became_active":"2024-05-09T01:22:39.828803+0000","last_became_peered":"2024-05-09T01:22:39.828803+0000","last_unstale":"2024-05-09T01:24:21.131431+0000","last_undegraded":"2024-05-09T01:24:21.131431+0000","last_fullsized":"2024-05-09T01:24:21.131431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:23.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306039+0000","last_change":"2024-05-09T01:22:36.794171+0000","last_active":"2024-05-09T01:23:22.306039+0000","last_peered":"2024-05-09T01:23:22.306039+0000","last_clean":"2024-05-09T01:23:22.306039+0000","last_became_active":"2024-05-09T01:22:36.793713+0000","last_became_peered":"2024-05-09T01:22:36.793713+0000","last_unstale":"2024-05-09T01:23:22.306039+0000","last_undegraded":"2024-05-09T01:23:22.306039+0000","last_fullsized":"2024-05-09T01:23:22.306039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:06.654667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153390+0000","last_change":"2024-05-09T01:22:39.838266+0000","last_active":"2024-05-09T01:24:21.153390+0000","last_peered":"2024-05-09T01:24:21.153390+0000","last_clean":"2024-05-09T01:24:21.153390+0000","last_became_active":"2024-05-09T01:22:39.837911+0000","last_became_peered":"2024-05-09T01:22:39.837911+0000","last_unstale":"2024-05-09T01:24:21.153390+0000","last_undegraded":"2024-05-09T01:24:21.153390+0000","last_fullsized":"2024-05-09T01:24:21.153390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:39.414176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308603+0000","last_change":"2024-05-09T01:22:36.799766+0000","last_active":"2024-05-09T01:23:22.308603+0000","last_peered":"2024-05-09T01:23:22.308603+0000","last_clean":"2024-05-09T01:23:22.308603+0000","last_became_active":"2024-05-09T01:22:36.799388+0000","last_became_peered":"2024-05-09T01:22:36.799388+0000","last_unstale":"2024-05-09T01:23:22.308603+0000","last_undegraded":"2024-05-09T01:23:22.308603+0000","last_fullsized":"2024-05-09T01:23:22.308603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:13.401659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141757+0000","last_change":"2024-05-09T01:22:39.838493+0000","last_active":"2024-05-09T01:24:21.141757+0000","last_peered":"2024-05-09T01:24:21.141757+0000","last_clean":"2024-05-09T01:24:21.141757+0000","last_became_active":"2024-05-09T01:22:39.838086+0000","last_became_peered":"2024-05-09T01:22:39.838086+0000","last_unstale":"2024-05-09T01:24:21.141757+0000","last_undegraded":"2024-05-09T01:24:21.141757+0000","last_fullsized":"2024-05-09T01:24:21.141757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:06.340693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421578+0000","last_change":"2024-05-09T01:22:36.804872+0000","last_active":"2024-05-09T01:23:22.421578+0000","last_peered":"2024-05-09T01:23:22.421578+0000","last_clean":"2024-05-09T01:23:22.421578+0000","last_became_active":"2024-05-09T01:22:36.804439+0000","last_became_peered":"2024-05-09T01:22:36.804439+0000","last_unstale":"2024-05-09T01:23:22.421578+0000","last_undegraded":"2024-05-09T01:23:22.421578+0000","last_fullsized":"2024-05-09T01:23:22.421578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:08.584749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.162012+0000","last_change":"2024-05-09T01:22:39.836579+0000","last_active":"2024-05-09T01:24:21.162012+0000","last_peered":"2024-05-09T01:24:21.162012+0000","last_clean":"2024-05-09T01:24:21.162012+0000","last_became_active":"2024-05-09T01:22:39.836244+0000","last_became_peered":"2024-05-09T01:22:39.836244+0000","last_unstale":"2024-05-09T01:24:21.162012+0000","last_undegraded":"2024-05-09T01:24:21.162012+0000","last_fullsized":"2024-05-09T01:24:21.162012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:43.329448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":395,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:59.590521+0000","last_change":"2024-05-09T01:22:36.796639+0000","last_active":"2024-05-09T01:25:59.590521+0000","last_peered":"2024-05-09T01:25:59.590521+0000","last_clean":"2024-05-09T01:25:59.590521+0000","last_became_active":"2024-05-09T01:22:36.796227+0000","last_became_peered":"2024-05-09T01:22:36.796227+0000","last_unstale":"2024-05-09T01:25:59.590521+0000","last_undegraded":"2024-05-09T01:25:59.590521+0000","last_fullsized":"2024-05-09T01:25:59.590521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:56.219281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":363,"num_read_kb":363,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.162895+0000","last_change":"2024-05-09T01:22:39.841515+0000","last_active":"2024-05-09T01:24:21.162895+0000","last_peered":"2024-05-09T01:24:21.162895+0000","last_clean":"2024-05-09T01:24:21.162895+0000","last_became_active":"2024-05-09T01:22:39.840057+0000","last_became_peered":"2024-05-09T01:22:39.840057+0000","last_unstale":"2024-05-09T01:24:21.162895+0000","last_undegraded":"2024-05-09T01:24:21.162895+0000","last_fullsized":"2024-05-09T01:24:21.162895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:43.258466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309168+0000","last_change":"2024-05-09T01:22:36.801937+0000","last_active":"2024-05-09T01:23:22.309168+0000","last_peered":"2024-05-09T01:23:22.309168+0000","last_clean":"2024-05-09T01:23:22.309168+0000","last_became_active":"2024-05-09T01:22:36.801563+0000","last_became_peered":"2024-05-09T01:22:36.801563+0000","last_unstale":"2024-05-09T01:23:22.309168+0000","last_undegraded":"2024-05-09T01:23:22.309168+0000","last_fullsized":"2024-05-09T01:23:22.309168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:00:42.170156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151364+0000","last_change":"2024-05-09T01:22:39.849154+0000","last_active":"2024-05-09T01:24:21.151364+0000","last_peered":"2024-05-09T01:24:21.151364+0000","last_clean":"2024-05-09T01:24:21.151364+0000","last_became_active":"2024-05-09T01:22:39.848815+0000","last_became_peered":"2024-05-09T01:22:39.848815+0000","last_unstale":"2024-05-09T01:24:21.151364+0000","last_undegraded":"2024-05-09T01:24:21.151364+0000","last_fullsized":"2024-05-09T01:24:21.151364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:00.300208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.422043+0000","last_change":"2024-05-09T01:22:36.843201+0000","last_active":"2024-05-09T01:23:22.422043+0000","last_peered":"2024-05-09T01:23:22.422043+0000","last_clean":"2024-05-09T01:23:22.422043+0000","last_became_active":"2024-05-09T01:22:36.842500+0000","last_became_peered":"2024-05-09T01:22:36.842500+0000","last_unstale":"2024-05-09T01:23:22.422043+0000","last_undegraded":"2024-05-09T01:23:22.422043+0000","last_fullsized":"2024-05-09T01:23:22.422043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:35.092246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159850+0000","last_change":"2024-05-09T01:22:39.835120+0000","last_active":"2024-05-09T01:24:21.159850+0000","last_peered":"2024-05-09T01:24:21.159850+0000","last_clean":"2024-05-09T01:24:21.159850+0000","last_became_active":"2024-05-09T01:22:39.834613+0000","last_became_peered":"2024-05-09T01:22:39.834613+0000","last_unstale":"2024-05-09T01:24:21.159850+0000","last_undegraded":"2024-05-09T01:24:21.159850+0000","last_fullsized":"2024-05-09T01:24:21.159850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:08.380445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":443,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:59.930492+0000","last_change":"2024-05-09T01:22:36.800816+0000","last_active":"2024-05-09T01:25:59.930492+0000","last_peered":"2024-05-09T01:25:59.930492+0000","last_clean":"2024-05-09T01:25:59.930492+0000","last_became_active":"2024-05-09T01:22:36.798797+0000","last_became_peered":"2024-05-09T01:22:36.798797+0000","last_unstale":"2024-05-09T01:25:59.930492+0000","last_undegraded":"2024-05-09T01:25:59.930492+0000","last_fullsized":"2024-05-09T01:25:59.930492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:37.670163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68071,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":377,"num_read_kb":255,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134714+0000","last_change":"2024-05-09T01:22:39.831504+0000","last_active":"2024-05-09T01:24:21.134714+0000","last_peered":"2024-05-09T01:24:21.134714+0000","last_clean":"2024-05-09T01:24:21.134714+0000","last_became_active":"2024-05-09T01:22:39.831129+0000","last_became_peered":"2024-05-09T01:22:39.831129+0000","last_unstale":"2024-05-09T01:24:21.134714+0000","last_undegraded":"2024-05-09T01:24:21.134714+0000","last_fullsized":"2024-05-09T01:24:21.134714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:19.212473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304494+0000","last_change":"2024-05-09T01:22:36.803447+0000","last_active":"2024-05-09T01:23:22.304494+0000","last_peered":"2024-05-09T01:23:22.304494+0000","last_clean":"2024-05-09T01:23:22.304494+0000","last_became_active":"2024-05-09T01:22:36.802986+0000","last_became_peered":"2024-05-09T01:22:36.802986+0000","last_unstale":"2024-05-09T01:23:22.304494+0000","last_undegraded":"2024-05-09T01:23:22.304494+0000","last_fullsized":"2024-05-09T01:23:22.304494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:28.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163307+0000","last_change":"2024-05-09T01:22:39.836546+0000","last_active":"2024-05-09T01:24:21.163307+0000","last_peered":"2024-05-09T01:24:21.163307+0000","last_clean":"2024-05-09T01:24:21.163307+0000","last_became_active":"2024-05-09T01:22:39.836168+0000","last_became_peered":"2024-05-09T01:22:39.836168+0000","last_unstale":"2024-05-09T01:24:21.163307+0000","last_undegraded":"2024-05-09T01:24:21.163307+0000","last_fullsized":"2024-05-09T01:24:21.163307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:45.522279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421659+0000","last_change":"2024-05-09T01:22:36.798703+0000","last_active":"2024-05-09T01:23:22.421659+0000","last_peered":"2024-05-09T01:23:22.421659+0000","last_clean":"2024-05-09T01:23:22.421659+0000","last_became_active":"2024-05-09T01:22:36.798280+0000","last_became_peered":"2024-05-09T01:22:36.798280+0000","last_unstale":"2024-05-09T01:23:22.421659+0000","last_undegraded":"2024-05-09T01:23:22.421659+0000","last_fullsized":"2024-05-09T01:23:22.421659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:33.794488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166230+0000","last_change":"2024-05-09T01:22:39.831223+0000","last_active":"2024-05-09T01:24:21.166230+0000","last_peered":"2024-05-09T01:24:21.166230+0000","last_clean":"2024-05-09T01:24:21.166230+0000","last_became_active":"2024-05-09T01:22:39.830845+0000","last_became_peered":"2024-05-09T01:22:39.830845+0000","last_unstale":"2024-05-09T01:24:21.166230+0000","last_undegraded":"2024-05-09T01:24:21.166230+0000","last_fullsized":"2024-05-09T01:24:21.166230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:23.365950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":220,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.952967+0000","last_change":"2024-05-09T01:22:36.802327+0000","last_active":"2024-05-09T01:25:57.952967+0000","last_peered":"2024-05-09T01:25:57.952967+0000","last_clean":"2024-05-09T01:25:57.952967+0000","last_became_active":"2024-05-09T01:22:36.801951+0000","last_became_peered":"2024-05-09T01:22:36.801951+0000","last_unstale":"2024-05-09T01:25:57.952967+0000","last_undegraded":"2024-05-09T01:25:57.952967+0000","last_fullsized":"2024-05-09T01:25:57.952967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:12.282424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154280+0000","last_change":"2024-05-09T01:22:39.833626+0000","last_active":"2024-05-09T01:24:21.154280+0000","last_peered":"2024-05-09T01:24:21.154280+0000","last_clean":"2024-05-09T01:24:21.154280+0000","last_became_active":"2024-05-09T01:22:39.833176+0000","last_became_peered":"2024-05-09T01:22:39.833176+0000","last_unstale":"2024-05-09T01:24:21.154280+0000","last_undegraded":"2024-05-09T01:24:21.154280+0000","last_fullsized":"2024-05-09T01:24:21.154280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:25.082300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:26.194840+0000","last_change":"2024-05-09T01:23:22.328011+0000","last_active":"2024-05-09T01:23:26.194840+0000","last_peered":"2024-05-09T01:23:26.194840+0000","last_clean":"2024-05-09T01:23:26.194840+0000","last_became_active":"2024-05-09T01:23:22.313049+0000","last_became_peered":"2024-05-09T01:23:22.313049+0000","last_unstale":"2024-05-09T01:23:26.194840+0000","last_undegraded":"2024-05-09T01:23:26.194840+0000","last_fullsized":"2024-05-09T01:23:26.194840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:17.854491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153329+0000","last_change":"2024-05-09T01:22:39.837529+0000","last_active":"2024-05-09T01:24:21.153329+0000","last_peered":"2024-05-09T01:24:21.153329+0000","last_clean":"2024-05-09T01:24:21.153329+0000","last_became_active":"2024-05-09T01:22:39.837155+0000","last_became_peered":"2024-05-09T01:22:39.837155+0000","last_unstale":"2024-05-09T01:24:21.153329+0000","last_undegraded":"2024-05-09T01:24:21.153329+0000","last_fullsized":"2024-05-09T01:24:21.153329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:44.485629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303600+0000","last_change":"2024-05-09T01:22:36.800073+0000","last_active":"2024-05-09T01:23:22.303600+0000","last_peered":"2024-05-09T01:23:22.303600+0000","last_clean":"2024-05-09T01:23:22.303600+0000","last_became_active":"2024-05-09T01:22:36.799463+0000","last_became_peered":"2024-05-09T01:22:36.799463+0000","last_unstale":"2024-05-09T01:23:22.303600+0000","last_undegraded":"2024-05-09T01:23:22.303600+0000","last_fullsized":"2024-05-09T01:23:22.303600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.942945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150713+0000","last_change":"2024-05-09T01:22:39.833615+0000","last_active":"2024-05-09T01:24:21.150713+0000","last_peered":"2024-05-09T01:24:21.150713+0000","last_clean":"2024-05-09T01:24:21.150713+0000","last_became_active":"2024-05-09T01:22:39.833212+0000","last_became_peered":"2024-05-09T01:22:39.833212+0000","last_unstale":"2024-05-09T01:24:21.150713+0000","last_undegraded":"2024-05-09T01:24:21.150713+0000","last_fullsized":"2024-05-09T01:24:21.150713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:35.387442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305695+0000","last_change":"2024-05-09T01:22:36.843143+0000","last_active":"2024-05-09T01:23:22.305695+0000","last_peered":"2024-05-09T01:23:22.305695+0000","last_clean":"2024-05-09T01:23:22.305695+0000","last_became_active":"2024-05-09T01:22:36.842759+0000","last_became_peered":"2024-05-09T01:22:36.842759+0000","last_unstale":"2024-05-09T01:23:22.305695+0000","last_undegraded":"2024-05-09T01:23:22.305695+0000","last_fullsized":"2024-05-09T01:23:22.305695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:42.666186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.143568+0000","last_change":"2024-05-09T01:22:39.838267+0000","last_active":"2024-05-09T01:24:21.143568+0000","last_peered":"2024-05-09T01:24:21.143568+0000","last_clean":"2024-05-09T01:24:21.143568+0000","last_became_active":"2024-05-09T01:22:39.837910+0000","last_became_peered":"2024-05-09T01:22:39.837910+0000","last_unstale":"2024-05-09T01:24:21.143568+0000","last_undegraded":"2024-05-09T01:24:21.143568+0000","last_fullsized":"2024-05-09T01:24:21.143568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:34.620332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'8","reported_seq":219,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.647507+0000","last_change":"2024-05-09T01:22:36.800515+0000","last_active":"2024-05-09T01:25:57.647507+0000","last_peered":"2024-05-09T01:25:57.647507+0000","last_clean":"2024-05-09T01:25:57.647507+0000","last_became_active":"2024-05-09T01:22:36.800065+0000","last_became_peered":"2024-05-09T01:22:36.800065+0000","last_unstale":"2024-05-09T01:25:57.647507+0000","last_undegraded":"2024-05-09T01:25:57.647507+0000","last_fullsized":"2024-05-09T01:25:57.647507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:50.160463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145505+0000","last_change":"2024-05-09T01:23:22.313627+0000","last_active":"2024-05-09T01:24:21.145505+0000","last_peered":"2024-05-09T01:24:21.145505+0000","last_clean":"2024-05-09T01:24:21.145505+0000","last_became_active":"2024-05-09T01:23:22.313138+0000","last_became_peered":"2024-05-09T01:23:22.313138+0000","last_unstale":"2024-05-09T01:24:21.145505+0000","last_undegraded":"2024-05-09T01:24:21.145505+0000","last_fullsized":"2024-05-09T01:24:21.145505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:07.216086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153456+0000","last_change":"2024-05-09T01:22:39.832188+0000","last_active":"2024-05-09T01:24:21.153456+0000","last_peered":"2024-05-09T01:24:21.153456+0000","last_clean":"2024-05-09T01:24:21.153456+0000","last_became_active":"2024-05-09T01:22:39.831302+0000","last_became_peered":"2024-05-09T01:22:39.831302+0000","last_unstale":"2024-05-09T01:24:21.153456+0000","last_undegraded":"2024-05-09T01:24:21.153456+0000","last_fullsized":"2024-05-09T01:24:21.153456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:06.703950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'7","reported_seq":216,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.952377+0000","last_change":"2024-05-09T01:22:36.841446+0000","last_active":"2024-05-09T01:25:57.952377+0000","last_peered":"2024-05-09T01:25:57.952377+0000","last_clean":"2024-05-09T01:25:57.952377+0000","last_became_active":"2024-05-09T01:22:36.841071+0000","last_became_peered":"2024-05-09T01:22:36.841071+0000","last_unstale":"2024-05-09T01:25:57.952377+0000","last_undegraded":"2024-05-09T01:25:57.952377+0000","last_fullsized":"2024-05-09T01:25:57.952377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:20.777482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163865+0000","last_change":"2024-05-09T01:22:39.846711+0000","last_active":"2024-05-09T01:24:21.163865+0000","last_peered":"2024-05-09T01:24:21.163865+0000","last_clean":"2024-05-09T01:24:21.163865+0000","last_became_active":"2024-05-09T01:22:39.846293+0000","last_became_peered":"2024-05-09T01:22:39.846293+0000","last_unstale":"2024-05-09T01:24:21.163865+0000","last_undegraded":"2024-05-09T01:24:21.163865+0000","last_fullsized":"2024-05-09T01:24:21.163865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:27.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303855+0000","last_change":"2024-05-09T01:22:36.798302+0000","last_active":"2024-05-09T01:23:22.303855+0000","last_peered":"2024-05-09T01:23:22.303855+0000","last_clean":"2024-05-09T01:23:22.303855+0000","last_became_active":"2024-05-09T01:22:36.797906+0000","last_became_peered":"2024-05-09T01:22:36.797906+0000","last_unstale":"2024-05-09T01:23:22.303855+0000","last_undegraded":"2024-05-09T01:23:22.303855+0000","last_fullsized":"2024-05-09T01:23:22.303855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:13.599240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:59.123370+0000","last_change":"2024-05-09T01:21:12.781366+0000","last_active":"2024-05-09T01:23:59.123370+0000","last_peered":"2024-05-09T01:23:59.123370+0000","last_clean":"2024-05-09T01:23:59.123370+0000","last_became_active":"2024-05-09T01:21:12.774967+0000","last_became_peered":"2024-05-09T01:21:12.774967+0000","last_unstale":"2024-05-09T01:23:59.123370+0000","last_undegraded":"2024-05-09T01:23:59.123370+0000","last_fullsized":"2024-05-09T01:23:59.123370+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_clean_scrub_stamp":"2024-05-09T01:17:49.682508+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:08.075365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155221+0000","last_change":"2024-05-09T01:22:39.840952+0000","last_active":"2024-05-09T01:24:21.155221+0000","last_peered":"2024-05-09T01:24:21.155221+0000","last_clean":"2024-05-09T01:24:21.155221+0000","last_became_active":"2024-05-09T01:22:39.840323+0000","last_became_peered":"2024-05-09T01:22:39.840323+0000","last_unstale":"2024-05-09T01:24:21.155221+0000","last_undegraded":"2024-05-09T01:24:21.155221+0000","last_fullsized":"2024-05-09T01:24:21.155221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:23.837123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308860+0000","last_change":"2024-05-09T01:22:36.798631+0000","last_active":"2024-05-09T01:23:22.308860+0000","last_peered":"2024-05-09T01:23:22.308860+0000","last_clean":"2024-05-09T01:23:22.308860+0000","last_became_active":"2024-05-09T01:22:36.798242+0000","last_became_peered":"2024-05-09T01:22:36.798242+0000","last_unstale":"2024-05-09T01:23:22.308860+0000","last_undegraded":"2024-05-09T01:23:22.308860+0000","last_fullsized":"2024-05-09T01:23:22.308860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:18.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303800+0000","last_change":"2024-05-09T01:22:36.843779+0000","last_active":"2024-05-09T01:23:22.303800+0000","last_peered":"2024-05-09T01:23:22.303800+0000","last_clean":"2024-05-09T01:23:22.303800+0000","last_became_active":"2024-05-09T01:22:36.842084+0000","last_became_peered":"2024-05-09T01:22:36.842084+0000","last_unstale":"2024-05-09T01:23:22.303800+0000","last_undegraded":"2024-05-09T01:23:22.303800+0000","last_fullsized":"2024-05-09T01:23:22.303800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:34.743372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146032+0000","last_change":"2024-05-09T01:22:39.842266+0000","last_active":"2024-05-09T01:24:21.146032+0000","last_peered":"2024-05-09T01:24:21.146032+0000","last_clean":"2024-05-09T01:24:21.146032+0000","last_became_active":"2024-05-09T01:22:39.841927+0000","last_became_peered":"2024-05-09T01:22:39.841927+0000","last_unstale":"2024-05-09T01:24:21.146032+0000","last_undegraded":"2024-05-09T01:24:21.146032+0000","last_fullsized":"2024-05-09T01:24:21.146032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:26.774250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'19","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:56.860389+0000","last_change":"2024-05-09T01:22:36.796929+0000","last_active":"2024-05-09T01:25:56.860389+0000","last_peered":"2024-05-09T01:25:56.860389+0000","last_clean":"2024-05-09T01:25:56.860389+0000","last_became_active":"2024-05-09T01:22:36.796480+0000","last_became_peered":"2024-05-09T01:22:36.796480+0000","last_unstale":"2024-05-09T01:25:56.860389+0000","last_undegraded":"2024-05-09T01:25:56.860389+0000","last_fullsized":"2024-05-09T01:25:56.860389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:45.168305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140120+0000","last_change":"2024-05-09T01:22:39.843093+0000","last_active":"2024-05-09T01:24:21.140120+0000","last_peered":"2024-05-09T01:24:21.140120+0000","last_clean":"2024-05-09T01:24:21.140120+0000","last_became_active":"2024-05-09T01:22:39.842729+0000","last_became_peered":"2024-05-09T01:22:39.842729+0000","last_unstale":"2024-05-09T01:24:21.140120+0000","last_undegraded":"2024-05-09T01:24:21.140120+0000","last_fullsized":"2024-05-09T01:24:21.140120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:02.074843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303716+0000","last_change":"2024-05-09T01:22:36.798166+0000","last_active":"2024-05-09T01:23:22.303716+0000","last_peered":"2024-05-09T01:23:22.303716+0000","last_clean":"2024-05-09T01:23:22.303716+0000","last_became_active":"2024-05-09T01:22:36.797752+0000","last_became_peered":"2024-05-09T01:22:36.797752+0000","last_unstale":"2024-05-09T01:23:22.303716+0000","last_undegraded":"2024-05-09T01:23:22.303716+0000","last_fullsized":"2024-05-09T01:23:22.303716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:05.211720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142128+0000","last_change":"2024-05-09T01:23:22.316228+0000","last_active":"2024-05-09T01:24:21.142128+0000","last_peered":"2024-05-09T01:24:21.142128+0000","last_clean":"2024-05-09T01:24:21.142128+0000","last_became_active":"2024-05-09T01:23:22.315683+0000","last_became_peered":"2024-05-09T01:23:22.315683+0000","last_unstale":"2024-05-09T01:24:21.142128+0000","last_undegraded":"2024-05-09T01:24:21.142128+0000","last_fullsized":"2024-05-09T01:24:21.142128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:04.623701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421356+0000","last_change":"2024-05-09T01:22:36.841999+0000","last_active":"2024-05-09T01:23:22.421356+0000","last_peered":"2024-05-09T01:23:22.421356+0000","last_clean":"2024-05-09T01:23:22.421356+0000","last_became_active":"2024-05-09T01:22:36.841664+0000","last_became_peered":"2024-05-09T01:22:36.841664+0000","last_unstale":"2024-05-09T01:23:22.421356+0000","last_undegraded":"2024-05-09T01:23:22.421356+0000","last_fullsized":"2024-05-09T01:23:22.421356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:27.110258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134153+0000","last_change":"2024-05-09T01:22:39.839553+0000","last_active":"2024-05-09T01:24:21.134153+0000","last_peered":"2024-05-09T01:24:21.134153+0000","last_clean":"2024-05-09T01:24:21.134153+0000","last_became_active":"2024-05-09T01:22:39.839189+0000","last_became_peered":"2024-05-09T01:22:39.839189+0000","last_unstale":"2024-05-09T01:24:21.134153+0000","last_undegraded":"2024-05-09T01:24:21.134153+0000","last_fullsized":"2024-05-09T01:24:21.134153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:02.794073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309908+0000","last_change":"2024-05-09T01:22:36.792516+0000","last_active":"2024-05-09T01:23:22.309908+0000","last_peered":"2024-05-09T01:23:22.309908+0000","last_clean":"2024-05-09T01:23:22.309908+0000","last_became_active":"2024-05-09T01:22:36.792129+0000","last_became_peered":"2024-05-09T01:22:36.792129+0000","last_unstale":"2024-05-09T01:23:22.309908+0000","last_undegraded":"2024-05-09T01:23:22.309908+0000","last_fullsized":"2024-05-09T01:23:22.309908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:46.596723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148884+0000","last_change":"2024-05-09T01:22:39.841523+0000","last_active":"2024-05-09T01:24:21.148884+0000","last_peered":"2024-05-09T01:24:21.148884+0000","last_clean":"2024-05-09T01:24:21.148884+0000","last_became_active":"2024-05-09T01:22:39.840667+0000","last_became_peered":"2024-05-09T01:22:39.840667+0000","last_unstale":"2024-05-09T01:24:21.148884+0000","last_undegraded":"2024-05-09T01:24:21.148884+0000","last_fullsized":"2024-05-09T01:24:21.148884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:39.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:26.194870+0000","last_change":"2024-05-09T01:23:22.331855+0000","last_active":"2024-05-09T01:23:26.194870+0000","last_peered":"2024-05-09T01:23:26.194870+0000","last_clean":"2024-05-09T01:23:26.194870+0000","last_became_active":"2024-05-09T01:23:22.313059+0000","last_became_peered":"2024-05-09T01:23:22.313059+0000","last_unstale":"2024-05-09T01:23:26.194870+0000","last_undegraded":"2024-05-09T01:23:26.194870+0000","last_fullsized":"2024-05-09T01:23:26.194870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:16.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142481+0000","last_change":"2024-05-09T01:22:39.845031+0000","last_active":"2024-05-09T01:24:21.142481+0000","last_peered":"2024-05-09T01:24:21.142481+0000","last_clean":"2024-05-09T01:24:21.142481+0000","last_became_active":"2024-05-09T01:22:39.844653+0000","last_became_peered":"2024-05-09T01:22:39.844653+0000","last_unstale":"2024-05-09T01:24:21.142481+0000","last_undegraded":"2024-05-09T01:24:21.142481+0000","last_fullsized":"2024-05-09T01:24:21.142481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:14.382445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303240+0000","last_change":"2024-05-09T01:22:36.801395+0000","last_active":"2024-05-09T01:23:22.303240+0000","last_peered":"2024-05-09T01:23:22.303240+0000","last_clean":"2024-05-09T01:23:22.303240+0000","last_became_active":"2024-05-09T01:22:36.800992+0000","last_became_peered":"2024-05-09T01:22:36.800992+0000","last_unstale":"2024-05-09T01:23:22.303240+0000","last_undegraded":"2024-05-09T01:23:22.303240+0000","last_fullsized":"2024-05-09T01:23:22.303240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:16.089068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166011+0000","last_change":"2024-05-09T01:22:39.835842+0000","last_active":"2024-05-09T01:24:21.166011+0000","last_peered":"2024-05-09T01:24:21.166011+0000","last_clean":"2024-05-09T01:24:21.166011+0000","last_became_active":"2024-05-09T01:22:39.835372+0000","last_became_peered":"2024-05-09T01:22:39.835372+0000","last_unstale":"2024-05-09T01:24:21.166011+0000","last_undegraded":"2024-05-09T01:24:21.166011+0000","last_fullsized":"2024-05-09T01:24:21.166011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:13.197946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":179,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.588036+0000","last_change":"2024-05-09T01:23:22.332514+0000","last_active":"2024-05-09T01:25:57.588036+0000","last_peered":"2024-05-09T01:25:57.588036+0000","last_clean":"2024-05-09T01:25:57.588036+0000","last_became_active":"2024-05-09T01:23:22.312966+0000","last_became_peered":"2024-05-09T01:23:22.312966+0000","last_unstale":"2024-05-09T01:25:57.588036+0000","last_undegraded":"2024-05-09T01:25:57.588036+0000","last_fullsized":"2024-05-09T01:25:57.588036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:52.637108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":163,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166259+0000","last_change":"2024-05-09T01:22:39.845390+0000","last_active":"2024-05-09T01:24:21.166259+0000","last_peered":"2024-05-09T01:24:21.166259+0000","last_clean":"2024-05-09T01:24:21.166259+0000","last_became_active":"2024-05-09T01:22:39.844918+0000","last_became_peered":"2024-05-09T01:22:39.844918+0000","last_unstale":"2024-05-09T01:24:21.166259+0000","last_undegraded":"2024-05-09T01:24:21.166259+0000","last_fullsized":"2024-05-09T01:24:21.166259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:04.455794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421891+0000","last_change":"2024-05-09T01:22:36.803329+0000","last_active":"2024-05-09T01:23:22.421891+0000","last_peered":"2024-05-09T01:23:22.421891+0000","last_clean":"2024-05-09T01:23:22.421891+0000","last_became_active":"2024-05-09T01:22:36.802831+0000","last_became_peered":"2024-05-09T01:22:36.802831+0000","last_unstale":"2024-05-09T01:23:22.421891+0000","last_undegraded":"2024-05-09T01:23:22.421891+0000","last_fullsized":"2024-05-09T01:23:22.421891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:57.919840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154788+0000","last_change":"2024-05-09T01:22:39.837595+0000","last_active":"2024-05-09T01:24:21.154788+0000","last_peered":"2024-05-09T01:24:21.154788+0000","last_clean":"2024-05-09T01:24:21.154788+0000","last_became_active":"2024-05-09T01:22:39.837197+0000","last_became_peered":"2024-05-09T01:22:39.837197+0000","last_unstale":"2024-05-09T01:24:21.154788+0000","last_undegraded":"2024-05-09T01:24:21.154788+0000","last_fullsized":"2024-05-09T01:24:21.154788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:08.700741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.321791+0000","last_change":"2024-05-09T01:23:22.321286+0000","last_active":"2024-05-09T01:23:22.321791+0000","last_peered":"2024-05-09T01:23:22.321791+0000","last_clean":"2024-05-09T01:23:22.321791+0000","last_became_active":"2024-05-09T01:23:22.315810+0000","last_became_peered":"2024-05-09T01:23:22.315810+0000","last_unstale":"2024-05-09T01:23:22.321791+0000","last_undegraded":"2024-05-09T01:23:22.321791+0000","last_fullsized":"2024-05-09T01:23:22.321791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:43.100893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.165342+0000","last_change":"2024-05-09T01:22:39.827798+0000","last_active":"2024-05-09T01:24:21.165342+0000","last_peered":"2024-05-09T01:24:21.165342+0000","last_clean":"2024-05-09T01:24:21.165342+0000","last_became_active":"2024-05-09T01:22:39.827322+0000","last_became_peered":"2024-05-09T01:22:39.827322+0000","last_unstale":"2024-05-09T01:24:21.165342+0000","last_undegraded":"2024-05-09T01:24:21.165342+0000","last_fullsized":"2024-05-09T01:24:21.165342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:52.454033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.149749+0000","last_change":"2024-05-09T01:22:39.841885+0000","last_active":"2024-05-09T01:24:21.149749+0000","last_peered":"2024-05-09T01:24:21.149749+0000","last_clean":"2024-05-09T01:24:21.149749+0000","last_became_active":"2024-05-09T01:22:39.841160+0000","last_became_peered":"2024-05-09T01:22:39.841160+0000","last_unstale":"2024-05-09T01:24:21.149749+0000","last_undegraded":"2024-05-09T01:24:21.149749+0000","last_fullsized":"2024-05-09T01:24:21.149749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:57.290465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146092+0000","last_change":"2024-05-09T01:22:39.846495+0000","last_active":"2024-05-09T01:24:21.146092+0000","last_peered":"2024-05-09T01:24:21.146092+0000","last_clean":"2024-05-09T01:24:21.146092+0000","last_became_active":"2024-05-09T01:22:39.846108+0000","last_became_peered":"2024-05-09T01:22:39.846108+0000","last_unstale":"2024-05-09T01:24:21.146092+0000","last_undegraded":"2024-05-09T01:24:21.146092+0000","last_fullsized":"2024-05-09T01:24:21.146092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:39.958174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.132056+0000","last_change":"2024-05-09T01:22:39.841288+0000","last_active":"2024-05-09T01:24:21.132056+0000","last_peered":"2024-05-09T01:24:21.132056+0000","last_clean":"2024-05-09T01:24:21.132056+0000","last_became_active":"2024-05-09T01:22:39.840900+0000","last_became_peered":"2024-05-09T01:22:39.840900+0000","last_unstale":"2024-05-09T01:24:21.132056+0000","last_undegraded":"2024-05-09T01:24:21.132056+0000","last_fullsized":"2024-05-09T01:24:21.132056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:30.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141526+0000","last_change":"2024-05-09T01:23:22.313135+0000","last_active":"2024-05-09T01:24:21.141526+0000","last_peered":"2024-05-09T01:24:21.141526+0000","last_clean":"2024-05-09T01:24:21.141526+0000","last_became_active":"2024-05-09T01:23:22.312710+0000","last_became_peered":"2024-05-09T01:23:22.312710+0000","last_unstale":"2024-05-09T01:24:21.141526+0000","last_undegraded":"2024-05-09T01:24:21.141526+0000","last_fullsized":"2024-05-09T01:24:21.141526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:47.725790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.156722+0000","last_change":"2024-05-09T01:22:39.838037+0000","last_active":"2024-05-09T01:24:21.156722+0000","last_peered":"2024-05-09T01:24:21.156722+0000","last_clean":"2024-05-09T01:24:21.156722+0000","last_became_active":"2024-05-09T01:22:39.837362+0000","last_became_peered":"2024-05-09T01:22:39.837362+0000","last_unstale":"2024-05-09T01:24:21.156722+0000","last_undegraded":"2024-05-09T01:24:21.156722+0000","last_fullsized":"2024-05-09T01:24:21.156722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.781318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146468+0000","last_change":"2024-05-09T01:23:22.310046+0000","last_active":"2024-05-09T01:24:21.146468+0000","last_peered":"2024-05-09T01:24:21.146468+0000","last_clean":"2024-05-09T01:24:21.146468+0000","last_became_active":"2024-05-09T01:23:22.309541+0000","last_became_peered":"2024-05-09T01:23:22.309541+0000","last_unstale":"2024-05-09T01:24:21.146468+0000","last_undegraded":"2024-05-09T01:24:21.146468+0000","last_fullsized":"2024-05-09T01:24:21.146468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:30.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.161756+0000","last_change":"2024-05-09T01:22:39.840453+0000","last_active":"2024-05-09T01:24:21.161756+0000","last_peered":"2024-05-09T01:24:21.161756+0000","last_clean":"2024-05-09T01:24:21.161756+0000","last_became_active":"2024-05-09T01:22:39.840077+0000","last_became_peered":"2024-05-09T01:22:39.840077+0000","last_unstale":"2024-05-09T01:24:21.161756+0000","last_undegraded":"2024-05-09T01:24:21.161756+0000","last_fullsized":"2024-05-09T01:24:21.161756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:20.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.160942+0000","last_change":"2024-05-09T01:22:39.844554+0000","last_active":"2024-05-09T01:24:21.160942+0000","last_peered":"2024-05-09T01:24:21.160942+0000","last_clean":"2024-05-09T01:24:21.160942+0000","last_became_active":"2024-05-09T01:22:39.843640+0000","last_became_peered":"2024-05-09T01:22:39.843640+0000","last_unstale":"2024-05-09T01:24:21.160942+0000","last_undegraded":"2024-05-09T01:24:21.160942+0000","last_fullsized":"2024-05-09T01:24:21.160942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:27.801565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140795+0000","last_change":"2024-05-09T01:22:39.839043+0000","last_active":"2024-05-09T01:24:21.140795+0000","last_peered":"2024-05-09T01:24:21.140795+0000","last_clean":"2024-05-09T01:24:21.140795+0000","last_became_active":"2024-05-09T01:22:39.838662+0000","last_became_peered":"2024-05-09T01:22:39.838662+0000","last_unstale":"2024-05-09T01:24:21.140795+0000","last_undegraded":"2024-05-09T01:24:21.140795+0000","last_fullsized":"2024-05-09T01:24:21.140795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:16.117689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.139048+0000","last_change":"2024-05-09T01:23:22.309945+0000","last_active":"2024-05-09T01:24:21.139048+0000","last_peered":"2024-05-09T01:24:21.139048+0000","last_clean":"2024-05-09T01:24:21.139048+0000","last_became_active":"2024-05-09T01:23:22.309543+0000","last_became_peered":"2024-05-09T01:23:22.309543+0000","last_unstale":"2024-05-09T01:24:21.139048+0000","last_undegraded":"2024-05-09T01:24:21.139048+0000","last_fullsized":"2024-05-09T01:24:21.139048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:47.464524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164552+0000","last_change":"2024-05-09T01:22:39.838911+0000","last_active":"2024-05-09T01:24:21.164552+0000","last_peered":"2024-05-09T01:24:21.164552+0000","last_clean":"2024-05-09T01:24:21.164552+0000","last_became_active":"2024-05-09T01:22:39.838293+0000","last_became_peered":"2024-05-09T01:22:39.838293+0000","last_unstale":"2024-05-09T01:24:21.164552+0000","last_undegraded":"2024-05-09T01:24:21.164552+0000","last_fullsized":"2024-05-09T01:24:21.164552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:32.596306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163721+0000","last_change":"2024-05-09T01:22:39.838185+0000","last_active":"2024-05-09T01:24:21.163721+0000","last_peered":"2024-05-09T01:24:21.163721+0000","last_clean":"2024-05-09T01:24:21.163721+0000","last_became_active":"2024-05-09T01:22:39.837530+0000","last_became_peered":"2024-05-09T01:22:39.837530+0000","last_unstale":"2024-05-09T01:24:21.163721+0000","last_undegraded":"2024-05-09T01:24:21.163721+0000","last_fullsized":"2024-05-09T01:24:21.163721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:22.556710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150895+0000","last_change":"2024-05-09T01:22:39.834124+0000","last_active":"2024-05-09T01:24:21.150895+0000","last_peered":"2024-05-09T01:24:21.150895+0000","last_clean":"2024-05-09T01:24:21.150895+0000","last_became_active":"2024-05-09T01:22:39.833716+0000","last_became_peered":"2024-05-09T01:22:39.833716+0000","last_unstale":"2024-05-09T01:24:21.150895+0000","last_undegraded":"2024-05-09T01:24:21.150895+0000","last_fullsized":"2024-05-09T01:24:21.150895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:54:42.337900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76871,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1834,"num_read_kb":1012,"num_write":273,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2345,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":230613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612664,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":437733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808893,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1097112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972418,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":437019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168648,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":913408,"data_stored":436613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1031370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528403,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":437659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":507315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":438607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":504770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117089,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":436561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":439222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724631,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1540096,"data_stored":1029202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T01:26:02.539 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-09T01:26:02.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T01:26:02.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:02 smithi196 ceph-mon[35041]: pgmap v393: 97 pgs: 97 active+clean; 652 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-05-09T01:26:02.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:02 smithi059 ceph-mon[34351]: pgmap v393: 97 pgs: 97 active+clean; 652 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 -- 172.21.15.19:0/3732735060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc108a70 msgr2=0x7f01cc10ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 --2- 172.21.15.19:0/3732735060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc108a70 0x7f01cc10ae90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009ca0 tx=0x7f01c402a8c0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 -- 172.21.15.19:0/3732735060 shutdown_connections 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 --2- 172.21.15.19:0/3732735060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc108a70 0x7f01cc10ae90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 --2- 172.21.15.19:0/3732735060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01cc103d70 0x7f01cc104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 --2- 172.21.15.19:0/3732735060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc102b30 0x7f01cc103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 -- 172.21.15.19:0/3732735060 >> 172.21.15.19:0/3732735060 conn(0x7f01cc0fdd20 msgr2=0x7f01cc1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 -- 172.21.15.19:0/3732735060 shutdown_connections 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 -- 172.21.15.19:0/3732735060 wait complete. 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.694+0000 7f01d4642640 1 Processor -- start 2024-05-09T01:26:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d4642640 1 -- start start 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d4642640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc102b30 0x7f01cc19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d4642640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc103d70 0x7f01cc19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d4642640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01cc108a70 0x7f01cc1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d4642640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc10f7b0 con 0x7f01cc103d70 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d4642640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc10f630 con 0x7f01cc102b30 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d4642640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc10f930 con 0x7f01cc108a70 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d1bdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc103d70 0x7f01cc19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d1bdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc103d70 0x7f01cc19a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58422/0 (socket says 172.21.15.19:58422) 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d1bdd640 1 -- 172.21.15.19:0/2890334445 learned_addr learned my addr 172.21.15.19:0/2890334445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d23de640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc102b30 0x7f01cc19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d2bdf640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01cc108a70 0x7f01cc1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d1bdd640 1 -- 172.21.15.19:0/2890334445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01cc108a70 msgr2=0x7f01cc1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d1bdd640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01cc108a70 0x7f01cc1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d1bdd640 1 -- 172.21.15.19:0/2890334445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc102b30 msgr2=0x7f01cc19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d1bdd640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc102b30 0x7f01cc19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d1bdd640 1 -- 172.21.15.19:0/2890334445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc1a1e00 con 0x7f01cc103d70 2024-05-09T01:26:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d2bdf640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01cc108a70 0x7f01cc1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.695+0000 7f01d1bdd640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc103d70 0x7f01cc19a800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00e9f0 tx=0x7f01bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.696+0000 7f01c37fe640 1 -- 172.21.15.19:0/2890334445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc00cd80 con 0x7f01cc103d70 2024-05-09T01:26:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.696+0000 7f01c37fe640 1 -- 172.21.15.19:0/2890334445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01bc0054d0 con 0x7f01cc103d70 2024-05-09T01:26:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.696+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01cc1a20f0 con 0x7f01cc103d70 2024-05-09T01:26:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.696+0000 7f01c37fe640 1 -- 172.21.15.19:0/2890334445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc010690 con 0x7f01cc103d70 2024-05-09T01:26:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.696+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01cc1a25f0 con 0x7f01cc103d70 2024-05-09T01:26:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.697+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01cc107880 con 0x7f01cc103d70 2024-05-09T01:26:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.698+0000 7f01c37fe640 1 -- 172.21.15.19:0/2890334445 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f01bc004e30 con 0x7f01cc103d70 2024-05-09T01:26:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.699+0000 7f01c37fe640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01b80b2880 0x7f01b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.699+0000 7f01d23de640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01b80b2880 0x7f01b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.699+0000 7f01c37fe640 1 -- 172.21.15.19:0/2890334445 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f01bc0e2b90 con 0x7f01cc103d70 2024-05-09T01:26:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.700+0000 7f01d23de640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01b80b2880 0x7f01b80b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f01cc103730 tx=0x7f01c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:02.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.703+0000 7f01c37fe640 1 -- 172.21.15.19:0/2890334445 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f01bc054df0 con 0x7f01cc103d70 2024-05-09T01:26:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.893+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 --> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01cc063110 con 0x7f01b80b2880 2024-05-09T01:26:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.900+0000 7f01c37fe640 1 -- 172.21.15.19:0/2890334445 <== mgr.14150 v2:172.21.15.19:6800/2331192605 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7f01cc063110 con 0x7f01b80b2880 2024-05-09T01:26:02.900 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:26:02.901 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T01:26:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01b80b2880 msgr2=0x7f01b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01b80b2880 0x7f01b80b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f01cc103730 tx=0x7f01c8005f90 comp rx=0 tx=0).stop 2024-05-09T01:26:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc103d70 msgr2=0x7f01cc19a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc103d70 0x7f01cc19a800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00e9f0 tx=0x7f01bc00edf0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 shutdown_connections 2024-05-09T01:26:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01b80b2880 0x7f01b80b4d90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01cc108a70 0x7f01cc1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01cc103d70 0x7f01cc19a800 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 --2- 172.21.15.19:0/2890334445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc102b30 0x7f01cc19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 >> 172.21.15.19:0/2890334445 conn(0x7f01cc0fdd20 msgr2=0x7f01cc104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 shutdown_connections 2024-05-09T01:26:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:02.903+0000 7f01d4642640 1 -- 172.21.15.19:0/2890334445 wait complete. 2024-05-09T01:26:02.912 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-05-09T01:26:01.178394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667473,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1910,"num_read_kb":1076,"num_write":398,"num_write_kb":2315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":2954185,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":828,"ondisk_log_size":828,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386964,"kb_used_data":12852,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554860,"statfs":{"total":1151940427776,"available":1151544176640,"internally_reserved":0,"allocated":13160448,"data_stored":7233183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6854,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":85,"num_write":14,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003416"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158424+0000","last_change":"2024-05-09T01:22:39.836756+0000","last_active":"2024-05-09T01:24:21.158424+0000","last_peered":"2024-05-09T01:24:21.158424+0000","last_clean":"2024-05-09T01:24:21.158424+0000","last_became_active":"2024-05-09T01:22:39.835505+0000","last_became_peered":"2024-05-09T01:22:39.835505+0000","last_unstale":"2024-05-09T01:24:21.158424+0000","last_undegraded":"2024-05-09T01:24:21.158424+0000","last_fullsized":"2024-05-09T01:24:21.158424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:29.109762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148168+0000","last_change":"2024-05-09T01:22:39.842897+0000","last_active":"2024-05-09T01:24:21.148168+0000","last_peered":"2024-05-09T01:24:21.148168+0000","last_clean":"2024-05-09T01:24:21.148168+0000","last_became_active":"2024-05-09T01:22:39.842537+0000","last_became_peered":"2024-05-09T01:22:39.842537+0000","last_unstale":"2024-05-09T01:24:21.148168+0000","last_undegraded":"2024-05-09T01:24:21.148168+0000","last_fullsized":"2024-05-09T01:24:21.148168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:20.429654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157917+0000","last_change":"2024-05-09T01:22:39.847176+0000","last_active":"2024-05-09T01:24:21.157917+0000","last_peered":"2024-05-09T01:24:21.157917+0000","last_clean":"2024-05-09T01:24:21.157917+0000","last_became_active":"2024-05-09T01:22:39.846782+0000","last_became_peered":"2024-05-09T01:22:39.846782+0000","last_unstale":"2024-05-09T01:24:21.157917+0000","last_undegraded":"2024-05-09T01:24:21.157917+0000","last_fullsized":"2024-05-09T01:24:21.157917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:06.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.137185+0000","last_change":"2024-05-09T01:22:39.842399+0000","last_active":"2024-05-09T01:24:21.137185+0000","last_peered":"2024-05-09T01:24:21.137185+0000","last_clean":"2024-05-09T01:24:21.137185+0000","last_became_active":"2024-05-09T01:22:39.842035+0000","last_became_peered":"2024-05-09T01:22:39.842035+0000","last_unstale":"2024-05-09T01:24:21.137185+0000","last_undegraded":"2024-05-09T01:24:21.137185+0000","last_fullsized":"2024-05-09T01:24:21.137185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:35.011920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.124665+0000","last_change":"2024-05-09T01:22:39.838574+0000","last_active":"2024-05-09T01:24:21.124665+0000","last_peered":"2024-05-09T01:24:21.124665+0000","last_clean":"2024-05-09T01:24:21.124665+0000","last_became_active":"2024-05-09T01:22:39.838059+0000","last_became_peered":"2024-05-09T01:22:39.838059+0000","last_unstale":"2024-05-09T01:24:21.124665+0000","last_undegraded":"2024-05-09T01:24:21.124665+0000","last_fullsized":"2024-05-09T01:24:21.124665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:20.968526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153364+0000","last_change":"2024-05-09T01:22:39.842407+0000","last_active":"2024-05-09T01:24:21.153364+0000","last_peered":"2024-05-09T01:24:21.153364+0000","last_clean":"2024-05-09T01:24:21.153364+0000","last_became_active":"2024-05-09T01:22:39.841982+0000","last_became_peered":"2024-05-09T01:22:39.841982+0000","last_unstale":"2024-05-09T01:24:21.153364+0000","last_undegraded":"2024-05-09T01:24:21.153364+0000","last_fullsized":"2024-05-09T01:24:21.153364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:17.086304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155290+0000","last_change":"2024-05-09T01:22:39.833800+0000","last_active":"2024-05-09T01:24:21.155290+0000","last_peered":"2024-05-09T01:24:21.155290+0000","last_clean":"2024-05-09T01:24:21.155290+0000","last_became_active":"2024-05-09T01:22:39.833388+0000","last_became_peered":"2024-05-09T01:22:39.833388+0000","last_unstale":"2024-05-09T01:24:21.155290+0000","last_undegraded":"2024-05-09T01:24:21.155290+0000","last_fullsized":"2024-05-09T01:24:21.155290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:35.341428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142200+0000","last_change":"2024-05-09T01:22:39.845395+0000","last_active":"2024-05-09T01:24:21.142200+0000","last_peered":"2024-05-09T01:24:21.142200+0000","last_clean":"2024-05-09T01:24:21.142200+0000","last_became_active":"2024-05-09T01:22:39.844946+0000","last_became_peered":"2024-05-09T01:22:39.844946+0000","last_unstale":"2024-05-09T01:24:21.142200+0000","last_undegraded":"2024-05-09T01:24:21.142200+0000","last_fullsized":"2024-05-09T01:24:21.142200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:46:44.750614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159768+0000","last_change":"2024-05-09T01:22:39.832894+0000","last_active":"2024-05-09T01:24:21.159768+0000","last_peered":"2024-05-09T01:24:21.159768+0000","last_clean":"2024-05-09T01:24:21.159768+0000","last_became_active":"2024-05-09T01:22:39.829743+0000","last_became_peered":"2024-05-09T01:22:39.829743+0000","last_unstale":"2024-05-09T01:24:21.159768+0000","last_undegraded":"2024-05-09T01:24:21.159768+0000","last_fullsized":"2024-05-09T01:24:21.159768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:11.463833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155902+0000","last_change":"2024-05-09T01:22:39.843131+0000","last_active":"2024-05-09T01:24:21.155902+0000","last_peered":"2024-05-09T01:24:21.155902+0000","last_clean":"2024-05-09T01:24:21.155902+0000","last_became_active":"2024-05-09T01:22:39.842690+0000","last_became_peered":"2024-05-09T01:22:39.842690+0000","last_unstale":"2024-05-09T01:24:21.155902+0000","last_undegraded":"2024-05-09T01:24:21.155902+0000","last_fullsized":"2024-05-09T01:24:21.155902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:14.172365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166267+0000","last_change":"2024-05-09T01:22:39.845557+0000","last_active":"2024-05-09T01:24:21.166267+0000","last_peered":"2024-05-09T01:24:21.166267+0000","last_clean":"2024-05-09T01:24:21.166267+0000","last_became_active":"2024-05-09T01:22:39.845059+0000","last_became_peered":"2024-05-09T01:22:39.845059+0000","last_unstale":"2024-05-09T01:24:21.166267+0000","last_undegraded":"2024-05-09T01:24:21.166267+0000","last_fullsized":"2024-05-09T01:24:21.166267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:47.238080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.129595+0000","last_change":"2024-05-09T01:22:39.842237+0000","last_active":"2024-05-09T01:24:21.129595+0000","last_peered":"2024-05-09T01:24:21.129595+0000","last_clean":"2024-05-09T01:24:21.129595+0000","last_became_active":"2024-05-09T01:22:39.841849+0000","last_became_peered":"2024-05-09T01:22:39.841849+0000","last_unstale":"2024-05-09T01:24:21.129595+0000","last_undegraded":"2024-05-09T01:24:21.129595+0000","last_fullsized":"2024-05-09T01:24:21.129595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:40.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150804+0000","last_change":"2024-05-09T01:22:39.839943+0000","last_active":"2024-05-09T01:24:21.150804+0000","last_peered":"2024-05-09T01:24:21.150804+0000","last_clean":"2024-05-09T01:24:21.150804+0000","last_became_active":"2024-05-09T01:22:39.839604+0000","last_became_peered":"2024-05-09T01:22:39.839604+0000","last_unstale":"2024-05-09T01:24:21.150804+0000","last_undegraded":"2024-05-09T01:24:21.150804+0000","last_fullsized":"2024-05-09T01:24:21.150804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:26:34.854263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145018+0000","last_change":"2024-05-09T01:22:39.846981+0000","last_active":"2024-05-09T01:24:21.145018+0000","last_peered":"2024-05-09T01:24:21.145018+0000","last_clean":"2024-05-09T01:24:21.145018+0000","last_became_active":"2024-05-09T01:22:39.846586+0000","last_became_peered":"2024-05-09T01:22:39.846586+0000","last_unstale":"2024-05-09T01:24:21.145018+0000","last_undegraded":"2024-05-09T01:24:21.145018+0000","last_fullsized":"2024-05-09T01:24:21.145018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:50.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151239+0000","last_change":"2024-05-09T01:22:39.848964+0000","last_active":"2024-05-09T01:24:21.151239+0000","last_peered":"2024-05-09T01:24:21.151239+0000","last_clean":"2024-05-09T01:24:21.151239+0000","last_became_active":"2024-05-09T01:22:39.848442+0000","last_became_peered":"2024-05-09T01:22:39.848442+0000","last_unstale":"2024-05-09T01:24:21.151239+0000","last_undegraded":"2024-05-09T01:24:21.151239+0000","last_fullsized":"2024-05-09T01:24:21.151239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:35.802739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164175+0000","last_change":"2024-05-09T01:22:39.840586+0000","last_active":"2024-05-09T01:24:21.164175+0000","last_peered":"2024-05-09T01:24:21.164175+0000","last_clean":"2024-05-09T01:24:21.164175+0000","last_became_active":"2024-05-09T01:22:39.839887+0000","last_became_peered":"2024-05-09T01:22:39.839887+0000","last_unstale":"2024-05-09T01:24:21.164175+0000","last_undegraded":"2024-05-09T01:24:21.164175+0000","last_fullsized":"2024-05-09T01:24:21.164175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:36.510039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.158620+0000","last_change":"2024-05-09T01:22:39.836152+0000","last_active":"2024-05-09T01:24:21.158620+0000","last_peered":"2024-05-09T01:24:21.158620+0000","last_clean":"2024-05-09T01:24:21.158620+0000","last_became_active":"2024-05-09T01:22:39.834691+0000","last_became_peered":"2024-05-09T01:22:39.834691+0000","last_unstale":"2024-05-09T01:24:21.158620+0000","last_undegraded":"2024-05-09T01:24:21.158620+0000","last_fullsized":"2024-05-09T01:24:21.158620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:31.331096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163485+0000","last_change":"2024-05-09T01:22:39.839760+0000","last_active":"2024-05-09T01:24:21.163485+0000","last_peered":"2024-05-09T01:24:21.163485+0000","last_clean":"2024-05-09T01:24:21.163485+0000","last_became_active":"2024-05-09T01:22:39.839403+0000","last_became_peered":"2024-05-09T01:22:39.839403+0000","last_unstale":"2024-05-09T01:24:21.163485+0000","last_undegraded":"2024-05-09T01:24:21.163485+0000","last_fullsized":"2024-05-09T01:24:21.163485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:19.248368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142981+0000","last_change":"2024-05-09T01:22:39.837538+0000","last_active":"2024-05-09T01:24:21.142981+0000","last_peered":"2024-05-09T01:24:21.142981+0000","last_clean":"2024-05-09T01:24:21.142981+0000","last_became_active":"2024-05-09T01:22:39.837167+0000","last_became_peered":"2024-05-09T01:22:39.837167+0000","last_unstale":"2024-05-09T01:24:21.142981+0000","last_undegraded":"2024-05-09T01:24:21.142981+0000","last_fullsized":"2024-05-09T01:24:21.142981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:35.695077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'15","reported_seq":244,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:58.648201+0000","last_change":"2024-05-09T01:23:22.433852+0000","last_active":"2024-05-09T01:25:58.648201+0000","last_peered":"2024-05-09T01:25:58.648201+0000","last_clean":"2024-05-09T01:25:58.648201+0000","last_became_active":"2024-05-09T01:23:22.423465+0000","last_became_peered":"2024-05-09T01:23:22.423465+0000","last_unstale":"2024-05-09T01:25:58.648201+0000","last_undegraded":"2024-05-09T01:25:58.648201+0000","last_fullsized":"2024-05-09T01:25:58.648201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:58.578798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.137595+0000","last_change":"2024-05-09T01:22:39.842667+0000","last_active":"2024-05-09T01:24:21.137595+0000","last_peered":"2024-05-09T01:24:21.137595+0000","last_clean":"2024-05-09T01:24:21.137595+0000","last_became_active":"2024-05-09T01:22:39.842256+0000","last_became_peered":"2024-05-09T01:22:39.842256+0000","last_unstale":"2024-05-09T01:24:21.137595+0000","last_undegraded":"2024-05-09T01:24:21.137595+0000","last_fullsized":"2024-05-09T01:24:21.137595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:28.525790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.314581+0000","last_change":"2024-05-09T01:22:36.841515+0000","last_active":"2024-05-09T01:23:22.314581+0000","last_peered":"2024-05-09T01:23:22.314581+0000","last_clean":"2024-05-09T01:23:22.314581+0000","last_became_active":"2024-05-09T01:22:36.841128+0000","last_became_peered":"2024-05-09T01:22:36.841128+0000","last_unstale":"2024-05-09T01:23:22.314581+0000","last_undegraded":"2024-05-09T01:23:22.314581+0000","last_fullsized":"2024-05-09T01:23:22.314581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:13.598177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.157725+0000","last_change":"2024-05-09T01:22:39.843491+0000","last_active":"2024-05-09T01:24:21.157725+0000","last_peered":"2024-05-09T01:24:21.157725+0000","last_clean":"2024-05-09T01:24:21.157725+0000","last_became_active":"2024-05-09T01:22:39.843098+0000","last_became_peered":"2024-05-09T01:22:39.843098+0000","last_unstale":"2024-05-09T01:24:21.157725+0000","last_undegraded":"2024-05-09T01:24:21.157725+0000","last_fullsized":"2024-05-09T01:24:21.157725+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:03:33.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:08.922898+0000","last_change":"2024-05-09T01:22:36.798217+0000","last_active":"2024-05-09T01:25:08.922898+0000","last_peered":"2024-05-09T01:25:08.922898+0000","last_clean":"2024-05-09T01:25:08.922898+0000","last_became_active":"2024-05-09T01:22:36.797847+0000","last_became_peered":"2024-05-09T01:22:36.797847+0000","last_unstale":"2024-05-09T01:25:08.922898+0000","last_undegraded":"2024-05-09T01:25:08.922898+0000","last_fullsized":"2024-05-09T01:25:08.922898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:21.116535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164715+0000","last_change":"2024-05-09T01:22:39.837568+0000","last_active":"2024-05-09T01:24:21.164715+0000","last_peered":"2024-05-09T01:24:21.164715+0000","last_clean":"2024-05-09T01:24:21.164715+0000","last_became_active":"2024-05-09T01:22:39.836159+0000","last_became_peered":"2024-05-09T01:22:39.836159+0000","last_unstale":"2024-05-09T01:24:21.164715+0000","last_undegraded":"2024-05-09T01:24:21.164715+0000","last_fullsized":"2024-05-09T01:24:21.164715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:28.089830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303449+0000","last_change":"2024-05-09T01:22:36.798185+0000","last_active":"2024-05-09T01:23:22.303449+0000","last_peered":"2024-05-09T01:23:22.303449+0000","last_clean":"2024-05-09T01:23:22.303449+0000","last_became_active":"2024-05-09T01:22:36.797522+0000","last_became_peered":"2024-05-09T01:22:36.797522+0000","last_unstale":"2024-05-09T01:23:22.303449+0000","last_undegraded":"2024-05-09T01:23:22.303449+0000","last_fullsized":"2024-05-09T01:23:22.303449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:40.960006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153901+0000","last_change":"2024-05-09T01:22:39.837436+0000","last_active":"2024-05-09T01:24:21.153901+0000","last_peered":"2024-05-09T01:24:21.153901+0000","last_clean":"2024-05-09T01:24:21.153901+0000","last_became_active":"2024-05-09T01:22:39.837050+0000","last_became_peered":"2024-05-09T01:22:39.837050+0000","last_unstale":"2024-05-09T01:24:21.153901+0000","last_undegraded":"2024-05-09T01:24:21.153901+0000","last_fullsized":"2024-05-09T01:24:21.153901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:07.149311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":214,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:56.900380+0000","last_change":"2024-05-09T01:22:36.800767+0000","last_active":"2024-05-09T01:25:56.900380+0000","last_peered":"2024-05-09T01:25:56.900380+0000","last_clean":"2024-05-09T01:25:56.900380+0000","last_became_active":"2024-05-09T01:22:36.800183+0000","last_became_peered":"2024-05-09T01:22:36.800183+0000","last_unstale":"2024-05-09T01:25:56.900380+0000","last_undegraded":"2024-05-09T01:25:56.900380+0000","last_fullsized":"2024-05-09T01:25:56.900380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:05.682889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.131431+0000","last_change":"2024-05-09T01:22:39.832798+0000","last_active":"2024-05-09T01:24:21.131431+0000","last_peered":"2024-05-09T01:24:21.131431+0000","last_clean":"2024-05-09T01:24:21.131431+0000","last_became_active":"2024-05-09T01:22:39.828803+0000","last_became_peered":"2024-05-09T01:22:39.828803+0000","last_unstale":"2024-05-09T01:24:21.131431+0000","last_undegraded":"2024-05-09T01:24:21.131431+0000","last_fullsized":"2024-05-09T01:24:21.131431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:02:23.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.306039+0000","last_change":"2024-05-09T01:22:36.794171+0000","last_active":"2024-05-09T01:23:22.306039+0000","last_peered":"2024-05-09T01:23:22.306039+0000","last_clean":"2024-05-09T01:23:22.306039+0000","last_became_active":"2024-05-09T01:22:36.793713+0000","last_became_peered":"2024-05-09T01:22:36.793713+0000","last_unstale":"2024-05-09T01:23:22.306039+0000","last_undegraded":"2024-05-09T01:23:22.306039+0000","last_fullsized":"2024-05-09T01:23:22.306039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:06.654667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153390+0000","last_change":"2024-05-09T01:22:39.838266+0000","last_active":"2024-05-09T01:24:21.153390+0000","last_peered":"2024-05-09T01:24:21.153390+0000","last_clean":"2024-05-09T01:24:21.153390+0000","last_became_active":"2024-05-09T01:22:39.837911+0000","last_became_peered":"2024-05-09T01:22:39.837911+0000","last_unstale":"2024-05-09T01:24:21.153390+0000","last_undegraded":"2024-05-09T01:24:21.153390+0000","last_fullsized":"2024-05-09T01:24:21.153390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:39.414176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308603+0000","last_change":"2024-05-09T01:22:36.799766+0000","last_active":"2024-05-09T01:23:22.308603+0000","last_peered":"2024-05-09T01:23:22.308603+0000","last_clean":"2024-05-09T01:23:22.308603+0000","last_became_active":"2024-05-09T01:22:36.799388+0000","last_became_peered":"2024-05-09T01:22:36.799388+0000","last_unstale":"2024-05-09T01:23:22.308603+0000","last_undegraded":"2024-05-09T01:23:22.308603+0000","last_fullsized":"2024-05-09T01:23:22.308603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:13.401659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141757+0000","last_change":"2024-05-09T01:22:39.838493+0000","last_active":"2024-05-09T01:24:21.141757+0000","last_peered":"2024-05-09T01:24:21.141757+0000","last_clean":"2024-05-09T01:24:21.141757+0000","last_became_active":"2024-05-09T01:22:39.838086+0000","last_became_peered":"2024-05-09T01:22:39.838086+0000","last_unstale":"2024-05-09T01:24:21.141757+0000","last_undegraded":"2024-05-09T01:24:21.141757+0000","last_fullsized":"2024-05-09T01:24:21.141757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:06.340693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421578+0000","last_change":"2024-05-09T01:22:36.804872+0000","last_active":"2024-05-09T01:23:22.421578+0000","last_peered":"2024-05-09T01:23:22.421578+0000","last_clean":"2024-05-09T01:23:22.421578+0000","last_became_active":"2024-05-09T01:22:36.804439+0000","last_became_peered":"2024-05-09T01:22:36.804439+0000","last_unstale":"2024-05-09T01:23:22.421578+0000","last_undegraded":"2024-05-09T01:23:22.421578+0000","last_fullsized":"2024-05-09T01:23:22.421578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:08.584749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.162012+0000","last_change":"2024-05-09T01:22:39.836579+0000","last_active":"2024-05-09T01:24:21.162012+0000","last_peered":"2024-05-09T01:24:21.162012+0000","last_clean":"2024-05-09T01:24:21.162012+0000","last_became_active":"2024-05-09T01:22:39.836244+0000","last_became_peered":"2024-05-09T01:22:39.836244+0000","last_unstale":"2024-05-09T01:24:21.162012+0000","last_undegraded":"2024-05-09T01:24:21.162012+0000","last_fullsized":"2024-05-09T01:24:21.162012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:43.329448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":395,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:59.590521+0000","last_change":"2024-05-09T01:22:36.796639+0000","last_active":"2024-05-09T01:25:59.590521+0000","last_peered":"2024-05-09T01:25:59.590521+0000","last_clean":"2024-05-09T01:25:59.590521+0000","last_became_active":"2024-05-09T01:22:36.796227+0000","last_became_peered":"2024-05-09T01:22:36.796227+0000","last_unstale":"2024-05-09T01:25:59.590521+0000","last_undegraded":"2024-05-09T01:25:59.590521+0000","last_fullsized":"2024-05-09T01:25:59.590521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:56.219281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":363,"num_read_kb":363,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.162895+0000","last_change":"2024-05-09T01:22:39.841515+0000","last_active":"2024-05-09T01:24:21.162895+0000","last_peered":"2024-05-09T01:24:21.162895+0000","last_clean":"2024-05-09T01:24:21.162895+0000","last_became_active":"2024-05-09T01:22:39.840057+0000","last_became_peered":"2024-05-09T01:22:39.840057+0000","last_unstale":"2024-05-09T01:24:21.162895+0000","last_undegraded":"2024-05-09T01:24:21.162895+0000","last_fullsized":"2024-05-09T01:24:21.162895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:21:43.258466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309168+0000","last_change":"2024-05-09T01:22:36.801937+0000","last_active":"2024-05-09T01:23:22.309168+0000","last_peered":"2024-05-09T01:23:22.309168+0000","last_clean":"2024-05-09T01:23:22.309168+0000","last_became_active":"2024-05-09T01:22:36.801563+0000","last_became_peered":"2024-05-09T01:22:36.801563+0000","last_unstale":"2024-05-09T01:23:22.309168+0000","last_undegraded":"2024-05-09T01:23:22.309168+0000","last_fullsized":"2024-05-09T01:23:22.309168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:00:42.170156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.151364+0000","last_change":"2024-05-09T01:22:39.849154+0000","last_active":"2024-05-09T01:24:21.151364+0000","last_peered":"2024-05-09T01:24:21.151364+0000","last_clean":"2024-05-09T01:24:21.151364+0000","last_became_active":"2024-05-09T01:22:39.848815+0000","last_became_peered":"2024-05-09T01:22:39.848815+0000","last_unstale":"2024-05-09T01:24:21.151364+0000","last_undegraded":"2024-05-09T01:24:21.151364+0000","last_fullsized":"2024-05-09T01:24:21.151364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:00.300208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.422043+0000","last_change":"2024-05-09T01:22:36.843201+0000","last_active":"2024-05-09T01:23:22.422043+0000","last_peered":"2024-05-09T01:23:22.422043+0000","last_clean":"2024-05-09T01:23:22.422043+0000","last_became_active":"2024-05-09T01:22:36.842500+0000","last_became_peered":"2024-05-09T01:22:36.842500+0000","last_unstale":"2024-05-09T01:23:22.422043+0000","last_undegraded":"2024-05-09T01:23:22.422043+0000","last_fullsized":"2024-05-09T01:23:22.422043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:35.092246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.159850+0000","last_change":"2024-05-09T01:22:39.835120+0000","last_active":"2024-05-09T01:24:21.159850+0000","last_peered":"2024-05-09T01:24:21.159850+0000","last_clean":"2024-05-09T01:24:21.159850+0000","last_became_active":"2024-05-09T01:22:39.834613+0000","last_became_peered":"2024-05-09T01:22:39.834613+0000","last_unstale":"2024-05-09T01:24:21.159850+0000","last_undegraded":"2024-05-09T01:24:21.159850+0000","last_fullsized":"2024-05-09T01:24:21.159850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:08.380445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":443,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:59.930492+0000","last_change":"2024-05-09T01:22:36.800816+0000","last_active":"2024-05-09T01:25:59.930492+0000","last_peered":"2024-05-09T01:25:59.930492+0000","last_clean":"2024-05-09T01:25:59.930492+0000","last_became_active":"2024-05-09T01:22:36.798797+0000","last_became_peered":"2024-05-09T01:22:36.798797+0000","last_unstale":"2024-05-09T01:25:59.930492+0000","last_undegraded":"2024-05-09T01:25:59.930492+0000","last_fullsized":"2024-05-09T01:25:59.930492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:37.670163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68071,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":377,"num_read_kb":255,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134714+0000","last_change":"2024-05-09T01:22:39.831504+0000","last_active":"2024-05-09T01:24:21.134714+0000","last_peered":"2024-05-09T01:24:21.134714+0000","last_clean":"2024-05-09T01:24:21.134714+0000","last_became_active":"2024-05-09T01:22:39.831129+0000","last_became_peered":"2024-05-09T01:22:39.831129+0000","last_unstale":"2024-05-09T01:24:21.134714+0000","last_undegraded":"2024-05-09T01:24:21.134714+0000","last_fullsized":"2024-05-09T01:24:21.134714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:19.212473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.304494+0000","last_change":"2024-05-09T01:22:36.803447+0000","last_active":"2024-05-09T01:23:22.304494+0000","last_peered":"2024-05-09T01:23:22.304494+0000","last_clean":"2024-05-09T01:23:22.304494+0000","last_became_active":"2024-05-09T01:22:36.802986+0000","last_became_peered":"2024-05-09T01:22:36.802986+0000","last_unstale":"2024-05-09T01:23:22.304494+0000","last_undegraded":"2024-05-09T01:23:22.304494+0000","last_fullsized":"2024-05-09T01:23:22.304494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:28.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163307+0000","last_change":"2024-05-09T01:22:39.836546+0000","last_active":"2024-05-09T01:24:21.163307+0000","last_peered":"2024-05-09T01:24:21.163307+0000","last_clean":"2024-05-09T01:24:21.163307+0000","last_became_active":"2024-05-09T01:22:39.836168+0000","last_became_peered":"2024-05-09T01:22:39.836168+0000","last_unstale":"2024-05-09T01:24:21.163307+0000","last_undegraded":"2024-05-09T01:24:21.163307+0000","last_fullsized":"2024-05-09T01:24:21.163307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:45.522279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421659+0000","last_change":"2024-05-09T01:22:36.798703+0000","last_active":"2024-05-09T01:23:22.421659+0000","last_peered":"2024-05-09T01:23:22.421659+0000","last_clean":"2024-05-09T01:23:22.421659+0000","last_became_active":"2024-05-09T01:22:36.798280+0000","last_became_peered":"2024-05-09T01:22:36.798280+0000","last_unstale":"2024-05-09T01:23:22.421659+0000","last_undegraded":"2024-05-09T01:23:22.421659+0000","last_fullsized":"2024-05-09T01:23:22.421659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:33.794488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166230+0000","last_change":"2024-05-09T01:22:39.831223+0000","last_active":"2024-05-09T01:24:21.166230+0000","last_peered":"2024-05-09T01:24:21.166230+0000","last_clean":"2024-05-09T01:24:21.166230+0000","last_became_active":"2024-05-09T01:22:39.830845+0000","last_became_peered":"2024-05-09T01:22:39.830845+0000","last_unstale":"2024-05-09T01:24:21.166230+0000","last_undegraded":"2024-05-09T01:24:21.166230+0000","last_fullsized":"2024-05-09T01:24:21.166230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:23.365950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":220,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.952967+0000","last_change":"2024-05-09T01:22:36.802327+0000","last_active":"2024-05-09T01:25:57.952967+0000","last_peered":"2024-05-09T01:25:57.952967+0000","last_clean":"2024-05-09T01:25:57.952967+0000","last_became_active":"2024-05-09T01:22:36.801951+0000","last_became_peered":"2024-05-09T01:22:36.801951+0000","last_unstale":"2024-05-09T01:25:57.952967+0000","last_undegraded":"2024-05-09T01:25:57.952967+0000","last_fullsized":"2024-05-09T01:25:57.952967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:12.282424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154280+0000","last_change":"2024-05-09T01:22:39.833626+0000","last_active":"2024-05-09T01:24:21.154280+0000","last_peered":"2024-05-09T01:24:21.154280+0000","last_clean":"2024-05-09T01:24:21.154280+0000","last_became_active":"2024-05-09T01:22:39.833176+0000","last_became_peered":"2024-05-09T01:22:39.833176+0000","last_unstale":"2024-05-09T01:24:21.154280+0000","last_undegraded":"2024-05-09T01:24:21.154280+0000","last_fullsized":"2024-05-09T01:24:21.154280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:25.082300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:26.194840+0000","last_change":"2024-05-09T01:23:22.328011+0000","last_active":"2024-05-09T01:23:26.194840+0000","last_peered":"2024-05-09T01:23:26.194840+0000","last_clean":"2024-05-09T01:23:26.194840+0000","last_became_active":"2024-05-09T01:23:22.313049+0000","last_became_peered":"2024-05-09T01:23:22.313049+0000","last_unstale":"2024-05-09T01:23:26.194840+0000","last_undegraded":"2024-05-09T01:23:26.194840+0000","last_fullsized":"2024-05-09T01:23:26.194840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:17.854491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153329+0000","last_change":"2024-05-09T01:22:39.837529+0000","last_active":"2024-05-09T01:24:21.153329+0000","last_peered":"2024-05-09T01:24:21.153329+0000","last_clean":"2024-05-09T01:24:21.153329+0000","last_became_active":"2024-05-09T01:22:39.837155+0000","last_became_peered":"2024-05-09T01:22:39.837155+0000","last_unstale":"2024-05-09T01:24:21.153329+0000","last_undegraded":"2024-05-09T01:24:21.153329+0000","last_fullsized":"2024-05-09T01:24:21.153329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:44.485629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303600+0000","last_change":"2024-05-09T01:22:36.800073+0000","last_active":"2024-05-09T01:23:22.303600+0000","last_peered":"2024-05-09T01:23:22.303600+0000","last_clean":"2024-05-09T01:23:22.303600+0000","last_became_active":"2024-05-09T01:22:36.799463+0000","last_became_peered":"2024-05-09T01:22:36.799463+0000","last_unstale":"2024-05-09T01:23:22.303600+0000","last_undegraded":"2024-05-09T01:23:22.303600+0000","last_fullsized":"2024-05-09T01:23:22.303600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:42.942945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150713+0000","last_change":"2024-05-09T01:22:39.833615+0000","last_active":"2024-05-09T01:24:21.150713+0000","last_peered":"2024-05-09T01:24:21.150713+0000","last_clean":"2024-05-09T01:24:21.150713+0000","last_became_active":"2024-05-09T01:22:39.833212+0000","last_became_peered":"2024-05-09T01:22:39.833212+0000","last_unstale":"2024-05-09T01:24:21.150713+0000","last_undegraded":"2024-05-09T01:24:21.150713+0000","last_fullsized":"2024-05-09T01:24:21.150713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:35.387442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.305695+0000","last_change":"2024-05-09T01:22:36.843143+0000","last_active":"2024-05-09T01:23:22.305695+0000","last_peered":"2024-05-09T01:23:22.305695+0000","last_clean":"2024-05-09T01:23:22.305695+0000","last_became_active":"2024-05-09T01:22:36.842759+0000","last_became_peered":"2024-05-09T01:22:36.842759+0000","last_unstale":"2024-05-09T01:23:22.305695+0000","last_undegraded":"2024-05-09T01:23:22.305695+0000","last_fullsized":"2024-05-09T01:23:22.305695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:42.666186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.143568+0000","last_change":"2024-05-09T01:22:39.838267+0000","last_active":"2024-05-09T01:24:21.143568+0000","last_peered":"2024-05-09T01:24:21.143568+0000","last_clean":"2024-05-09T01:24:21.143568+0000","last_became_active":"2024-05-09T01:22:39.837910+0000","last_became_peered":"2024-05-09T01:22:39.837910+0000","last_unstale":"2024-05-09T01:24:21.143568+0000","last_undegraded":"2024-05-09T01:24:21.143568+0000","last_fullsized":"2024-05-09T01:24:21.143568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:34.620332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'8","reported_seq":219,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.647507+0000","last_change":"2024-05-09T01:22:36.800515+0000","last_active":"2024-05-09T01:25:57.647507+0000","last_peered":"2024-05-09T01:25:57.647507+0000","last_clean":"2024-05-09T01:25:57.647507+0000","last_became_active":"2024-05-09T01:22:36.800065+0000","last_became_peered":"2024-05-09T01:22:36.800065+0000","last_unstale":"2024-05-09T01:25:57.647507+0000","last_undegraded":"2024-05-09T01:25:57.647507+0000","last_fullsized":"2024-05-09T01:25:57.647507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:50.160463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.145505+0000","last_change":"2024-05-09T01:23:22.313627+0000","last_active":"2024-05-09T01:24:21.145505+0000","last_peered":"2024-05-09T01:24:21.145505+0000","last_clean":"2024-05-09T01:24:21.145505+0000","last_became_active":"2024-05-09T01:23:22.313138+0000","last_became_peered":"2024-05-09T01:23:22.313138+0000","last_unstale":"2024-05-09T01:24:21.145505+0000","last_undegraded":"2024-05-09T01:24:21.145505+0000","last_fullsized":"2024-05-09T01:24:21.145505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:07.216086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.153456+0000","last_change":"2024-05-09T01:22:39.832188+0000","last_active":"2024-05-09T01:24:21.153456+0000","last_peered":"2024-05-09T01:24:21.153456+0000","last_clean":"2024-05-09T01:24:21.153456+0000","last_became_active":"2024-05-09T01:22:39.831302+0000","last_became_peered":"2024-05-09T01:22:39.831302+0000","last_unstale":"2024-05-09T01:24:21.153456+0000","last_undegraded":"2024-05-09T01:24:21.153456+0000","last_fullsized":"2024-05-09T01:24:21.153456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:06.703950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'7","reported_seq":216,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.952377+0000","last_change":"2024-05-09T01:22:36.841446+0000","last_active":"2024-05-09T01:25:57.952377+0000","last_peered":"2024-05-09T01:25:57.952377+0000","last_clean":"2024-05-09T01:25:57.952377+0000","last_became_active":"2024-05-09T01:22:36.841071+0000","last_became_peered":"2024-05-09T01:22:36.841071+0000","last_unstale":"2024-05-09T01:25:57.952377+0000","last_undegraded":"2024-05-09T01:25:57.952377+0000","last_fullsized":"2024-05-09T01:25:57.952377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:20.777482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163865+0000","last_change":"2024-05-09T01:22:39.846711+0000","last_active":"2024-05-09T01:24:21.163865+0000","last_peered":"2024-05-09T01:24:21.163865+0000","last_clean":"2024-05-09T01:24:21.163865+0000","last_became_active":"2024-05-09T01:22:39.846293+0000","last_became_peered":"2024-05-09T01:22:39.846293+0000","last_unstale":"2024-05-09T01:24:21.163865+0000","last_undegraded":"2024-05-09T01:24:21.163865+0000","last_fullsized":"2024-05-09T01:24:21.163865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:12:27.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303855+0000","last_change":"2024-05-09T01:22:36.798302+0000","last_active":"2024-05-09T01:23:22.303855+0000","last_peered":"2024-05-09T01:23:22.303855+0000","last_clean":"2024-05-09T01:23:22.303855+0000","last_became_active":"2024-05-09T01:22:36.797906+0000","last_became_peered":"2024-05-09T01:22:36.797906+0000","last_unstale":"2024-05-09T01:23:22.303855+0000","last_undegraded":"2024-05-09T01:23:22.303855+0000","last_fullsized":"2024-05-09T01:23:22.303855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:13.599240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:59.123370+0000","last_change":"2024-05-09T01:21:12.781366+0000","last_active":"2024-05-09T01:23:59.123370+0000","last_peered":"2024-05-09T01:23:59.123370+0000","last_clean":"2024-05-09T01:23:59.123370+0000","last_became_active":"2024-05-09T01:21:12.774967+0000","last_became_peered":"2024-05-09T01:21:12.774967+0000","last_unstale":"2024-05-09T01:23:59.123370+0000","last_undegraded":"2024-05-09T01:23:59.123370+0000","last_fullsized":"2024-05-09T01:23:59.123370+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:17:49.682508+0000","last_clean_scrub_stamp":"2024-05-09T01:17:49.682508+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:08.075365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.155221+0000","last_change":"2024-05-09T01:22:39.840952+0000","last_active":"2024-05-09T01:24:21.155221+0000","last_peered":"2024-05-09T01:24:21.155221+0000","last_clean":"2024-05-09T01:24:21.155221+0000","last_became_active":"2024-05-09T01:22:39.840323+0000","last_became_peered":"2024-05-09T01:22:39.840323+0000","last_unstale":"2024-05-09T01:24:21.155221+0000","last_undegraded":"2024-05-09T01:24:21.155221+0000","last_fullsized":"2024-05-09T01:24:21.155221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:23.837123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.308860+0000","last_change":"2024-05-09T01:22:36.798631+0000","last_active":"2024-05-09T01:23:22.308860+0000","last_peered":"2024-05-09T01:23:22.308860+0000","last_clean":"2024-05-09T01:23:22.308860+0000","last_became_active":"2024-05-09T01:22:36.798242+0000","last_became_peered":"2024-05-09T01:22:36.798242+0000","last_unstale":"2024-05-09T01:23:22.308860+0000","last_undegraded":"2024-05-09T01:23:22.308860+0000","last_fullsized":"2024-05-09T01:23:22.308860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:18.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303800+0000","last_change":"2024-05-09T01:22:36.843779+0000","last_active":"2024-05-09T01:23:22.303800+0000","last_peered":"2024-05-09T01:23:22.303800+0000","last_clean":"2024-05-09T01:23:22.303800+0000","last_became_active":"2024-05-09T01:22:36.842084+0000","last_became_peered":"2024-05-09T01:22:36.842084+0000","last_unstale":"2024-05-09T01:23:22.303800+0000","last_undegraded":"2024-05-09T01:23:22.303800+0000","last_fullsized":"2024-05-09T01:23:22.303800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:34.743372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146032+0000","last_change":"2024-05-09T01:22:39.842266+0000","last_active":"2024-05-09T01:24:21.146032+0000","last_peered":"2024-05-09T01:24:21.146032+0000","last_clean":"2024-05-09T01:24:21.146032+0000","last_became_active":"2024-05-09T01:22:39.841927+0000","last_became_peered":"2024-05-09T01:22:39.841927+0000","last_unstale":"2024-05-09T01:24:21.146032+0000","last_undegraded":"2024-05-09T01:24:21.146032+0000","last_fullsized":"2024-05-09T01:24:21.146032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:26.774250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'19","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:56.860389+0000","last_change":"2024-05-09T01:22:36.796929+0000","last_active":"2024-05-09T01:25:56.860389+0000","last_peered":"2024-05-09T01:25:56.860389+0000","last_clean":"2024-05-09T01:25:56.860389+0000","last_became_active":"2024-05-09T01:22:36.796480+0000","last_became_peered":"2024-05-09T01:22:36.796480+0000","last_unstale":"2024-05-09T01:25:56.860389+0000","last_undegraded":"2024-05-09T01:25:56.860389+0000","last_fullsized":"2024-05-09T01:25:56.860389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:45.168305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140120+0000","last_change":"2024-05-09T01:22:39.843093+0000","last_active":"2024-05-09T01:24:21.140120+0000","last_peered":"2024-05-09T01:24:21.140120+0000","last_clean":"2024-05-09T01:24:21.140120+0000","last_became_active":"2024-05-09T01:22:39.842729+0000","last_became_peered":"2024-05-09T01:22:39.842729+0000","last_unstale":"2024-05-09T01:24:21.140120+0000","last_undegraded":"2024-05-09T01:24:21.140120+0000","last_fullsized":"2024-05-09T01:24:21.140120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:02.074843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303716+0000","last_change":"2024-05-09T01:22:36.798166+0000","last_active":"2024-05-09T01:23:22.303716+0000","last_peered":"2024-05-09T01:23:22.303716+0000","last_clean":"2024-05-09T01:23:22.303716+0000","last_became_active":"2024-05-09T01:22:36.797752+0000","last_became_peered":"2024-05-09T01:22:36.797752+0000","last_unstale":"2024-05-09T01:23:22.303716+0000","last_undegraded":"2024-05-09T01:23:22.303716+0000","last_fullsized":"2024-05-09T01:23:22.303716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:05.211720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142128+0000","last_change":"2024-05-09T01:23:22.316228+0000","last_active":"2024-05-09T01:24:21.142128+0000","last_peered":"2024-05-09T01:24:21.142128+0000","last_clean":"2024-05-09T01:24:21.142128+0000","last_became_active":"2024-05-09T01:23:22.315683+0000","last_became_peered":"2024-05-09T01:23:22.315683+0000","last_unstale":"2024-05-09T01:24:21.142128+0000","last_undegraded":"2024-05-09T01:24:21.142128+0000","last_fullsized":"2024-05-09T01:24:21.142128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:04.623701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421356+0000","last_change":"2024-05-09T01:22:36.841999+0000","last_active":"2024-05-09T01:23:22.421356+0000","last_peered":"2024-05-09T01:23:22.421356+0000","last_clean":"2024-05-09T01:23:22.421356+0000","last_became_active":"2024-05-09T01:22:36.841664+0000","last_became_peered":"2024-05-09T01:22:36.841664+0000","last_unstale":"2024-05-09T01:23:22.421356+0000","last_undegraded":"2024-05-09T01:23:22.421356+0000","last_fullsized":"2024-05-09T01:23:22.421356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:27.110258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.134153+0000","last_change":"2024-05-09T01:22:39.839553+0000","last_active":"2024-05-09T01:24:21.134153+0000","last_peered":"2024-05-09T01:24:21.134153+0000","last_clean":"2024-05-09T01:24:21.134153+0000","last_became_active":"2024-05-09T01:22:39.839189+0000","last_became_peered":"2024-05-09T01:22:39.839189+0000","last_unstale":"2024-05-09T01:24:21.134153+0000","last_undegraded":"2024-05-09T01:24:21.134153+0000","last_fullsized":"2024-05-09T01:24:21.134153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:02.794073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.309908+0000","last_change":"2024-05-09T01:22:36.792516+0000","last_active":"2024-05-09T01:23:22.309908+0000","last_peered":"2024-05-09T01:23:22.309908+0000","last_clean":"2024-05-09T01:23:22.309908+0000","last_became_active":"2024-05-09T01:22:36.792129+0000","last_became_peered":"2024-05-09T01:22:36.792129+0000","last_unstale":"2024-05-09T01:23:22.309908+0000","last_undegraded":"2024-05-09T01:23:22.309908+0000","last_fullsized":"2024-05-09T01:23:22.309908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:42:46.596723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.148884+0000","last_change":"2024-05-09T01:22:39.841523+0000","last_active":"2024-05-09T01:24:21.148884+0000","last_peered":"2024-05-09T01:24:21.148884+0000","last_clean":"2024-05-09T01:24:21.148884+0000","last_became_active":"2024-05-09T01:22:39.840667+0000","last_became_peered":"2024-05-09T01:22:39.840667+0000","last_unstale":"2024-05-09T01:24:21.148884+0000","last_undegraded":"2024-05-09T01:24:21.148884+0000","last_fullsized":"2024-05-09T01:24:21.148884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:39.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:26.194870+0000","last_change":"2024-05-09T01:23:22.331855+0000","last_active":"2024-05-09T01:23:26.194870+0000","last_peered":"2024-05-09T01:23:26.194870+0000","last_clean":"2024-05-09T01:23:26.194870+0000","last_became_active":"2024-05-09T01:23:22.313059+0000","last_became_peered":"2024-05-09T01:23:22.313059+0000","last_unstale":"2024-05-09T01:23:26.194870+0000","last_undegraded":"2024-05-09T01:23:26.194870+0000","last_fullsized":"2024-05-09T01:23:26.194870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:16.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.142481+0000","last_change":"2024-05-09T01:22:39.845031+0000","last_active":"2024-05-09T01:24:21.142481+0000","last_peered":"2024-05-09T01:24:21.142481+0000","last_clean":"2024-05-09T01:24:21.142481+0000","last_became_active":"2024-05-09T01:22:39.844653+0000","last_became_peered":"2024-05-09T01:22:39.844653+0000","last_unstale":"2024-05-09T01:24:21.142481+0000","last_undegraded":"2024-05-09T01:24:21.142481+0000","last_fullsized":"2024-05-09T01:24:21.142481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:14.382445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.303240+0000","last_change":"2024-05-09T01:22:36.801395+0000","last_active":"2024-05-09T01:23:22.303240+0000","last_peered":"2024-05-09T01:23:22.303240+0000","last_clean":"2024-05-09T01:23:22.303240+0000","last_became_active":"2024-05-09T01:22:36.800992+0000","last_became_peered":"2024-05-09T01:22:36.800992+0000","last_unstale":"2024-05-09T01:23:22.303240+0000","last_undegraded":"2024-05-09T01:23:22.303240+0000","last_fullsized":"2024-05-09T01:23:22.303240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:16.089068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166011+0000","last_change":"2024-05-09T01:22:39.835842+0000","last_active":"2024-05-09T01:24:21.166011+0000","last_peered":"2024-05-09T01:24:21.166011+0000","last_clean":"2024-05-09T01:24:21.166011+0000","last_became_active":"2024-05-09T01:22:39.835372+0000","last_became_peered":"2024-05-09T01:22:39.835372+0000","last_unstale":"2024-05-09T01:24:21.166011+0000","last_undegraded":"2024-05-09T01:24:21.166011+0000","last_fullsized":"2024-05-09T01:24:21.166011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:09:13.197946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":179,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:25:57.588036+0000","last_change":"2024-05-09T01:23:22.332514+0000","last_active":"2024-05-09T01:25:57.588036+0000","last_peered":"2024-05-09T01:25:57.588036+0000","last_clean":"2024-05-09T01:25:57.588036+0000","last_became_active":"2024-05-09T01:23:22.312966+0000","last_became_peered":"2024-05-09T01:23:22.312966+0000","last_unstale":"2024-05-09T01:25:57.588036+0000","last_undegraded":"2024-05-09T01:25:57.588036+0000","last_fullsized":"2024-05-09T01:25:57.588036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:52.637108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":163,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.166259+0000","last_change":"2024-05-09T01:22:39.845390+0000","last_active":"2024-05-09T01:24:21.166259+0000","last_peered":"2024-05-09T01:24:21.166259+0000","last_clean":"2024-05-09T01:24:21.166259+0000","last_became_active":"2024-05-09T01:22:39.844918+0000","last_became_peered":"2024-05-09T01:22:39.844918+0000","last_unstale":"2024-05-09T01:24:21.166259+0000","last_undegraded":"2024-05-09T01:24:21.166259+0000","last_fullsized":"2024-05-09T01:24:21.166259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:04.455794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.421891+0000","last_change":"2024-05-09T01:22:36.803329+0000","last_active":"2024-05-09T01:23:22.421891+0000","last_peered":"2024-05-09T01:23:22.421891+0000","last_clean":"2024-05-09T01:23:22.421891+0000","last_became_active":"2024-05-09T01:22:36.802831+0000","last_became_peered":"2024-05-09T01:22:36.802831+0000","last_unstale":"2024-05-09T01:23:22.421891+0000","last_undegraded":"2024-05-09T01:23:22.421891+0000","last_fullsized":"2024-05-09T01:23:22.421891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:57.919840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.154788+0000","last_change":"2024-05-09T01:22:39.837595+0000","last_active":"2024-05-09T01:24:21.154788+0000","last_peered":"2024-05-09T01:24:21.154788+0000","last_clean":"2024-05-09T01:24:21.154788+0000","last_became_active":"2024-05-09T01:22:39.837197+0000","last_became_peered":"2024-05-09T01:22:39.837197+0000","last_unstale":"2024-05-09T01:24:21.154788+0000","last_undegraded":"2024-05-09T01:24:21.154788+0000","last_fullsized":"2024-05-09T01:24:21.154788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:08.700741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:23:22.321791+0000","last_change":"2024-05-09T01:23:22.321286+0000","last_active":"2024-05-09T01:23:22.321791+0000","last_peered":"2024-05-09T01:23:22.321791+0000","last_clean":"2024-05-09T01:23:22.321791+0000","last_became_active":"2024-05-09T01:23:22.315810+0000","last_became_peered":"2024-05-09T01:23:22.315810+0000","last_unstale":"2024-05-09T01:23:22.321791+0000","last_undegraded":"2024-05-09T01:23:22.321791+0000","last_fullsized":"2024-05-09T01:23:22.321791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:35.778642+0000","last_clean_scrub_stamp":"2024-05-09T01:22:35.778642+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:43.100893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.165342+0000","last_change":"2024-05-09T01:22:39.827798+0000","last_active":"2024-05-09T01:24:21.165342+0000","last_peered":"2024-05-09T01:24:21.165342+0000","last_clean":"2024-05-09T01:24:21.165342+0000","last_became_active":"2024-05-09T01:22:39.827322+0000","last_became_peered":"2024-05-09T01:22:39.827322+0000","last_unstale":"2024-05-09T01:24:21.165342+0000","last_undegraded":"2024-05-09T01:24:21.165342+0000","last_fullsized":"2024-05-09T01:24:21.165342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:52.454033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.149749+0000","last_change":"2024-05-09T01:22:39.841885+0000","last_active":"2024-05-09T01:24:21.149749+0000","last_peered":"2024-05-09T01:24:21.149749+0000","last_clean":"2024-05-09T01:24:21.149749+0000","last_became_active":"2024-05-09T01:22:39.841160+0000","last_became_peered":"2024-05-09T01:22:39.841160+0000","last_unstale":"2024-05-09T01:24:21.149749+0000","last_undegraded":"2024-05-09T01:24:21.149749+0000","last_fullsized":"2024-05-09T01:24:21.149749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:57.290465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146092+0000","last_change":"2024-05-09T01:22:39.846495+0000","last_active":"2024-05-09T01:24:21.146092+0000","last_peered":"2024-05-09T01:24:21.146092+0000","last_clean":"2024-05-09T01:24:21.146092+0000","last_became_active":"2024-05-09T01:22:39.846108+0000","last_became_peered":"2024-05-09T01:22:39.846108+0000","last_unstale":"2024-05-09T01:24:21.146092+0000","last_undegraded":"2024-05-09T01:24:21.146092+0000","last_fullsized":"2024-05-09T01:24:21.146092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:21:39.958174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.132056+0000","last_change":"2024-05-09T01:22:39.841288+0000","last_active":"2024-05-09T01:24:21.132056+0000","last_peered":"2024-05-09T01:24:21.132056+0000","last_clean":"2024-05-09T01:24:21.132056+0000","last_became_active":"2024-05-09T01:22:39.840900+0000","last_became_peered":"2024-05-09T01:22:39.840900+0000","last_unstale":"2024-05-09T01:24:21.132056+0000","last_undegraded":"2024-05-09T01:24:21.132056+0000","last_fullsized":"2024-05-09T01:24:21.132056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:30.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.141526+0000","last_change":"2024-05-09T01:23:22.313135+0000","last_active":"2024-05-09T01:24:21.141526+0000","last_peered":"2024-05-09T01:24:21.141526+0000","last_clean":"2024-05-09T01:24:21.141526+0000","last_became_active":"2024-05-09T01:23:22.312710+0000","last_became_peered":"2024-05-09T01:23:22.312710+0000","last_unstale":"2024-05-09T01:24:21.141526+0000","last_undegraded":"2024-05-09T01:24:21.141526+0000","last_fullsized":"2024-05-09T01:24:21.141526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:47.725790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.156722+0000","last_change":"2024-05-09T01:22:39.838037+0000","last_active":"2024-05-09T01:24:21.156722+0000","last_peered":"2024-05-09T01:24:21.156722+0000","last_clean":"2024-05-09T01:24:21.156722+0000","last_became_active":"2024-05-09T01:22:39.837362+0000","last_became_peered":"2024-05-09T01:22:39.837362+0000","last_unstale":"2024-05-09T01:24:21.156722+0000","last_undegraded":"2024-05-09T01:24:21.156722+0000","last_fullsized":"2024-05-09T01:24:21.156722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.781318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.146468+0000","last_change":"2024-05-09T01:23:22.310046+0000","last_active":"2024-05-09T01:24:21.146468+0000","last_peered":"2024-05-09T01:24:21.146468+0000","last_clean":"2024-05-09T01:24:21.146468+0000","last_became_active":"2024-05-09T01:23:22.309541+0000","last_became_peered":"2024-05-09T01:23:22.309541+0000","last_unstale":"2024-05-09T01:24:21.146468+0000","last_undegraded":"2024-05-09T01:24:21.146468+0000","last_fullsized":"2024-05-09T01:24:21.146468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:30.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.161756+0000","last_change":"2024-05-09T01:22:39.840453+0000","last_active":"2024-05-09T01:24:21.161756+0000","last_peered":"2024-05-09T01:24:21.161756+0000","last_clean":"2024-05-09T01:24:21.161756+0000","last_became_active":"2024-05-09T01:22:39.840077+0000","last_became_peered":"2024-05-09T01:22:39.840077+0000","last_unstale":"2024-05-09T01:24:21.161756+0000","last_undegraded":"2024-05-09T01:24:21.161756+0000","last_fullsized":"2024-05-09T01:24:21.161756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:20.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.160942+0000","last_change":"2024-05-09T01:22:39.844554+0000","last_active":"2024-05-09T01:24:21.160942+0000","last_peered":"2024-05-09T01:24:21.160942+0000","last_clean":"2024-05-09T01:24:21.160942+0000","last_became_active":"2024-05-09T01:22:39.843640+0000","last_became_peered":"2024-05-09T01:22:39.843640+0000","last_unstale":"2024-05-09T01:24:21.160942+0000","last_undegraded":"2024-05-09T01:24:21.160942+0000","last_fullsized":"2024-05-09T01:24:21.160942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:27.801565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.140795+0000","last_change":"2024-05-09T01:22:39.839043+0000","last_active":"2024-05-09T01:24:21.140795+0000","last_peered":"2024-05-09T01:24:21.140795+0000","last_clean":"2024-05-09T01:24:21.140795+0000","last_became_active":"2024-05-09T01:22:39.838662+0000","last_became_peered":"2024-05-09T01:22:39.838662+0000","last_unstale":"2024-05-09T01:24:21.140795+0000","last_undegraded":"2024-05-09T01:24:21.140795+0000","last_fullsized":"2024-05-09T01:24:21.140795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:21:16.117689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.139048+0000","last_change":"2024-05-09T01:23:22.309945+0000","last_active":"2024-05-09T01:24:21.139048+0000","last_peered":"2024-05-09T01:24:21.139048+0000","last_clean":"2024-05-09T01:24:21.139048+0000","last_became_active":"2024-05-09T01:23:22.309543+0000","last_became_peered":"2024-05-09T01:23:22.309543+0000","last_unstale":"2024-05-09T01:24:21.139048+0000","last_undegraded":"2024-05-09T01:24:21.139048+0000","last_fullsized":"2024-05-09T01:24:21.139048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:47.464524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.164552+0000","last_change":"2024-05-09T01:22:39.838911+0000","last_active":"2024-05-09T01:24:21.164552+0000","last_peered":"2024-05-09T01:24:21.164552+0000","last_clean":"2024-05-09T01:24:21.164552+0000","last_became_active":"2024-05-09T01:22:39.838293+0000","last_became_peered":"2024-05-09T01:22:39.838293+0000","last_unstale":"2024-05-09T01:24:21.164552+0000","last_undegraded":"2024-05-09T01:24:21.164552+0000","last_fullsized":"2024-05-09T01:24:21.164552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:04:32.596306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.163721+0000","last_change":"2024-05-09T01:22:39.838185+0000","last_active":"2024-05-09T01:24:21.163721+0000","last_peered":"2024-05-09T01:24:21.163721+0000","last_clean":"2024-05-09T01:24:21.163721+0000","last_became_active":"2024-05-09T01:22:39.837530+0000","last_became_peered":"2024-05-09T01:22:39.837530+0000","last_unstale":"2024-05-09T01:24:21.163721+0000","last_undegraded":"2024-05-09T01:24:21.163721+0000","last_fullsized":"2024-05-09T01:24:21.163721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:22.556710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T01:24:21.150895+0000","last_change":"2024-05-09T01:22:39.834124+0000","last_active":"2024-05-09T01:24:21.150895+0000","last_peered":"2024-05-09T01:24:21.150895+0000","last_clean":"2024-05-09T01:24:21.150895+0000","last_became_active":"2024-05-09T01:22:39.833716+0000","last_became_peered":"2024-05-09T01:22:39.833716+0000","last_unstale":"2024-05-09T01:24:21.150895+0000","last_undegraded":"2024-05-09T01:24:21.150895+0000","last_fullsized":"2024-05-09T01:24:21.150895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T01:22:38.793040+0000","last_clean_scrub_stamp":"2024-05-09T01:22:38.793040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:54:42.337900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76871,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1834,"num_read_kb":1012,"num_write":273,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2345,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":230613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612664,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":437733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808893,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1097112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972418,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":437019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168648,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":913408,"data_stored":436613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1031370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528403,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":437659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":507315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":438607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":504770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117089,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":436561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":439222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724631,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1540096,"data_stored":1029202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T01:26:02.917 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:26:03.201 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:03.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:03 smithi019 ceph-mon[30158]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:26:03.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:03 smithi019 ceph-mon[30158]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:26:03.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:03 smithi059 ceph-mon[34351]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:26:03.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:03 smithi059 ceph-mon[34351]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:26:03.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:03 smithi196 ceph-mon[35041]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:26:03.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:03 smithi196 ceph-mon[35041]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:26:04.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:04 smithi019 ceph-mon[30158]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:26:04.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:04 smithi019 ceph-mon[30158]: pgmap v394: 97 pgs: 97 active+clean; 652 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-09T01:26:04.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:04 smithi059 ceph-mon[34351]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:26:04.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:04 smithi059 ceph-mon[34351]: pgmap v394: 97 pgs: 97 active+clean; 652 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-09T01:26:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:04 smithi196 ceph-mon[35041]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T01:26:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:04 smithi196 ceph-mon[35041]: pgmap v394: 97 pgs: 97 active+clean; 652 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-09T01:26:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 -- 172.21.15.19:0/2267629385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab0105e60 msgr2=0x7fbab010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 --2- 172.21.15.19:0/2267629385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab0105e60 0x7fbab010e320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fba98009990 tx=0x7fba98009ca0 comp rx=0 tx=0).stop 2024-05-09T01:26:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 -- 172.21.15.19:0/2267629385 shutdown_connections 2024-05-09T01:26:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 --2- 172.21.15.19:0/2267629385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbab010e940 0x7fbab0110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 --2- 172.21.15.19:0/2267629385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab0105480 0x7fbab0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 --2- 172.21.15.19:0/2267629385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab0105e60 0x7fbab010e320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 -- 172.21.15.19:0/2267629385 >> 172.21.15.19:0/2267629385 conn(0x7fbab00fef50 msgr2=0x7fbab0101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 -- 172.21.15.19:0/2267629385 shutdown_connections 2024-05-09T01:26:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 -- 172.21.15.19:0/2267629385 wait complete. 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 Processor -- start 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.857+0000 7fbab6f48700 1 -- start start 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab6f48700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbab0105480 0x7fbab019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab6f48700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab0105e60 0x7fbab019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab6f48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab010e940 0x7fbab019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab6f48700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbab01144d0 con 0x7fbab010e940 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab6f48700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0114350 con 0x7fbab0105480 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab6f48700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0114650 con 0x7fbab0105e60 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab54e5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab010e940 0x7fbab019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab54e5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab010e940 0x7fbab019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58438/0 (socket says 172.21.15.19:58438) 2024-05-09T01:26:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab54e5700 1 -- 172.21.15.19:0/3303298257 learned_addr learned my addr 172.21.15.19:0/3303298257 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab4ce4700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbab0105480 0x7fbab019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbaaffff700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab0105e60 0x7fbab019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab54e5700 1 -- 172.21.15.19:0/3303298257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab0105e60 msgr2=0x7fbab019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab54e5700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab0105e60 0x7fbab019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab54e5700 1 -- 172.21.15.19:0/3303298257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbab0105480 msgr2=0x7fbab019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab54e5700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbab0105480 0x7fbab019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab54e5700 1 -- 172.21.15.19:0/3303298257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab019fcc0 con 0x7fbab010e940 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.858+0000 7fbab54e5700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab010e940 0x7fbab019f6d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8007d70 tx=0x7fbaa800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.859+0000 7fbaadffb700 1 -- 172.21.15.19:0/3303298257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8016020 con 0x7fbab010e940 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.859+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab019ffb0 con 0x7fbab010e940 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.859+0000 7fbaadffb700 1 -- 172.21.15.19:0/3303298257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa800ad40 con 0x7fbab010e940 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.859+0000 7fbaadffb700 1 -- 172.21.15.19:0/3303298257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa80115e0 con 0x7fbab010e940 2024-05-09T01:26:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.859+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab01a04f0 con 0x7fbab010e940 2024-05-09T01:26:04.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.860+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab00512c0 con 0x7fbab010e940 2024-05-09T01:26:04.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.862+0000 7fbaadffb700 1 -- 172.21.15.19:0/3303298257 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fbaa8019020 con 0x7fbab010e940 2024-05-09T01:26:04.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.863+0000 7fbaadffb700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba9c0b2910 0x7fba9c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:04.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.863+0000 7fbab4ce4700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba9c0b2910 0x7fba9c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:04.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.863+0000 7fbaadffb700 1 -- 172.21.15.19:0/3303298257 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbaa80e2750 con 0x7fbab010e940 2024-05-09T01:26:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.863+0000 7fbab4ce4700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba9c0b2910 0x7fba9c0b4e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0005950 tx=0x7fbaa000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:04.867+0000 7fbaadffb700 1 -- 172.21.15.19:0/3303298257 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbaa8020020 con 0x7fbab010e940 2024-05-09T01:26:05.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.127+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbab019c200 con 0x7fbab010e940 2024-05-09T01:26:05.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.128+0000 7fbaadffb700 1 -- 172.21.15.19:0/3303298257 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fbaa80aa400 con 0x7fbab010e940 2024-05-09T01:26:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:26:05.126 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:26:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba9c0b2910 msgr2=0x7fba9c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba9c0b2910 0x7fba9c0b4e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0005950 tx=0x7fbaa000b410 comp rx=0 tx=0).stop 2024-05-09T01:26:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab010e940 msgr2=0x7fbab019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab010e940 0x7fbab019f6d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8007d70 tx=0x7fbaa800a340 comp rx=0 tx=0).stop 2024-05-09T01:26:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 shutdown_connections 2024-05-09T01:26:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba9c0b2910 0x7fba9c0b4e20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab010e940 0x7fbab019f6d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbab0105480 0x7fbab019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:05.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 --2- 172.21.15.19:0/3303298257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbab0105e60 0x7fbab019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:05.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 >> 172.21.15.19:0/3303298257 conn(0x7fbab00fef50 msgr2=0x7fbab00ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:05.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.130+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 shutdown_connections 2024-05-09T01:26:05.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:05.131+0000 7fbab6f48700 1 -- 172.21.15.19:0/3303298257 wait complete. 2024-05-09T01:26:05.130 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:26:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:05 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3303298257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:26:05.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:05 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3303298257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:26:05.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:05 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3303298257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:26:05.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:26:05.990 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:06.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:06 smithi019 ceph-mon[30158]: pgmap v395: 97 pgs: 97 active+clean; 652 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-09T01:26:06.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:06 smithi059 ceph-mon[34351]: pgmap v395: 97 pgs: 97 active+clean; 652 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-09T01:26:06.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:06 smithi196 ceph-mon[35041]: pgmap v395: 97 pgs: 97 active+clean; 652 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-09T01:26:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.721+0000 7fc016af5700 1 -- 172.21.15.19:0/2152203083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 msgr2=0x7fc0101038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.721+0000 7fc016af5700 1 --2- 172.21.15.19:0/2152203083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 0x7fc0101038d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc0000099c0 tx=0x7fc000009cd0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.721+0000 7fc016af5700 1 -- 172.21.15.19:0/2152203083 shutdown_connections 2024-05-09T01:26:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.721+0000 7fc016af5700 1 --2- 172.21.15.19:0/2152203083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc010108740 0x7fc0101100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.721+0000 7fc016af5700 1 --2- 172.21.15.19:0/2152203083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 0x7fc0101038d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.721+0000 7fc016af5700 1 --2- 172.21.15.19:0/2152203083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc010107df0 0x7fc010108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.721+0000 7fc016af5700 1 -- 172.21.15.19:0/2152203083 >> 172.21.15.19:0/2152203083 conn(0x7fc0100fedc0 msgr2=0x7fc0101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.721+0000 7fc016af5700 1 -- 172.21.15.19:0/2152203083 shutdown_connections 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.721+0000 7fc016af5700 1 -- 172.21.15.19:0/2152203083 wait complete. 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.722+0000 7fc016af5700 1 Processor -- start 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.722+0000 7fc016af5700 1 -- start start 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc016af5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 0x7fc01019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc016af5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc010107df0 0x7fc01019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc016af5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc010108740 0x7fc010199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc016af5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc010113370 con 0x7fc0101034c0 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc016af5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0101131f0 con 0x7fc010108740 2024-05-09T01:26:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc016af5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0101134f0 con 0x7fc010107df0 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc014891700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 0x7fc01019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc014891700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 0x7fc01019f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58452/0 (socket says 172.21.15.19:58452) 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc014891700 1 -- 172.21.15.19:0/3013575871 learned_addr learned my addr 172.21.15.19:0/3013575871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc00ffff700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc010107df0 0x7fc01019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc015092700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc010108740 0x7fc010199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc014891700 1 -- 172.21.15.19:0/3013575871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc010107df0 msgr2=0x7fc01019ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc014891700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc010107df0 0x7fc01019ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc014891700 1 -- 172.21.15.19:0/3013575871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc010108740 msgr2=0x7fc010199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc014891700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc010108740 0x7fc010199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc014891700 1 -- 172.21.15.19:0/3013575871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01019a280 con 0x7fc0101034c0 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc015092700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc010108740 0x7fc010199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc014891700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 0x7fc01019f9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b580 tx=0x7fc0000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.723+0000 7fc00dffb700 1 -- 172.21.15.19:0/3013575871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000027070 con 0x7fc0101034c0 2024-05-09T01:26:07.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.724+0000 7fc00dffb700 1 -- 172.21.15.19:0/3013575871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc000003f20 con 0x7fc0101034c0 2024-05-09T01:26:07.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.724+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01019a510 con 0x7fc0101034c0 2024-05-09T01:26:07.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.724+0000 7fc00dffb700 1 -- 172.21.15.19:0/3013575871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000022d70 con 0x7fc0101034c0 2024-05-09T01:26:07.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.724+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01019a900 con 0x7fc0101034c0 2024-05-09T01:26:07.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.725+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc010104970 con 0x7fc0101034c0 2024-05-09T01:26:07.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.727+0000 7fc00dffb700 1 -- 172.21.15.19:0/3013575871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fc000022500 con 0x7fc0101034c0 2024-05-09T01:26:07.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.727+0000 7fc00dffb700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbff80b2910 0x7fbff80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:07.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.727+0000 7fc00ffff700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbff80b2910 0x7fbff80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:07.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.727+0000 7fc00dffb700 1 -- 172.21.15.19:0/3013575871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc0000e34a0 con 0x7fc0101034c0 2024-05-09T01:26:07.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.728+0000 7fc00ffff700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbff80b2910 0x7fbff80b4e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc01019b280 tx=0x7fc00400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:07.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.731+0000 7fc00dffb700 1 -- 172.21.15.19:0/3013575871 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc000013070 con 0x7fc0101034c0 2024-05-09T01:26:07.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.949+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc01010b170 con 0x7fc0101034c0 2024-05-09T01:26:07.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.951+0000 7fc00dffb700 1 -- 172.21.15.19:0/3013575871 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22716 (secure 0 0 0) 0x7fc0000ab070 con 0x7fc0101034c0 2024-05-09T01:26:07.949 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:26:07.949 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:23:22.298486+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbff80b2910 msgr2=0x7fbff80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbff80b2910 0x7fbff80b4e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc01019b280 tx=0x7fc00400b410 comp rx=0 tx=0).stop 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 msgr2=0x7fc01019f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 0x7fc01019f9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b580 tx=0x7fc0000047b0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 shutdown_connections 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc010108740 0x7fc010199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbff80b2910 0x7fbff80b4e20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0101034c0 0x7fc01019f9c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 --2- 172.21.15.19:0/3013575871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc010107df0 0x7fc01019ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 >> 172.21.15.19:0/3013575871 conn(0x7fc0100fedc0 msgr2=0x7fc0101096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 shutdown_connections 2024-05-09T01:26:07.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:07.953+0000 7fc016af5700 1 -- 172.21.15.19:0/3013575871 wait complete. 2024-05-09T01:26:08.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:08 smithi059 ceph-mon[34351]: pgmap v396: 97 pgs: 97 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T01:26:08.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:08 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3013575871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:26:08.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:08 smithi019 ceph-mon[30158]: pgmap v396: 97 pgs: 97 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T01:26:08.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:08 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3013575871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:26:08.637 DEBUG:tasks.fwd_scrub:Yielding 2024-05-09T01:26:08.638 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-09T01:26:08.651 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T01:26:08.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-09T01:26:08.652 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-09T01:26:08.652 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:26:08.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:26:08.655 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T01:26:08.668 INFO:tasks.workunit:Pulling workunits from ref c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:26:08.669 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T01:26:08.669 INFO:tasks.workunit:timeout=3h 2024-05-09T01:26:08.669 INFO:tasks.workunit:cleanup=True 2024-05-09T01:26:08.669 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T01:26:08.689 INFO:teuthology.orchestra.run.smithi019.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T01:26:08.689 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-09T01:26:08.689 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 67h/103d Inode: 1099511628283 Links: 2 2024-05-09T01:26:08.689 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T01:26:08.690 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-09T01:26:08.690 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-09 01:23:14.274236856 +0000 2024-05-09T01:26:08.690 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-09 01:23:14.274236856 +0000 2024-05-09T01:26:08.690 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-09 01:25:55.901672558 +0000 2024-05-09T01:26:08.690 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-09 01:23:14.274236856 +0000 2024-05-09T01:26:08.690 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T01:26:08.690 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T01:26:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:08 smithi196 ceph-mon[35041]: pgmap v396: 97 pgs: 97 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T01:26:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:08 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3013575871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:26:08.765 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:26:08.822 INFO:tasks.workunit.client.0.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T01:26:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:10.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:10 smithi019 ceph-mon[30158]: pgmap v397: 97 pgs: 97 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-09T01:26:10.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.573+0000 7f14e85f6700 1 -- 172.21.15.19:0/2294127268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 msgr2=0x7f14e0108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:10.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.573+0000 7f14e85f6700 1 --2- 172.21.15.19:0/2294127268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 0x7f14e0108250 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f14d00099c0 tx=0x7f14d0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.573+0000 7f14e85f6700 1 -- 172.21.15.19:0/2294127268 shutdown_connections 2024-05-09T01:26:10.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.574+0000 7f14e85f6700 1 --2- 172.21.15.19:0/2294127268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14e0108790 0x7f14e01100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.574+0000 7f14e85f6700 1 --2- 172.21.15.19:0/2294127268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e0103c40 0x7f14e0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.574+0000 7f14e85f6700 1 --2- 172.21.15.19:0/2294127268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 0x7f14e0108250 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.574+0000 7f14e85f6700 1 -- 172.21.15.19:0/2294127268 >> 172.21.15.19:0/2294127268 conn(0x7f14e00fe6f0 msgr2=0x7f14e0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:10.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.574+0000 7f14e85f6700 1 -- 172.21.15.19:0/2294127268 shutdown_connections 2024-05-09T01:26:10.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.574+0000 7f14e85f6700 1 -- 172.21.15.19:0/2294127268 wait complete. 2024-05-09T01:26:10.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.574+0000 7f14e85f6700 1 Processor -- start 2024-05-09T01:26:10.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.574+0000 7f14e85f6700 1 -- start start 2024-05-09T01:26:10.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e85f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14e0103c40 0x7f14e019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e85f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 0x7f14e019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e85f6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e0108790 0x7f14e019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e85f6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0113390 con 0x7f14e0107e40 2024-05-09T01:26:10.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e85f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0113210 con 0x7f14e0103c40 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e85f6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0113510 con 0x7f14e0108790 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e5b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 0x7f14e019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e5b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 0x7f14e019b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46926/0 (socket says 172.21.15.19:46926) 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e5b91700 1 -- 172.21.15.19:0/1965112539 learned_addr learned my addr 172.21.15.19:0/1965112539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e6b93700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e0108790 0x7f14e019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e6392700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14e0103c40 0x7f14e019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e5b91700 1 -- 172.21.15.19:0/1965112539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e0108790 msgr2=0x7f14e019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e5b91700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e0108790 0x7f14e019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e5b91700 1 -- 172.21.15.19:0/1965112539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14e0103c40 msgr2=0x7f14e019aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e5b91700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14e0103c40 0x7f14e019aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e5b91700 1 -- 172.21.15.19:0/1965112539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e019fdc0 con 0x7f14e0107e40 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.575+0000 7f14e5b91700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 0x7f14e019b490 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f14d00045e0 tx=0x7f14d0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.576+0000 7f14d77fe700 1 -- 172.21.15.19:0/1965112539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0026070 con 0x7f14e0107e40 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.576+0000 7f14d77fe700 1 -- 172.21.15.19:0/1965112539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d000f460 con 0x7f14e0107e40 2024-05-09T01:26:10.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.576+0000 7f14e85f6700 1 -- 172.21.15.19:0/1965112539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e01a0050 con 0x7f14e0107e40 2024-05-09T01:26:10.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.576+0000 7f14d77fe700 1 -- 172.21.15.19:0/1965112539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0009e60 con 0x7f14e0107e40 2024-05-09T01:26:10.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.576+0000 7f14e85f6700 1 -- 172.21.15.19:0/1965112539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e01a79d0 con 0x7f14e0107e40 2024-05-09T01:26:10.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.579+0000 7f14d77fe700 1 -- 172.21.15.19:0/1965112539 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f14d0029070 con 0x7f14e0107e40 2024-05-09T01:26:10.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.580+0000 7f14d77fe700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f14cc0b2990 0x7f14cc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.580+0000 7f14f700cb80 1 Processor -- start 2024-05-09T01:26:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.580+0000 7f14e6392700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f14cc0b2990 0x7f14cc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.580+0000 7f14d77fe700 1 -- 172.21.15.19:0/1965112539 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f14d00e2b80 con 0x7f14e0107e40 2024-05-09T01:26:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.580+0000 7f14f700cb80 1 -- start start 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c75fc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e6392700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f14cc0b2990 0x7f14cc0b4ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f14e006afd0 tx=0x7f14dc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c75fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 0x55c4c75f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14f700cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c4c7519070 con 0x55c4c7479520 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14f700cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c4c74b7870 con 0x55c4c75f6720 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14f700cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c4c7515030 con 0x55c4c75631e0 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e5b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c75fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e5b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c75fcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46930/0 (socket says 172.21.15.19:46930) 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e5b91700 1 -- 172.21.15.19:0/1197899268 learned_addr learned my addr 172.21.15.19:0/1197899268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e6b93700 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 0x55c4c75f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e6392700 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c75fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e5b91700 1 -- 172.21.15.19:0/1197899268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 msgr2=0x55c4c75fc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e5b91700 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c75fc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e5b91700 1 -- 172.21.15.19:0/1197899268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 msgr2=0x55c4c75f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e5b91700 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 0x55c4c75f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e5b91700 1 -- 172.21.15.19:0/1197899268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4c74af3f0 con 0x55c4c7479520 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e6b93700 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 0x55c4c75f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14e5b91700 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c75fcab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14d00aa040 tx=0x7f14d00a9ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc35edb6805893f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14d4ff9700 1 -- 172.21.15.19:0/1197899268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d00aa0b0 con 0x55c4c7479520 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14d4ff9700 1 -- 172.21.15.19:0/1197899268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d00ae460 con 0x55c4c7479520 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.581+0000 7f14d4ff9700 1 -- 172.21.15.19:0/1197899268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d00aa570 con 0x55c4c7479520 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.582+0000 7f14f700cb80 1 -- 172.21.15.19:0/1197899268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 msgr2=0x55c4c75fcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.582+0000 7f14f700cb80 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c75fcab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14d00aa040 tx=0x7f14d00a9ad0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.582+0000 7f14f700cb80 1 -- 172.21.15.19:0/1197899268 shutdown_connections 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.582+0000 7f14f700cb80 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 0x55c4c75f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.582+0000 7f14f700cb80 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c75fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.582+0000 7f14f700cb80 1 --2- 172.21.15.19:0/1197899268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c75fcab0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.582+0000 7f14f700cb80 1 -- 172.21.15.19:0/1197899268 >> 172.21.15.19:0/1197899268 conn(0x55c4c74d9720 msgr2=0x55c4c75f1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 -- 172.21.15.19:0/1197899268 shutdown_connections 2024-05-09T01:26:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 -- 172.21.15.19:0/1197899268 wait complete. 2024-05-09T01:26:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 -- start start 2024-05-09T01:26:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 Processor -- start 2024-05-09T01:26:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 0x55c4c7605e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75631e0 0x55c4c7606360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75f6720 0x55c4c7600220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c4c7394c20 con 0x55c4c75631e0 2024-05-09T01:26:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c4c74d31d0 con 0x55c4c7479520 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c4c75f73e0 con 0x55c4c75f6720 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.583+0000 7f14f700cb80 10 client.0 _ll_register_callbacks cb 0x55c4c738a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e5b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75631e0 0x55c4c7606360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e5b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75631e0 0x55c4c7606360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46936/0 (socket says 172.21.15.19:46936) 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e5b91700 1 -- 172.21.15.19:0/2335428186 learned_addr learned my addr 172.21.15.19:0/2335428186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e6392700 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 0x55c4c7605e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e6b93700 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75f6720 0x55c4c7600220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e5b91700 1 -- 172.21.15.19:0/2335428186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75f6720 msgr2=0x55c4c7600220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e5b91700 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75f6720 0x55c4c7600220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e5b91700 1 -- 172.21.15.19:0/2335428186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 msgr2=0x55c4c7605e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e5b91700 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 0x55c4c7605e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e5b91700 1 -- 172.21.15.19:0/2335428186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4c7394510 con 0x55c4c75631e0 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e6b93700 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75f6720 0x55c4c7600220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14e5b91700 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75631e0 0x55c4c7606360 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009d40 tx=0x7f14d00e0990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc98bdb34d3cd118 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14c9ffb700 1 -- 172.21.15.19:0/2335428186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d00c3060 con 0x55c4c75631e0 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14c9ffb700 1 -- 172.21.15.19:0/2335428186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f14d00aa310 con 0x55c4c75631e0 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14f700cb80 1 -- 172.21.15.19:0/2335428186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d00ae460 con 0x55c4c75631e0 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.584+0000 7f14c9ffb700 1 -- 172.21.15.19:0/2335428186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14d00b2e80 con 0x55c4c75631e0 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14f700cb80 10 client.15318 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14f700cb80 1 -- 172.21.15.19:0/2335428186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4c760da40 con 0x55c4c75631e0 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14c9ffb700 1 -- 172.21.15.19:0/2335428186 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d00ae460 con 0x55c4c75631e0 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14c9ffb700 1 -- 172.21.15.19:0/2335428186 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f14d00a5a20 con 0x55c4c75631e0 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14f700cb80 10 client.15318 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14f700cb80 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14f700cb80 20 client.15318 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14f700cb80 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c4c760ff10 0x55c4c7612330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14f700cb80 4 client.15318 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14f700cb80 1 -- 172.21.15.19:0/2335428186 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4c7526a50 con 0x55c4c760ff10 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.585+0000 7f14e6392700 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c4c760ff10 0x55c4c7612330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.586+0000 7f14e6392700 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c4c760ff10 0x55c4c7612330 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc18f4efa4babb8 server_cookie=f964f9f6c31d785a in_seq=0 out_seq=0 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.586+0000 7f14c9ffb700 10 client.15318 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.586+0000 7f14c9ffb700 1 -- 172.21.15.19:0/2335428186 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4c7526a50 con 0x55c4c760ff10 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.586+0000 7f14c9ffb700 10 client.15318 handle_command_reply: tid=0 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.586+0000 7f14f700cb80 1 client.15318 shutdown 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 -- 172.21.15.19:0/2335428186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75631e0 msgr2=0x55c4c7606360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75631e0 0x55c4c7606360 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009d40 tx=0x7f14d00e0990 comp rx=0 tx=0).stop 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 -- 172.21.15.19:0/2335428186 shutdown_connections 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c4c760ff10 0x55c4c7612330 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75f6720 0x55c4c7600220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 0x55c4c7605e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14c9ffb700 0 client.15318 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 --2- 172.21.15.19:0/2335428186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75631e0 0x55c4c7606360 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 -- 172.21.15.19:0/2335428186 >> 172.21.15.19:0/2335428186 conn(0x55c4c74d9720 msgr2=0x55c4c75f1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 -- 172.21.15.19:0/2335428186 shutdown_connections 2024-05-09T01:26:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 1 -- 172.21.15.19:0/2335428186 wait complete. 2024-05-09T01:26:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.587+0000 7f14f700cb80 20 client.15318 trim_cache size 0 max 16384 2024-05-09T01:26:10.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:10 smithi059 ceph-mon[34351]: pgmap v397: 97 pgs: 97 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-09T01:26:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.607+0000 7f14f700cb80 1 Processor -- start 2024-05-09T01:26:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.607+0000 7f14f700cb80 1 -- start start 2024-05-09T01:26:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 0x55c4c75ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c7617960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75f6720 0x55c4c7617ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14f700cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01aecc0 con 0x55c4c75f6720 2024-05-09T01:26:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14f700cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01b0800 con 0x55c4c7479520 2024-05-09T01:26:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14f700cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01af390 con 0x55c4c75631e0 2024-05-09T01:26:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6b93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75f6720 0x55c4c7617ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6b93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75f6720 0x55c4c7617ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46942/0 (socket says 172.21.15.19:46942) 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6b93700 1 -- 172.21.15.19:0/789023030 learned_addr learned my addr 172.21.15.19:0/789023030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e5b91700 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c7617960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6392700 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 0x55c4c75ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6b93700 1 -- 172.21.15.19:0/789023030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 msgr2=0x55c4c7617960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6b93700 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c7617960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6b93700 1 -- 172.21.15.19:0/789023030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 msgr2=0x55c4c75ff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6b93700 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 0x55c4c75ff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6b93700 1 -- 172.21.15.19:0/789023030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4c75f2f30 con 0x55c4c75f6720 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e5b91700 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c7617960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.608+0000 7f14e6b93700 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75f6720 0x55c4c7617ea0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f14d800ac20 tx=0x7f14d800af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b78b3db34eeb460 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:10.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.609+0000 7f14cbfff700 1 -- 172.21.15.19:0/789023030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8012cc0 con 0x55c4c75f6720 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.609+0000 7f14cbfff700 1 -- 172.21.15.19:0/789023030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d80044d0 con 0x55c4c75f6720 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.609+0000 7f14cbfff700 1 -- 172.21.15.19:0/789023030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8003e60 con 0x55c4c75f6720 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 -- 172.21.15.19:0/789023030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75f6720 msgr2=0x55c4c7617ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75f6720 0x55c4c7617ea0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f14d800ac20 tx=0x7f14d800af30 comp rx=0 tx=0).stop 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 -- 172.21.15.19:0/789023030 shutdown_connections 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c75f6720 0x55c4c7617ea0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c7479520 0x55c4c75ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 --2- 172.21.15.19:0/789023030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c7617960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 -- 172.21.15.19:0/789023030 >> 172.21.15.19:0/789023030 conn(0x55c4c74d9720 msgr2=0x55c4c75fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 -- 172.21.15.19:0/789023030 shutdown_connections 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 -- 172.21.15.19:0/789023030 wait complete. 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 -- start start 2024-05-09T01:26:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.610+0000 7f14f700cb80 1 Processor -- start 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c760c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c760a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14f700cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 0x55c4c760ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14f700cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01b0130 con 0x55c4c7479520 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14f700cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01afa60 con 0x55c4c75f6720 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14f700cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01acc60 con 0x55c4c75631e0 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14f700cb80 10 client.0 _ll_register_callbacks cb 0x55c4c75fbe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c760c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c760c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46956/0 (socket says 172.21.15.19:46956) 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6392700 1 -- 172.21.15.19:0/577285354 learned_addr learned my addr 172.21.15.19:0/577285354 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6b93700 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 0x55c4c760ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e5b91700 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c760a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6392700 1 -- 172.21.15.19:0/577285354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 msgr2=0x55c4c760a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6392700 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c760a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6392700 1 -- 172.21.15.19:0/577285354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 msgr2=0x55c4c760ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6392700 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 0x55c4c760ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6392700 1 -- 172.21.15.19:0/577285354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4c760b340 con 0x55c4c7479520 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14e6392700 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c760c460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55c4c75ffaa0 tx=0x7f14dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9824ae22bff0cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.611+0000 7f14c9ffb700 1 -- 172.21.15.19:0/577285354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc01c040 con 0x55c4c7479520 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14c9ffb700 1 -- 172.21.15.19:0/577285354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f14dc02b720 con 0x55c4c7479520 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14f700cb80 1 -- 172.21.15.19:0/577285354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d80044d0 con 0x55c4c7479520 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14c9ffb700 1 -- 172.21.15.19:0/577285354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14dc02b920 con 0x55c4c7479520 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14f700cb80 10 client.15324 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14f700cb80 1 -- 172.21.15.19:0/577285354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4c767b320 con 0x55c4c7479520 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14c9ffb700 1 -- 172.21.15.19:0/577285354 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d80044d0 con 0x55c4c7479520 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14c9ffb700 1 -- 172.21.15.19:0/577285354 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f14dc02bce0 con 0x55c4c7479520 2024-05-09T01:26:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14f700cb80 10 client.15324 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14f700cb80 10 client.15324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14f700cb80 20 client.15324 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14f700cb80 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c4c767ba90 0x55c4c767deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14f700cb80 4 client.15324 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:26:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14f700cb80 1 -- 172.21.15.19:0/577285354 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4c7520af0 con 0x55c4c767ba90 2024-05-09T01:26:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.612+0000 7f14e5b91700 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c4c767ba90 0x55c4c767deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.613+0000 7f14e5b91700 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c4c767ba90 0x55c4c767deb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ff461630ab4477 server_cookie=1e7ea7aa206917b8 in_seq=0 out_seq=0 2024-05-09T01:26:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.613+0000 7f14c9ffb700 10 client.15324 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:10.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14c9ffb700 1 -- 172.21.15.19:0/577285354 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4c7520af0 con 0x55c4c767ba90 2024-05-09T01:26:10.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14c9ffb700 10 client.15324 handle_command_reply: tid=0 2024-05-09T01:26:10.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 client.15324 shutdown 2024-05-09T01:26:10.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 -- 172.21.15.19:0/577285354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 msgr2=0x55c4c760c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c760c460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55c4c75ffaa0 tx=0x7f14dc004ad0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 -- 172.21.15.19:0/577285354 shutdown_connections 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c4c767ba90 0x55c4c767deb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c4c75f6720 0x55c4c760ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4c7479520 0x55c4c760c460 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 --2- 172.21.15.19:0/577285354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4c75631e0 0x55c4c760a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 -- 172.21.15.19:0/577285354 >> 172.21.15.19:0/577285354 conn(0x55c4c74d9720 msgr2=0x55c4c75fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14c9ffb700 0 client.15324 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 -- 172.21.15.19:0/577285354 shutdown_connections 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.614+0000 7f14f700cb80 1 -- 172.21.15.19:0/577285354 wait complete. 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14f700cb80 20 client.15324 trim_cache size 0 max 16384 2024-05-09T01:26:10.616 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:26:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:26:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "5bb75e53-bbd7-432f-9483-060c84275400", 2024-05-09T01:26:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:26:10.617 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:26:10.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 -- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f14cc0b2990 msgr2=0x7f14cc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:10.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f14cc0b2990 0x7f14cc0b4ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f14e006afd0 tx=0x7f14dc00b040 comp rx=0 tx=0).stop 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 -- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 msgr2=0x7f14e019b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 0x7f14e019b490 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f14d00045e0 tx=0x7f14d0004610 comp rx=0 tx=0).stop 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 -- 172.21.15.19:0/1965112539 shutdown_connections 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e0108790 0x7f14e019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14e0103c40 0x7f14e019aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f14cc0b2990 0x7f14cc0b4ea0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 --2- 172.21.15.19:0/1965112539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14e0107e40 0x7f14e019b490 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 -- 172.21.15.19:0/1965112539 >> 172.21.15.19:0/1965112539 conn(0x7f14e00fe6f0 msgr2=0x7f14e0102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 -- 172.21.15.19:0/1965112539 shutdown_connections 2024-05-09T01:26:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:10.615+0000 7f14e85f6700 1 -- 172.21.15.19:0/1965112539 wait complete. 2024-05-09T01:26:10.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:10 smithi196 ceph-mon[35041]: pgmap v397: 97 pgs: 97 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-09T01:26:11.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:11 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:26:11.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:11 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:26:11.330 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:11 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:26:11.330 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:11 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:26:11.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:26:11.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:11 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:26:11.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:11 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:26:11.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:11 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:26:11.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:11 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:26:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:11.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:11 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:26:11.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:11 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:26:11.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:11 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:26:11.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:11 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:26:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:12 smithi019 ceph-mon[30158]: pgmap v398: 97 pgs: 97 active+clean; 660 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-09T01:26:12.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:12 smithi059 ceph-mon[34351]: pgmap v398: 97 pgs: 97 active+clean; 660 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-09T01:26:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:12 smithi196 ceph-mon[35041]: pgmap v398: 97 pgs: 97 active+clean; 660 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-09T01:26:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.288+0000 7f73180b1700 1 -- 172.21.15.19:0/3945132754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310107df0 msgr2=0x7f7310108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.288+0000 7f73180b1700 1 --2- 172.21.15.19:0/3945132754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310107df0 0x7f7310108200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f73000099c0 tx=0x7f7300009cd0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.288+0000 7f73180b1700 1 -- 172.21.15.19:0/3945132754 shutdown_connections 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.288+0000 7f73180b1700 1 --2- 172.21.15.19:0/3945132754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7310108740 0x7f73101100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.288+0000 7f73180b1700 1 --2- 172.21.15.19:0/3945132754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73101034c0 0x7f73101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.288+0000 7f73180b1700 1 --2- 172.21.15.19:0/3945132754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310107df0 0x7f7310108200 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.288+0000 7f73180b1700 1 -- 172.21.15.19:0/3945132754 >> 172.21.15.19:0/3945132754 conn(0x7f73100fedc0 msgr2=0x7f73101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.288+0000 7f73180b1700 1 -- 172.21.15.19:0/3945132754 shutdown_connections 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.288+0000 7f73180b1700 1 -- 172.21.15.19:0/3945132754 wait complete. 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.289+0000 7f73180b1700 1 Processor -- start 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.289+0000 7f73180b1700 1 -- start start 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.289+0000 7f73180b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73101034c0 0x7f73100ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.289+0000 7f73180b1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7310107df0 0x7f73100ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.289+0000 7f73180b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310108740 0x7f73100ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.289+0000 7f73180b1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7310113350 con 0x7f7310108740 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.289+0000 7f73180b1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73101131d0 con 0x7f73101034c0 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.289+0000 7f73180b1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73101134d0 con 0x7f7310107df0 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731664e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310108740 0x7f73100ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f7315e4d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73101034c0 0x7f73100ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731664e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310108740 0x7f73100ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46976/0 (socket says 172.21.15.19:46976) 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731664e700 1 -- 172.21.15.19:0/2956419729 learned_addr learned my addr 172.21.15.19:0/2956419729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731564c700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7310107df0 0x7f73100ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731664e700 1 -- 172.21.15.19:0/2956419729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7310107df0 msgr2=0x7f73100ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731664e700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7310107df0 0x7f73100ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731664e700 1 -- 172.21.15.19:0/2956419729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73101034c0 msgr2=0x7f73100ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731664e700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73101034c0 0x7f73100ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731664e700 1 -- 172.21.15.19:0/2956419729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73101a9170 con 0x7f7310108740 2024-05-09T01:26:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f7315e4d700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73101034c0 0x7f73100ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:26:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.290+0000 7f731664e700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310108740 0x7f73100ffd20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7308007ee0 tx=0x7f730800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.291+0000 7f7306ffd700 1 -- 172.21.15.19:0/2956419729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308004bb0 con 0x7f7310108740 2024-05-09T01:26:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.291+0000 7f7306ffd700 1 -- 172.21.15.19:0/2956419729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7308004d50 con 0x7f7310108740 2024-05-09T01:26:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.291+0000 7f73180b1700 1 -- 172.21.15.19:0/2956419729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73101a9460 con 0x7f7310108740 2024-05-09T01:26:13.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.291+0000 7f7306ffd700 1 -- 172.21.15.19:0/2956419729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308005920 con 0x7f7310108740 2024-05-09T01:26:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.291+0000 7f73180b1700 1 -- 172.21.15.19:0/2956419729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73101a9970 con 0x7f7310108740 2024-05-09T01:26:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.293+0000 7f7306ffd700 1 -- 172.21.15.19:0/2956419729 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f730801a020 con 0x7f7310108740 2024-05-09T01:26:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.294+0000 7f7306ffd700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72fc0b2990 0x7f72fc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.294+0000 7f7315e4d700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72fc0b2990 0x7f72fc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.294+0000 7f7306ffd700 1 -- 172.21.15.19:0/2956419729 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f73080e1940 con 0x7f7310108740 2024-05-09T01:26:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.295+0000 7f7315e4d700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72fc0b2990 0x7f72fc0b4ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f730c00afd0 tx=0x7f730c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.295+0000 7f7326ac7b80 1 Processor -- start 2024-05-09T01:26:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.295+0000 7f7326ac7b80 1 -- start start 2024-05-09T01:26:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb65a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb5fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ab1ba81ff0 con 0x55ab1bacc160 2024-05-09T01:26:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ab1ba207f0 con 0x55ab1b9e24a0 2024-05-09T01:26:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab1ba7dfb0 con 0x55ab1bb5f6a0 2024-05-09T01:26:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7315e4d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7315e4d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb654f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46988/0 (socket says 172.21.15.19:46988) 2024-05-09T01:26:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7315e4d700 1 -- 172.21.15.19:0/5323265 learned_addr learned my addr 172.21.15.19:0/5323265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f731564c700 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb65a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f731664e700 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb5fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7315e4d700 1 -- 172.21.15.19:0/5323265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 msgr2=0x55ab1bb5fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7315e4d700 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb5fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7315e4d700 1 -- 172.21.15.19:0/5323265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 msgr2=0x55ab1bb65a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7315e4d700 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb65a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7315e4d700 1 -- 172.21.15.19:0/5323265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1ba18370 con 0x55ab1bacc160 2024-05-09T01:26:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f731664e700 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb5fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.296+0000 7f7315e4d700 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb654f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f730c010780 tx=0x7f730c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a0d9a5d7e15586 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.297+0000 7f72fbfff700 1 -- 172.21.15.19:0/5323265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c016070 con 0x55ab1bacc160 2024-05-09T01:26:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.297+0000 7f72fbfff700 1 -- 172.21.15.19:0/5323265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f730c00fe50 con 0x55ab1bacc160 2024-05-09T01:26:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.297+0000 7f72fbfff700 1 -- 172.21.15.19:0/5323265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c01b710 con 0x55ab1bacc160 2024-05-09T01:26:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 -- 172.21.15.19:0/5323265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 msgr2=0x55ab1bb654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb654f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f730c010780 tx=0x7f730c012000 comp rx=0 tx=0).stop 2024-05-09T01:26:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 -- 172.21.15.19:0/5323265 shutdown_connections 2024-05-09T01:26:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb5fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb654f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/5323265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb65a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 -- 172.21.15.19:0/5323265 >> 172.21.15.19:0/5323265 conn(0x55ab1ba426a0 msgr2=0x55ab1bb5a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 -- 172.21.15.19:0/5323265 shutdown_connections 2024-05-09T01:26:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 -- 172.21.15.19:0/5323265 wait complete. 2024-05-09T01:26:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 -- start start 2024-05-09T01:26:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.298+0000 7f7326ac7b80 1 Processor -- start 2024-05-09T01:26:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb69730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb69c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ab1b8fdba0 con 0x55ab1bacc160 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ab1ba3c150 con 0x55ab1bb5f6a0 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab1bb6c1d0 con 0x55ab1b9e24a0 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f7326ac7b80 10 client.0 _ll_register_callbacks cb 0x55ab1b8f36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f731564c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb69730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f731564c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb69730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46998/0 (socket says 172.21.15.19:46998) 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f731564c700 1 -- 172.21.15.19:0/3457124888 learned_addr learned my addr 172.21.15.19:0/3457124888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f7315e4d700 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f731664e700 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb69c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f731564c700 1 -- 172.21.15.19:0/3457124888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1b9e24a0 msgr2=0x55ab1bb691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f731564c700 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f731564c700 1 -- 172.21.15.19:0/3457124888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 msgr2=0x55ab1bb69c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f731564c700 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb69c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.299+0000 7f731564c700 1 -- 172.21.15.19:0/3457124888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1b8fd490 con 0x55ab1bacc160 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.300+0000 7f731564c700 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb69730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7310076ab0 tx=0x7f7300005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b2dc2fec61e670 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.300+0000 7f72f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.300+0000 7f72f97fa700 1 -- 172.21.15.19:0/3457124888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300005580 con 0x55ab1bacc160 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.300+0000 7f72f97fa700 1 -- 172.21.15.19:0/3457124888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f730000b790 con 0x55ab1bacc160 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.300+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3457124888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f730c00fe50 con 0x55ab1bacc160 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.300+0000 7f72f97fa700 1 -- 172.21.15.19:0/3457124888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f730000b990 con 0x55ab1bacc160 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.300+0000 7f7326ac7b80 10 client.15336 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.300+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3457124888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab1bb76b80 con 0x55ab1bacc160 2024-05-09T01:26:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.300+0000 7f72f97fa700 1 -- 172.21.15.19:0/3457124888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f730c00fe50 con 0x55ab1bacc160 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f72f97fa700 1 -- 172.21.15.19:0/3457124888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7300023bd0 con 0x55ab1bacc160 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f7326ac7b80 10 client.15336 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f7326ac7b80 10 client.15336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f7326ac7b80 20 client.15336 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ab1bb79290 0x55ab1bb7b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f7326ac7b80 4 client.15336 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3457124888 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab1ba8f9d0 con 0x55ab1bb79290 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f7315e4d700 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ab1bb79290 0x55ab1bb7b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f72f97fa700 10 client.15336 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.301+0000 7f7315e4d700 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ab1bb79290 0x55ab1bb7b6e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c821a87c12f5859e server_cookie=44ac3f47fe3083b in_seq=0 out_seq=0 2024-05-09T01:26:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f72f97fa700 1 -- 172.21.15.19:0/3457124888 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ab1ba8f9d0 con 0x55ab1bb79290 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f72f97fa700 10 client.15336 handle_command_reply: tid=0 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f7326ac7b80 1 client.15336 shutdown 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3457124888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 msgr2=0x55ab1bb69730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb69730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7310076ab0 tx=0x7f7300005af0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3457124888 shutdown_connections 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ab1bb79290 0x55ab1bb7b6e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb69c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f72f97fa700 0 client.15336 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3457124888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb69730 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.302+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3457124888 >> 172.21.15.19:0/3457124888 conn(0x55ab1ba426a0 msgr2=0x55ab1bb5a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.303+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3457124888 shutdown_connections 2024-05-09T01:26:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.303+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3457124888 wait complete. 2024-05-09T01:26:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.303+0000 7f7326ac7b80 20 client.15336 trim_cache size 0 max 16384 2024-05-09T01:26:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7326ac7b80 1 Processor -- start 2024-05-09T01:26:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7326ac7b80 1 -- start start 2024-05-09T01:26:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb64900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bacc160 0x55ab1bb74fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb69760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b0c10 con 0x55ab1b9e24a0 2024-05-09T01:26:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b2750 con 0x55ab1bb5f6a0 2024-05-09T01:26:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b12e0 con 0x55ab1bacc160 2024-05-09T01:26:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7315e4d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb64900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7315e4d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb64900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47000/0 (socket says 172.21.15.19:47000) 2024-05-09T01:26:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.324+0000 7f7315e4d700 1 -- 172.21.15.19:0/3337296147 learned_addr learned my addr 172.21.15.19:0/3337296147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f731564c700 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bacc160 0x55ab1bb74fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f731664e700 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb69760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f7315e4d700 1 -- 172.21.15.19:0/3337296147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bacc160 msgr2=0x55ab1bb74fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f7315e4d700 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bacc160 0x55ab1bb74fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f7315e4d700 1 -- 172.21.15.19:0/3337296147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 msgr2=0x55ab1bb69760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f7315e4d700 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb69760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f7315e4d700 1 -- 172.21.15.19:0/3337296147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1bb68510 con 0x55ab1b9e24a0 2024-05-09T01:26:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f731664e700 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb69760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f7315e4d700 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb64900 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f730c025f00 tx=0x7f730c00adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea19344c913b1ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f72fb7fe700 1 -- 172.21.15.19:0/3337296147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c016040 con 0x55ab1b9e24a0 2024-05-09T01:26:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f72fb7fe700 1 -- 172.21.15.19:0/3337296147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f730c0044d0 con 0x55ab1b9e24a0 2024-05-09T01:26:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.325+0000 7f72fb7fe700 1 -- 172.21.15.19:0/3337296147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c0038e0 con 0x55ab1b9e24a0 2024-05-09T01:26:13.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.326+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3337296147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1b9e24a0 msgr2=0x55ab1bb64900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:13.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.326+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb64900 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f730c025f00 tx=0x7f730c00adb0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.326+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3337296147 shutdown_connections 2024-05-09T01:26:13.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.326+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb69760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.326+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb64900 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.326+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/3337296147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bacc160 0x55ab1bb74fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.326+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3337296147 >> 172.21.15.19:0/3337296147 conn(0x55ab1ba426a0 msgr2=0x55ab1bb641f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.327+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3337296147 shutdown_connections 2024-05-09T01:26:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.327+0000 7f7326ac7b80 1 -- 172.21.15.19:0/3337296147 wait complete. 2024-05-09T01:26:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.327+0000 7f7326ac7b80 1 -- start start 2024-05-09T01:26:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.327+0000 7f7326ac7b80 1 Processor -- start 2024-05-09T01:26:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb80650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb80b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7326ac7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b2080 con 0x55ab1bacc160 2024-05-09T01:26:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b19b0 con 0x55ab1b9e24a0 2024-05-09T01:26:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7326ac7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73101aebb0 con 0x55ab1bb5f6a0 2024-05-09T01:26:13.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7326ac7b80 10 client.0 _ll_register_callbacks cb 0x55ab1b8f36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:13.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7315e4d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb80650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7315e4d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb80650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:59136/0 (socket says 172.21.15.19:59136) 2024-05-09T01:26:13.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f7315e4d700 1 -- 172.21.15.19:0/2367642456 learned_addr learned my addr 172.21.15.19:0/2367642456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:13.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f731664e700 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.328+0000 7f731564c700 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb80b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f7315e4d700 1 -- 172.21.15.19:0/2367642456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 msgr2=0x55ab1bb810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:13.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f7315e4d700 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f7315e4d700 1 -- 172.21.15.19:0/2367642456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 msgr2=0x55ab1bb80b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:13.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f7315e4d700 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb80b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f7315e4d700 1 -- 172.21.15.19:0/2367642456 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1bbe2a10 con 0x55ab1b9e24a0 2024-05-09T01:26:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f731564c700 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb80b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f7315e4d700 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb80650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f730c0046a0 tx=0x7f730c01bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f1ac8a7326c187a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f72f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:26:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f72f97fa700 1 -- 172.21.15.19:0/2367642456 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c01b620 con 0x55ab1b9e24a0 2024-05-09T01:26:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f7326ac7b80 1 -- 172.21.15.19:0/2367642456 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f730c0044d0 con 0x55ab1b9e24a0 2024-05-09T01:26:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.329+0000 7f72f97fa700 1 -- 172.21.15.19:0/2367642456 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f730c025470 con 0x55ab1b9e24a0 2024-05-09T01:26:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f72f97fa700 1 -- 172.21.15.19:0/2367642456 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f730c025670 con 0x55ab1b9e24a0 2024-05-09T01:26:13.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f7326ac7b80 10 client.25031 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:13.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f7326ac7b80 1 -- 172.21.15.19:0/2367642456 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab1bbe3bb0 con 0x55ab1b9e24a0 2024-05-09T01:26:13.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f72f97fa700 1 -- 172.21.15.19:0/2367642456 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f730c0044d0 con 0x55ab1b9e24a0 2024-05-09T01:26:13.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f72f97fa700 1 -- 172.21.15.19:0/2367642456 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f730c0339e0 con 0x55ab1b9e24a0 2024-05-09T01:26:13.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f7326ac7b80 10 client.25031 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:13.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f7326ac7b80 10 client.25031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f7326ac7b80 20 client.25031 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ab1bbe4000 0x55ab1bbe6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f7326ac7b80 4 client.25031 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:26:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f7326ac7b80 1 -- 172.21.15.19:0/2367642456 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab1ba89a70 con 0x55ab1bbe4000 2024-05-09T01:26:13.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.330+0000 7f731564c700 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ab1bbe4000 0x55ab1bbe6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.331+0000 7f72f97fa700 10 client.25031 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.331+0000 7f731564c700 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ab1bbe4000 0x55ab1bbe6420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58029002c07214b server_cookie=9669f24fb1a160f5 in_seq=0 out_seq=0 2024-05-09T01:26:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.331+0000 7f72f97fa700 1 -- 172.21.15.19:0/2367642456 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ab1ba89a70 con 0x55ab1bbe4000 2024-05-09T01:26:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.331+0000 7f72f97fa700 10 client.25031 handle_command_reply: tid=0 2024-05-09T01:26:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.331+0000 7f7326ac7b80 1 client.25031 shutdown 2024-05-09T01:26:13.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 -- 172.21.15.19:0/2367642456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 msgr2=0x55ab1bb80650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:13.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb80650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f730c0046a0 tx=0x7f730c01bf80 comp rx=0 tx=0).stop 2024-05-09T01:26:13.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 -- 172.21.15.19:0/2367642456 shutdown_connections 2024-05-09T01:26:13.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ab1bbe4000 0x55ab1bbe6420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab1bb5f6a0 0x55ab1bb810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab1b9e24a0 0x55ab1bb80650 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f72f97fa700 0 client.25031 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:13.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 --2- 172.21.15.19:0/2367642456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab1bacc160 0x55ab1bb80b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 -- 172.21.15.19:0/2367642456 >> 172.21.15.19:0/2367642456 conn(0x55ab1ba426a0 msgr2=0x55ab1bb62e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:13.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 -- 172.21.15.19:0/2367642456 shutdown_connections 2024-05-09T01:26:13.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 1 -- 172.21.15.19:0/2367642456 wait complete. 2024-05-09T01:26:13.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.332+0000 7f7326ac7b80 20 client.25031 trim_cache size 0 max 16384 2024-05-09T01:26:13.346 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:26:13.350 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T01:26:13.350 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T01:26:13.350 INFO:teuthology.orchestra.run.smithi019.stdout: "5bb75e53-bbd7-432f-9483-060c84275400": { 2024-05-09T01:26:13.350 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T01:26:13.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "5bb75e53-bbd7-432f-9483-060c84275400", 2024-05-09T01:26:13.351 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T01:26:13.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:26:13.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:26:13.351 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:26:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 -- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72fc0b2990 msgr2=0x7f72fc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72fc0b2990 0x7f72fc0b4ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f730c00afd0 tx=0x7f730c00a380 comp rx=0 tx=0).stop 2024-05-09T01:26:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 -- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310108740 msgr2=0x7f73100ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310108740 0x7f73100ffd20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7308007ee0 tx=0x7f730800d3b0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 -- 172.21.15.19:0/2956419729 shutdown_connections 2024-05-09T01:26:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72fc0b2990 0x7f72fc0b4ea0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7310108740 0x7f73100ffd20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73101034c0 0x7f73100ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 --2- 172.21.15.19:0/2956419729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7310107df0 0x7f73100ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 -- 172.21.15.19:0/2956419729 >> 172.21.15.19:0/2956419729 conn(0x7f73100fedc0 msgr2=0x7f731006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 -- 172.21.15.19:0/2956419729 shutdown_connections 2024-05-09T01:26:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:13.333+0000 7f73180b1700 1 -- 172.21.15.19:0/2956419729 wait complete. 2024-05-09T01:26:14.048 INFO:tasks.cephfs.filesystem:scrub status for tag:5bb75e53-bbd7-432f-9483-060c84275400 - {'path': '/', 'tag': '5bb75e53-bbd7-432f-9483-060c84275400', 'options': 'recursive,force'} 2024-05-09T01:26:14.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:14 smithi059 ceph-mon[34351]: pgmap v399: 97 pgs: 97 active+clean; 660 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-09T01:26:14.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:14 smithi019 ceph-mon[30158]: pgmap v399: 97 pgs: 97 active+clean; 660 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-09T01:26:14.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:14 smithi196 ceph-mon[35041]: pgmap v399: 97 pgs: 97 active+clean; 660 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-09T01:26:16.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:16 smithi059 ceph-mon[34351]: pgmap v400: 97 pgs: 97 active+clean; 660 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-09T01:26:16.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:16 smithi019 ceph-mon[30158]: pgmap v400: 97 pgs: 97 active+clean; 660 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-09T01:26:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:16 smithi196 ceph-mon[35041]: pgmap v400: 97 pgs: 97 active+clean; 660 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-09T01:26:18.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:18 smithi059 ceph-mon[34351]: pgmap v401: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T01:26:18.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:18 smithi019 ceph-mon[30158]: pgmap v401: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T01:26:18.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:18 smithi196 ceph-mon[35041]: pgmap v401: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T01:26:19.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:19 smithi196 ceph-mon[35041]: pgmap v402: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T01:26:20.099 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:19 smithi059 ceph-mon[34351]: pgmap v402: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T01:26:20.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:19 smithi019 ceph-mon[30158]: pgmap v402: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T01:26:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.819+0000 7f41e118d640 1 -- 172.21.15.19:0/2242547273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc104380 msgr2=0x7f41dc1067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.819+0000 7f41e118d640 1 --2- 172.21.15.19:0/2242547273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc104380 0x7f41dc1067a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009900 tx=0x7f41cc02a3f0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.820+0000 7f41e118d640 1 -- 172.21.15.19:0/2242547273 shutdown_connections 2024-05-09T01:26:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.820+0000 7f41e118d640 1 --2- 172.21.15.19:0/2242547273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41dc1098e0 0x7f41dc10bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.820+0000 7f41e118d640 1 --2- 172.21.15.19:0/2242547273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41dc106d70 0x7f41dc109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.820+0000 7f41e118d640 1 --2- 172.21.15.19:0/2242547273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc104380 0x7f41dc1067a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.820+0000 7f41e118d640 1 -- 172.21.15.19:0/2242547273 >> 172.21.15.19:0/2242547273 conn(0x7f41dc0fdd40 msgr2=0x7f41dc1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.820+0000 7f41e118d640 1 -- 172.21.15.19:0/2242547273 shutdown_connections 2024-05-09T01:26:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.820+0000 7f41e118d640 1 -- 172.21.15.19:0/2242547273 wait complete. 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41e118d640 1 Processor -- start 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41e118d640 1 -- start start 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41e118d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41dc104380 0x7f41dc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41e118d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc106d70 0x7f41dc19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41e118d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41dc1098e0 0x7f41dc1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41e118d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc10f720 con 0x7f41dc106d70 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41e118d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc10f5a0 con 0x7f41dc104380 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41e118d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc10f8a0 con 0x7f41dc1098e0 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc106d70 0x7f41dc19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc106d70 0x7f41dc19a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46318/0 (socket says 172.21.15.19:46318) 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.821+0000 7f41d259c640 1 -- 172.21.15.19:0/3094737822 learned_addr learned my addr 172.21.15.19:0/3094737822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41dad9d640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41dc104380 0x7f41dc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41db59e640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41dc1098e0 0x7f41dc1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41d259c640 1 -- 172.21.15.19:0/3094737822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41dc1098e0 msgr2=0x7f41dc1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41d259c640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41dc1098e0 0x7f41dc1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41d259c640 1 -- 172.21.15.19:0/3094737822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41dc104380 msgr2=0x7f41dc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41d259c640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41dc104380 0x7f41dc19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41d259c640 1 -- 172.21.15.19:0/3094737822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc1a1e10 con 0x7f41dc106d70 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41db59e640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41dc1098e0 0x7f41dc1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41d259c640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc106d70 0x7f41dc19a8c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41c400d950 tx=0x7f41c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41d8d99640 1 -- 172.21.15.19:0/3094737822 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4017070 con 0x7f41dc106d70 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41e118d640 1 -- 172.21.15.19:0/3094737822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc1a2100 con 0x7f41dc106d70 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41d8d99640 1 -- 172.21.15.19:0/3094737822 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c4009c30 con 0x7f41dc106d70 2024-05-09T01:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41d8d99640 1 -- 172.21.15.19:0/3094737822 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c400f620 con 0x7f41dc106d70 2024-05-09T01:26:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.822+0000 7f41e118d640 1 -- 172.21.15.19:0/3094737822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc1a26a0 con 0x7f41dc106d70 2024-05-09T01:26:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.825+0000 7f41e804a740 1 Processor -- start 2024-05-09T01:26:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.825+0000 7f41e804a740 1 -- start start 2024-05-09T01:26:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.825+0000 7f41d8d99640 1 -- 172.21.15.19:0/3094737822 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f41c400f7c0 con 0x7f41dc106d70 2024-05-09T01:26:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.825+0000 7f41e804a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9ab27500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.825+0000 7f41e804a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab27a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.825+0000 7f41e804a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab2e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.825+0000 7f41e804a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aa9aa8b7c0 con 0x55aa9a7831b0 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.825+0000 7f41e804a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55aa9aa270e0 con 0x55aa9a7624d0 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.825+0000 7f41e804a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aa9aa87780 con 0x55aa9a75a2f0 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9ab27500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9ab27500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46322/0 (socket says 172.21.15.19:46322) 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41db59e640 1 -- 172.21.15.19:0/844383888 learned_addr learned my addr 172.21.15.19:0/844383888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41d259c640 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab2e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41dad9d640 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab27a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41db59e640 1 -- 172.21.15.19:0/844383888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a75a2f0 msgr2=0x55aa9ab27a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41db59e640 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab27a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41db59e640 1 -- 172.21.15.19:0/844383888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 msgr2=0x55aa9ab2e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41db59e640 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab2e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41db59e640 1 -- 172.21.15.19:0/844383888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9aa507c0 con 0x55aa9a7831b0 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41d259c640 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab2e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41d8d99640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41c80b2870 0x7f41c80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41db59e640 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9ab27500 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f41d4008e60 tx=0x7f41d400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed8657f867eecd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41dad9d640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41c80b2870 0x7f41c80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41d159a640 1 -- 172.21.15.19:0/844383888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d400f830 con 0x55aa9a7831b0 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41d8d99640 1 -- 172.21.15.19:0/3094737822 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f41c40e1fd0 con 0x7f41dc106d70 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41d159a640 1 -- 172.21.15.19:0/844383888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41d4018480 con 0x55aa9a7831b0 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.826+0000 7f41d159a640 1 -- 172.21.15.19:0/844383888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d400e610 con 0x55aa9a7831b0 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.827+0000 7f41e804a740 1 -- 172.21.15.19:0/844383888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 msgr2=0x55aa9ab27500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.827+0000 7f41dad9d640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41c80b2870 0x7f41c80b4d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009230 tx=0x7f41cc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.827+0000 7f41e804a740 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9ab27500 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f41d4008e60 tx=0x7f41d400d710 comp rx=0 tx=0).stop 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.827+0000 7f41e804a740 1 -- 172.21.15.19:0/844383888 shutdown_connections 2024-05-09T01:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.827+0000 7f41e804a740 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab2e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.827+0000 7f41e804a740 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab27a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.827+0000 7f41e804a740 1 --2- 172.21.15.19:0/844383888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9ab27500 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.827+0000 7f41e804a740 1 -- 172.21.15.19:0/844383888 >> 172.21.15.19:0/844383888 conn(0x55aa9a777300 msgr2=0x55aa9ab22e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 -- 172.21.15.19:0/844383888 shutdown_connections 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 -- 172.21.15.19:0/844383888 wait complete. 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 -- start start 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 Processor -- start 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab31370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 0x55aa9ab318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7831b0 0x55aa9ab387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aa9ab2f3d0 con 0x55aa9a75a2f0 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55aa9aa129c0 con 0x55aa9a7831b0 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aa9ab2f550 con 0x55aa9a7624d0 2024-05-09T01:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41e804a740 10 client.0 _ll_register_callbacks cb 0x55aa9aa3f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41dad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab31370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41dad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab31370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46332/0 (socket says 172.21.15.19:46332) 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.828+0000 7f41dad9d640 1 -- 172.21.15.19:0/716396134 learned_addr learned my addr 172.21.15.19:0/716396134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41db59e640 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7831b0 0x55aa9ab387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41d259c640 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 0x55aa9ab318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41dad9d640 1 -- 172.21.15.19:0/716396134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 msgr2=0x55aa9ab318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41dad9d640 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 0x55aa9ab318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41dad9d640 1 -- 172.21.15.19:0/716396134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7831b0 msgr2=0x55aa9ab387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41dad9d640 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7831b0 0x55aa9ab387a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41dad9d640 1 -- 172.21.15.19:0/716396134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa9aa391f0 con 0x55aa9a75a2f0 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41dad9d640 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab31370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009900 tx=0x7f41cc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6607543f423303 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41c27fc640 1 -- 172.21.15.19:0/716396134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc035040 con 0x55aa9a75a2f0 2024-05-09T01:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41c27fc640 1 -- 172.21.15.19:0/716396134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f41cc003a00 con 0x55aa9a75a2f0 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41e804a740 1 -- 172.21.15.19:0/716396134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41d4018480 con 0x55aa9a75a2f0 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41c27fc640 1 -- 172.21.15.19:0/716396134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41cc056070 con 0x55aa9a75a2f0 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41e804a740 10 client.15354 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.829+0000 7f41e804a740 1 -- 172.21.15.19:0/716396134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa9a7ee790 con 0x55aa9a75a2f0 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.830+0000 7f41c27fc640 1 -- 172.21.15.19:0/716396134 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41d4018480 con 0x55aa9a75a2f0 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.830+0000 7f41c27fc640 1 -- 172.21.15.19:0/716396134 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f41cc044070 con 0x55aa9a75a2f0 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.830+0000 7f41e804a740 10 client.15354 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.830+0000 7f41e804a740 10 client.15354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.830+0000 7f41e804a740 20 client.15354 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.830+0000 7f41e804a740 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa9a741d80 0x55aa9ab3ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.830+0000 7f41e804a740 4 client.15354 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.830+0000 7f41e804a740 1 -- 172.21.15.19:0/716396134 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa9aa993d0 con 0x55aa9a741d80 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.830+0000 7f41d259c640 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa9a741d80 0x55aa9ab3ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.831+0000 7f41d259c640 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa9a741d80 0x55aa9ab3ebc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf98b9f8a95350c3 server_cookie=fe24f3cf5f757f4e in_seq=0 out_seq=0 2024-05-09T01:26:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.831+0000 7f41c27fc640 10 client.15354 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.831+0000 7f41c27fc640 1 -- 172.21.15.19:0/716396134 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa9aa993d0 con 0x55aa9a741d80 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.831+0000 7f41c27fc640 10 client.15354 handle_command_reply: tid=0 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.831+0000 7f41e804a740 1 client.15354 shutdown 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.831+0000 7f41e804a740 1 -- 172.21.15.19:0/716396134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 msgr2=0x55aa9ab31370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab31370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009900 tx=0x7f41cc02a3f0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 1 -- 172.21.15.19:0/716396134 shutdown_connections 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa9a741d80 0x55aa9ab3ebc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7831b0 0x55aa9ab387a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 0x55aa9ab318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 1 --2- 172.21.15.19:0/716396134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab31370 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41c27fc640 0 client.15354 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 1 -- 172.21.15.19:0/716396134 >> 172.21.15.19:0/716396134 conn(0x55aa9a777300 msgr2=0x55aa9ab21700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 1 -- 172.21.15.19:0/716396134 shutdown_connections 2024-05-09T01:26:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 1 -- 172.21.15.19:0/716396134 wait complete. 2024-05-09T01:26:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.832+0000 7f41e804a740 20 client.15354 trim_cache size 0 max 16384 2024-05-09T01:26:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41e804a740 1 Processor -- start 2024-05-09T01:26:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41e804a740 1 -- start start 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41e804a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab22760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41e804a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab22ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41e804a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7831b0 0x55aa9ab30f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41e804a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1ab430 con 0x55aa9a75a2f0 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41e804a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1acf70 con 0x55aa9a7624d0 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41e804a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1abb00 con 0x55aa9a7831b0 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41dad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab22760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41dad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab22760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46338/0 (socket says 172.21.15.19:46338) 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.847+0000 7f41dad9d640 1 -- 172.21.15.19:0/1873292058 learned_addr learned my addr 172.21.15.19:0/1873292058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41d259c640 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab22ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41db59e640 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7831b0 0x55aa9ab30f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41dad9d640 1 -- 172.21.15.19:0/1873292058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7831b0 msgr2=0x55aa9ab30f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41dad9d640 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7831b0 0x55aa9ab30f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41dad9d640 1 -- 172.21.15.19:0/1873292058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 msgr2=0x55aa9ab22ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41dad9d640 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab22ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41dad9d640 1 -- 172.21.15.19:0/1873292058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9a8655f0 con 0x55aa9a75a2f0 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41d259c640 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab22ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41db59e640 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7831b0 0x55aa9ab30f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41dad9d640 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab22760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f41cc031e30 tx=0x7f41cc004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb338a177c9c3b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41d0d99640 1 -- 172.21.15.19:0/1873292058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc035040 con 0x55aa9a75a2f0 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41d0d99640 1 -- 172.21.15.19:0/1873292058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41cc04b070 con 0x55aa9a75a2f0 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.848+0000 7f41d0d99640 1 -- 172.21.15.19:0/1873292058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc03d590 con 0x55aa9a75a2f0 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 -- 172.21.15.19:0/1873292058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 msgr2=0x55aa9ab22760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab22760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f41cc031e30 tx=0x7f41cc004410 comp rx=0 tx=0).stop 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 -- 172.21.15.19:0/1873292058 shutdown_connections 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7831b0 0x55aa9ab30f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a7624d0 0x55aa9ab22ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 --2- 172.21.15.19:0/1873292058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a75a2f0 0x55aa9ab22760 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 -- 172.21.15.19:0/1873292058 >> 172.21.15.19:0/1873292058 conn(0x55aa9a777300 msgr2=0x55aa9ab38b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 -- 172.21.15.19:0/1873292058 shutdown_connections 2024-05-09T01:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 -- 172.21.15.19:0/1873292058 wait complete. 2024-05-09T01:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 -- start start 2024-05-09T01:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.849+0000 7f41e804a740 1 Processor -- start 2024-05-09T01:26:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41e804a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a75a2f0 0x55aa9aba27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41e804a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 0x55aa9aba2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41e804a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9aba9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41e804a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1ac8a0 con 0x55aa9a7831b0 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41e804a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1ac1d0 con 0x55aa9a75a2f0 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41e804a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1a9410 con 0x55aa9a7624d0 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41e804a740 10 client.0 _ll_register_callbacks cb 0x7f41cc005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9aba9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41db59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9aba9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46340/0 (socket says 172.21.15.19:46340) 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41db59e640 1 -- 172.21.15.19:0/3209271835 learned_addr learned my addr 172.21.15.19:0/3209271835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41d259c640 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 0x55aa9aba2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41dad9d640 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a75a2f0 0x55aa9aba27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41db59e640 1 -- 172.21.15.19:0/3209271835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 msgr2=0x55aa9aba2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41db59e640 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 0x55aa9aba2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41db59e640 1 -- 172.21.15.19:0/3209271835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a75a2f0 msgr2=0x55aa9aba27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41db59e640 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a75a2f0 0x55aa9aba27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41db59e640 1 -- 172.21.15.19:0/3209271835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa9aa4fed0 con 0x55aa9a7831b0 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41dad9d640 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a75a2f0 0x55aa9aba27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41d259c640 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 0x55aa9aba2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41db59e640 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9aba9c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41d4008e60 tx=0x7f41d4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9eeaea8228a691 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.850+0000 7f41c27fc640 1 -- 172.21.15.19:0/3209271835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d40087a0 con 0x55aa9a7831b0 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41c27fc640 1 -- 172.21.15.19:0/3209271835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f41d400f840 con 0x55aa9a7831b0 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41e804a740 1 -- 172.21.15.19:0/3209271835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41cc04b070 con 0x55aa9a7831b0 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41c27fc640 1 -- 172.21.15.19:0/3209271835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41d400fa40 con 0x55aa9a7831b0 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41e804a740 10 client.15360 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41e804a740 1 -- 172.21.15.19:0/3209271835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa9abab2e0 con 0x55aa9a7831b0 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41c27fc640 1 -- 172.21.15.19:0/3209271835 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41cc04b070 con 0x55aa9a7831b0 2024-05-09T01:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41c27fc640 1 -- 172.21.15.19:0/3209271835 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f41d4005850 con 0x55aa9a7831b0 2024-05-09T01:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41e804a740 10 client.15360 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41e804a740 10 client.15360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41e804a740 20 client.15360 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41e804a740 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa9a7171c0 0x55aa9abadb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41e804a740 4 client.15360 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41e804a740 1 -- 172.21.15.19:0/3209271835 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa9aa93240 con 0x55aa9a7171c0 2024-05-09T01:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.851+0000 7f41dad9d640 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa9a7171c0 0x55aa9abadb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.852+0000 7f41c27fc640 10 client.15360 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.852+0000 7f41dad9d640 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa9a7171c0 0x55aa9abadb60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3dc50ed9b30a9a server_cookie=ae71326dd9899dec in_seq=0 out_seq=0 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41c27fc640 1 -- 172.21.15.19:0/3209271835 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55aa9aa93240 con 0x55aa9a7171c0 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41c27fc640 10 client.15360 handle_command_reply: tid=0 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 client.15360 shutdown 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 -- 172.21.15.19:0/3209271835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 msgr2=0x55aa9aba9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9aba9c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41d4008e60 tx=0x7f41d4004f90 comp rx=0 tx=0).stop 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 -- 172.21.15.19:0/3209271835 shutdown_connections 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa9a7171c0 0x55aa9abadb60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa9a7831b0 0x55aa9aba9c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa9a7624d0 0x55aa9aba2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41c27fc640 0 client.15360 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 --2- 172.21.15.19:0/3209271835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa9a75a2f0 0x55aa9aba27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 -- 172.21.15.19:0/3209271835 >> 172.21.15.19:0/3209271835 conn(0x55aa9a777300 msgr2=0x55aa9ab30f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 -- 172.21.15.19:0/3209271835 shutdown_connections 2024-05-09T01:26:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 1 -- 172.21.15.19:0/3209271835 wait complete. 2024-05-09T01:26:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.865+0000 7f41e804a740 20 client.15360 trim_cache size 0 max 16384 2024-05-09T01:26:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:26:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:26:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:26:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 -- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41c80b2870 msgr2=0x7f41c80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41c80b2870 0x7f41c80b4d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009230 tx=0x7f41cc031400 comp rx=0 tx=0).stop 2024-05-09T01:26:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 -- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc106d70 msgr2=0x7f41dc19a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc106d70 0x7f41dc19a8c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f41c400d950 tx=0x7f41c400de10 comp rx=0 tx=0).stop 2024-05-09T01:26:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 -- 172.21.15.19:0/3094737822 shutdown_connections 2024-05-09T01:26:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f41c80b2870 0x7f41c80b4d80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41dc1098e0 0x7f41dc1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41dc106d70 0x7f41dc19a8c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 --2- 172.21.15.19:0/3094737822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41dc104380 0x7f41dc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:21.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 -- 172.21.15.19:0/3094737822 >> 172.21.15.19:0/3094737822 conn(0x7f41dc0fdd40 msgr2=0x7f41dc104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:21.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 -- 172.21.15.19:0/3094737822 shutdown_connections 2024-05-09T01:26:21.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:21.866+0000 7f41e118d640 1 -- 172.21.15.19:0/3094737822 wait complete. 2024-05-09T01:26:22.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:22 smithi059 ceph-mon[34351]: pgmap v403: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-09T01:26:22.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:22 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:26:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:22 smithi019 ceph-mon[30158]: pgmap v403: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-09T01:26:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:22 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:26:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:22 smithi196 ceph-mon[35041]: pgmap v403: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-09T01:26:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:22 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:26:24.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:24 smithi059 ceph-mon[34351]: pgmap v404: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:26:24.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:24 smithi019 ceph-mon[30158]: pgmap v404: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:26:24.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:24 smithi196 ceph-mon[35041]: pgmap v404: 97 pgs: 97 active+clean; 661 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:26:26.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:26 smithi059 ceph-mon[34351]: pgmap v405: 97 pgs: 97 active+clean; 661 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 13 op/s 2024-05-09T01:26:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:26 smithi019 ceph-mon[30158]: pgmap v405: 97 pgs: 97 active+clean; 661 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 13 op/s 2024-05-09T01:26:26.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:26 smithi196 ceph-mon[35041]: pgmap v405: 97 pgs: 97 active+clean; 661 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 13 op/s 2024-05-09T01:26:28.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:28 smithi059 ceph-mon[34351]: pgmap v406: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-05-09T01:26:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:28 smithi019 ceph-mon[30158]: pgmap v406: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-05-09T01:26:28.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:28 smithi196 ceph-mon[35041]: pgmap v406: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-05-09T01:26:30.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:30 smithi059 ceph-mon[34351]: pgmap v407: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-09T01:26:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:30 smithi019 ceph-mon[30158]: pgmap v407: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-09T01:26:30.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:30 smithi196 ceph-mon[35041]: pgmap v407: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-09T01:26:32.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:32 smithi059 ceph-mon[34351]: pgmap v408: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-05-09T01:26:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:32 smithi019 ceph-mon[30158]: pgmap v408: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-05-09T01:26:32.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:32 smithi196 ceph-mon[35041]: pgmap v408: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-05-09T01:26:34.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:34 smithi059 ceph-mon[34351]: pgmap v409: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T01:26:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:34 smithi019 ceph-mon[30158]: pgmap v409: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T01:26:34.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:34 smithi196 ceph-mon[35041]: pgmap v409: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T01:26:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.980+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/2632890765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc109bb0 msgr2=0x7f3bcc10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.980+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/2632890765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc109bb0 0x7f3bcc10bfd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009ca0 tx=0x7f3bc402a8c0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.980+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/2632890765 shutdown_connections 2024-05-09T01:26:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.980+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/2632890765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc109bb0 0x7f3bcc10bfd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.980+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/2632890765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bcc104eb0 0x7f3bcc1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.980+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/2632890765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bcc103c70 0x7f3bcc104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.980+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/2632890765 >> 172.21.15.19:0/2632890765 conn(0x7f3bcc0fee40 msgr2=0x7f3bcc101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.980+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/2632890765 shutdown_connections 2024-05-09T01:26:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.980+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/2632890765 wait complete. 2024-05-09T01:26:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd3ad8640 1 Processor -- start 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd3ad8640 1 -- start start 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd3ad8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bcc103c70 0x7f3bcc19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd3ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc104eb0 0x7f3bcc19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd3ad8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bcc109bb0 0x7f3bcc1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd3ad8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10f780 con 0x7f3bcc104eb0 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd3ad8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10f600 con 0x7f3bcc109bb0 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd3ad8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10f900 con 0x7f3bcc103c70 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd1073640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc104eb0 0x7f3bcc19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd1073640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc104eb0 0x7f3bcc19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46222/0 (socket says 172.21.15.19:46222) 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd1073640 1 -- 172.21.15.19:0/3904815728 learned_addr learned my addr 172.21.15.19:0/3904815728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd1874640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bcc103c70 0x7f3bcc19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd2075640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bcc109bb0 0x7f3bcc1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd1073640 1 -- 172.21.15.19:0/3904815728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bcc103c70 msgr2=0x7f3bcc19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd1073640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bcc103c70 0x7f3bcc19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd1073640 1 -- 172.21.15.19:0/3904815728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bcc109bb0 msgr2=0x7f3bcc1a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd1073640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bcc109bb0 0x7f3bcc1a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.981+0000 7f3bd1073640 1 -- 172.21.15.19:0/3904815728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc1a1e60 con 0x7f3bcc104eb0 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.982+0000 7f3bd1073640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc104eb0 0x7f3bcc19a860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00d950 tx=0x7f3bbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.982+0000 7f3bc2ffd640 1 -- 172.21.15.19:0/3904815728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc017070 con 0x7f3bcc104eb0 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.982+0000 7f3bc2ffd640 1 -- 172.21.15.19:0/3904815728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bbc009c30 con 0x7f3bcc104eb0 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.982+0000 7f3bc2ffd640 1 -- 172.21.15.19:0/3904815728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00f620 con 0x7f3bcc104eb0 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.982+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/3904815728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc1a2150 con 0x7f3bcc104eb0 2024-05-09T01:26:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.982+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/3904815728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1a2650 con 0x7f3bcc104eb0 2024-05-09T01:26:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.985+0000 7f3bda983740 1 Processor -- start 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.985+0000 7f3bda983740 1 -- start start 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.985+0000 7f3bc2ffd640 1 -- 172.21.15.19:0/3904815728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f3bbc010460 con 0x7f3bcc104eb0 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bda983740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d461b0 0x5633350ee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bda983740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350eee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bda983740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 0x5633350f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bda983740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56333504e7c0 con 0x563334d1d2f0 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bda983740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563334fea0e0 con 0x563334d254d0 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bda983740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56333504a780 con 0x563334d461b0 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1874640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350eee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1874640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350eee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46228/0 (socket says 172.21.15.19:46228) 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1874640 1 -- 172.21.15.19:0/2982016353 learned_addr learned my addr 172.21.15.19:0/2982016353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd2075640 1 --2- 172.21.15.19:0/2982016353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d461b0 0x5633350ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1073640 1 --2- 172.21.15.19:0/2982016353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 0x5633350f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1874640 1 -- 172.21.15.19:0/2982016353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d461b0 msgr2=0x5633350ee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1874640 1 --2- 172.21.15.19:0/2982016353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d461b0 0x5633350ee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1874640 1 -- 172.21.15.19:0/2982016353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 msgr2=0x5633350f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1874640 1 --2- 172.21.15.19:0/2982016353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 0x5633350f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1874640 1 -- 172.21.15.19:0/2982016353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633350137c0 con 0x563334d1d2f0 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.986+0000 7f3bd1874640 1 --2- 172.21.15.19:0/2982016353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350eee90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800bc70 tx=0x7f3bc80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ef2ee7ea26b7af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.987+0000 7f3bc2ffd640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3bb80b28f0 0x7f3bb80b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.987+0000 7f3bb7fff640 1 -- 172.21.15.19:0/2982016353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8004220 con 0x563334d1d2f0 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.987+0000 7f3bb7fff640 1 -- 172.21.15.19:0/2982016353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc800be20 con 0x563334d1d2f0 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.987+0000 7f3bb7fff640 1 -- 172.21.15.19:0/2982016353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8004c20 con 0x563334d1d2f0 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.987+0000 7f3bd2075640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3bb80b28f0 0x7f3bb80b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.987+0000 7f3bc2ffd640 1 -- 172.21.15.19:0/3904815728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3bbc0e1f90 con 0x7f3bcc104eb0 2024-05-09T01:26:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 -- 172.21.15.19:0/2982016353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 msgr2=0x5633350eee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bd2075640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3bb80b28f0 0x7f3bb80b4e00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5633350efae0 tx=0x7f3bc4031af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 --2- 172.21.15.19:0/2982016353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350eee90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800bc70 tx=0x7f3bc80107a0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 -- 172.21.15.19:0/2982016353 shutdown_connections 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 --2- 172.21.15.19:0/2982016353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 0x5633350f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 --2- 172.21.15.19:0/2982016353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350eee90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 --2- 172.21.15.19:0/2982016353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d461b0 0x5633350ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 -- 172.21.15.19:0/2982016353 >> 172.21.15.19:0/2982016353 conn(0x563334d3a300 msgr2=0x5633350e5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 -- 172.21.15.19:0/2982016353 shutdown_connections 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 -- 172.21.15.19:0/2982016353 wait complete. 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 -- start start 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.988+0000 7f3bda983740 1 Processor -- start 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bda983740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d1d2f0 0x5633350f8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bda983740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d254d0 0x5633350f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bda983740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d461b0 0x5633350ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bda983740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5633350f67e0 con 0x563334d461b0 2024-05-09T01:26:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bda983740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563334fd59c0 con 0x563334d1d2f0 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bda983740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5633350f6960 con 0x563334d254d0 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bda983740 10 client.0 _ll_register_callbacks cb 0x563335002020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd2075640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d461b0 0x5633350ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd1874640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d1d2f0 0x5633350f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd2075640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d461b0 0x5633350ffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46242/0 (socket says 172.21.15.19:46242) 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd1874640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d1d2f0 0x5633350f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:33742/0 (socket says 172.21.15.19:33742) 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd2075640 1 -- 172.21.15.19:0/1333476600 learned_addr learned my addr 172.21.15.19:0/1333476600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd1073640 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d254d0 0x5633350f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd2075640 1 -- 172.21.15.19:0/1333476600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d254d0 msgr2=0x5633350f8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd2075640 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d254d0 0x5633350f8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd2075640 1 -- 172.21.15.19:0/1333476600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d1d2f0 msgr2=0x5633350f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd2075640 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d1d2f0 0x5633350f8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd2075640 1 -- 172.21.15.19:0/1333476600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563334ffc1f0 con 0x563334d461b0 2024-05-09T01:26:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.989+0000 7f3bd1874640 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d1d2f0 0x5633350f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.990+0000 7f3bd2075640 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d461b0 0x5633350ffab0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009ca0 tx=0x7f3bc402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e59a856c70fe0fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.990+0000 7f3bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.990+0000 7f3bb57fa640 1 -- 172.21.15.19:0/1333476600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4004250 con 0x563334d461b0 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.990+0000 7f3bb57fa640 1 -- 172.21.15.19:0/1333476600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3bc40043f0 con 0x563334d461b0 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.990+0000 7f3bda983740 1 -- 172.21.15.19:0/1333476600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc800be20 con 0x563334d461b0 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.990+0000 7f3bb57fa640 1 -- 172.21.15.19:0/1333476600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3bc40045f0 con 0x563334d461b0 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.990+0000 7f3bda983740 10 client.15372 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.990+0000 7f3bda983740 1 -- 172.21.15.19:0/1333476600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563334db1790 con 0x563334d461b0 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.990+0000 7f3bb57fa640 1 -- 172.21.15.19:0/1333476600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc800be20 con 0x563334d461b0 2024-05-09T01:26:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bb57fa640 1 -- 172.21.15.19:0/1333476600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3bc4051070 con 0x563334d461b0 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bda983740 10 client.15372 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bda983740 10 client.15372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bda983740 20 client.15372 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bda983740 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563334d04d80 0x563335105fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bda983740 4 client.15372 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bda983740 1 -- 172.21.15.19:0/1333476600 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56333505c3d0 con 0x563334d04d80 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bd1874640 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563334d04d80 0x563335105fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bd1874640 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563334d04d80 0x563335105fa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e1012354a2edb7 server_cookie=3f2591dcfa8104f4 in_seq=0 out_seq=0 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.991+0000 7f3bb57fa640 10 client.15372 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bb57fa640 1 -- 172.21.15.19:0/1333476600 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56333505c3d0 con 0x563334d04d80 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bb57fa640 10 client.15372 handle_command_reply: tid=0 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 client.15372 shutdown 2024-05-09T01:26:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 -- 172.21.15.19:0/1333476600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d461b0 msgr2=0x5633350ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d461b0 0x5633350ffab0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009ca0 tx=0x7f3bc402a8c0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 -- 172.21.15.19:0/1333476600 shutdown_connections 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563334d04d80 0x563335105fa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d461b0 0x5633350ffab0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d254d0 0x5633350f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 --2- 172.21.15.19:0/1333476600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d1d2f0 0x5633350f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bb57fa640 0 client.15372 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 -- 172.21.15.19:0/1333476600 >> 172.21.15.19:0/1333476600 conn(0x563334d3a300 msgr2=0x5633350e4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.992+0000 7f3bda983740 1 -- 172.21.15.19:0/1333476600 shutdown_connections 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.993+0000 7f3bda983740 1 -- 172.21.15.19:0/1333476600 wait complete. 2024-05-09T01:26:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:34.993+0000 7f3bda983740 20 client.15372 trim_cache size 0 max 16384 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bda983740 1 Processor -- start 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bda983740 1 -- start start 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bda983740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d04d80 0x5633350f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bda983740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bda983740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 0x5633350e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bda983740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc1a92c0 con 0x563334d1d2f0 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bda983740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc1aae00 con 0x563334d254d0 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bda983740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc1a9990 con 0x563334d04d80 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bd1073640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bd1073640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350e93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46258/0 (socket says 172.21.15.19:46258) 2024-05-09T01:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.008+0000 7f3bd1073640 1 -- 172.21.15.19:0/2586667332 learned_addr learned my addr 172.21.15.19:0/2586667332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bd2075640 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 0x5633350e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bd1874640 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d04d80 0x5633350f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bd1073640 1 -- 172.21.15.19:0/2586667332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d04d80 msgr2=0x5633350f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bd1073640 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d04d80 0x5633350f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bd1073640 1 -- 172.21.15.19:0/2586667332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 msgr2=0x5633350e98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bd1073640 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 0x5633350e98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bd1073640 1 -- 172.21.15.19:0/2586667332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563334e285f0 con 0x563334d1d2f0 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bd1874640 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d04d80 0x5633350f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bd1073640 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350e93a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0a9130 tx=0x7f3bbc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2119b0ff37004bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bb77fe640 1 -- 172.21.15.19:0/2586667332 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc020040 con 0x563334d1d2f0 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bb77fe640 1 -- 172.21.15.19:0/2586667332 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bbc0a8f40 con 0x563334d1d2f0 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.009+0000 7f3bb77fe640 1 -- 172.21.15.19:0/2586667332 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0ad7e0 con 0x563334d1d2f0 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 -- 172.21.15.19:0/2586667332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 msgr2=0x5633350e93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350e93a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0a9130 tx=0x7f3bbc00bf60 comp rx=0 tx=0).stop 2024-05-09T01:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 -- 172.21.15.19:0/2586667332 shutdown_connections 2024-05-09T01:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d254d0 0x5633350e98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d1d2f0 0x5633350e93a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 --2- 172.21.15.19:0/2586667332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d04d80 0x5633350f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 -- 172.21.15.19:0/2586667332 >> 172.21.15.19:0/2586667332 conn(0x563334d3a300 msgr2=0x5633350e8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 -- 172.21.15.19:0/2586667332 shutdown_connections 2024-05-09T01:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 -- 172.21.15.19:0/2586667332 wait complete. 2024-05-09T01:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 -- start start 2024-05-09T01:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.010+0000 7f3bda983740 1 Processor -- start 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bda983740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d04d80 0x5633350fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bda983740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d1d2f0 0x5633350fd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bda983740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d254d0 0x56333516c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bda983740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc1aa730 con 0x563334d254d0 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bda983740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc1aa060 con 0x563334d04d80 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bda983740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc1a72a0 con 0x563334d1d2f0 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bda983740 10 client.0 _ll_register_callbacks cb 0x7f3bc4005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd2075640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d254d0 0x56333516c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd2075640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d254d0 0x56333516c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46272/0 (socket says 172.21.15.19:46272) 2024-05-09T01:26:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd2075640 1 -- 172.21.15.19:0/536579169 learned_addr learned my addr 172.21.15.19:0/536579169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd1874640 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d04d80 0x5633350fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd1073640 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d1d2f0 0x5633350fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd2075640 1 -- 172.21.15.19:0/536579169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d1d2f0 msgr2=0x5633350fd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd2075640 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d1d2f0 0x5633350fd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd2075640 1 -- 172.21.15.19:0/536579169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d04d80 msgr2=0x5633350fd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd2075640 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d04d80 0x5633350fd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd2075640 1 -- 172.21.15.19:0/536579169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563335012ed0 con 0x563334d254d0 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd1874640 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d04d80 0x5633350fd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bd2075640 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d254d0 0x56333516c340 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006010 tx=0x7f3bc40317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d263836f3d097507 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.011+0000 7f3bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bb57fa640 1 -- 172.21.15.19:0/536579169 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4062030 con 0x563334d254d0 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bb57fa640 1 -- 172.21.15.19:0/536579169 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3bc4030b50 con 0x563334d254d0 2024-05-09T01:26:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bda983740 1 -- 172.21.15.19:0/536579169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bbc0a8f40 con 0x563334d254d0 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bb57fa640 1 -- 172.21.15.19:0/536579169 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3bc4030d50 con 0x563334d254d0 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bda983740 10 client.15378 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bda983740 1 -- 172.21.15.19:0/536579169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56333516d9c0 con 0x563334d254d0 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bb57fa640 1 -- 172.21.15.19:0/536579169 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bbc0a8f40 con 0x563334d254d0 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bb57fa640 1 -- 172.21.15.19:0/536579169 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3bc4051040 con 0x563334d254d0 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bda983740 10 client.15378 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bda983740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bda983740 20 client.15378 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bda983740 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563334d461b0 0x5633351703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bda983740 4 client.15378 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.012+0000 7f3bda983740 1 -- 172.21.15.19:0/536579169 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563335056240 con 0x563334d461b0 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.013+0000 7f3bd1874640 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563334d461b0 0x5633351703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.013+0000 7f3bd1874640 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563334d461b0 0x5633351703e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b54a8c058d3f8e4 server_cookie=7a42978d6acefd0e in_seq=0 out_seq=0 2024-05-09T01:26:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.013+0000 7f3bb57fa640 10 client.15378 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bb57fa640 1 -- 172.21.15.19:0/536579169 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563335056240 con 0x563334d461b0 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bb57fa640 10 client.15378 handle_command_reply: tid=0 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 client.15378 shutdown 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 -- 172.21.15.19:0/536579169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d254d0 msgr2=0x56333516c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d254d0 0x56333516c340 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006010 tx=0x7f3bc40317f0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 -- 172.21.15.19:0/536579169 shutdown_connections 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563334d461b0 0x5633351703e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563334d254d0 0x56333516c340 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563334d1d2f0 0x5633350fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bb57fa640 0 client.15378 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 --2- 172.21.15.19:0/536579169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563334d04d80 0x5633350fd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 -- 172.21.15.19:0/536579169 >> 172.21.15.19:0/536579169 conn(0x563334d3a300 msgr2=0x5633350e96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 -- 172.21.15.19:0/536579169 shutdown_connections 2024-05-09T01:26:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 1 -- 172.21.15.19:0/536579169 wait complete. 2024-05-09T01:26:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.017+0000 7f3bda983740 20 client.15378 trim_cache size 0 max 16384 2024-05-09T01:26:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:26:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:26:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3bb80b28f0 msgr2=0x7f3bb80b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3bb80b28f0 0x7f3bb80b4e00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5633350efae0 tx=0x7f3bc4031af0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc104eb0 msgr2=0x7f3bcc19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc104eb0 0x7f3bcc19a860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00d950 tx=0x7f3bbc00de10 comp rx=0 tx=0).stop 2024-05-09T01:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/3904815728 shutdown_connections 2024-05-09T01:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3bb80b28f0 0x7f3bb80b4e00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bcc109bb0 0x7f3bcc1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bcc104eb0 0x7f3bcc19a860 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 --2- 172.21.15.19:0/3904815728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bcc103c70 0x7f3bcc19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/3904815728 >> 172.21.15.19:0/3904815728 conn(0x7f3bcc0fee40 msgr2=0x7f3bcc105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/3904815728 shutdown_connections 2024-05-09T01:26:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:35.018+0000 7f3bd3ad8640 1 -- 172.21.15.19:0/3904815728 wait complete. 2024-05-09T01:26:36.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:36 smithi059 ceph-mon[34351]: pgmap v410: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-09T01:26:36.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:36 smithi019 ceph-mon[30158]: pgmap v410: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-09T01:26:36.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:36 smithi196 ceph-mon[35041]: pgmap v410: 97 pgs: 97 active+clean; 662 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-09T01:26:38.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:38 smithi059 ceph-mon[34351]: pgmap v411: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T01:26:38.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:38 smithi019 ceph-mon[30158]: pgmap v411: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T01:26:38.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:38 smithi196 ceph-mon[35041]: pgmap v411: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T01:26:40.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:40 smithi059 ceph-mon[34351]: pgmap v412: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:26:40.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:40 smithi019 ceph-mon[30158]: pgmap v412: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:26:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:40 smithi196 ceph-mon[35041]: pgmap v412: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:26:42.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:42 smithi059 ceph-mon[34351]: pgmap v413: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:26:42.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:42 smithi019 ceph-mon[30158]: pgmap v413: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:26:42.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:42 smithi196 ceph-mon[35041]: pgmap v413: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:26:44.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:26:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:44.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:44 smithi019 ceph-mon[30158]: pgmap v414: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:26:44.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:44 smithi059 ceph-mon[34351]: pgmap v414: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:26:44.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:44 smithi196 ceph-mon[35041]: pgmap v414: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:26:46.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.128+0000 7fa3bac4a700 1 -- 172.21.15.19:0/2182189792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 msgr2=0x7fa3b4105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:46.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.128+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/2182189792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 0x7fa3b4105890 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009990 tx=0x7fa3a8009ca0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.129+0000 7fa3bac4a700 1 -- 172.21.15.19:0/2182189792 shutdown_connections 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.129+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/2182189792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b410e8f0 0x7fa3b4110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.129+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/2182189792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 0x7fa3b4105890 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.129+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/2182189792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b4105dd0 0x7fa3b410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.129+0000 7fa3bac4a700 1 -- 172.21.15.19:0/2182189792 >> 172.21.15.19:0/2182189792 conn(0x7fa3b40fef30 msgr2=0x7fa3b4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.129+0000 7fa3bac4a700 1 -- 172.21.15.19:0/2182189792 shutdown_connections 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.129+0000 7fa3bac4a700 1 -- 172.21.15.19:0/2182189792 wait complete. 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.129+0000 7fa3bac4a700 1 Processor -- start 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.129+0000 7fa3bac4a700 1 -- start start 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3bac4a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b4105480 0x7fa3b419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3bac4a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105dd0 0x7fa3b419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3bac4a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b410e8f0 0x7fa3b41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3bac4a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4112310 con 0x7fa3b4105dd0 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3bac4a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4112190 con 0x7fa3b4105480 2024-05-09T01:26:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3bac4a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4112490 con 0x7fa3b410e8f0 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105dd0 0x7fa3b419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105dd0 0x7fa3b419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43730/0 (socket says 172.21.15.19:43730) 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b3fff700 1 -- 172.21.15.19:0/684659289 learned_addr learned my addr 172.21.15.19:0/684659289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b91e7700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b410e8f0 0x7fa3b41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b4105480 0x7fa3b419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b3fff700 1 -- 172.21.15.19:0/684659289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b410e8f0 msgr2=0x7fa3b41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b410e8f0 0x7fa3b41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b3fff700 1 -- 172.21.15.19:0/684659289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b4105480 msgr2=0x7fa3b419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b4105480 0x7fa3b419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b3fff700 1 -- 172.21.15.19:0/684659289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b41990a0 con 0x7fa3b4105dd0 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b4105480 0x7fa3b419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.130+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105dd0 0x7fa3b419ed10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000d980 tx=0x7fa3a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.131+0000 7fa3b1ffb700 1 -- 172.21.15.19:0/684659289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0016070 con 0x7fa3b4105dd0 2024-05-09T01:26:46.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.131+0000 7fa3bac4a700 1 -- 172.21.15.19:0/684659289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b4199390 con 0x7fa3b4105dd0 2024-05-09T01:26:46.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.131+0000 7fa3b1ffb700 1 -- 172.21.15.19:0/684659289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a0004830 con 0x7fa3b4105dd0 2024-05-09T01:26:46.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.131+0000 7fa3b1ffb700 1 -- 172.21.15.19:0/684659289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00054c0 con 0x7fa3b4105dd0 2024-05-09T01:26:46.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.131+0000 7fa3bac4a700 1 -- 172.21.15.19:0/684659289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b41a5660 con 0x7fa3b4105dd0 2024-05-09T01:26:46.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.134+0000 7fa3b1ffb700 1 -- 172.21.15.19:0/684659289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fa3a0004220 con 0x7fa3b4105dd0 2024-05-09T01:26:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.134+0000 7fa3b1ffb700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3a40b2910 0x7fa3a40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.134+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3a40b2910 0x7fa3a40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.134+0000 7fa3b1ffb700 1 -- 172.21.15.19:0/684659289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa3a00e22b0 con 0x7fa3b4105dd0 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3c9660b80 1 Processor -- start 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3c9660b80 1 -- start start 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3a40b2910 0x7fa3a40b4e20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800b2e0 tx=0x7fa3a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dff1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dff1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638dfeb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55638df0e0a0 con 0x55638de6e4a0 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55638deb39a0 con 0x55638dee8d00 2024-05-09T01:26:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55638df0a060 con 0x55638dfeb400 2024-05-09T01:26:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dff1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dff1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43740/0 (socket says 172.21.15.19:43740) 2024-05-09T01:26:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3b89e6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dff1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.135+0000 7fa3b3fff700 1 -- 172.21.15.19:0/1928549761 learned_addr learned my addr 172.21.15.19:0/1928549761 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa3b91e7700 1 --2- 172.21.15.19:0/1928549761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638dfeb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa3b3fff700 1 -- 172.21.15.19:0/1928549761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 msgr2=0x55638dfeb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1928549761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638dfeb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa3b3fff700 1 -- 172.21.15.19:0/1928549761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 msgr2=0x55638dff1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1928549761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dff1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa3b3fff700 1 -- 172.21.15.19:0/1928549761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55638dea4500 con 0x55638de6e4a0 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/1928549761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dff1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1928549761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dff1780 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00a5770 tx=0x7fa3a00a8f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e461a97bff5739 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa39effd700 1 -- 172.21.15.19:0/1928549761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00adaa0 con 0x55638de6e4a0 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa39effd700 1 -- 172.21.15.19:0/1928549761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a00b2550 con 0x55638de6e4a0 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.136+0000 7fa39effd700 1 -- 172.21.15.19:0/1928549761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00910e0 con 0x55638de6e4a0 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1928549761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 msgr2=0x55638dff1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1928549761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dff1780 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00a5770 tx=0x7fa3a00a8f70 comp rx=0 tx=0).stop 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1928549761 shutdown_connections 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1928549761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638dfeb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1928549761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dff1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1928549761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dff1780 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1928549761 >> 172.21.15.19:0/1928549761 conn(0x55638de71f70 msgr2=0x55638dfe6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:46.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1928549761 shutdown_connections 2024-05-09T01:26:46.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1928549761 wait complete. 2024-05-09T01:26:46.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.137+0000 7fa3c9660b80 1 -- start start 2024-05-09T01:26:46.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3c9660b80 1 Processor -- start 2024-05-09T01:26:46.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 0x55638dff5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dee8d00 0x55638dff5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dfeb400 0x55638dff5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55638dfec100 con 0x55638dfeb400 2024-05-09T01:26:46.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55638dea8b20 con 0x55638de6e4a0 2024-05-09T01:26:46.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55638e0011d0 con 0x55638dee8d00 2024-05-09T01:26:46.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3c9660b80 10 client.0 _ll_register_callbacks cb 0x55638dd7f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3b91e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dfeb400 0x55638dff5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3b91e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dfeb400 0x55638dff5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43744/0 (socket says 172.21.15.19:43744) 2024-05-09T01:26:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3b91e7700 1 -- 172.21.15.19:0/3544418642 learned_addr learned my addr 172.21.15.19:0/3544418642 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dee8d00 0x55638dff5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.138+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 0x55638dff5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa3b91e7700 1 -- 172.21.15.19:0/3544418642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dee8d00 msgr2=0x55638dff5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa3b91e7700 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dee8d00 0x55638dff5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa3b91e7700 1 -- 172.21.15.19:0/3544418642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 msgr2=0x55638dff5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa3b91e7700 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 0x55638dff5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa3b91e7700 1 -- 172.21.15.19:0/3544418642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638dd89490 con 0x55638dfeb400 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa3b91e7700 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dfeb400 0x55638dff5aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac007f40 tx=0x7fa3ac007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d76f58ca40ae41b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa393fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa393fff700 1 -- 172.21.15.19:0/3544418642 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac01a030 con 0x55638dfeb400 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa393fff700 1 -- 172.21.15.19:0/3544418642 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa3ac0136f0 con 0x55638dfeb400 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa3c9660b80 1 -- 172.21.15.19:0/3544418642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a00b2550 con 0x55638dfeb400 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa393fff700 1 -- 172.21.15.19:0/3544418642 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa3ac025070 con 0x55638dfeb400 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa3c9660b80 10 client.15390 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa3c9660b80 1 -- 172.21.15.19:0/3544418642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55638e002b60 con 0x55638dfeb400 2024-05-09T01:26:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.139+0000 7fa393fff700 1 -- 172.21.15.19:0/3544418642 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a00b2550 con 0x55638dfeb400 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.140+0000 7fa393fff700 1 -- 172.21.15.19:0/3544418642 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa3ac016a40 con 0x55638dfeb400 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.140+0000 7fa3c9660b80 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.140+0000 7fa3c9660b80 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.140+0000 7fa3c9660b80 20 client.15390 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.140+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55638e0051a0 0x55638e0075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.140+0000 7fa3c9660b80 4 client.15390 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.140+0000 7fa3c9660b80 1 -- 172.21.15.19:0/3544418642 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55638df1ba80 con 0x55638e0051a0 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.140+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55638e0051a0 0x55638e0075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.141+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55638e0051a0 0x55638e0075c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c1558910a92f0f server_cookie=d4989b038b730412 in_seq=0 out_seq=0 2024-05-09T01:26:46.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.141+0000 7fa393fff700 10 client.15390 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.141+0000 7fa393fff700 1 -- 172.21.15.19:0/3544418642 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55638df1ba80 con 0x55638e0051a0 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.141+0000 7fa393fff700 10 client.15390 handle_command_reply: tid=0 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 client.15390 shutdown 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 -- 172.21.15.19:0/3544418642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dfeb400 msgr2=0x55638dff5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dfeb400 0x55638dff5aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac007f40 tx=0x7fa3ac007f70 comp rx=0 tx=0).stop 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 -- 172.21.15.19:0/3544418642 shutdown_connections 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55638e0051a0 0x55638e0075c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dfeb400 0x55638dff5aa0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 0x55638dff5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/3544418642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dee8d00 0x55638dff5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 -- 172.21.15.19:0/3544418642 >> 172.21.15.19:0/3544418642 conn(0x55638de71f70 msgr2=0x55638dfdfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa393fff700 0 client.15390 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 -- 172.21.15.19:0/3544418642 shutdown_connections 2024-05-09T01:26:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 1 -- 172.21.15.19:0/3544418642 wait complete. 2024-05-09T01:26:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.142+0000 7fa3c9660b80 20 client.15390 trim_cache size 0 max 16384 2024-05-09T01:26:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.162+0000 7fa3c9660b80 1 Processor -- start 2024-05-09T01:26:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.162+0000 7fa3c9660b80 1 -- start start 2024-05-09T01:26:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 0x55638dfe0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dee8d00 0x55638dfef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638e00cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41acb70 con 0x55638dee8d00 2024-05-09T01:26:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41ae6b0 con 0x55638de6e4a0 2024-05-09T01:26:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41ad240 con 0x55638dfeb400 2024-05-09T01:26:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dee8d00 0x55638dfef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dee8d00 0x55638dfef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43750/0 (socket says 172.21.15.19:43750) 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b3fff700 1 -- 172.21.15.19:0/1142956763 learned_addr learned my addr 172.21.15.19:0/1142956763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/1142956763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 0x55638dfe0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b3fff700 1 -- 172.21.15.19:0/1142956763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 msgr2=0x55638e00cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1142956763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638e00cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b3fff700 1 -- 172.21.15.19:0/1142956763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 msgr2=0x55638dfe0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1142956763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 0x55638dfe0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b3fff700 1 -- 172.21.15.19:0/1142956763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55638dff3590 con 0x55638dee8d00 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/1142956763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 0x55638dfe0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1142956763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dee8d00 0x55638dfef3e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0012040 tx=0x7fa3a0091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b27960c7b3b18cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.163+0000 7fa39e7fc700 1 -- 172.21.15.19:0/1142956763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0004510 con 0x55638dee8d00 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.164+0000 7fa39e7fc700 1 -- 172.21.15.19:0/1142956763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a00adaa0 con 0x55638dee8d00 2024-05-09T01:26:46.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.164+0000 7fa39e7fc700 1 -- 172.21.15.19:0/1142956763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00912f0 con 0x55638dee8d00 2024-05-09T01:26:46.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.164+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1142956763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dee8d00 msgr2=0x55638dfef3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.164+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1142956763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dee8d00 0x55638dfef3e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0012040 tx=0x7fa3a0091040 comp rx=0 tx=0).stop 2024-05-09T01:26:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.164+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1142956763 shutdown_connections 2024-05-09T01:26:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.164+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1142956763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638e00cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.164+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1142956763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638de6e4a0 0x55638dfe0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.164+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1142956763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638dee8d00 0x55638dfef3e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.164+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1142956763 >> 172.21.15.19:0/1142956763 conn(0x55638de71f70 msgr2=0x55638dfeecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:46.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.165+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1142956763 shutdown_connections 2024-05-09T01:26:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.165+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1142956763 wait complete. 2024-05-09T01:26:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.165+0000 7fa3c9660b80 1 -- start start 2024-05-09T01:26:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.165+0000 7fa3c9660b80 1 Processor -- start 2024-05-09T01:26:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.165+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dffcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.165+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dffd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3c9660b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638dffb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41adfe0 con 0x55638de6e4a0 2024-05-09T01:26:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41ad910 con 0x55638dee8d00 2024-05-09T01:26:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3c9660b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41aab10 con 0x55638dfeb400 2024-05-09T01:26:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3c9660b80 10 client.0 _ll_register_callbacks cb 0x55638dd7f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b89e6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dffcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b89e6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dffcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43754/0 (socket says 172.21.15.19:43754) 2024-05-09T01:26:46.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b89e6700 1 -- 172.21.15.19:0/1777101568 learned_addr learned my addr 172.21.15.19:0/1777101568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dffd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b91e7700 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638dffb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b89e6700 1 -- 172.21.15.19:0/1777101568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 msgr2=0x55638dffb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638dffb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b89e6700 1 -- 172.21.15.19:0/1777101568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 msgr2=0x55638dffd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:46.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dffd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b89e6700 1 -- 172.21.15.19:0/1777101568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638dffbae0 con 0x55638de6e4a0 2024-05-09T01:26:46.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dffd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:46.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3b89e6700 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dffcfa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8003ec0 tx=0x7fa3a8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90bde1530164fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:46.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa393fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:46.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa393fff700 1 -- 172.21.15.19:0/1777101568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8005210 con 0x55638de6e4a0 2024-05-09T01:26:46.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa393fff700 1 -- 172.21.15.19:0/1777101568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa3a80053b0 con 0x55638de6e4a0 2024-05-09T01:26:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.166+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1777101568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a00adaa0 con 0x55638de6e4a0 2024-05-09T01:26:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa393fff700 1 -- 172.21.15.19:0/1777101568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa3a80055b0 con 0x55638de6e4a0 2024-05-09T01:26:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa3c9660b80 10 client.15396 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1777101568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55638e070220 con 0x55638de6e4a0 2024-05-09T01:26:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa393fff700 1 -- 172.21.15.19:0/1777101568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a00adaa0 con 0x55638de6e4a0 2024-05-09T01:26:46.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa393fff700 1 -- 172.21.15.19:0/1777101568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa3a8030040 con 0x55638de6e4a0 2024-05-09T01:26:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa3c9660b80 10 client.15396 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa3c9660b80 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa3c9660b80 20 client.15396 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55638e070910 0x55638e072d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa3c9660b80 4 client.15396 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:26:46.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1777101568 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55638df15b20 con 0x55638e070910 2024-05-09T01:26:46.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.167+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55638e070910 0x55638e072d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:46.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa393fff700 10 client.15396 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:46.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3b3fff700 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55638e070910 0x55638e072d30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba86f8e57c642f5c server_cookie=aec68a621e37ed7 in_seq=0 out_seq=0 2024-05-09T01:26:46.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa393fff700 1 -- 172.21.15.19:0/1777101568 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55638df15b20 con 0x55638e070910 2024-05-09T01:26:46.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa393fff700 10 client.15396 handle_command_reply: tid=0 2024-05-09T01:26:46.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 client.15396 shutdown 2024-05-09T01:26:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1777101568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 msgr2=0x55638dffcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dffcfa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8003ec0 tx=0x7fa3a8009310 comp rx=0 tx=0).stop 2024-05-09T01:26:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1777101568 shutdown_connections 2024-05-09T01:26:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55638e070910 0x55638e072d30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638dfeb400 0x55638dffb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55638de6e4a0 0x55638dffcfa0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa393fff700 0 client.15396 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 --2- 172.21.15.19:0/1777101568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55638dee8d00 0x55638dffd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1777101568 >> 172.21.15.19:0/1777101568 conn(0x55638de71f70 msgr2=0x55638dfe0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1777101568 shutdown_connections 2024-05-09T01:26:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.168+0000 7fa3c9660b80 1 -- 172.21.15.19:0/1777101568 wait complete. 2024-05-09T01:26:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3c9660b80 20 client.15396 trim_cache size 0 max 16384 2024-05-09T01:26:46.177 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:26:46.177 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:26:46.177 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:26:46.177 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:26:46.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 -- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3a40b2910 msgr2=0x7fa3a40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:46.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3a40b2910 0x7fa3a40b4e20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800b2e0 tx=0x7fa3a8005fb0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 -- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105dd0 msgr2=0x7fa3b419ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:46.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105dd0 0x7fa3b419ed10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000d980 tx=0x7fa3a000dc90 comp rx=0 tx=0).stop 2024-05-09T01:26:46.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 -- 172.21.15.19:0/684659289 shutdown_connections 2024-05-09T01:26:46.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b410e8f0 0x7fa3b41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b4105480 0x7fa3b419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3a40b2910 0x7fa3a40b4e20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 --2- 172.21.15.19:0/684659289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105dd0 0x7fa3b419ed10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:46.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 -- 172.21.15.19:0/684659289 >> 172.21.15.19:0/684659289 conn(0x7fa3b40fef30 msgr2=0x7fa3b40ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:46.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 -- 172.21.15.19:0/684659289 shutdown_connections 2024-05-09T01:26:46.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:46.169+0000 7fa3bac4a700 1 -- 172.21.15.19:0/684659289 wait complete. 2024-05-09T01:26:46.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:46 smithi019 ceph-mon[30158]: pgmap v415: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:26:46.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:46 smithi059 ceph-mon[34351]: pgmap v415: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:26:46.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:46 smithi196 ceph-mon[35041]: pgmap v415: 97 pgs: 97 active+clean; 663 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:26:46.880 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:26:46.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:26:47.132 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.127+0000 7eff8a605640 1 -- 172.21.15.19:0/114460072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84103d70 msgr2=0x7eff84104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.127+0000 7eff8a605640 1 --2- 172.21.15.19:0/114460072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84103d70 0x7eff84104180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009900 tx=0x7eff6c02a410 comp rx=0 tx=0).stop 2024-05-09T01:26:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.128+0000 7eff8a605640 1 -- 172.21.15.19:0/114460072 shutdown_connections 2024-05-09T01:26:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.128+0000 7eff8a605640 1 --2- 172.21.15.19:0/114460072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff84108a70 0x7eff8410ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.128+0000 7eff8a605640 1 --2- 172.21.15.19:0/114460072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84103d70 0x7eff84104180 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.128+0000 7eff8a605640 1 --2- 172.21.15.19:0/114460072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff84102b30 0x7eff84103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.128+0000 7eff8a605640 1 -- 172.21.15.19:0/114460072 >> 172.21.15.19:0/114460072 conn(0x7eff840fdd00 msgr2=0x7eff841001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.128+0000 7eff8a605640 1 -- 172.21.15.19:0/114460072 shutdown_connections 2024-05-09T01:26:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.128+0000 7eff8a605640 1 -- 172.21.15.19:0/114460072 wait complete. 2024-05-09T01:26:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.128+0000 7eff8a605640 1 Processor -- start 2024-05-09T01:26:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff8a605640 1 -- start start 2024-05-09T01:26:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff8a605640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84102b30 0x7eff84199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff8a605640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff84103d70 0x7eff84199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff8a605640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff84108a70 0x7eff841a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff8a605640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff8410e680 con 0x7eff84102b30 2024-05-09T01:26:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff8a605640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7eff8410e500 con 0x7eff84103d70 2024-05-09T01:26:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff8a605640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff8410e800 con 0x7eff84108a70 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84102b30 0x7eff84199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84102b30 0x7eff84199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43762/0 (socket says 172.21.15.19:43762) 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff83fff640 1 -- 172.21.15.19:0/1774726407 learned_addr learned my addr 172.21.15.19:0/1774726407 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff88ba2640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff84108a70 0x7eff841a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff7bfff640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff84103d70 0x7eff84199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff83fff640 1 -- 172.21.15.19:0/1774726407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff84108a70 msgr2=0x7eff841a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff83fff640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff84108a70 0x7eff841a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff83fff640 1 -- 172.21.15.19:0/1774726407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff84103d70 msgr2=0x7eff84199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.129+0000 7eff83fff640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff84103d70 0x7eff84199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.130+0000 7eff83fff640 1 -- 172.21.15.19:0/1774726407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff84075000 con 0x7eff84102b30 2024-05-09T01:26:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.130+0000 7eff88ba2640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff84108a70 0x7eff841a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.130+0000 7eff83fff640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84102b30 0x7eff84199230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7eff7400d740 tx=0x7eff7400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.130+0000 7eff81ffb640 1 -- 172.21.15.19:0/1774726407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74017070 con 0x7eff84102b30 2024-05-09T01:26:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.130+0000 7eff8a605640 1 -- 172.21.15.19:0/1774726407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff840752f0 con 0x7eff84102b30 2024-05-09T01:26:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.130+0000 7eff81ffb640 1 -- 172.21.15.19:0/1774726407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff7400de50 con 0x7eff84102b30 2024-05-09T01:26:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.130+0000 7eff81ffb640 1 -- 172.21.15.19:0/1774726407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74003c30 con 0x7eff84102b30 2024-05-09T01:26:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.130+0000 7eff8a605640 1 -- 172.21.15.19:0/1774726407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff84075840 con 0x7eff84102b30 2024-05-09T01:26:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff91593740 1 Processor -- start 2024-05-09T01:26:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff91593740 1 -- start start 2024-05-09T01:26:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff81ffb640 1 -- 172.21.15.19:0/1774726407 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7eff74003dd0 con 0x7eff84102b30 2024-05-09T01:26:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff91593740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b05011b0 0x5631b08a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff91593740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04d82f0 0x5631b08a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff91593740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08ac980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff91593740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5631b08097c0 con 0x5631b04e04d0 2024-05-09T01:26:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff91593740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5631b07a50e0 con 0x5631b04d82f0 2024-05-09T01:26:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff91593740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5631b0805780 con 0x5631b05011b0 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08ac980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43776/0 (socket says 172.21.15.19:43776) 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff7bfff640 1 -- 172.21.15.19:0/2209430160 learned_addr learned my addr 172.21.15.19:0/2209430160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff88ba2640 1 --2- 172.21.15.19:0/2209430160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b05011b0 0x5631b08a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.133+0000 7eff83fff640 1 --2- 172.21.15.19:0/2209430160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04d82f0 0x5631b08a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff7bfff640 1 -- 172.21.15.19:0/2209430160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b05011b0 msgr2=0x5631b08a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff7bfff640 1 --2- 172.21.15.19:0/2209430160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b05011b0 0x5631b08a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff7bfff640 1 -- 172.21.15.19:0/2209430160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04d82f0 msgr2=0x5631b08a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff7bfff640 1 --2- 172.21.15.19:0/2209430160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04d82f0 0x5631b08a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff7bfff640 1 -- 172.21.15.19:0/2209430160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631b07ce7c0 con 0x5631b04e04d0 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff81ffb640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff700b2870 0x7eff700b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff7bfff640 1 --2- 172.21.15.19:0/2209430160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08ac980 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009900 tx=0x7eff6c004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea8060d97acb436 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff88ba2640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff700b2870 0x7eff700b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff81ffb640 1 -- 172.21.15.19:0/1774726407 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7eff740e1ed0 con 0x7eff84102b30 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff7a7fc640 1 -- 172.21.15.19:0/2209430160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c035030 con 0x5631b04e04d0 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff7a7fc640 1 -- 172.21.15.19:0/2209430160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6c0054f0 con 0x5631b04e04d0 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.134+0000 7eff7a7fc640 1 -- 172.21.15.19:0/2209430160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0035f0 con 0x5631b04e04d0 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff91593740 1 -- 172.21.15.19:0/2209430160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 msgr2=0x5631b08ac980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff88ba2640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff700b2870 0x7eff700b4d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5631b08a66d0 tx=0x7eff7c009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff91593740 1 --2- 172.21.15.19:0/2209430160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08ac980 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009900 tx=0x7eff6c004170 comp rx=0 tx=0).stop 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff91593740 1 -- 172.21.15.19:0/2209430160 shutdown_connections 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff91593740 1 --2- 172.21.15.19:0/2209430160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08ac980 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff91593740 1 --2- 172.21.15.19:0/2209430160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04d82f0 0x5631b08a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff91593740 1 --2- 172.21.15.19:0/2209430160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b05011b0 0x5631b08a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff91593740 1 -- 172.21.15.19:0/2209430160 >> 172.21.15.19:0/2209430160 conn(0x5631b04f5300 msgr2=0x5631b08a0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff91593740 1 -- 172.21.15.19:0/2209430160 shutdown_connections 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.135+0000 7eff91593740 1 -- 172.21.15.19:0/2209430160 wait complete. 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff91593740 1 -- start start 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff91593740 1 Processor -- start 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff91593740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff91593740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff91593740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 0x5631b08b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff91593740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5631b08ad370 con 0x5631b04e04d0 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff91593740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5631b07909c0 con 0x5631b05011b0 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff91593740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5631b08ad4f0 con 0x5631b04d82f0 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff91593740 10 client.0 _ll_register_callbacks cb 0x5631b07bd020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43778/0 (socket says 172.21.15.19:43778) 2024-05-09T01:26:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff7bfff640 1 -- 172.21.15.19:0/769165491 learned_addr learned my addr 172.21.15.19:0/769165491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.136+0000 7eff88ba2640 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 0x5631b08b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff83fff640 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff7bfff640 1 -- 172.21.15.19:0/769165491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 msgr2=0x5631b08af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff7bfff640 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff7bfff640 1 -- 172.21.15.19:0/769165491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 msgr2=0x5631b08b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff7bfff640 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 0x5631b08b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff7bfff640 1 -- 172.21.15.19:0/769165491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631b07b71f0 con 0x5631b04e04d0 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff7bfff640 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08af710 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7eff6c02a930 tx=0x7eff6c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60aa9f6dc0e8c98d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff637fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff637fe640 1 -- 172.21.15.19:0/769165491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c035030 con 0x5631b04e04d0 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff637fe640 1 -- 172.21.15.19:0/769165491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7eff6c004f10 con 0x5631b04e04d0 2024-05-09T01:26:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff91593740 1 -- 172.21.15.19:0/769165491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6c0054f0 con 0x5631b04e04d0 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff637fe640 1 -- 172.21.15.19:0/769165491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff6c005110 con 0x5631b04e04d0 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff91593740 10 client.15408 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff91593740 1 -- 172.21.15.19:0/769165491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631b056c790 con 0x5631b04e04d0 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.137+0000 7eff637fe640 1 -- 172.21.15.19:0/769165491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6c0054f0 con 0x5631b04e04d0 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.138+0000 7eff637fe640 1 -- 172.21.15.19:0/769165491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7eff6c0316c0 con 0x5631b04e04d0 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.138+0000 7eff91593740 10 client.15408 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.138+0000 7eff91593740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.138+0000 7eff91593740 20 client.15408 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.138+0000 7eff91593740 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5631b04bfd80 0x5631b08bc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.138+0000 7eff91593740 4 client.15408 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.138+0000 7eff91593740 1 -- 172.21.15.19:0/769165491 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631b08173d0 con 0x5631b04bfd80 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.138+0000 7eff83fff640 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5631b04bfd80 0x5631b08bc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.139+0000 7eff637fe640 10 client.15408 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.139+0000 7eff83fff640 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5631b04bfd80 0x5631b08bc730 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c17ae577d4c302 server_cookie=928c7fc9c203b042 in_seq=0 out_seq=0 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.139+0000 7eff637fe640 1 -- 172.21.15.19:0/769165491 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5631b08173d0 con 0x5631b04bfd80 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.139+0000 7eff637fe640 10 client.15408 handle_command_reply: tid=0 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.139+0000 7eff91593740 1 client.15408 shutdown 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 -- 172.21.15.19:0/769165491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 msgr2=0x5631b08af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08af710 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7eff6c02a930 tx=0x7eff6c0048f0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 -- 172.21.15.19:0/769165491 shutdown_connections 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5631b04bfd80 0x5631b08bc730 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 0x5631b08b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08af710 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff637fe640 0 client.15408 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 --2- 172.21.15.19:0/769165491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 -- 172.21.15.19:0/769165491 >> 172.21.15.19:0/769165491 conn(0x5631b04f5300 msgr2=0x5631b08a3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 -- 172.21.15.19:0/769165491 shutdown_connections 2024-05-09T01:26:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 1 -- 172.21.15.19:0/769165491 wait complete. 2024-05-09T01:26:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.140+0000 7eff91593740 20 client.15408 trim_cache size 0 max 16384 2024-05-09T01:26:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff91593740 1 Processor -- start 2024-05-09T01:26:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff91593740 1 -- start start 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff91593740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff91593740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff91593740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 0x5631b08c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff91593740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff841ab450 con 0x5631b04e04d0 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff91593740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7eff841acf90 con 0x5631b05011b0 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff91593740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff841abb20 con 0x5631b04d82f0 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43790/0 (socket says 172.21.15.19:43790) 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.155+0000 7eff7bfff640 1 -- 172.21.15.19:0/1917167341 learned_addr learned my addr 172.21.15.19:0/1917167341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff88ba2640 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 0x5631b08c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff83fff640 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff7bfff640 1 -- 172.21.15.19:0/1917167341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 msgr2=0x5631b08ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff7bfff640 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff7bfff640 1 -- 172.21.15.19:0/1917167341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 msgr2=0x5631b08c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff7bfff640 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 0x5631b08c0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff7bfff640 1 -- 172.21.15.19:0/1917167341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631b05e35f0 con 0x5631b04e04d0 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff83fff640 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff7bfff640 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08addd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7eff6c031ae0 tx=0x7eff6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bea32c449ae4d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff79ffb640 1 -- 172.21.15.19:0/1917167341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c035070 con 0x5631b04e04d0 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff79ffb640 1 -- 172.21.15.19:0/1917167341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6c004f10 con 0x5631b04e04d0 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.156+0000 7eff79ffb640 1 -- 172.21.15.19:0/1917167341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c03bb40 con 0x5631b04e04d0 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 -- 172.21.15.19:0/1917167341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 msgr2=0x5631b08addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08addd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7eff6c031ae0 tx=0x7eff6c000bc0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 -- 172.21.15.19:0/1917167341 shutdown_connections 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b05011b0 0x5631b08c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b04e04d0 0x5631b08addd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 --2- 172.21.15.19:0/1917167341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 -- 172.21.15.19:0/1917167341 >> 172.21.15.19:0/1917167341 conn(0x5631b04f5300 msgr2=0x5631b08a8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 -- 172.21.15.19:0/1917167341 shutdown_connections 2024-05-09T01:26:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 -- 172.21.15.19:0/1917167341 wait complete. 2024-05-09T01:26:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 -- start start 2024-05-09T01:26:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.157+0000 7eff91593740 1 Processor -- start 2024-05-09T01:26:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff91593740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff91593740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04e04d0 0x5631b08b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff91593740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b05011b0 0x5631b0922a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff91593740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff841ac8c0 con 0x5631b05011b0 2024-05-09T01:26:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff91593740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7eff841ac1f0 con 0x5631b04e04d0 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff91593740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff841a9430 con 0x5631b04d82f0 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff91593740 10 client.0 _ll_register_callbacks cb 0x7eff740e58f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff88ba2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b05011b0 0x5631b0922a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff88ba2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b05011b0 0x5631b0922a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43792/0 (socket says 172.21.15.19:43792) 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff88ba2640 1 -- 172.21.15.19:0/3955765640 learned_addr learned my addr 172.21.15.19:0/3955765640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff7bfff640 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04e04d0 0x5631b08b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff83fff640 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff88ba2640 1 -- 172.21.15.19:0/3955765640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 msgr2=0x5631b08b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff88ba2640 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff88ba2640 1 -- 172.21.15.19:0/3955765640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04e04d0 msgr2=0x5631b08b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff88ba2640 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04e04d0 0x5631b08b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff88ba2640 1 -- 172.21.15.19:0/3955765640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631b07cded0 con 0x5631b05011b0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff83fff640 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff7bfff640 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04e04d0 0x5631b08b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff88ba2640 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b05011b0 0x5631b0922a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff7c011d00 tx=0x7eff7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3afacc35648ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.158+0000 7eff637fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff637fe640 1 -- 172.21.15.19:0/3955765640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c02f060 con 0x5631b05011b0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff637fe640 1 -- 172.21.15.19:0/3955765640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7eff7c020d30 con 0x5631b05011b0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff91593740 1 -- 172.21.15.19:0/3955765640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6c004f10 con 0x5631b05011b0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff637fe640 1 -- 172.21.15.19:0/3955765640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff7c035400 con 0x5631b05011b0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff91593740 10 client.15414 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff91593740 1 -- 172.21.15.19:0/3955765640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631b0924100 con 0x5631b05011b0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff637fe640 1 -- 172.21.15.19:0/3955765640 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6c004f10 con 0x5631b05011b0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff637fe640 1 -- 172.21.15.19:0/3955765640 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7eff7c029070 con 0x5631b05011b0 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff91593740 10 client.15414 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff91593740 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:26:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff91593740 20 client.15414 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff91593740 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5631b04951c0 0x5631b0926b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff91593740 4 client.15414 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:26:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.159+0000 7eff91593740 1 -- 172.21.15.19:0/3955765640 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631b0811240 con 0x5631b04951c0 2024-05-09T01:26:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.160+0000 7eff83fff640 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5631b04951c0 0x5631b0926b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.160+0000 7eff83fff640 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5631b04951c0 0x5631b0926b50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c2f926fbdd5d62 server_cookie=399891ceb8b7d137 in_seq=0 out_seq=0 2024-05-09T01:26:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.160+0000 7eff637fe640 10 client.15414 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.168+0000 7eff637fe640 1 -- 172.21.15.19:0/3955765640 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5631b0811240 con 0x5631b04951c0 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.168+0000 7eff637fe640 10 client.15414 handle_command_reply: tid=0 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.168+0000 7eff91593740 1 client.15414 shutdown 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 -- 172.21.15.19:0/3955765640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b05011b0 msgr2=0x5631b0922a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b05011b0 0x5631b0922a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff7c011d00 tx=0x7eff7c004a40 comp rx=0 tx=0).stop 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 -- 172.21.15.19:0/3955765640 shutdown_connections 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5631b04951c0 0x5631b0926b50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631b05011b0 0x5631b0922a10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631b04e04d0 0x5631b08b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 --2- 172.21.15.19:0/3955765640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631b04d82f0 0x5631b08b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff637fe640 0 client.15414 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 -- 172.21.15.19:0/3955765640 >> 172.21.15.19:0/3955765640 conn(0x5631b04f5300 msgr2=0x5631b08ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 -- 172.21.15.19:0/3955765640 shutdown_connections 2024-05-09T01:26:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 1 -- 172.21.15.19:0/3955765640 wait complete. 2024-05-09T01:26:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff91593740 20 client.15414 trim_cache size 0 max 16384 2024-05-09T01:26:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:26:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:26:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:26:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff8a605640 1 -- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff700b2870 msgr2=0x7eff700b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff8a605640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff700b2870 0x7eff700b4d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5631b08a66d0 tx=0x7eff7c009800 comp rx=0 tx=0).stop 2024-05-09T01:26:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.169+0000 7eff8a605640 1 -- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84102b30 msgr2=0x7eff84199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.170+0000 7eff8a605640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84102b30 0x7eff84199230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7eff7400d740 tx=0x7eff7400dc00 comp rx=0 tx=0).stop 2024-05-09T01:26:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.170+0000 7eff8a605640 1 -- 172.21.15.19:0/1774726407 shutdown_connections 2024-05-09T01:26:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.170+0000 7eff8a605640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7eff700b2870 0x7eff700b4d80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.170+0000 7eff8a605640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff84108a70 0x7eff841a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.170+0000 7eff8a605640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff84103d70 0x7eff84199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.170+0000 7eff8a605640 1 --2- 172.21.15.19:0/1774726407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff84102b30 0x7eff84199230 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.170+0000 7eff8a605640 1 -- 172.21.15.19:0/1774726407 >> 172.21.15.19:0/1774726407 conn(0x7eff840fdd00 msgr2=0x7eff84104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.170+0000 7eff8a605640 1 -- 172.21.15.19:0/1774726407 shutdown_connections 2024-05-09T01:26:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:26:48.170+0000 7eff8a605640 1 -- 172.21.15.19:0/1774726407 wait complete. 2024-05-09T01:26:48.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:48 smithi019 ceph-mon[30158]: pgmap v416: 97 pgs: 97 active+clean; 672 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T01:26:48.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:48 smithi059 ceph-mon[34351]: pgmap v416: 97 pgs: 97 active+clean; 672 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T01:26:48.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:48 smithi196 ceph-mon[35041]: pgmap v416: 97 pgs: 97 active+clean; 672 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T01:26:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.785+0000 7f0304449700 1 -- 172.21.15.19:0/1606121344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 msgr2=0x7f02fc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.785+0000 7f0304449700 1 --2- 172.21.15.19:0/1606121344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 0x7f02fc1038d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f02f00099c0 tx=0x7f02f0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.785+0000 7f0304449700 1 -- 172.21.15.19:0/1606121344 shutdown_connections 2024-05-09T01:26:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.785+0000 7f0304449700 1 --2- 172.21.15.19:0/1606121344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc108740 0x7f02fc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.785+0000 7f0304449700 1 --2- 172.21.15.19:0/1606121344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 0x7f02fc1038d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.785+0000 7f0304449700 1 --2- 172.21.15.19:0/1606121344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc107df0 0x7f02fc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.785+0000 7f0304449700 1 -- 172.21.15.19:0/1606121344 >> 172.21.15.19:0/1606121344 conn(0x7f02fc0fedc0 msgr2=0x7f02fc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.785+0000 7f0304449700 1 -- 172.21.15.19:0/1606121344 shutdown_connections 2024-05-09T01:26:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.785+0000 7f0304449700 1 -- 172.21.15.19:0/1606121344 wait complete. 2024-05-09T01:26:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.786+0000 7f0304449700 1 Processor -- start 2024-05-09T01:26:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.786+0000 7f0304449700 1 -- start start 2024-05-09T01:26:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.786+0000 7f0304449700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 0x7f02fc19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.786+0000 7f0304449700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc107df0 0x7f02fc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.786+0000 7f0304449700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc108740 0x7f02fc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.786+0000 7f0304449700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1122d0 con 0x7f02fc1034c0 2024-05-09T01:26:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.786+0000 7f0304449700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc112150 con 0x7f02fc108740 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.786+0000 7f0304449700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc112450 con 0x7f02fc107df0 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.786+0000 7f03021e5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 0x7f02fc19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03021e5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 0x7f02fc19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43816/0 (socket says 172.21.15.19:43816) 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03021e5700 1 -- 172.21.15.19:0/704809605 learned_addr learned my addr 172.21.15.19:0/704809605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03029e6700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc108740 0x7f02fc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03019e4700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc107df0 0x7f02fc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03021e5700 1 -- 172.21.15.19:0/704809605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc107df0 msgr2=0x7f02fc19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03021e5700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc107df0 0x7f02fc19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03021e5700 1 -- 172.21.15.19:0/704809605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc108740 msgr2=0x7f02fc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03021e5700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc108740 0x7f02fc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03021e5700 1 -- 172.21.15.19:0/704809605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc1990e0 con 0x7f02fc1034c0 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03029e6700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc108740 0x7f02fc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f03021e5700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 0x7f02fc19e810 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f02f0006010 tx=0x7f02f0005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f02ef7fe700 1 -- 172.21.15.19:0/704809605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0026070 con 0x7f02fc1034c0 2024-05-09T01:26:48.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc199370 con 0x7f02fc1034c0 2024-05-09T01:26:48.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f02ef7fe700 1 -- 172.21.15.19:0/704809605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f0003c70 con 0x7f02fc1034c0 2024-05-09T01:26:48.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f02ef7fe700 1 -- 172.21.15.19:0/704809605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f000fae0 con 0x7f02fc1034c0 2024-05-09T01:26:48.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.787+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc1a56a0 con 0x7f02fc1034c0 2024-05-09T01:26:48.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.789+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02fc0512c0 con 0x7f02fc1034c0 2024-05-09T01:26:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.790+0000 7f02ef7fe700 1 -- 172.21.15.19:0/704809605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f02f0029070 con 0x7f02fc1034c0 2024-05-09T01:26:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.791+0000 7f02ef7fe700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02e80b2910 0x7f02e80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.791+0000 7f03019e4700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02e80b2910 0x7f02e80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.791+0000 7f02ef7fe700 1 -- 172.21.15.19:0/704809605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f02f00e3290 con 0x7f02fc1034c0 2024-05-09T01:26:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.792+0000 7f03019e4700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02e80b2910 0x7f02e80b4e20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f02fc102360 tx=0x7f02f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:48.795+0000 7f02ef7fe700 1 -- 172.21.15.19:0/704809605 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02f0030080 con 0x7f02fc1034c0 2024-05-09T01:26:49.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.067+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02fc199f60 con 0x7f02fc1034c0 2024-05-09T01:26:49.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.068+0000 7f02ef7fe700 1 -- 172.21.15.19:0/704809605 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f02f0013070 con 0x7f02fc1034c0 2024-05-09T01:26:49.066 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:26:49.066 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02e80b2910 msgr2=0x7f02e80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02e80b2910 0x7f02e80b4e20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f02fc102360 tx=0x7f02f8009260 comp rx=0 tx=0).stop 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 msgr2=0x7f02fc19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 0x7f02fc19e810 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f02f0006010 tx=0x7f02f0005280 comp rx=0 tx=0).stop 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 shutdown_connections 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc108740 0x7f02fc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02e80b2910 0x7f02e80b4e20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02fc1034c0 0x7f02fc19e810 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 --2- 172.21.15.19:0/704809605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc107df0 0x7f02fc19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 >> 172.21.15.19:0/704809605 conn(0x7f02fc0fedc0 msgr2=0x7f02fc1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 shutdown_connections 2024-05-09T01:26:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:49.070+0000 7f0304449700 1 -- 172.21.15.19:0/704809605 wait complete. 2024-05-09T01:26:49.069 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:26:49.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:49 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:26:49.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:49 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:26:49.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:49 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:26:49.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:49 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/704809605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:26:49.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:49 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:26:49.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:49 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:26:49.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:49 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:26:49.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:49 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/704809605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:26:49.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:49 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:26:49.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:49 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:26:49.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:49 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:26:49.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:49 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/704809605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:26:49.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:26:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:50.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:50 smithi019 ceph-mon[30158]: pgmap v417: 97 pgs: 97 active+clean; 672 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T01:26:50.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:26:50.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:26:50.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:26:50.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:50 smithi059 ceph-mon[34351]: pgmap v417: 97 pgs: 97 active+clean; 672 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T01:26:50.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:50 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:26:50.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:50 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:26:50.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:50 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:26:50.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:50 smithi196 ceph-mon[35041]: pgmap v417: 97 pgs: 97 active+clean; 672 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T01:26:50.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:50 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:26:50.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:50 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:26:50.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:50 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:26:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.920+0000 7fdc319ed700 1 -- 172.21.15.19:0/1745440295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c10e920 msgr2=0x7fdc2c110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.920+0000 7fdc319ed700 1 --2- 172.21.15.19:0/1745440295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c10e920 0x7fdc2c110e00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b0a0 tx=0x7fdc2400b3b0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.920+0000 7fdc319ed700 1 -- 172.21.15.19:0/1745440295 shutdown_connections 2024-05-09T01:26:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.920+0000 7fdc319ed700 1 --2- 172.21.15.19:0/1745440295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c10e920 0x7fdc2c110e00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.920+0000 7fdc319ed700 1 --2- 172.21.15.19:0/1745440295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2c1054f0 0x7fdc2c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.920+0000 7fdc319ed700 1 --2- 172.21.15.19:0/1745440295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc2c105e40 0x7fdc2c10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.920+0000 7fdc319ed700 1 -- 172.21.15.19:0/1745440295 >> 172.21.15.19:0/1745440295 conn(0x7fdc2c078e00 msgr2=0x7fdc2c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.920+0000 7fdc319ed700 1 -- 172.21.15.19:0/1745440295 shutdown_connections 2024-05-09T01:26:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.920+0000 7fdc319ed700 1 -- 172.21.15.19:0/1745440295 wait complete. 2024-05-09T01:26:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.921+0000 7fdc319ed700 1 Processor -- start 2024-05-09T01:26:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.921+0000 7fdc319ed700 1 -- start start 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.921+0000 7fdc319ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c1054f0 0x7fdc2c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.921+0000 7fdc319ed700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2c105e40 0x7fdc2c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.921+0000 7fdc319ed700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc2c10e920 0x7fdc2c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.921+0000 7fdc319ed700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c114550 con 0x7fdc2c1054f0 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.921+0000 7fdc319ed700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1143d0 con 0x7fdc2c105e40 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.921+0000 7fdc319ed700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1146d0 con 0x7fdc2c10e920 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c1054f0 0x7fdc2c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c1054f0 0x7fdc2c19aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47536/0 (socket says 172.21.15.19:47536) 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2affd700 1 -- 172.21.15.19:0/448508152 learned_addr learned my addr 172.21.15.19:0/448508152 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2c105e40 0x7fdc2c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc2c10e920 0x7fdc2c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2affd700 1 -- 172.21.15.19:0/448508152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc2c10e920 msgr2=0x7fdc2c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2affd700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc2c10e920 0x7fdc2c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2affd700 1 -- 172.21.15.19:0/448508152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2c105e40 msgr2=0x7fdc2c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2affd700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2c105e40 0x7fdc2c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2affd700 1 -- 172.21.15.19:0/448508152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c19fce0 con 0x7fdc2c1054f0 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc2c10e920 0x7fdc2c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc2affd700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c1054f0 0x7fdc2c19aec0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c9b0 tx=0x7fdc1c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.922+0000 7fdc309eb700 1 -- 172.21.15.19:0/448508152 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c007b00 con 0x7fdc2c1054f0 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.923+0000 7fdc309eb700 1 -- 172.21.15.19:0/448508152 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc1c007ca0 con 0x7fdc2c1054f0 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.923+0000 7fdc319ed700 1 -- 172.21.15.19:0/448508152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c19ffd0 con 0x7fdc2c1054f0 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.923+0000 7fdc309eb700 1 -- 172.21.15.19:0/448508152 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c018660 con 0x7fdc2c1054f0 2024-05-09T01:26:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.923+0000 7fdc319ed700 1 -- 172.21.15.19:0/448508152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c1a0510 con 0x7fdc2c1054f0 2024-05-09T01:26:51.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.925+0000 7fdc309eb700 1 -- 172.21.15.19:0/448508152 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fdc1c004830 con 0x7fdc2c1054f0 2024-05-09T01:26:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.926+0000 7fdc309eb700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc180b2910 0x7fdc180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.926+0000 7fdc309eb700 1 -- 172.21.15.19:0/448508152 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdc1c0e23c0 con 0x7fdc2c1054f0 2024-05-09T01:26:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.926+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc180b2910 0x7fdc180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc40403b80 1 Processor -- start 2024-05-09T01:26:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc40403b80 1 -- start start 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc180b2910 0x7fdc180b4e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c079800 tx=0x7fdc14009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e69c4a0 0x55a32e81fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 0x55a32e819ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc40403b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a32e73bff0 con 0x55a32e69c4a0 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc40403b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a32e6da7f0 con 0x55a32e786160 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc40403b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a32e737fb0 con 0x55a32e8196a0 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc2a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e69c4a0 0x55a32e81fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc2a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e69c4a0 0x55a32e81fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47542/0 (socket says 172.21.15.19:47542) 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc2a7fc700 1 -- 172.21.15.19:0/1911641870 learned_addr learned my addr 172.21.15.19:0/1911641870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc2affd700 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.927+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 0x55a32e819ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc2a7fc700 1 -- 172.21.15.19:0/1911641870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 msgr2=0x55a32e819ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 0x55a32e819ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc2a7fc700 1 -- 172.21.15.19:0/1911641870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 msgr2=0x55a32e81f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc2a7fc700 1 -- 172.21.15.19:0/1911641870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32e6d2370 con 0x55a32e69c4a0 2024-05-09T01:26:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc2affd700 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e69c4a0 0x55a32e81fa30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009fd0 tx=0x7fdc14011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c6d623890fc47b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc21ffb700 1 -- 172.21.15.19:0/1911641870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1401b040 con 0x55a32e69c4a0 2024-05-09T01:26:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc21ffb700 1 -- 172.21.15.19:0/1911641870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc14024070 con 0x55a32e69c4a0 2024-05-09T01:26:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.928+0000 7fdc21ffb700 1 -- 172.21.15.19:0/1911641870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14013720 con 0x55a32e69c4a0 2024-05-09T01:26:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 -- 172.21.15.19:0/1911641870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e69c4a0 msgr2=0x55a32e81fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e69c4a0 0x55a32e81fa30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009fd0 tx=0x7fdc14011000 comp rx=0 tx=0).stop 2024-05-09T01:26:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 -- 172.21.15.19:0/1911641870 shutdown_connections 2024-05-09T01:26:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 0x55a32e819ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 --2- 172.21.15.19:0/1911641870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e69c4a0 0x55a32e81fa30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 -- 172.21.15.19:0/1911641870 >> 172.21.15.19:0/1911641870 conn(0x55a32e6fc6a0 msgr2=0x55a32e814400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 -- 172.21.15.19:0/1911641870 shutdown_connections 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 -- 172.21.15.19:0/1911641870 wait complete. 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.929+0000 7fdc40403b80 1 -- start start 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc40403b80 1 Processor -- start 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 0x55a32e823230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e823770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc40403b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a32e5b7ba0 con 0x55a32e8196a0 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc40403b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a32e6f6150 con 0x55a32e786160 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc40403b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a32e8261d0 con 0x55a32e69c4a0 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc40403b80 10 client.0 _ll_register_callbacks cb 0x55a32e5ad6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47558/0 (socket says 172.21.15.19:47558) 2024-05-09T01:26:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2b7fe700 1 -- 172.21.15.19:0/3894737694 learned_addr learned my addr 172.21.15.19:0/3894737694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2affd700 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 0x55a32e823230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e823770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2b7fe700 1 -- 172.21.15.19:0/3894737694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 msgr2=0x55a32e823230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 0x55a32e823230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2b7fe700 1 -- 172.21.15.19:0/3894737694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 msgr2=0x55a32e823770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e823770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.930+0000 7fdc2b7fe700 1 -- 172.21.15.19:0/3894737694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32e5b7490 con 0x55a32e8196a0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.931+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823cb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b0a0 tx=0x7fdc24004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e96950b25d3a7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.931+0000 7fdc0affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.931+0000 7fdc0affd700 1 -- 172.21.15.19:0/3894737694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24009cf0 con 0x55a32e8196a0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.931+0000 7fdc0affd700 1 -- 172.21.15.19:0/3894737694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdc240079d0 con 0x55a32e8196a0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.931+0000 7fdc40403b80 1 -- 172.21.15.19:0/3894737694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc14024070 con 0x55a32e8196a0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.931+0000 7fdc0affd700 1 -- 172.21.15.19:0/3894737694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc2402c070 con 0x55a32e8196a0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.931+0000 7fdc40403b80 10 client.15432 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.931+0000 7fdc40403b80 1 -- 172.21.15.19:0/3894737694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a32e830b30 con 0x55a32e8196a0 2024-05-09T01:26:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.931+0000 7fdc0affd700 1 -- 172.21.15.19:0/3894737694 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc14024070 con 0x55a32e8196a0 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc0affd700 1 -- 172.21.15.19:0/3894737694 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdc24011070 con 0x55a32e8196a0 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc40403b80 10 client.15432 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc40403b80 10 client.15432 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc40403b80 20 client.15432 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55a32e833170 0x55a32e8355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc40403b80 4 client.15432 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc40403b80 1 -- 172.21.15.19:0/3894737694 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a32e7499d0 con 0x55a32e833170 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc2affd700 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55a32e833170 0x55a32e8355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc0affd700 10 client.15432 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:26:51.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.932+0000 7fdc2affd700 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55a32e833170 0x55a32e8355e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=29a0f28664a19c61 server_cookie=b6861294de46667c in_seq=0 out_seq=0 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc0affd700 1 -- 172.21.15.19:0/3894737694 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a32e7499d0 con 0x55a32e833170 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc0affd700 10 client.15432 handle_command_reply: tid=0 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc40403b80 1 client.15432 shutdown 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc40403b80 1 -- 172.21.15.19:0/3894737694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 msgr2=0x55a32e823cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823cb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b0a0 tx=0x7fdc24004c80 comp rx=0 tx=0).stop 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc40403b80 1 -- 172.21.15.19:0/3894737694 shutdown_connections 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55a32e833170 0x55a32e8355e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc0affd700 0 client.15432 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823cb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 0x55a32e823230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3894737694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e823770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.933+0000 7fdc40403b80 1 -- 172.21.15.19:0/3894737694 >> 172.21.15.19:0/3894737694 conn(0x55a32e6fc6a0 msgr2=0x55a32e8141c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:51.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.934+0000 7fdc40403b80 1 -- 172.21.15.19:0/3894737694 shutdown_connections 2024-05-09T01:26:51.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.934+0000 7fdc40403b80 1 -- 172.21.15.19:0/3894737694 wait complete. 2024-05-09T01:26:51.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.934+0000 7fdc40403b80 20 client.15432 trim_cache size 0 max 16384 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc40403b80 1 Processor -- start 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc40403b80 1 -- start start 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 0x55a32e822550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc40403b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1aec60 con 0x55a32e8196a0 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc40403b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1b07a0 con 0x55a32e786160 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc40403b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1af330 con 0x55a32e69c4a0 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47570/0 (socket says 172.21.15.19:47570) 2024-05-09T01:26:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc2b7fe700 1 -- 172.21.15.19:0/1875171476 learned_addr learned my addr 172.21.15.19:0/1875171476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc2affd700 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 0x55a32e822550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.949+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc2b7fe700 1 -- 172.21.15.19:0/1875171476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 msgr2=0x55a32e822550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 0x55a32e822550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc2b7fe700 1 -- 172.21.15.19:0/1875171476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 msgr2=0x55a32e81e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc2b7fe700 1 -- 172.21.15.19:0/1875171476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32e8152e0 con 0x55a32e8196a0 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc2affd700 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 0x55a32e822550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823af0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b000 tx=0x7fdc24009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784eceac1a8b855e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc217fa700 1 -- 172.21.15.19:0/1875171476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24035070 con 0x55a32e8196a0 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc217fa700 1 -- 172.21.15.19:0/1875171476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc24007c80 con 0x55a32e8196a0 2024-05-09T01:26:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.950+0000 7fdc217fa700 1 -- 172.21.15.19:0/1875171476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24026b10 con 0x55a32e8196a0 2024-05-09T01:26:51.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 -- 172.21.15.19:0/1875171476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 msgr2=0x55a32e823af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:51.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823af0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b000 tx=0x7fdc24009580 comp rx=0 tx=0).stop 2024-05-09T01:26:51.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 -- 172.21.15.19:0/1875171476 shutdown_connections 2024-05-09T01:26:51.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e8196a0 0x55a32e823af0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e69c4a0 0x55a32e822550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 --2- 172.21.15.19:0/1875171476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e786160 0x55a32e81e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 -- 172.21.15.19:0/1875171476 >> 172.21.15.19:0/1875171476 conn(0x55a32e6fc6a0 msgr2=0x55a32e81ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 -- 172.21.15.19:0/1875171476 shutdown_connections 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 -- 172.21.15.19:0/1875171476 wait complete. 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.951+0000 7fdc40403b80 1 -- start start 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc40403b80 1 Processor -- start 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e69c4a0 0x55a32e82b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e786160 0x55a32e82b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc40403b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 0x55a32e839de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc40403b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1b00d0 con 0x55a32e786160 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc40403b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1afa00 con 0x55a32e69c4a0 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc40403b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1acc00 con 0x55a32e8196a0 2024-05-09T01:26:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc40403b80 10 client.0 _ll_register_callbacks cb 0x55a32e5ad6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e786160 0x55a32e82b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e786160 0x55a32e82b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47584/0 (socket says 172.21.15.19:47584) 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2a7fc700 1 -- 172.21.15.19:0/3828330645 learned_addr learned my addr 172.21.15.19:0/3828330645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2affd700 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e69c4a0 0x55a32e82b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2b7fe700 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 0x55a32e839de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2a7fc700 1 -- 172.21.15.19:0/3828330645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 msgr2=0x55a32e839de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 0x55a32e839de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2a7fc700 1 -- 172.21.15.19:0/3828330645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e69c4a0 msgr2=0x55a32e82b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e69c4a0 0x55a32e82b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.952+0000 7fdc2a7fc700 1 -- 172.21.15.19:0/3828330645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32e83a320 con 0x55a32e786160 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc2a7fc700 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e786160 0x55a32e82b840 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdc140048a0 tx=0x7fdc14015b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e99adfabc7585a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc0affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc0affd700 1 -- 172.21.15.19:0/3828330645 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14036060 con 0x55a32e786160 2024-05-09T01:26:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc0affd700 1 -- 172.21.15.19:0/3828330645 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdc14013660 con 0x55a32e786160 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc40403b80 1 -- 172.21.15.19:0/3828330645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc24007c80 con 0x55a32e786160 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc0affd700 1 -- 172.21.15.19:0/3828330645 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc14006d90 con 0x55a32e786160 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc40403b80 10 client.15438 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc40403b80 1 -- 172.21.15.19:0/3828330645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a32e89cf90 con 0x55a32e786160 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc0affd700 1 -- 172.21.15.19:0/3828330645 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc24007c80 con 0x55a32e786160 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc0affd700 1 -- 172.21.15.19:0/3828330645 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdc1401e030 con 0x55a32e786160 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc40403b80 10 client.15438 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.953+0000 7fdc40403b80 10 client.15438 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc40403b80 20 client.15438 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55a32e89d440 0x55a32e89f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc40403b80 4 client.15438 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:26:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc40403b80 1 -- 172.21.15.19:0/3828330645 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a32e743a70 con 0x55a32e89d440 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc2affd700 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55a32e89d440 0x55a32e89f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc0affd700 10 client.15438 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc2affd700 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55a32e89d440 0x55a32e89f860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de8f4c9529a0187f server_cookie=c16e07c6b7cc29c7 in_seq=0 out_seq=0 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc0affd700 1 -- 172.21.15.19:0/3828330645 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a32e743a70 con 0x55a32e89d440 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc0affd700 10 client.15438 handle_command_reply: tid=0 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.954+0000 7fdc40403b80 1 client.15438 shutdown 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 -- 172.21.15.19:0/3828330645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e786160 msgr2=0x55a32e82b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e786160 0x55a32e82b840 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdc140048a0 tx=0x7fdc14015b20 comp rx=0 tx=0).stop 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 -- 172.21.15.19:0/3828330645 shutdown_connections 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55a32e89d440 0x55a32e89f860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a32e8196a0 0x55a32e839de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a32e69c4a0 0x55a32e82b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 --2- 172.21.15.19:0/3828330645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a32e786160 0x55a32e82b840 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 -- 172.21.15.19:0/3828330645 >> 172.21.15.19:0/3828330645 conn(0x55a32e6fc6a0 msgr2=0x55a32e8222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:51.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc0affd700 0 client.15438 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:26:51.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 -- 172.21.15.19:0/3828330645 shutdown_connections 2024-05-09T01:26:51.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 1 -- 172.21.15.19:0/3828330645 wait complete. 2024-05-09T01:26:51.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.955+0000 7fdc40403b80 20 client.15438 trim_cache size 0 max 16384 2024-05-09T01:26:51.958 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:26:51.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 -- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc180b2910 msgr2=0x7fdc180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:51.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc180b2910 0x7fdc180b4e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c079800 tx=0x7fdc14009190 comp rx=0 tx=0).stop 2024-05-09T01:26:51.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 -- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c1054f0 msgr2=0x7fdc2c19aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:51.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c1054f0 0x7fdc2c19aec0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c9b0 tx=0x7fdc1c00ccc0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 -- 172.21.15.19:0/448508152 shutdown_connections 2024-05-09T01:26:51.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc2c10e920 0x7fdc2c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc180b2910 0x7fdc180b4e20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc2c1054f0 0x7fdc2c19aec0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 --2- 172.21.15.19:0/448508152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2c105e40 0x7fdc2c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:51.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 -- 172.21.15.19:0/448508152 >> 172.21.15.19:0/448508152 conn(0x7fdc2c078e00 msgr2=0x7fdc2c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:51.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 -- 172.21.15.19:0/448508152 shutdown_connections 2024-05-09T01:26:51.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:51.956+0000 7fdc319ed700 1 -- 172.21.15.19:0/448508152 wait complete. 2024-05-09T01:26:52.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:52 smithi019 ceph-mon[30158]: pgmap v418: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T01:26:52.579 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:26:52.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:52 smithi059 ceph-mon[34351]: pgmap v418: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T01:26:52.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:52 smithi196 ceph-mon[35041]: pgmap v418: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T01:26:52.833 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:54.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:54 smithi019 ceph-mon[30158]: pgmap v419: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-09T01:26:54.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:54 smithi059 ceph-mon[34351]: pgmap v419: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-09T01:26:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.606+0000 7f655f36f700 1 -- 172.21.15.19:0/3614858111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 msgr2=0x7f65581049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.606+0000 7f655f36f700 1 --2- 172.21.15.19:0/3614858111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 0x7f65581049f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f65540099c0 tx=0x7f6554009cd0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.606+0000 7f655f36f700 1 -- 172.21.15.19:0/3614858111 shutdown_connections 2024-05-09T01:26:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.606+0000 7f655f36f700 1 --2- 172.21.15.19:0/3614858111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6558109860 0x7f65581111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.606+0000 7f655f36f700 1 --2- 172.21.15.19:0/3614858111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 0x7f65581049f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.606+0000 7f655f36f700 1 --2- 172.21.15.19:0/3614858111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6558108f10 0x7f6558109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.606+0000 7f655f36f700 1 -- 172.21.15.19:0/3614858111 >> 172.21.15.19:0/3614858111 conn(0x7f65580ffe70 msgr2=0x7f65581022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.606+0000 7f655f36f700 1 -- 172.21.15.19:0/3614858111 shutdown_connections 2024-05-09T01:26:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.606+0000 7f655f36f700 1 -- 172.21.15.19:0/3614858111 wait complete. 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655f36f700 1 Processor -- start 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655f36f700 1 -- start start 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655f36f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 0x7f6558075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655f36f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6558108f10 0x7f6558076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655f36f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6558109860 0x7f6558076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655f36f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6558114490 con 0x7f65581045e0 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655f36f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6558114310 con 0x7f6558108f10 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655f36f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6558114610 con 0x7f6558109860 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655d10b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 0x7f6558075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655d10b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 0x7f6558075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47612/0 (socket says 172.21.15.19:47612) 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.607+0000 7f655d10b700 1 -- 172.21.15.19:0/2821017903 learned_addr learned my addr 172.21.15.19:0/2821017903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655c90a700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6558108f10 0x7f6558076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655d10b700 1 -- 172.21.15.19:0/2821017903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6558109860 msgr2=0x7f6558076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655d90c700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6558109860 0x7f6558076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655d10b700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6558109860 0x7f6558076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655d10b700 1 -- 172.21.15.19:0/2821017903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6558108f10 msgr2=0x7f6558076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655d10b700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6558108f10 0x7f6558076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655d10b700 1 -- 172.21.15.19:0/2821017903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65581a91c0 con 0x7f65581045e0 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655d10b700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 0x7f6558075ac0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6554006010 tx=0x7f65540045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f654e7fc700 1 -- 172.21.15.19:0/2821017903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554026070 con 0x7f65581045e0 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655f36f700 1 -- 172.21.15.19:0/2821017903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65581a9450 con 0x7f65581045e0 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f654e7fc700 1 -- 172.21.15.19:0/2821017903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655400f460 con 0x7f65581045e0 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f654e7fc700 1 -- 172.21.15.19:0/2821017903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554009e60 con 0x7f65581045e0 2024-05-09T01:26:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.608+0000 7f655f36f700 1 -- 172.21.15.19:0/2821017903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65581a9900 con 0x7f65581045e0 2024-05-09T01:26:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.611+0000 7f654e7fc700 1 -- 172.21.15.19:0/2821017903 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f6554029070 con 0x7f65581045e0 2024-05-09T01:26:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.612+0000 7f654e7fc700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65440b2910 0x7f65440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.612+0000 7f656dd85b80 1 Processor -- start 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.612+0000 7f654e7fc700 1 -- 172.21.15.19:0/2821017903 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f65540e2c50 con 0x7f65581045e0 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.612+0000 7f655c90a700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65440b2910 0x7f65440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.612+0000 7f656dd85b80 1 -- start start 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383f974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f97a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383f916a0 0x559383f91ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f656dd85b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559383eb3ff0 con 0x559383f916a0 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f656dd85b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559383e527f0 con 0x559383efe160 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f656dd85b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559383eaffb0 con 0x559383e144a0 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655c90a700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65440b2910 0x7f65440b4e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f65581034f0 tx=0x7f654800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655c90a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f97a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655d90c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383f916a0 0x559383f91ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655c90a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f97a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:52426/0 (socket says 172.21.15.19:52426) 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655c90a700 1 -- 172.21.15.19:0/2630380988 learned_addr learned my addr 172.21.15.19:0/2630380988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655d10b700 1 --2- 172.21.15.19:0/2630380988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383f974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655d90c700 1 -- 172.21.15.19:0/2630380988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 msgr2=0x559383f97a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655d90c700 1 --2- 172.21.15.19:0/2630380988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f97a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655d90c700 1 -- 172.21.15.19:0/2630380988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 msgr2=0x559383f974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655d90c700 1 --2- 172.21.15.19:0/2630380988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383f974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.613+0000 7f655d90c700 1 -- 172.21.15.19:0/2630380988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559383e4a370 con 0x559383f916a0 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.614+0000 7f655d90c700 1 --2- 172.21.15.19:0/2630380988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383f916a0 0x559383f91ad0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6550009370 tx=0x7f6550009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8351ba76c8485a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.614+0000 7f65437fe700 1 -- 172.21.15.19:0/2630380988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655001a070 con 0x559383f916a0 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.614+0000 7f65437fe700 1 -- 172.21.15.19:0/2630380988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6550004030 con 0x559383f916a0 2024-05-09T01:26:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.614+0000 7f65437fe700 1 -- 172.21.15.19:0/2630380988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550015650 con 0x559383f916a0 2024-05-09T01:26:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 -- 172.21.15.19:0/2630380988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383f916a0 msgr2=0x559383f91ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 --2- 172.21.15.19:0/2630380988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383f916a0 0x559383f91ad0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6550009370 tx=0x7f6550009920 comp rx=0 tx=0).stop 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 -- 172.21.15.19:0/2630380988 shutdown_connections 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 --2- 172.21.15.19:0/2630380988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383f916a0 0x559383f91ad0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 --2- 172.21.15.19:0/2630380988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383f974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 --2- 172.21.15.19:0/2630380988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f97a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 -- 172.21.15.19:0/2630380988 >> 172.21.15.19:0/2630380988 conn(0x559383e746a0 msgr2=0x559383f8c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 -- 172.21.15.19:0/2630380988 shutdown_connections 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 -- 172.21.15.19:0/2630380988 wait complete. 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.615+0000 7f656dd85b80 1 -- start start 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f656dd85b80 1 Processor -- start 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f9b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383efe160 0x559383f9b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383f916a0 0x559383f9bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f656dd85b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559383d2fba0 con 0x559383efe160 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f656dd85b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559383e6e150 con 0x559383f916a0 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f656dd85b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559383f9e1d0 con 0x559383e144a0 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f656dd85b80 10 client.0 _ll_register_callbacks cb 0x559383d256c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f655c90a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383efe160 0x559383f9b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f655c90a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383efe160 0x559383f9b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47628/0 (socket says 172.21.15.19:47628) 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f655c90a700 1 -- 172.21.15.19:0/542707265 learned_addr learned my addr 172.21.15.19:0/542707265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f655d90c700 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383f916a0 0x559383f9bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.616+0000 7f655d10b700 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f9b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f655c90a700 1 -- 172.21.15.19:0/542707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 msgr2=0x559383f9b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f655c90a700 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f9b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f655c90a700 1 -- 172.21.15.19:0/542707265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383f916a0 msgr2=0x559383f9bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f655c90a700 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383f916a0 0x559383f9bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f655c90a700 1 -- 172.21.15.19:0/542707265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559383d2f490 con 0x559383efe160 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f655d10b700 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f9b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f655d90c700 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383f916a0 0x559383f9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f655c90a700 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383efe160 0x559383f9b6f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6548011720 tx=0x7f65480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fee411c8649580 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f6540ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f6540ff9700 1 -- 172.21.15.19:0/542707265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548004a30 con 0x559383efe160 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f6540ff9700 1 -- 172.21.15.19:0/542707265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6548004bd0 con 0x559383efe160 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f656dd85b80 1 -- 172.21.15.19:0/542707265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6550004030 con 0x559383efe160 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f6540ff9700 1 -- 172.21.15.19:0/542707265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6548004dd0 con 0x559383efe160 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f656dd85b80 10 client.15450 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.617+0000 7f656dd85b80 1 -- 172.21.15.19:0/542707265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559383fa8bf0 con 0x559383efe160 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.618+0000 7f6540ff9700 1 -- 172.21.15.19:0/542707265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6550004030 con 0x559383efe160 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.618+0000 7f6540ff9700 1 -- 172.21.15.19:0/542707265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6548026030 con 0x559383efe160 2024-05-09T01:26:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.618+0000 7f656dd85b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.618+0000 7f656dd85b80 10 client.15450 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:26:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.618+0000 7f656dd85b80 20 client.15450 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.618+0000 7f656dd85b80 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x559383fab280 0x559383fad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.618+0000 7f656dd85b80 4 client.15450 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:26:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.618+0000 7f656dd85b80 1 -- 172.21.15.19:0/542707265 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559383ec19d0 con 0x559383fab280 2024-05-09T01:26:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.618+0000 7f655d10b700 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x559383fab280 0x559383fad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.619+0000 7f655d10b700 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x559383fab280 0x559383fad6a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8836d4d76ec6240f server_cookie=e7f9191405da3bdd in_seq=0 out_seq=0 2024-05-09T01:26:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.619+0000 7f6540ff9700 10 client.15450 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:26:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f6540ff9700 1 -- 172.21.15.19:0/542707265 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559383ec19d0 con 0x559383fab280 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f6540ff9700 10 client.15450 handle_command_reply: tid=0 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 client.15450 shutdown 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 -- 172.21.15.19:0/542707265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383efe160 msgr2=0x559383f9b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383efe160 0x559383f9b6f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6548011720 tx=0x7f65480047c0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 -- 172.21.15.19:0/542707265 shutdown_connections 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x559383fab280 0x559383fad6a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383f916a0 0x559383f9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f6540ff9700 0 client.15450 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383e144a0 0x559383f9b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 --2- 172.21.15.19:0/542707265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383efe160 0x559383f9b6f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 -- 172.21.15.19:0/542707265 >> 172.21.15.19:0/542707265 conn(0x559383e746a0 msgr2=0x559383f8c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 -- 172.21.15.19:0/542707265 shutdown_connections 2024-05-09T01:26:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 1 -- 172.21.15.19:0/542707265 wait complete. 2024-05-09T01:26:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.620+0000 7f656dd85b80 20 client.15450 trim_cache size 0 max 16384 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f656dd85b80 1 Processor -- start 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f656dd85b80 1 -- start start 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383f9ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383f9cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 0x559383fa6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f656dd85b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b0e20 con 0x559383e144a0 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f656dd85b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b2960 con 0x559383efe160 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f656dd85b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b14f0 con 0x559383f916a0 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f655d10b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383f9ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f655d10b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383f9ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47642/0 (socket says 172.21.15.19:47642) 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.636+0000 7f655d10b700 1 -- 172.21.15.19:0/680107300 learned_addr learned my addr 172.21.15.19:0/680107300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f655c90a700 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383f9cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f655d90c700 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 0x559383fa6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f655d10b700 1 -- 172.21.15.19:0/680107300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 msgr2=0x559383fa6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f655d10b700 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 0x559383fa6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f655d10b700 1 -- 172.21.15.19:0/680107300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 msgr2=0x559383f9cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f655d10b700 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383f9cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f655d10b700 1 -- 172.21.15.19:0/680107300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559383f85f60 con 0x559383e144a0 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f655d90c700 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 0x559383fa6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f655d10b700 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383f9ba70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f65540aa2d0 tx=0x7f65540a6310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0e130925d0d1a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f6542ffd700 1 -- 172.21.15.19:0/680107300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655400b730 con 0x559383e144a0 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f6542ffd700 1 -- 172.21.15.19:0/680107300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655400f610 con 0x559383e144a0 2024-05-09T01:26:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.637+0000 7f6542ffd700 1 -- 172.21.15.19:0/680107300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540ae3c0 con 0x559383e144a0 2024-05-09T01:26:54.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.638+0000 7f656dd85b80 1 -- 172.21.15.19:0/680107300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 msgr2=0x559383f9ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:54.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.638+0000 7f656dd85b80 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383f9ba70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f65540aa2d0 tx=0x7f65540a6310 comp rx=0 tx=0).stop 2024-05-09T01:26:54.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.638+0000 7f656dd85b80 1 -- 172.21.15.19:0/680107300 shutdown_connections 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.638+0000 7f656dd85b80 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 0x559383fa6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.638+0000 7f656dd85b80 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383f9ba70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.638+0000 7f656dd85b80 1 --2- 172.21.15.19:0/680107300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383f9cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.638+0000 7f656dd85b80 1 -- 172.21.15.19:0/680107300 >> 172.21.15.19:0/680107300 conn(0x559383e746a0 msgr2=0x559383f9c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.638+0000 7f656dd85b80 1 -- 172.21.15.19:0/680107300 shutdown_connections 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.638+0000 7f656dd85b80 1 -- 172.21.15.19:0/680107300 wait complete. 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.639+0000 7f656dd85b80 1 -- start start 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.639+0000 7f656dd85b80 1 Processor -- start 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.639+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383fb1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.639+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383fb23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.639+0000 7f656dd85b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 0x559383fb2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.639+0000 7f656dd85b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b2290 con 0x559383e144a0 2024-05-09T01:26:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.639+0000 7f656dd85b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65581b1bc0 con 0x559383efe160 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.639+0000 7f656dd85b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65581aedc0 con 0x559383f916a0 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d10b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383fb1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d10b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383fb1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47646/0 (socket says 172.21.15.19:47646) 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d10b700 1 -- 172.21.15.19:0/3695443419 learned_addr learned my addr 172.21.15.19:0/3695443419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f656dd85b80 10 client.0 _ll_register_callbacks cb 0x559383d256c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d90c700 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 0x559383fb2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655c90a700 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383fb23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d10b700 1 -- 172.21.15.19:0/3695443419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 msgr2=0x559383fb2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d10b700 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 0x559383fb2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d10b700 1 -- 172.21.15.19:0/3695443419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 msgr2=0x559383fb23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d10b700 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383fb23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d10b700 1 -- 172.21.15.19:0/3695443419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559384013b10 con 0x559383e144a0 2024-05-09T01:26:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f655d10b700 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383fb1e90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f65540a63c0 tx=0x7f65540057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aec7bc76bb49417 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f6540ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f6540ff9700 1 -- 172.21.15.19:0/3695443419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540c4060 con 0x559383e144a0 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f6540ff9700 1 -- 172.21.15.19:0/3695443419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f65540b2cc0 con 0x559383e144a0 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f656dd85b80 1 -- 172.21.15.19:0/3695443419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f655400f610 con 0x559383e144a0 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.640+0000 7f6540ff9700 1 -- 172.21.15.19:0/3695443419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65540ae160 con 0x559383e144a0 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f656dd85b80 10 client.15456 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f656dd85b80 1 -- 172.21.15.19:0/3695443419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559384014d00 con 0x559383e144a0 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f6540ff9700 1 -- 172.21.15.19:0/3695443419 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655400f610 con 0x559383e144a0 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f6540ff9700 1 -- 172.21.15.19:0/3695443419 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f65540a5cb0 con 0x559383e144a0 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f656dd85b80 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f656dd85b80 10 client.15456 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f656dd85b80 20 client.15456 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f656dd85b80 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5593840152c0 0x5593840176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f656dd85b80 4 client.15456 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f656dd85b80 1 -- 172.21.15.19:0/3695443419 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559383ebba70 con 0x5593840152c0 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.641+0000 7f655c90a700 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5593840152c0 0x5593840176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.642+0000 7f6540ff9700 10 client.15456 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.642+0000 7f655c90a700 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5593840152c0 0x5593840176e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8162144e96fad25 server_cookie=2f859ad29ad92c64 in_seq=0 out_seq=0 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.642+0000 7f6540ff9700 1 -- 172.21.15.19:0/3695443419 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559383ebba70 con 0x5593840152c0 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.642+0000 7f6540ff9700 10 client.15456 handle_command_reply: tid=0 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.642+0000 7f656dd85b80 1 client.15456 shutdown 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.642+0000 7f656dd85b80 1 -- 172.21.15.19:0/3695443419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 msgr2=0x559383fb1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383fb1e90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f65540a63c0 tx=0x7f65540057d0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 1 -- 172.21.15.19:0/3695443419 shutdown_connections 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5593840152c0 0x5593840176e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559383f916a0 0x559383fb2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559383e144a0 0x559383fb1e90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 1 --2- 172.21.15.19:0/3695443419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559383efe160 0x559383fb23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 1 -- 172.21.15.19:0/3695443419 >> 172.21.15.19:0/3695443419 conn(0x559383e746a0 msgr2=0x559383f96a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f6540ff9700 0 client.15456 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:26:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 1 -- 172.21.15.19:0/3695443419 shutdown_connections 2024-05-09T01:26:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 1 -- 172.21.15.19:0/3695443419 wait complete. 2024-05-09T01:26:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f656dd85b80 20 client.15456 trim_cache size 0 max 16384 2024-05-09T01:26:54.645 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:26:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f655f36f700 1 -- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65440b2910 msgr2=0x7f65440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f655f36f700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65440b2910 0x7f65440b4e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f65581034f0 tx=0x7f654800b410 comp rx=0 tx=0).stop 2024-05-09T01:26:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f655f36f700 1 -- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 msgr2=0x7f6558075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.643+0000 7f655f36f700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 0x7f6558075ac0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6554006010 tx=0x7f65540045e0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.644+0000 7f655f36f700 1 -- 172.21.15.19:0/2821017903 shutdown_connections 2024-05-09T01:26:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.644+0000 7f655f36f700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6558109860 0x7f6558076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.644+0000 7f655f36f700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65440b2910 0x7f65440b4e20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.644+0000 7f655f36f700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65581045e0 0x7f6558075ac0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.644+0000 7f655f36f700 1 --2- 172.21.15.19:0/2821017903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6558108f10 0x7f6558076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.644+0000 7f655f36f700 1 -- 172.21.15.19:0/2821017903 >> 172.21.15.19:0/2821017903 conn(0x7f65580ffe70 msgr2=0x7f65581086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.644+0000 7f655f36f700 1 -- 172.21.15.19:0/2821017903 shutdown_connections 2024-05-09T01:26:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:54.644+0000 7f655f36f700 1 -- 172.21.15.19:0/2821017903 wait complete. 2024-05-09T01:26:54.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:54 smithi196 ceph-mon[35041]: pgmap v419: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-09T01:26:55.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:26:55.740 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:56.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:56 smithi019 ceph-mon[30158]: pgmap v420: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T01:26:56.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:56 smithi059 ceph-mon[34351]: pgmap v420: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T01:26:56.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:56 smithi196 ceph-mon[35041]: pgmap v420: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T01:26:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.594+0000 7f8ef8286700 1 -- 172.21.15.19:0/2950128374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0108f10 msgr2=0x7f8ef0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.594+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2950128374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0108f10 0x7f8ef0109320 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00099c0 tx=0x7f8ee0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.594+0000 7f8ef8286700 1 -- 172.21.15.19:0/2950128374 shutdown_connections 2024-05-09T01:26:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.594+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2950128374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef0109860 0x7f8ef01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.595+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2950128374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef01045e0 0x7f8ef01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.595+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2950128374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0108f10 0x7f8ef0109320 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.595+0000 7f8ef8286700 1 -- 172.21.15.19:0/2950128374 >> 172.21.15.19:0/2950128374 conn(0x7f8ef00ffe70 msgr2=0x7f8ef01022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.595+0000 7f8ef8286700 1 -- 172.21.15.19:0/2950128374 shutdown_connections 2024-05-09T01:26:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.595+0000 7f8ef8286700 1 -- 172.21.15.19:0/2950128374 wait complete. 2024-05-09T01:26:57.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.596+0000 7f8ef8286700 1 Processor -- start 2024-05-09T01:26:57.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.596+0000 7f8ef8286700 1 -- start start 2024-05-09T01:26:57.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.596+0000 7f8ef8286700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef01045e0 0x7f8ef019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.596+0000 7f8ef8286700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef0108f10 0x7f8ef019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.596+0000 7f8ef8286700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0109860 0x7f8ef019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.596+0000 7f8ef8286700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0114470 con 0x7f8ef0109860 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.596+0000 7f8ef8286700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01142f0 con 0x7f8ef0108f10 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.596+0000 7f8ef8286700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01145f0 con 0x7f8ef01045e0 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6823700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0109860 0x7f8ef019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6823700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0109860 0x7f8ef019f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47678/0 (socket says 172.21.15.19:47678) 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6823700 1 -- 172.21.15.19:0/2665716433 learned_addr learned my addr 172.21.15.19:0/2665716433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6022700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef01045e0 0x7f8ef019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef5821700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef0108f10 0x7f8ef019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6823700 1 -- 172.21.15.19:0/2665716433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef01045e0 msgr2=0x7f8ef019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6823700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef01045e0 0x7f8ef019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6823700 1 -- 172.21.15.19:0/2665716433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef0108f10 msgr2=0x7f8ef019b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6823700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef0108f10 0x7f8ef019b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6823700 1 -- 172.21.15.19:0/2665716433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef019fd10 con 0x7f8ef0109860 2024-05-09T01:26:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef6823700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0109860 0x7f8ef019f670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800ef30 tx=0x7f8ee800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ee77fe700 1 -- 172.21.15.19:0/2665716433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80092e0 con 0x7f8ef0109860 2024-05-09T01:26:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ee77fe700 1 -- 172.21.15.19:0/2665716433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee8004bb0 con 0x7f8ef0109860 2024-05-09T01:26:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ef8286700 1 -- 172.21.15.19:0/2665716433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef019ffa0 con 0x7f8ef0109860 2024-05-09T01:26:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.597+0000 7f8ee77fe700 1 -- 172.21.15.19:0/2665716433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8004020 con 0x7f8ef0109860 2024-05-09T01:26:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.598+0000 7f8ef8286700 1 -- 172.21.15.19:0/2665716433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef01a0460 con 0x7f8ef0109860 2024-05-09T01:26:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.601+0000 7f8f06c9cb80 1 Processor -- start 2024-05-09T01:26:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.602+0000 7f8ee77fe700 1 -- 172.21.15.19:0/2665716433 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f8ee801b030 con 0x7f8ef0109860 2024-05-09T01:26:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.602+0000 7f8f06c9cb80 1 -- start start 2024-05-09T01:26:57.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.602+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a92a160 0x55906a9c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.602+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a8404a0 0x55906a9c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a9bd6a0 0x55906a9bdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8f06c9cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55906a8dfff0 con 0x55906a8404a0 2024-05-09T01:26:57.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8f06c9cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55906a87e7f0 con 0x55906a92a160 2024-05-09T01:26:57.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8f06c9cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55906a8dbfb0 con 0x55906a9bd6a0 2024-05-09T01:26:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef5821700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a92a160 0x55906a9c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef6022700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a8404a0 0x55906a9c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef5821700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a92a160 0x55906a9c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47236/0 (socket says 172.21.15.19:47236) 2024-05-09T01:26:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef6022700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a8404a0 0x55906a9c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47686/0 (socket says 172.21.15.19:47686) 2024-05-09T01:26:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef5821700 1 -- 172.21.15.19:0/3552549536 learned_addr learned my addr 172.21.15.19:0/3552549536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ee77fe700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8edc0b2910 0x7f8edc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef5821700 1 --2- 172.21.15.19:0/3552549536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a9bd6a0 0x55906a9bdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef6823700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8edc0b2910 0x7f8edc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef5821700 1 -- 172.21.15.19:0/3552549536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a9bd6a0 msgr2=0x55906a9bdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef5821700 1 --2- 172.21.15.19:0/3552549536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a9bd6a0 0x55906a9bdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef5821700 1 -- 172.21.15.19:0/3552549536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a8404a0 msgr2=0x55906a9c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef5821700 1 --2- 172.21.15.19:0/3552549536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a8404a0 0x55906a9c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ef5821700 1 -- 172.21.15.19:0/3552549536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906a876370 con 0x55906a92a160 2024-05-09T01:26:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.603+0000 7f8ee77fe700 1 -- 172.21.15.19:0/2665716433 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8ee80e1e10 con 0x7f8ef0109860 2024-05-09T01:26:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.604+0000 7f8ef5821700 1 --2- 172.21.15.19:0/3552549536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a92a160 0x55906a9c34f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8ef006b520 tx=0x7f8ee0005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3368f97614003624 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.604+0000 7f8ee4d47700 1 -- 172.21.15.19:0/3552549536 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0021040 con 0x55906a92a160 2024-05-09T01:26:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.604+0000 7f8ef6823700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8edc0b2910 0x7f8edc0b4e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8ee801b000 tx=0x7f8ee8018d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.604+0000 7f8ee4d47700 1 -- 172.21.15.19:0/3552549536 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee00055e0 con 0x55906a92a160 2024-05-09T01:26:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.604+0000 7f8ee4d47700 1 -- 172.21.15.19:0/3552549536 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000bbf0 con 0x55906a92a160 2024-05-09T01:26:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.605+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3552549536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a92a160 msgr2=0x55906a9c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.605+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3552549536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a92a160 0x55906a9c34f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8ef006b520 tx=0x7f8ee0005040 comp rx=0 tx=0).stop 2024-05-09T01:26:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.605+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3552549536 shutdown_connections 2024-05-09T01:26:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.605+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3552549536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a9bd6a0 0x55906a9bdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.605+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3552549536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a92a160 0x55906a9c34f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.605+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3552549536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a8404a0 0x55906a9c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.605+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3552549536 >> 172.21.15.19:0/3552549536 conn(0x55906a8a06a0 msgr2=0x55906a9b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.605+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3552549536 shutdown_connections 2024-05-09T01:26:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.605+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3552549536 wait complete. 2024-05-09T01:26:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.606+0000 7f8f06c9cb80 1 -- start start 2024-05-09T01:26:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.606+0000 7f8f06c9cb80 1 Processor -- start 2024-05-09T01:26:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.606+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 0x55906a9c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 0x55906a9c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef5821700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef5821700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47702/0 (socket says 172.21.15.19:47702) 2024-05-09T01:26:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef5821700 1 -- 172.21.15.19:0/2304454073 learned_addr learned my addr 172.21.15.19:0/2304454073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55906a75bba0 con 0x55906a9bd6a0 2024-05-09T01:26:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55906a89a150 con 0x55906a8404a0 2024-05-09T01:26:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef6022700 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 0x55906a9c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55906a9ca1d0 con 0x55906a92a160 2024-05-09T01:26:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef5821700 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 0x55906a9c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef6022700 1 -- 172.21.15.19:0/2304454073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 msgr2=0x55906a9c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef6022700 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 0x55906a9c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef6022700 1 -- 172.21.15.19:0/2304454073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 msgr2=0x55906a9c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef6022700 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8ef6022700 1 -- 172.21.15.19:0/2304454073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906a75b490 con 0x55906a92a160 2024-05-09T01:26:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8f06c9cb80 10 client.0 _ll_register_callbacks cb 0x55906a7516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.607+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee00055e0 con 0x55906a92a160 2024-05-09T01:26:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.608+0000 7f8ef6022700 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 0x55906a9c7750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55906a9b9ed0 tx=0x7f8eec00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d88387f5649b5c0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.608+0000 7f8ed9ffb700 10 client.25162 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T01:26:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.608+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/2304454073 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec004520 con 0x55906a92a160 2024-05-09T01:26:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.608+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/2304454073 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8eec003cd0 con 0x55906a92a160 2024-05-09T01:26:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.608+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/2304454073 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8eec004830 con 0x55906a92a160 2024-05-09T01:26:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.608+0000 7f8f06c9cb80 10 client.25162 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.608+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55906a9d4bb0 con 0x55906a92a160 2024-05-09T01:26:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.609+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/2304454073 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8eec011440 con 0x55906a92a160 2024-05-09T01:26:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.609+0000 7f8f06c9cb80 10 client.25162 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.609+0000 7f8f06c9cb80 10 client.25162 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.609+0000 7f8f06c9cb80 20 client.25162 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.609+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55906a9d7080 0x55906a9d94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.609+0000 7f8f06c9cb80 4 client.25162 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.609+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55906a8ed9d0 con 0x55906a9d7080 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.609+0000 7f8ef5821700 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55906a9d7080 0x55906a9d94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.610+0000 7f8ed9ffb700 10 client.25162 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.610+0000 7f8ef5821700 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55906a9d7080 0x55906a9d94a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5640974ffc55841 server_cookie=d7c2d36710844bd in_seq=0 out_seq=0 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.610+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/2304454073 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55906a8ed9d0 con 0x55906a9d7080 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.610+0000 7f8ed9ffb700 10 client.25162 handle_command_reply: tid=0 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.610+0000 7f8f06c9cb80 1 client.25162 shutdown 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 msgr2=0x55906a9c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 0x55906a9c7750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55906a9b9ed0 tx=0x7f8eec00be20 comp rx=0 tx=0).stop 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 shutdown_connections 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55906a9d7080 0x55906a9d94a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 0x55906a9c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/2304454073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 0x55906a9c7750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 >> 172.21.15.19:0/2304454073 conn(0x55906a8a06a0 msgr2=0x55906a9b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8ed9ffb700 0 client.25162 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 shutdown_connections 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/2304454073 wait complete. 2024-05-09T01:26:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.611+0000 7f8f06c9cb80 20 client.25162 trim_cache size 0 max 16384 2024-05-09T01:26:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.626+0000 7f8f06c9cb80 1 Processor -- start 2024-05-09T01:26:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.626+0000 7f8f06c9cb80 1 -- start start 2024-05-09T01:26:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 0x55906a9bd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 0x55906a9d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8f06c9cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b51a0 con 0x55906a9bd6a0 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8f06c9cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b6ce0 con 0x55906a8404a0 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8f06c9cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b5870 con 0x55906a92a160 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9debf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47710/0 (socket says 172.21.15.19:47710) 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 -- 172.21.15.19:0/1576998424 learned_addr learned my addr 172.21.15.19:0/1576998424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 --2- 172.21.15.19:0/1576998424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 0x55906a9bd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef6022700 1 --2- 172.21.15.19:0/1576998424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 0x55906a9d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 -- 172.21.15.19:0/1576998424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 msgr2=0x55906a9d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 --2- 172.21.15.19:0/1576998424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 0x55906a9d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 -- 172.21.15.19:0/1576998424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 msgr2=0x55906a9bd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 --2- 172.21.15.19:0/1576998424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 0x55906a9bd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 -- 172.21.15.19:0/1576998424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906a9b1f60 con 0x55906a9bd6a0 2024-05-09T01:26:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.627+0000 7f8ef5821700 1 --2- 172.21.15.19:0/1576998424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9debf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000b620 tx=0x7f8ee0005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9067f86af256af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.628+0000 7f8edbfff700 1 -- 172.21.15.19:0/1576998424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000b8d0 con 0x55906a9bd6a0 2024-05-09T01:26:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.628+0000 7f8edbfff700 1 -- 172.21.15.19:0/1576998424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee00037c0 con 0x55906a9bd6a0 2024-05-09T01:26:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.628+0000 7f8edbfff700 1 -- 172.21.15.19:0/1576998424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee003b630 con 0x55906a9bd6a0 2024-05-09T01:26:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/1576998424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 msgr2=0x55906a9debf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/1576998424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9debf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000b620 tx=0x7f8ee0005fb0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/1576998424 shutdown_connections 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/1576998424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a9bd6a0 0x55906a9debf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/1576998424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a8404a0 0x55906a9bd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/1576998424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a92a160 0x55906a9d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/1576998424 >> 172.21.15.19:0/1576998424 conn(0x55906a8a06a0 msgr2=0x55906a9c2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/1576998424 shutdown_connections 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/1576998424 wait complete. 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 -- start start 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.629+0000 7f8f06c9cb80 1 Processor -- start 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a8404a0 0x55906a9cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a92a160 0x55906a9ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8f06c9cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a9bd6a0 0x55906aa3f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8f06c9cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b6610 con 0x55906a92a160 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8f06c9cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b5f40 con 0x55906a9bd6a0 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8f06c9cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01aa920 con 0x55906a8404a0 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8f06c9cb80 10 client.0 _ll_register_callbacks cb 0x55906a7516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef6022700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a92a160 0x55906a9ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef6022700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a92a160 0x55906a9ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47720/0 (socket says 172.21.15.19:47720) 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef6022700 1 -- 172.21.15.19:0/3627325611 learned_addr learned my addr 172.21.15.19:0/3627325611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef5821700 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a9bd6a0 0x55906aa3f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef5821700 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a8404a0 0x55906a9cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef6022700 1 -- 172.21.15.19:0/3627325611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a8404a0 msgr2=0x55906a9cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef6022700 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a8404a0 0x55906a9cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef6022700 1 -- 172.21.15.19:0/3627325611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a9bd6a0 msgr2=0x55906aa3f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:26:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef6022700 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a9bd6a0 0x55906aa3f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef6022700 1 -- 172.21.15.19:0/3627325611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906aa3fce0 con 0x55906a92a160 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ef6022700 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a92a160 0x55906a9ccc00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0043b0 tx=0x7f8eec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8609888d1fa3f4ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.630+0000 7f8ed9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/3627325611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0118f0 con 0x55906a92a160 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/3627325611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8eec011a90 con 0x55906a92a160 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3627325611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee00037c0 con 0x55906a92a160 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/3627325611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8eec011c90 con 0x55906a92a160 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8f06c9cb80 10 client.15474 fetch_fsmap learned FSMap version 28 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3627325611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55906aa40fb0 con 0x55906a92a160 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/3627325611 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee00037c0 con 0x55906a92a160 2024-05-09T01:26:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/3627325611 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8eec02e020 con 0x55906a92a160 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8f06c9cb80 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8f06c9cb80 10 client.15474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8f06c9cb80 20 client.15474 populate_metadata read hostname 'smithi019' 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55906aa416f0 0x55906aa43b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8f06c9cb80 4 client.15474 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.631+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3627325611 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55906a8e7a70 con 0x55906aa416f0 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8ef5821700 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55906aa416f0 0x55906aa43b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8ef5821700 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55906aa416f0 0x55906aa43b10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3684af99a8374bbd server_cookie=5924c344148d9fb1 in_seq=0 out_seq=0 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8ed9ffb700 10 client.15474 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8ed9ffb700 1 -- 172.21.15.19:0/3627325611 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55906a8e7a70 con 0x55906aa416f0 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8ed9ffb700 10 client.15474 handle_command_reply: tid=0 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8f06c9cb80 1 client.15474 shutdown 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3627325611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a92a160 msgr2=0x55906a9ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a92a160 0x55906a9ccc00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0043b0 tx=0x7f8eec004750 comp rx=0 tx=0).stop 2024-05-09T01:26:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3627325611 shutdown_connections 2024-05-09T01:26:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55906aa416f0 0x55906aa43b10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55906a9bd6a0 0x55906aa3f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55906a8404a0 0x55906a9cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8ed9ffb700 0 client.15474 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:26:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8f06c9cb80 1 --2- 172.21.15.19:0/3627325611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55906a92a160 0x55906a9ccc00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.632+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3627325611 >> 172.21.15.19:0/3627325611 conn(0x55906a8a06a0 msgr2=0x55906a9c2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3627325611 shutdown_connections 2024-05-09T01:26:57.635 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:26:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8f06c9cb80 1 -- 172.21.15.19:0/3627325611 wait complete. 2024-05-09T01:26:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8f06c9cb80 20 client.15474 trim_cache size 0 max 16384 2024-05-09T01:26:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 -- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8edc0b2910 msgr2=0x7f8edc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8edc0b2910 0x7f8edc0b4e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8ee801b000 tx=0x7f8ee8018d90 comp rx=0 tx=0).stop 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 -- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0109860 msgr2=0x7f8ef019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0109860 0x7f8ef019f670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800ef30 tx=0x7f8ee800c3a0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 -- 172.21.15.19:0/2665716433 shutdown_connections 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8edc0b2910 0x7f8edc0b4e20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ef0109860 0x7f8ef019f670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef01045e0 0x7f8ef019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 --2- 172.21.15.19:0/2665716433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef0108f10 0x7f8ef019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.633+0000 7f8ef8286700 1 -- 172.21.15.19:0/2665716433 >> 172.21.15.19:0/2665716433 conn(0x7f8ef00ffe70 msgr2=0x7f8ef01086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.634+0000 7f8ef8286700 1 -- 172.21.15.19:0/2665716433 shutdown_connections 2024-05-09T01:26:57.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:26:57.634+0000 7f8ef8286700 1 -- 172.21.15.19:0/2665716433 wait complete. 2024-05-09T01:26:58.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:26:58.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:26:58 smithi059 ceph-mon[34351]: pgmap v421: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T01:26:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:26:58.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:26:58 smithi019 ceph-mon[30158]: pgmap v421: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T01:26:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:26:58 smithi196 ceph-mon[35041]: pgmap v421: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T01:27:00.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:00 smithi019 ceph-mon[30158]: pgmap v422: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-09T01:27:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.537+0000 7f669546b700 1 -- 172.21.15.19:0/2783671556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6690105dd0 msgr2=0x7f669010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.537+0000 7f669546b700 1 --2- 172.21.15.19:0/2783671556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6690105dd0 0x7f669010e320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6678009990 tx=0x7f6678009ca0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.537+0000 7f669546b700 1 -- 172.21.15.19:0/2783671556 shutdown_connections 2024-05-09T01:27:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.537+0000 7f669546b700 1 --2- 172.21.15.19:0/2783671556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f669010e8f0 0x7f6690110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.537+0000 7f669546b700 1 --2- 172.21.15.19:0/2783671556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6690105480 0x7f6690105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.537+0000 7f669546b700 1 --2- 172.21.15.19:0/2783671556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6690105dd0 0x7f669010e320 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.537+0000 7f669546b700 1 -- 172.21.15.19:0/2783671556 >> 172.21.15.19:0/2783671556 conn(0x7f66900fef30 msgr2=0x7f6690101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.538+0000 7f669546b700 1 -- 172.21.15.19:0/2783671556 shutdown_connections 2024-05-09T01:27:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.538+0000 7f669546b700 1 -- 172.21.15.19:0/2783671556 wait complete. 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.538+0000 7f669546b700 1 Processor -- start 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.538+0000 7f669546b700 1 -- start start 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f669546b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6690105480 0x7f6690103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f669546b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6690105dd0 0x7f66901036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f669546b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f669010e8f0 0x7f6690103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f669546b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6690114450 con 0x7f669010e8f0 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f669546b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66901142d0 con 0x7f6690105dd0 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f669546b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66901145d0 con 0x7f6690105480 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f668f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f669010e8f0 0x7f6690103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f668f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f669010e8f0 0x7f6690103bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59378/0 (socket says 172.21.15.19:59378) 2024-05-09T01:27:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f668f7fe700 1 -- 172.21.15.19:0/3580482801 learned_addr learned my addr 172.21.15.19:0/3580482801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f66867fc700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6690105dd0 0x7f66901036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f668effd700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6690105480 0x7f6690103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f668f7fe700 1 -- 172.21.15.19:0/3580482801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6690105480 msgr2=0x7f6690103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f668f7fe700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6690105480 0x7f6690103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f668f7fe700 1 -- 172.21.15.19:0/3580482801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6690105dd0 msgr2=0x7f66901036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f668f7fe700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6690105dd0 0x7f66901036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f668f7fe700 1 -- 172.21.15.19:0/3580482801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66901a90f0 con 0x7f669010e8f0 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.539+0000 7f66867fc700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6690105dd0 0x7f66901036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.540+0000 7f668f7fe700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f669010e8f0 0x7f6690103bf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6688007ee0 tx=0x7f668800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.540+0000 7f668cff9700 1 -- 172.21.15.19:0/3580482801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688004bb0 con 0x7f669010e8f0 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.540+0000 7f669546b700 1 -- 172.21.15.19:0/3580482801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66901a93e0 con 0x7f669010e8f0 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.540+0000 7f668cff9700 1 -- 172.21.15.19:0/3580482801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6688004d50 con 0x7f669010e8f0 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.540+0000 7f668cff9700 1 -- 172.21.15.19:0/3580482801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688005920 con 0x7f669010e8f0 2024-05-09T01:27:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.540+0000 7f669546b700 1 -- 172.21.15.19:0/3580482801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66901a98f0 con 0x7f669010e8f0 2024-05-09T01:27:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.543+0000 7f668cff9700 1 -- 172.21.15.19:0/3580482801 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f668801a020 con 0x7f669010e8f0 2024-05-09T01:27:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.544+0000 7f668cff9700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f667c0b2990 0x7f667c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.544+0000 7f66a3e81b80 1 Processor -- start 2024-05-09T01:27:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.544+0000 7f66a3e81b80 1 -- start start 2024-05-09T01:27:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.544+0000 7f668cff9700 1 -- 172.21.15.19:0/3580482801 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f66880e19a0 con 0x7f669010e8f0 2024-05-09T01:27:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.544+0000 7f668effd700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f667c0b2990 0x7f667c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557840051160 0x5578400ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400eaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578400e46a0 0x5578400e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557840006ff0 con 0x5578400e46a0 2024-05-09T01:27:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55783ffa57f0 con 0x55783ff674a0 2024-05-09T01:27:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557840002fb0 con 0x557840051160 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578400e46a0 0x5578400e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578400e46a0 0x5578400e4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59392/0 (socket says 172.21.15.19:59392) 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668f7fe700 1 -- 172.21.15.19:0/4075493152 learned_addr learned my addr 172.21.15.19:0/4075493152 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668effd700 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557840051160 0x5578400ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668effd700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f667c0b2990 0x7f667c0b4ea0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f668000ac60 tx=0x7f668000f6a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f66867fc700 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400eaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668f7fe700 1 -- 172.21.15.19:0/4075493152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557840051160 msgr2=0x5578400ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668f7fe700 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557840051160 0x5578400ea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668f7fe700 1 -- 172.21.15.19:0/4075493152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 msgr2=0x5578400eaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668f7fe700 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400eaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668f7fe700 1 -- 172.21.15.19:0/4075493152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783ff9d370 con 0x5578400e46a0 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668effd700 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557840051160 0x5578400ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f66867fc700 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.545+0000 7f668f7fe700 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578400e46a0 0x5578400e4ad0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f66880e1c80 tx=0x7f66880a4d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e074fbeec18c98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.546+0000 7f66857fa700 1 -- 172.21.15.19:0/4075493152 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66880ab030 con 0x5578400e46a0 2024-05-09T01:27:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.546+0000 7f66857fa700 1 -- 172.21.15.19:0/4075493152 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66880a9540 con 0x5578400e46a0 2024-05-09T01:27:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.546+0000 7f66857fa700 1 -- 172.21.15.19:0/4075493152 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66880ae980 con 0x5578400e46a0 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4075493152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578400e46a0 msgr2=0x5578400e4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578400e46a0 0x5578400e4ad0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f66880e1c80 tx=0x7f66880a4d80 comp rx=0 tx=0).stop 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4075493152 shutdown_connections 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578400e46a0 0x5578400e4ad0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557840051160 0x5578400ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4075493152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4075493152 >> 172.21.15.19:0/4075493152 conn(0x55783ffc76a0 msgr2=0x5578400df400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4075493152 shutdown_connections 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4075493152 wait complete. 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 -- start start 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.547+0000 7f66a3e81b80 1 Processor -- start 2024-05-09T01:27:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400ee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578400ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 0x5578400eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55783fe82ba0 con 0x557840051160 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55783ffc1150 con 0x55783ff674a0 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578400f11d0 con 0x5578400e46a0 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66a3e81b80 10 client.0 _ll_register_callbacks cb 0x55783fe786c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66867fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578400ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66867fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578400ee7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59398/0 (socket says 172.21.15.19:59398) 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66867fc700 1 -- 172.21.15.19:0/4099445315 learned_addr learned my addr 172.21.15.19:0/4099445315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f668f7fe700 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 0x5578400eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f668effd700 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66867fc700 1 -- 172.21.15.19:0/4099445315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 msgr2=0x5578400eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66867fc700 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 0x5578400eecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66867fc700 1 -- 172.21.15.19:0/4099445315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 msgr2=0x5578400ee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66867fc700 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400ee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f66867fc700 1 -- 172.21.15.19:0/4099445315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783fe82490 con 0x557840051160 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.548+0000 7f668effd700 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66867fc700 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578400ee7b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6678000c00 tx=0x7f6678003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3b8aa535292501 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66767fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66767fc700 1 -- 172.21.15.19:0/4099445315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678003d30 con 0x557840051160 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66767fc700 1 -- 172.21.15.19:0/4099445315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6678005070 con 0x557840051160 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4099445315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66880a9540 con 0x557840051160 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66767fc700 1 -- 172.21.15.19:0/4099445315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6678005270 con 0x557840051160 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66a3e81b80 10 client.15486 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4099445315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578400fbc40 con 0x557840051160 2024-05-09T01:27:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66767fc700 1 -- 172.21.15.19:0/4099445315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66880a9540 con 0x557840051160 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.549+0000 7f66767fc700 1 -- 172.21.15.19:0/4099445315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f667802b400 con 0x557840051160 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.550+0000 7f66a3e81b80 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.550+0000 7f66a3e81b80 10 client.15486 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.550+0000 7f66a3e81b80 20 client.15486 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.550+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5578400fe280 0x5578401006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.550+0000 7f66a3e81b80 4 client.15486 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.550+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4099445315 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578400149d0 con 0x5578400fe280 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.550+0000 7f668effd700 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5578400fe280 0x5578401006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.550+0000 7f668effd700 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5578400fe280 0x5578401006a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7387ebaa72e41d89 server_cookie=bc0e8759cb7a02c in_seq=0 out_seq=0 2024-05-09T01:27:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.550+0000 7f66767fc700 10 client.15486 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:27:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.551+0000 7f66767fc700 1 -- 172.21.15.19:0/4099445315 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578400149d0 con 0x5578400fe280 2024-05-09T01:27:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.551+0000 7f66767fc700 10 client.15486 handle_command_reply: tid=0 2024-05-09T01:27:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.551+0000 7f66a3e81b80 1 client.15486 shutdown 2024-05-09T01:27:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.551+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4099445315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 msgr2=0x5578400ee7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.551+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578400ee7b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6678000c00 tx=0x7f6678003730 comp rx=0 tx=0).stop 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4099445315 shutdown_connections 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5578400fe280 0x5578401006a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 0x5578400eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66767fc700 0 client.15486 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55783ff674a0 0x5578400ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/4099445315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578400ee7b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4099445315 >> 172.21.15.19:0/4099445315 conn(0x55783ffc76a0 msgr2=0x5578400df1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4099445315 shutdown_connections 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66a3e81b80 1 -- 172.21.15.19:0/4099445315 wait complete. 2024-05-09T01:27:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.552+0000 7f66a3e81b80 20 client.15486 trim_cache size 0 max 16384 2024-05-09T01:27:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.567+0000 7f66a3e81b80 1 Processor -- start 2024-05-09T01:27:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.567+0000 7f66a3e81b80 1 -- start start 2024-05-09T01:27:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55783ff674a0 0x5578400e9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557840051160 0x5578400e8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 0x557840105e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66901b0dd0 con 0x55783ff674a0 2024-05-09T01:27:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66901b2910 con 0x557840051160 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66901b14a0 con 0x5578400e46a0 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55783ff674a0 0x5578400e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55783ff674a0 0x5578400e9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59414/0 (socket says 172.21.15.19:59414) 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668effd700 1 -- 172.21.15.19:0/566379241 learned_addr learned my addr 172.21.15.19:0/566379241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f66867fc700 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557840051160 0x5578400e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668f7fe700 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 0x557840105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668effd700 1 -- 172.21.15.19:0/566379241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 msgr2=0x557840105e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668effd700 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 0x557840105e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668effd700 1 -- 172.21.15.19:0/566379241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557840051160 msgr2=0x5578400e8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668effd700 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557840051160 0x5578400e8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668effd700 1 -- 172.21.15.19:0/566379241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578400ea050 con 0x55783ff674a0 2024-05-09T01:27:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668f7fe700 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 0x557840105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:00.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.568+0000 7f668effd700 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55783ff674a0 0x5578400e9570 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6680010430 tx=0x7f6680018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3b5c886984f458 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:00.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.569+0000 7f6684ff9700 1 -- 172.21.15.19:0/566379241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668002d070 con 0x55783ff674a0 2024-05-09T01:27:00.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.569+0000 7f6684ff9700 1 -- 172.21.15.19:0/566379241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66800044e0 con 0x55783ff674a0 2024-05-09T01:27:00.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.569+0000 7f6684ff9700 1 -- 172.21.15.19:0/566379241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680018070 con 0x55783ff674a0 2024-05-09T01:27:00.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 -- 172.21.15.19:0/566379241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55783ff674a0 msgr2=0x5578400e9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:00.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55783ff674a0 0x5578400e9570 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6680010430 tx=0x7f6680018000 comp rx=0 tx=0).stop 2024-05-09T01:27:00.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 -- 172.21.15.19:0/566379241 shutdown_connections 2024-05-09T01:27:00.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578400e46a0 0x557840105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55783ff674a0 0x5578400e9570 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/566379241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557840051160 0x5578400e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 -- 172.21.15.19:0/566379241 >> 172.21.15.19:0/566379241 conn(0x55783ffc76a0 msgr2=0x5578400e7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:00.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 -- 172.21.15.19:0/566379241 shutdown_connections 2024-05-09T01:27:00.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 -- 172.21.15.19:0/566379241 wait complete. 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 -- start start 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.570+0000 7f66a3e81b80 1 Processor -- start 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783ff674a0 0x5578400f6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578401054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66a3e81b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578400e46a0 0x557840105ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66901b2240 con 0x557840051160 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66901b1b70 con 0x5578400e46a0 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66a3e81b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66901aed70 con 0x55783ff674a0 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66867fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578401054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66867fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578401054d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59426/0 (socket says 172.21.15.19:59426) 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66867fc700 1 -- 172.21.15.19:0/2940276052 learned_addr learned my addr 172.21.15.19:0/2940276052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66a3e81b80 10 client.0 _ll_register_callbacks cb 0x55783fe786c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f668effd700 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783ff674a0 0x5578400f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f668f7fe700 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578400e46a0 0x557840105ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66867fc700 1 -- 172.21.15.19:0/2940276052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783ff674a0 msgr2=0x5578400f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66867fc700 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783ff674a0 0x5578400f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66867fc700 1 -- 172.21.15.19:0/2940276052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578400e46a0 msgr2=0x557840105ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66867fc700 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578400e46a0 0x557840105ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66867fc700 1 -- 172.21.15.19:0/2940276052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557840106240 con 0x557840051160 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66867fc700 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578401054d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f667802b870 tx=0x7f6678009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898903e55e9be976 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.571+0000 7f66767fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66767fc700 1 -- 172.21.15.19:0/2940276052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678005410 con 0x557840051160 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66767fc700 1 -- 172.21.15.19:0/2940276052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f66780055b0 con 0x557840051160 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66a3e81b80 1 -- 172.21.15.19:0/2940276052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66800044e0 con 0x557840051160 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66767fc700 1 -- 172.21.15.19:0/2940276052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6678045050 con 0x557840051160 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66a3e81b80 10 client.15492 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66a3e81b80 1 -- 172.21.15.19:0/2940276052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557840168960 con 0x557840051160 2024-05-09T01:27:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66767fc700 1 -- 172.21.15.19:0/2940276052 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66800044e0 con 0x557840051160 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66767fc700 1 -- 172.21.15.19:0/2940276052 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f66780414f0 con 0x557840051160 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66a3e81b80 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66a3e81b80 10 client.15492 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66a3e81b80 20 client.15492 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.572+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5578400fe280 0x55784016b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.573+0000 7f66a3e81b80 4 client.15492 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.573+0000 7f66a3e81b80 1 -- 172.21.15.19:0/2940276052 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55784000ea70 con 0x5578400fe280 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.573+0000 7f668effd700 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5578400fe280 0x55784016b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.573+0000 7f668effd700 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5578400fe280 0x55784016b0a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=752c0413e4d12a37 server_cookie=7afb7219b8140fe9 in_seq=0 out_seq=0 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66767fc700 10 client.15492 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66767fc700 1 -- 172.21.15.19:0/2940276052 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55784000ea70 con 0x5578400fe280 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66767fc700 10 client.15492 handle_command_reply: tid=0 2024-05-09T01:27:00.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 client.15492 shutdown 2024-05-09T01:27:00.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 -- 172.21.15.19:0/2940276052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 msgr2=0x5578401054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578401054d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f667802b870 tx=0x7f6678009f50 comp rx=0 tx=0).stop 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 -- 172.21.15.19:0/2940276052 shutdown_connections 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5578400fe280 0x55784016b0a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66767fc700 0 client.15492 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578400e46a0 0x557840105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55783ff674a0 0x5578400f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 --2- 172.21.15.19:0/2940276052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557840051160 0x5578401054d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 -- 172.21.15.19:0/2940276052 >> 172.21.15.19:0/2940276052 conn(0x55783ffc76a0 msgr2=0x5578400e3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 -- 172.21.15.19:0/2940276052 shutdown_connections 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 1 -- 172.21.15.19:0/2940276052 wait complete. 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.574+0000 7f66a3e81b80 20 client.15492 trim_cache size 0 max 16384 2024-05-09T01:27:00.576 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:27:00.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 -- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f667c0b2990 msgr2=0x7f667c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:00.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f667c0b2990 0x7f667c0b4ea0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f668000ac60 tx=0x7f668000f6a0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 -- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f669010e8f0 msgr2=0x7f6690103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:00.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f669010e8f0 0x7f6690103bf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6688007ee0 tx=0x7f668800d3b0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 -- 172.21.15.19:0/3580482801 shutdown_connections 2024-05-09T01:27:00.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f667c0b2990 0x7f667c0b4ea0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f669010e8f0 0x7f6690103bf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6690105480 0x7f6690103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 --2- 172.21.15.19:0/3580482801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6690105dd0 0x7f66901036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:00.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 -- 172.21.15.19:0/3580482801 >> 172.21.15.19:0/3580482801 conn(0x7f66900fef30 msgr2=0x7f66900ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:00.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 -- 172.21.15.19:0/3580482801 shutdown_connections 2024-05-09T01:27:00.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:00.575+0000 7f669546b700 1 -- 172.21.15.19:0/3580482801 wait complete. 2024-05-09T01:27:00.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:00 smithi059 ceph-mon[34351]: pgmap v422: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-09T01:27:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:00 smithi196 ceph-mon[35041]: pgmap v422: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 -- 172.21.15.19:0/2325269930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538108ac0 msgr2=0x7f753810aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 --2- 172.21.15.19:0/2325269930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538108ac0 0x7f753810aee0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7530009ca0 tx=0x7f753002a8c0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 -- 172.21.15.19:0/2325269930 shutdown_connections 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 --2- 172.21.15.19:0/2325269930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538108ac0 0x7f753810aee0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 --2- 172.21.15.19:0/2325269930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7538103dc0 0x7f75381041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 --2- 172.21.15.19:0/2325269930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7538102b80 0x7f7538103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 -- 172.21.15.19:0/2325269930 >> 172.21.15.19:0/2325269930 conn(0x7f75380fdd90 msgr2=0x7f7538100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 -- 172.21.15.19:0/2325269930 shutdown_connections 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 -- 172.21.15.19:0/2325269930 wait complete. 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 Processor -- start 2024-05-09T01:27:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.272+0000 7f753f53f640 1 -- start start 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753f53f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538102b80 0x7f75380ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753f53f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7538103dc0 0x7f75380fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753f53f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7538108ac0 0x7f75380fe800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753f53f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f753810e6b0 con 0x7f7538102b80 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753f53f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f753810e530 con 0x7f7538108ac0 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753f53f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f753810e830 con 0x7f7538103dc0 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753d2db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538102b80 0x7f75380ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753d2db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538102b80 0x7f75380ffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59448/0 (socket says 172.21.15.19:59448) 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753d2db640 1 -- 172.21.15.19:0/2296943890 learned_addr learned my addr 172.21.15.19:0/2296943890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753cada640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7538103dc0 0x7f75380fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753dadc640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7538108ac0 0x7f75380fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753d2db640 1 -- 172.21.15.19:0/2296943890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7538103dc0 msgr2=0x7f75380fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753d2db640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7538103dc0 0x7f75380fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753d2db640 1 -- 172.21.15.19:0/2296943890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7538108ac0 msgr2=0x7f75380fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753d2db640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7538108ac0 0x7f75380fe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753d2db640 1 -- 172.21.15.19:0/2296943890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75380fef30 con 0x7f7538102b80 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753cada640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7538103dc0 0x7f75380fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.273+0000 7f753d2db640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538102b80 0x7f75380ffc20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7534005d00 tx=0x7f753400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.274+0000 7f752e7fc640 1 -- 172.21.15.19:0/2296943890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534017070 con 0x7f7538102b80 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.274+0000 7f753f53f640 1 -- 172.21.15.19:0/2296943890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75381a43e0 con 0x7f7538102b80 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.274+0000 7f752e7fc640 1 -- 172.21.15.19:0/2296943890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7534009c30 con 0x7f7538102b80 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.274+0000 7f752e7fc640 1 -- 172.21.15.19:0/2296943890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753400f620 con 0x7f7538102b80 2024-05-09T01:27:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.274+0000 7f753f53f640 1 -- 172.21.15.19:0/2296943890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75381a48a0 con 0x7f7538102b80 2024-05-09T01:27:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.277+0000 7f75463f3740 1 Processor -- start 2024-05-09T01:27:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.277+0000 7f75463f3740 1 -- start start 2024-05-09T01:27:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.277+0000 7f75463f3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab05500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.277+0000 7f75463f3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.277+0000 7f75463f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab0c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.277+0000 7f75463f3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55701aa697c0 con 0x55701a7382f0 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.277+0000 7f75463f3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55701aa050e0 con 0x55701a7611b0 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.277+0000 7f75463f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55701aa65780 con 0x55701a7404d0 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753dadc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab05500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753dadc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab05500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38866/0 (socket says 172.21.15.19:38866) 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753dadc640 1 -- 172.21.15.19:0/3338024054 learned_addr learned my addr 172.21.15.19:0/3338024054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753cada640 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab0c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f752e7fc640 1 -- 172.21.15.19:0/2296943890 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f753401a070 con 0x7f7538102b80 2024-05-09T01:27:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753dadc640 1 -- 172.21.15.19:0/3338024054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 msgr2=0x55701ab0c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753d2db640 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753dadc640 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab0c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753dadc640 1 -- 172.21.15.19:0/3338024054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 msgr2=0x55701ab05a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753dadc640 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab05a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753dadc640 1 -- 172.21.15.19:0/3338024054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55701aa2e7c0 con 0x55701a7611b0 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753d2db640 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.278+0000 7f753dadc640 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab05500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7530006010 tx=0x7f753002f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dfebd2de6728fe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.279+0000 7f75237fe640 1 -- 172.21.15.19:0/3338024054 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530031650 con 0x55701a7611b0 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.279+0000 7f75237fe640 1 -- 172.21.15.19:0/3338024054 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7530036070 con 0x55701a7611b0 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.279+0000 7f75237fe640 1 -- 172.21.15.19:0/3338024054 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75300045e0 con 0x55701a7611b0 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.279+0000 7f752e7fc640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75240b2870 0x7f75240b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.279+0000 7f753d2db640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75240b2870 0x7f75240b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.279+0000 7f752e7fc640 1 -- 172.21.15.19:0/2296943890 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f75340e1ef0 con 0x7f7538102b80 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 -- 172.21.15.19:0/3338024054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 msgr2=0x55701ab05500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab05500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7530006010 tx=0x7f753002f3a0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 -- 172.21.15.19:0/3338024054 shutdown_connections 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab0c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 --2- 172.21.15.19:0/3338024054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab05500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 -- 172.21.15.19:0/3338024054 >> 172.21.15.19:0/3338024054 conn(0x55701a755300 msgr2=0x55701ab00e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 -- 172.21.15.19:0/3338024054 shutdown_connections 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f753d2db640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75240b2870 0x7f75240b4d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55701ab068e0 tx=0x7f7534010c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 -- 172.21.15.19:0/3338024054 wait complete. 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 -- start start 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.280+0000 7f75463f3740 1 Processor -- start 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f75463f3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab0f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f75463f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab0f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f75463f3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab16640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f75463f3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55701ab0d370 con 0x55701a7382f0 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f75463f3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55701a9f09c0 con 0x55701a7611b0 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f75463f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55701ab0d4f0 con 0x55701a7404d0 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f75463f3740 10 client.0 _ll_register_callbacks cb 0x55701aa1d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab0f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab0f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59472/0 (socket says 172.21.15.19:59472) 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 -- 172.21.15.19:0/2756958961 learned_addr learned my addr 172.21.15.19:0/2756958961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753cada640 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab0f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab16640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 -- 172.21.15.19:0/2756958961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 msgr2=0x55701ab0f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab0f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 -- 172.21.15.19:0/2756958961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 msgr2=0x55701ab16640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab16640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 -- 172.21.15.19:0/2756958961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55701aa171f0 con 0x55701a7382f0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753cada640 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab0f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f753dadc640 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab0f1a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7530005e30 tx=0x7f75300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c52942071acd831 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.281+0000 7f7520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f7520ff9640 1 -- 172.21.15.19:0/2756958961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530055060 con 0x55701a7382f0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f7520ff9640 1 -- 172.21.15.19:0/2756958961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7530004770 con 0x55701a7382f0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f75463f3740 1 -- 172.21.15.19:0/2756958961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7530036070 con 0x55701a7382f0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f7520ff9640 1 -- 172.21.15.19:0/2756958961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7530003970 con 0x55701a7382f0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f75463f3740 10 client.15501 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f75463f3740 1 -- 172.21.15.19:0/2756958961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55701a7cc790 con 0x55701a7382f0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f7520ff9640 1 -- 172.21.15.19:0/2756958961 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7530036070 con 0x55701a7382f0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f7520ff9640 1 -- 172.21.15.19:0/2756958961 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f753002fc30 con 0x55701a7382f0 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f75463f3740 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f75463f3740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f75463f3740 20 client.15501 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f75463f3740 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55701a71fd80 0x55701ab1c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f75463f3740 4 client.15501 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.282+0000 7f75463f3740 1 -- 172.21.15.19:0/2756958961 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55701aa773d0 con 0x55701a71fd80 2024-05-09T01:27:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.283+0000 7f753cada640 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55701a71fd80 0x55701ab1c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.283+0000 7f7520ff9640 10 client.15501 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.283+0000 7f753cada640 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55701a71fd80 0x55701ab1c670 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdde07b480236e37 server_cookie=d735280a68de5fc6 in_seq=0 out_seq=0 2024-05-09T01:27:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f7520ff9640 1 -- 172.21.15.19:0/2756958961 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55701aa773d0 con 0x55701a71fd80 2024-05-09T01:27:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f7520ff9640 10 client.15501 handle_command_reply: tid=0 2024-05-09T01:27:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 client.15501 shutdown 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 -- 172.21.15.19:0/2756958961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 msgr2=0x55701ab0f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab0f1a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7530005e30 tx=0x7f75300095c0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 -- 172.21.15.19:0/2756958961 shutdown_connections 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55701a71fd80 0x55701ab1c670 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab16640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab0f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 --2- 172.21.15.19:0/2756958961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab0f1a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 -- 172.21.15.19:0/2756958961 >> 172.21.15.19:0/2756958961 conn(0x55701a755300 msgr2=0x55701ab03da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f7520ff9640 0 client.15501 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 -- 172.21.15.19:0/2756958961 shutdown_connections 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 1 -- 172.21.15.19:0/2756958961 wait complete. 2024-05-09T01:27:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.284+0000 7f75463f3740 20 client.15501 trim_cache size 0 max 16384 2024-05-09T01:27:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.300+0000 7f75463f3740 1 Processor -- start 2024-05-09T01:27:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.300+0000 7f75463f3740 1 -- start start 2024-05-09T01:27:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f75463f3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab21df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f75463f3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7404d0 0x55701ab22330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f75463f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7611b0 0x55701ab165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f75463f3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75381ad5b0 con 0x55701a7382f0 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f75463f3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75381af0f0 con 0x55701a7404d0 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f75463f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75381adc80 con 0x55701a7611b0 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab21df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab21df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59486/0 (socket says 172.21.15.19:59486) 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 -- 172.21.15.19:0/1060166298 learned_addr learned my addr 172.21.15.19:0/1060166298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753cada640 1 --2- 172.21.15.19:0/1060166298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7404d0 0x55701ab22330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 --2- 172.21.15.19:0/1060166298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7611b0 0x55701ab165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 -- 172.21.15.19:0/1060166298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7611b0 msgr2=0x55701ab165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 --2- 172.21.15.19:0/1060166298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7611b0 0x55701ab165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 -- 172.21.15.19:0/1060166298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7404d0 msgr2=0x55701ab22330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 --2- 172.21.15.19:0/1060166298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7404d0 0x55701ab22330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 -- 172.21.15.19:0/1060166298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55701a8435f0 con 0x55701a7382f0 2024-05-09T01:27:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f753dadc640 1 --2- 172.21.15.19:0/1060166298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab21df0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55701ab01950 tx=0x7f753002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85af1e0340fc8a89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.301+0000 7f7522ffd640 1 -- 172.21.15.19:0/1060166298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530030780 con 0x55701a7382f0 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.302+0000 7f7522ffd640 1 -- 172.21.15.19:0/1060166298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7530061070 con 0x55701a7382f0 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.302+0000 7f7522ffd640 1 -- 172.21.15.19:0/1060166298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530004570 con 0x55701a7382f0 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.302+0000 7f75463f3740 1 -- 172.21.15.19:0/1060166298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 msgr2=0x55701ab21df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.302+0000 7f75463f3740 1 --2- 172.21.15.19:0/1060166298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab21df0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55701ab01950 tx=0x7f753002f3a0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.302+0000 7f75463f3740 1 -- 172.21.15.19:0/1060166298 shutdown_connections 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.302+0000 7f75463f3740 1 --2- 172.21.15.19:0/1060166298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7611b0 0x55701ab165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.302+0000 7f75463f3740 1 --2- 172.21.15.19:0/1060166298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7404d0 0x55701ab22330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.302+0000 7f75463f3740 1 --2- 172.21.15.19:0/1060166298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab21df0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.302+0000 7f75463f3740 1 -- 172.21.15.19:0/1060166298 >> 172.21.15.19:0/1060166298 conn(0x55701a755300 msgr2=0x55701ab095c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 -- 172.21.15.19:0/1060166298 shutdown_connections 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 -- 172.21.15.19:0/1060166298 wait complete. 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 -- start start 2024-05-09T01:27:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 Processor -- start 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab16af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab13670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab13bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75381aea20 con 0x55701a7382f0 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75381ae350 con 0x55701a7611b0 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75381ab590 con 0x55701a7404d0 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f75463f3740 10 client.0 _ll_register_callbacks cb 0x7f7528009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.303+0000 7f753dadc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab16af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753dadc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab16af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59494/0 (socket says 172.21.15.19:59494) 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753dadc640 1 -- 172.21.15.19:0/3012015480 learned_addr learned my addr 172.21.15.19:0/3012015480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753dadc640 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab13bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753cada640 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab13670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753dadc640 1 -- 172.21.15.19:0/3012015480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 msgr2=0x55701ab13670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753dadc640 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab13670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753dadc640 1 -- 172.21.15.19:0/3012015480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 msgr2=0x55701ab13bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753dadc640 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab13bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753dadc640 1 -- 172.21.15.19:0/3012015480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55701aa2ded0 con 0x55701a7382f0 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753cada640 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab13670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f753dadc640 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab16af0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7530009ca0 tx=0x7f75300099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cd755784711274 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f7520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f7520ff9640 1 -- 172.21.15.19:0/3012015480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530069020 con 0x55701a7382f0 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f7520ff9640 1 -- 172.21.15.19:0/3012015480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7530004770 con 0x55701a7382f0 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.304+0000 7f75463f3740 1 -- 172.21.15.19:0/3012015480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7530061070 con 0x55701a7382f0 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f7520ff9640 1 -- 172.21.15.19:0/3012015480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7530074050 con 0x55701a7382f0 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f75463f3740 10 client.15507 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f75463f3740 1 -- 172.21.15.19:0/3012015480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55701ab837b0 con 0x55701a7382f0 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f7520ff9640 1 -- 172.21.15.19:0/3012015480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7530061070 con 0x55701a7382f0 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f7520ff9640 1 -- 172.21.15.19:0/3012015480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f753002f7f0 con 0x55701a7382f0 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f75463f3740 10 client.15507 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f75463f3740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f75463f3740 20 client.15507 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f75463f3740 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55701a6f51c0 0x55701ab86130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f75463f3740 4 client.15507 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f75463f3740 1 -- 172.21.15.19:0/3012015480 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55701aa71240 con 0x55701a6f51c0 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.305+0000 7f753cada640 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55701a6f51c0 0x55701ab86130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.306+0000 7f753cada640 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55701a6f51c0 0x55701ab86130 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25fc97533c73e141 server_cookie=440c089dd3e4a42 in_seq=0 out_seq=0 2024-05-09T01:27:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.306+0000 7f7520ff9640 10 client.15507 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f7520ff9640 1 -- 172.21.15.19:0/3012015480 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55701aa71240 con 0x55701a6f51c0 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f7520ff9640 10 client.15507 handle_command_reply: tid=0 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 client.15507 shutdown 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 -- 172.21.15.19:0/3012015480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 msgr2=0x55701ab16af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab16af0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7530009ca0 tx=0x7f75300099c0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 -- 172.21.15.19:0/3012015480 shutdown_connections 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55701a6f51c0 0x55701ab86130 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701a7611b0 0x55701ab13bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701a7404d0 0x55701ab13670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f7520ff9640 0 client.15507 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 --2- 172.21.15.19:0/3012015480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701a7382f0 0x55701ab16af0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 -- 172.21.15.19:0/3012015480 >> 172.21.15.19:0/3012015480 conn(0x55701a755300 msgr2=0x55701ab0d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 -- 172.21.15.19:0/3012015480 shutdown_connections 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 1 -- 172.21.15.19:0/3012015480 wait complete. 2024-05-09T01:27:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.310+0000 7f75463f3740 20 client.15507 trim_cache size 0 max 16384 2024-05-09T01:27:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:27:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:27:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:27:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 -- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75240b2870 msgr2=0x7f75240b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75240b2870 0x7f75240b4d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55701ab068e0 tx=0x7f7534010c00 comp rx=0 tx=0).stop 2024-05-09T01:27:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 -- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538102b80 msgr2=0x7f75380ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538102b80 0x7f75380ffc20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7534005d00 tx=0x7f753400dde0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 -- 172.21.15.19:0/2296943890 shutdown_connections 2024-05-09T01:27:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75240b2870 0x7f75240b4d80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7538108ac0 0x7f75380fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7538103dc0 0x7f75380fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 --2- 172.21.15.19:0/2296943890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7538102b80 0x7f75380ffc20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 -- 172.21.15.19:0/2296943890 >> 172.21.15.19:0/2296943890 conn(0x7f75380fdd90 msgr2=0x7f753806bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 -- 172.21.15.19:0/2296943890 shutdown_connections 2024-05-09T01:27:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:01.311+0000 7f753f53f640 1 -- 172.21.15.19:0/2296943890 wait complete. 2024-05-09T01:27:01.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:27:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:27:02.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:02 smithi019 ceph-mon[30158]: pgmap v423: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-09T01:27:02.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:02 smithi059 ceph-mon[34351]: pgmap v423: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-09T01:27:02.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:02 smithi196 ceph-mon[35041]: pgmap v423: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-09T01:27:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.604+0000 7f9fd4a58700 1 -- 172.21.15.19:0/3360328495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 msgr2=0x7f9fd01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.604+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/3360328495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 0x7f9fd01038d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00099c0 tx=0x7f9fc0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.604+0000 7f9fd4a58700 1 -- 172.21.15.19:0/3360328495 shutdown_connections 2024-05-09T01:27:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.604+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/3360328495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd0108740 0x7f9fd01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.604+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/3360328495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 0x7f9fd01038d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.604+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/3360328495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd0107df0 0x7f9fd0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.604+0000 7f9fd4a58700 1 -- 172.21.15.19:0/3360328495 >> 172.21.15.19:0/3360328495 conn(0x7f9fd00fedc0 msgr2=0x7f9fd01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.604+0000 7f9fd4a58700 1 -- 172.21.15.19:0/3360328495 shutdown_connections 2024-05-09T01:27:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.604+0000 7f9fd4a58700 1 -- 172.21.15.19:0/3360328495 wait complete. 2024-05-09T01:27:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.605+0000 7f9fd4a58700 1 Processor -- start 2024-05-09T01:27:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.605+0000 7f9fd4a58700 1 -- start start 2024-05-09T01:27:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fd4a58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 0x7f9fd019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fd4a58700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd0107df0 0x7f9fd019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fd4a58700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd0108740 0x7f9fd0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fd4a58700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd0113370 con 0x7f9fd01034c0 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fd4a58700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01131f0 con 0x7f9fd0108740 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fd4a58700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01134f0 con 0x7f9fd0107df0 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 0x7f9fd019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 0x7f9fd019f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59514/0 (socket says 172.21.15.19:59514) 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fce59c700 1 -- 172.21.15.19:0/1790931692 learned_addr learned my addr 172.21.15.19:0/1790931692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fced9d700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd0108740 0x7f9fd0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fcdd9b700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd0107df0 0x7f9fd019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fce59c700 1 -- 172.21.15.19:0/1790931692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd0107df0 msgr2=0x7f9fd019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fce59c700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd0107df0 0x7f9fd019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fce59c700 1 -- 172.21.15.19:0/1790931692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd0108740 msgr2=0x7f9fd0199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fce59c700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd0108740 0x7f9fd0199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.606+0000 7f9fce59c700 1 -- 172.21.15.19:0/1790931692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd019a1d0 con 0x7f9fd01034c0 2024-05-09T01:27:03.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.607+0000 7f9fce59c700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 0x7f9fd019f8b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000b580 tx=0x7f9fc00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:03.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.607+0000 7f9fbf7fe700 1 -- 172.21.15.19:0/1790931692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0024070 con 0x7f9fd01034c0 2024-05-09T01:27:03.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.607+0000 7f9fbf7fe700 1 -- 172.21.15.19:0/1790931692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc0004120 con 0x7f9fd01034c0 2024-05-09T01:27:03.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.607+0000 7f9fd4a58700 1 -- 172.21.15.19:0/1790931692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd019a460 con 0x7f9fd01034c0 2024-05-09T01:27:03.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.607+0000 7f9fbf7fe700 1 -- 172.21.15.19:0/1790931692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0020440 con 0x7f9fd01034c0 2024-05-09T01:27:03.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.607+0000 7f9fd4a58700 1 -- 172.21.15.19:0/1790931692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd006b610 con 0x7f9fd01034c0 2024-05-09T01:27:03.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.610+0000 7f9fbf7fe700 1 -- 172.21.15.19:0/1790931692 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f9fc0027070 con 0x7f9fd01034c0 2024-05-09T01:27:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.610+0000 7f9fbf7fe700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fb80b2990 0x7f9fb80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.611+0000 7f9fcdd9b700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fb80b2990 0x7f9fb80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.611+0000 7f9fbf7fe700 1 -- 172.21.15.19:0/1790931692 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9fc00e3270 con 0x7f9fd01034c0 2024-05-09T01:27:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.611+0000 7f9fcdd9b700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fb80b2990 0x7f9fb80b4ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4005950 tx=0x7f9fc40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.612+0000 7f9fe346eb80 1 Processor -- start 2024-05-09T01:27:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.612+0000 7f9fe346eb80 1 -- start start 2024-05-09T01:27:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65b9f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 0x558f65b9fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65b99ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f65abbff0 con 0x558f65b996a0 2024-05-09T01:27:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558f65a5a7f0 con 0x558f65a1c4a0 2024-05-09T01:27:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558f65ab7fb0 con 0x558f65b06160 2024-05-09T01:27:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65b99ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65b99ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59516/0 (socket says 172.21.15.19:59516) 2024-05-09T01:27:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fced9d700 1 -- 172.21.15.19:0/1929188326 learned_addr learned my addr 172.21.15.19:0/1929188326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fce59c700 1 --2- 172.21.15.19:0/1929188326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65b9f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fcdd9b700 1 --2- 172.21.15.19:0/1929188326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 0x558f65b9fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fced9d700 1 -- 172.21.15.19:0/1929188326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 msgr2=0x558f65b9f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.613+0000 7f9fced9d700 1 --2- 172.21.15.19:0/1929188326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65b9f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.614+0000 7f9fced9d700 1 -- 172.21.15.19:0/1929188326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 msgr2=0x558f65b9fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.614+0000 7f9fced9d700 1 --2- 172.21.15.19:0/1929188326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 0x558f65b9fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.614+0000 7f9fced9d700 1 -- 172.21.15.19:0/1929188326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f65a52370 con 0x558f65b996a0 2024-05-09T01:27:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.614+0000 7f9fced9d700 1 --2- 172.21.15.19:0/1929188326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65b99ad0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8003fa0 tx=0x7f9fc8009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134a06c274dbd129 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.614+0000 7f9fbcff9700 1 -- 172.21.15.19:0/1929188326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8004bb0 con 0x558f65b996a0 2024-05-09T01:27:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.614+0000 7f9fbcff9700 1 -- 172.21.15.19:0/1929188326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc8004d50 con 0x558f65b996a0 2024-05-09T01:27:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.614+0000 7f9fbcff9700 1 -- 172.21.15.19:0/1929188326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8007d80 con 0x558f65b996a0 2024-05-09T01:27:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.615+0000 7f9fe346eb80 1 -- 172.21.15.19:0/1929188326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 msgr2=0x558f65b99ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.615+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/1929188326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65b99ad0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8003fa0 tx=0x7f9fc8009ba0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.615+0000 7f9fe346eb80 1 -- 172.21.15.19:0/1929188326 shutdown_connections 2024-05-09T01:27:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.615+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/1929188326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65b99ad0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.615+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/1929188326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65b9f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.615+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/1929188326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 0x558f65b9fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.615+0000 7f9fe346eb80 1 -- 172.21.15.19:0/1929188326 >> 172.21.15.19:0/1929188326 conn(0x558f65a7c6a0 msgr2=0x558f65b94400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.615+0000 7f9fe346eb80 1 -- 172.21.15.19:0/1929188326 shutdown_connections 2024-05-09T01:27:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.615+0000 7f9fe346eb80 1 -- 172.21.15.19:0/1929188326 wait complete. 2024-05-09T01:27:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fe346eb80 1 -- start start 2024-05-09T01:27:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fe346eb80 1 Processor -- start 2024-05-09T01:27:03.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65ba31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65ba3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b996a0 0x558f65ba3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f65937ba0 con 0x558f65a1c4a0 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558f65a76150 con 0x558f65b996a0 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558f65ba61d0 con 0x558f65b06160 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fe346eb80 10 client.0 _ll_register_callbacks cb 0x558f6592d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65ba31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65ba31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59528/0 (socket says 172.21.15.19:59528) 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.616+0000 7f9fce59c700 1 -- 172.21.15.19:0/232990948 learned_addr learned my addr 172.21.15.19:0/232990948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fced9d700 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b996a0 0x558f65ba3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fcdd9b700 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65ba3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fce59c700 1 -- 172.21.15.19:0/232990948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 msgr2=0x558f65ba3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fce59c700 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65ba3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fce59c700 1 -- 172.21.15.19:0/232990948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b996a0 msgr2=0x558f65ba3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:03.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fce59c700 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b996a0 0x558f65ba3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fce59c700 1 -- 172.21.15.19:0/232990948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f65937490 con 0x558f65a1c4a0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fce59c700 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65ba31d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0003800 tx=0x7f9fc00e1060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edbff381f32218d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/232990948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00aae80 con 0x558f65a1c4a0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/232990948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9fc00aa9e0 con 0x558f65a1c4a0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fe346eb80 1 -- 172.21.15.19:0/232990948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc8004d50 con 0x558f65a1c4a0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/232990948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9fc00aa780 con 0x558f65a1c4a0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fe346eb80 10 client.15519 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.617+0000 7f9fe346eb80 1 -- 172.21.15.19:0/232990948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f65bb0ba0 con 0x558f65a1c4a0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.618+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/232990948 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc8004d50 con 0x558f65a1c4a0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.618+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/232990948 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9fc0092110 con 0x558f65a1c4a0 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.618+0000 7f9fe346eb80 10 client.15519 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.618+0000 7f9fe346eb80 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:03.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.618+0000 7f9fe346eb80 20 client.15519 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.618+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558f65bb31b0 0x558f65bb5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.618+0000 7f9fe346eb80 4 client.15519 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.618+0000 7f9fe346eb80 1 -- 172.21.15.19:0/232990948 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f65ac99d0 con 0x558f65bb31b0 2024-05-09T01:27:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.618+0000 7f9fcdd9b700 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558f65bb31b0 0x558f65bb5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.619+0000 7f9fcdd9b700 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558f65bb31b0 0x558f65bb5660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98b7b7f5284035a9 server_cookie=21c97b2bde474ea in_seq=0 out_seq=0 2024-05-09T01:27:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.619+0000 7f9fb5ffb700 10 client.15519 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/232990948 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f65ac99d0 con 0x558f65bb31b0 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fb5ffb700 10 client.15519 handle_command_reply: tid=0 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 client.15519 shutdown 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 -- 172.21.15.19:0/232990948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 msgr2=0x558f65ba31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65ba31d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0003800 tx=0x7f9fc00e1060 comp rx=0 tx=0).stop 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 -- 172.21.15.19:0/232990948 shutdown_connections 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558f65bb31b0 0x558f65bb5660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b996a0 0x558f65ba3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65ba31d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/232990948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65ba3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fb5ffb700 0 client.15519 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 -- 172.21.15.19:0/232990948 >> 172.21.15.19:0/232990948 conn(0x558f65a7c6a0 msgr2=0x558f65b941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 -- 172.21.15.19:0/232990948 shutdown_connections 2024-05-09T01:27:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 1 -- 172.21.15.19:0/232990948 wait complete. 2024-05-09T01:27:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.620+0000 7f9fe346eb80 20 client.15519 trim_cache size 0 max 16384 2024-05-09T01:27:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.636+0000 7f9fe346eb80 1 Processor -- start 2024-05-09T01:27:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.636+0000 7f9fe346eb80 1 -- start start 2024-05-09T01:27:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65b9d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b06160 0x558f65b9e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b996a0 0x558f65ba1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01b0dc0 con 0x558f65a1c4a0 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01b2900 con 0x558f65b06160 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01b1490 con 0x558f65b996a0 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65b9d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65b9d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59536/0 (socket says 172.21.15.19:59536) 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fce59c700 1 -- 172.21.15.19:0/3910257614 learned_addr learned my addr 172.21.15.19:0/3910257614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fced9d700 1 --2- 172.21.15.19:0/3910257614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b996a0 0x558f65ba1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fcdd9b700 1 --2- 172.21.15.19:0/3910257614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b06160 0x558f65b9e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fce59c700 1 -- 172.21.15.19:0/3910257614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b996a0 msgr2=0x558f65ba1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fce59c700 1 --2- 172.21.15.19:0/3910257614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b996a0 0x558f65ba1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fce59c700 1 -- 172.21.15.19:0/3910257614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b06160 msgr2=0x558f65b9e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fce59c700 1 --2- 172.21.15.19:0/3910257614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b06160 0x558f65b9e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.637+0000 7f9fce59c700 1 -- 172.21.15.19:0/3910257614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f65bb5bf0 con 0x558f65a1c4a0 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.638+0000 7f9fce59c700 1 --2- 172.21.15.19:0/3910257614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65b9d7c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00a6d20 tx=0x7f9fc000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ce90bef7ca5add server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.638+0000 7f9fb7fff700 1 -- 172.21.15.19:0/3910257614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00aeb30 con 0x558f65a1c4a0 2024-05-09T01:27:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.638+0000 7f9fb7fff700 1 -- 172.21.15.19:0/3910257614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc00b3550 con 0x558f65a1c4a0 2024-05-09T01:27:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.638+0000 7f9fb7fff700 1 -- 172.21.15.19:0/3910257614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00a9ad0 con 0x558f65a1c4a0 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 -- 172.21.15.19:0/3910257614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 msgr2=0x558f65b9d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/3910257614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65b9d7c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00a6d20 tx=0x7f9fc000fa40 comp rx=0 tx=0).stop 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 -- 172.21.15.19:0/3910257614 shutdown_connections 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/3910257614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b996a0 0x558f65ba1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/3910257614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65a1c4a0 0x558f65b9d7c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/3910257614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65b06160 0x558f65b9e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 -- 172.21.15.19:0/3910257614 >> 172.21.15.19:0/3910257614 conn(0x558f65a7c6a0 msgr2=0x558f65b9e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 -- 172.21.15.19:0/3910257614 shutdown_connections 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 -- 172.21.15.19:0/3910257614 wait complete. 2024-05-09T01:27:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 -- start start 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.639+0000 7f9fe346eb80 1 Processor -- start 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 0x558f65bab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65bae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fe346eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65bae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01b2230 con 0x558f65b996a0 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01b1b60 con 0x558f65a1c4a0 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fe346eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01aed60 con 0x558f65b06160 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fe346eb80 10 client.0 _ll_register_callbacks cb 0x558f6592d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65bae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fced9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65bae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59550/0 (socket says 172.21.15.19:59550) 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fced9d700 1 -- 172.21.15.19:0/4204712937 learned_addr learned my addr 172.21.15.19:0/4204712937 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fcdd9b700 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65bae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fce59c700 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 0x558f65bab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fced9d700 1 -- 172.21.15.19:0/4204712937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 msgr2=0x558f65bae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fced9d700 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65bae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fced9d700 1 -- 172.21.15.19:0/4204712937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 msgr2=0x558f65bab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fced9d700 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 0x558f65bab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.640+0000 7f9fced9d700 1 -- 172.21.15.19:0/4204712937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f65baea80 con 0x558f65b996a0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.641+0000 7f9fced9d700 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65bae540 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x558f65ba00c0 tx=0x7f9fc8008c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a2d706a91c1d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.641+0000 7f9fb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.641+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/4204712937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8008e60 con 0x558f65b996a0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.641+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/4204712937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9fc8015690 con 0x558f65b996a0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.641+0000 7f9fe346eb80 1 -- 172.21.15.19:0/4204712937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc00b3550 con 0x558f65b996a0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.641+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/4204712937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9fc8015890 con 0x558f65b996a0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.641+0000 7f9fe346eb80 10 client.15525 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.641+0000 7f9fe346eb80 1 -- 172.21.15.19:0/4204712937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f65bab080 con 0x558f65b996a0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.642+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/4204712937 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc00b3550 con 0x558f65b996a0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.642+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/4204712937 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9fc8028b60 con 0x558f65b996a0 2024-05-09T01:27:03.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.642+0000 7f9fe346eb80 10 client.15525 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.642+0000 7f9fe346eb80 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.642+0000 7f9fe346eb80 20 client.15525 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.642+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558f65c1da70 0x558f65c1fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.642+0000 7f9fe346eb80 4 client.15525 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.642+0000 7f9fe346eb80 1 -- 172.21.15.19:0/4204712937 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f65ac3a70 con 0x558f65c1da70 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.642+0000 7f9fce59c700 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558f65c1da70 0x558f65c1fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fb5ffb700 10 client.15525 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fce59c700 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558f65c1da70 0x558f65c1fe90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d004b77e8d9b6c server_cookie=195900755ef69e1e in_seq=0 out_seq=0 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fb5ffb700 1 -- 172.21.15.19:0/4204712937 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f65ac3a70 con 0x558f65c1da70 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fb5ffb700 10 client.15525 handle_command_reply: tid=0 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 client.15525 shutdown 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 -- 172.21.15.19:0/4204712937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 msgr2=0x558f65bae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65bae540 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x558f65ba00c0 tx=0x7f9fc8008c30 comp rx=0 tx=0).stop 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 -- 172.21.15.19:0/4204712937 shutdown_connections 2024-05-09T01:27:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558f65c1da70 0x558f65c1fe90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f65b996a0 0x558f65bae540 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f65a1c4a0 0x558f65bab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 --2- 172.21.15.19:0/4204712937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f65b06160 0x558f65bae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 -- 172.21.15.19:0/4204712937 >> 172.21.15.19:0/4204712937 conn(0x558f65a7c6a0 msgr2=0x558f65b9ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fb5ffb700 0 client.15525 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 -- 172.21.15.19:0/4204712937 shutdown_connections 2024-05-09T01:27:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 1 -- 172.21.15.19:0/4204712937 wait complete. 2024-05-09T01:27:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.643+0000 7f9fe346eb80 20 client.15525 trim_cache size 0 max 16384 2024-05-09T01:27:03.646 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 -- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fb80b2990 msgr2=0x7f9fb80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fb80b2990 0x7f9fb80b4ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4005950 tx=0x7f9fc40058e0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 -- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 msgr2=0x7f9fd019f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 0x7f9fd019f8b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000b580 tx=0x7f9fc00047b0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 -- 172.21.15.19:0/1790931692 shutdown_connections 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd0108740 0x7f9fd0199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fb80b2990 0x7f9fb80b4ea0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd01034c0 0x7f9fd019f8b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 --2- 172.21.15.19:0/1790931692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd0107df0 0x7f9fd019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.644+0000 7f9fd4a58700 1 -- 172.21.15.19:0/1790931692 >> 172.21.15.19:0/1790931692 conn(0x7f9fd00fedc0 msgr2=0x7f9fd0101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.645+0000 7f9fd4a58700 1 -- 172.21.15.19:0/1790931692 shutdown_connections 2024-05-09T01:27:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:03.645+0000 7f9fd4a58700 1 -- 172.21.15.19:0/1790931692 wait complete. 2024-05-09T01:27:04.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:04 smithi059 ceph-mon[34351]: pgmap v424: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T01:27:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:04 smithi019 ceph-mon[30158]: pgmap v424: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T01:27:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:04 smithi196 ceph-mon[35041]: pgmap v424: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T01:27:05.394 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:27:05.394 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:27:05.675 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:27:06.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:06 smithi019 ceph-mon[30158]: pgmap v425: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:27:06.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:06 smithi059 ceph-mon[34351]: pgmap v425: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:27:06.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:06 smithi196 ceph-mon[35041]: pgmap v425: 97 pgs: 97 active+clean; 673 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:27:07.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.613+0000 7ff77a846700 1 -- 172.21.15.19:0/1433549307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff774108760 msgr2=0x7ff7741100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.613+0000 7ff77a846700 1 --2- 172.21.15.19:0/1433549307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff774108760 0x7ff7741100c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b0d0 tx=0x7ff76c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.613+0000 7ff77a846700 1 -- 172.21.15.19:0/1433549307 shutdown_connections 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.613+0000 7ff77a846700 1 --2- 172.21.15.19:0/1433549307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff774108760 0x7ff7741100c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.613+0000 7ff77a846700 1 --2- 172.21.15.19:0/1433549307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7741034e0 0x7ff7741038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.613+0000 7ff77a846700 1 --2- 172.21.15.19:0/1433549307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff774107e10 0x7ff774108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.613+0000 7ff77a846700 1 -- 172.21.15.19:0/1433549307 >> 172.21.15.19:0/1433549307 conn(0x7ff7740fef50 msgr2=0x7ff774101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.613+0000 7ff77a846700 1 -- 172.21.15.19:0/1433549307 shutdown_connections 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.613+0000 7ff77a846700 1 -- 172.21.15.19:0/1433549307 wait complete. 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.614+0000 7ff77a846700 1 Processor -- start 2024-05-09T01:27:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.614+0000 7ff77a846700 1 -- start start 2024-05-09T01:27:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.614+0000 7ff77a846700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7741034e0 0x7ff77419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.614+0000 7ff77a846700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff774107e10 0x7ff77419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.614+0000 7ff77a846700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff774108760 0x7ff77419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.614+0000 7ff77a846700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff774113390 con 0x7ff7741034e0 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.614+0000 7ff77a846700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff774113210 con 0x7ff774108760 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.614+0000 7ff77a846700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff774113510 con 0x7ff774107e10 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff773fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7741034e0 0x7ff77419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff773fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7741034e0 0x7ff77419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59570/0 (socket says 172.21.15.19:59570) 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff773fff700 1 -- 172.21.15.19:0/1045805123 learned_addr learned my addr 172.21.15.19:0/1045805123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff7737fe700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff774107e10 0x7ff77419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff778de3700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff774108760 0x7ff77419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff773fff700 1 -- 172.21.15.19:0/1045805123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff774107e10 msgr2=0x7ff77419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff773fff700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff774107e10 0x7ff77419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff773fff700 1 -- 172.21.15.19:0/1045805123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff774108760 msgr2=0x7ff77419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff773fff700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff774108760 0x7ff77419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff773fff700 1 -- 172.21.15.19:0/1045805123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77419fc80 con 0x7ff7741034e0 2024-05-09T01:27:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff7737fe700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff774107e10 0x7ff77419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:07.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.615+0000 7ff773fff700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7741034e0 0x7ff77419adb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff76400d950 tx=0x7ff76400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:07.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.616+0000 7ff7717fa700 1 -- 172.21.15.19:0/1045805123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764016070 con 0x7ff7741034e0 2024-05-09T01:27:07.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.616+0000 7ff7717fa700 1 -- 172.21.15.19:0/1045805123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff764010460 con 0x7ff7741034e0 2024-05-09T01:27:07.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.616+0000 7ff7717fa700 1 -- 172.21.15.19:0/1045805123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76400f620 con 0x7ff7741034e0 2024-05-09T01:27:07.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.616+0000 7ff77a846700 1 -- 172.21.15.19:0/1045805123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77419ff70 con 0x7ff7741034e0 2024-05-09T01:27:07.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.616+0000 7ff77a846700 1 -- 172.21.15.19:0/1045805123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7741a7890 con 0x7ff7741034e0 2024-05-09T01:27:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.620+0000 7ff7717fa700 1 -- 172.21.15.19:0/1045805123 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7ff764019070 con 0x7ff7741034e0 2024-05-09T01:27:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.621+0000 7ff7717fa700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff75c0b2910 0x7ff75c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.621+0000 7ff78925cb80 1 Processor -- start 2024-05-09T01:27:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.621+0000 7ff7717fa700 1 -- 172.21.15.19:0/1045805123 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff7640e1fe0 con 0x7ff7741034e0 2024-05-09T01:27:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.621+0000 7ff78925cb80 1 -- start start 2024-05-09T01:27:07.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.621+0000 7ff7737fe700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff75c0b2910 0x7ff75c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.621+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cecb0520 0x55a8cee33800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 0x55a8cee2d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff78925cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8ced50120 con 0x55a8cecb0520 2024-05-09T01:27:07.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff78925cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a8cecf5a20 con 0x55a8cee2d480 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff78925cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a8ced4c0e0 con 0x55a8ced2ad80 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff7737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cecb0520 0x55a8cee33800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff7737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cecb0520 0x55a8cee33800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59576/0 (socket says 172.21.15.19:59576) 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff773fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff7737fe700 1 -- 172.21.15.19:0/1668086789 learned_addr learned my addr 172.21.15.19:0/1668086789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff7737fe700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff75c0b2910 0x7ff75c0b4e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff774109770 tx=0x7ff768009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff778de3700 1 --2- 172.21.15.19:0/1668086789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 0x55a8cee2d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff7737fe700 1 -- 172.21.15.19:0/1668086789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 msgr2=0x55a8cee332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff7737fe700 1 --2- 172.21.15.19:0/1668086789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff7737fe700 1 -- 172.21.15.19:0/1668086789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 msgr2=0x55a8cee2d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff7737fe700 1 --2- 172.21.15.19:0/1668086789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 0x55a8cee2d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff7737fe700 1 -- 172.21.15.19:0/1668086789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8cece6580 con 0x55a8cecb0520 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.622+0000 7ff773fff700 1 --2- 172.21.15.19:0/1668086789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.623+0000 7ff7737fe700 1 --2- 172.21.15.19:0/1668086789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cecb0520 0x55a8cee33800 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff76800fc70 tx=0x7ff768012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3caca733e5179e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:07.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.623+0000 7ff75a7fc700 1 -- 172.21.15.19:0/1668086789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768022070 con 0x55a8cecb0520 2024-05-09T01:27:07.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.623+0000 7ff75a7fc700 1 -- 172.21.15.19:0/1668086789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7680118d0 con 0x55a8cecb0520 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff75a7fc700 1 -- 172.21.15.19:0/1668086789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768003bf0 con 0x55a8cecb0520 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff78925cb80 1 -- 172.21.15.19:0/1668086789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cecb0520 msgr2=0x55a8cee33800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff78925cb80 1 --2- 172.21.15.19:0/1668086789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cecb0520 0x55a8cee33800 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff76800fc70 tx=0x7ff768012000 comp rx=0 tx=0).stop 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff78925cb80 1 -- 172.21.15.19:0/1668086789 shutdown_connections 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff78925cb80 1 --2- 172.21.15.19:0/1668086789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 0x55a8cee2d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff78925cb80 1 --2- 172.21.15.19:0/1668086789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff78925cb80 1 --2- 172.21.15.19:0/1668086789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cecb0520 0x55a8cee33800 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff78925cb80 1 -- 172.21.15.19:0/1668086789 >> 172.21.15.19:0/1668086789 conn(0x55a8cecb3ff0 msgr2=0x55a8cee282a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff78925cb80 1 -- 172.21.15.19:0/1668086789 shutdown_connections 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.624+0000 7ff78925cb80 1 -- 172.21.15.19:0/1668086789 wait complete. 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.625+0000 7ff78925cb80 1 -- start start 2024-05-09T01:27:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.625+0000 7ff78925cb80 1 Processor -- start 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee37150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 0x55a8cee37690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee477a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff773fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee37150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff773fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee37150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:47354/0 (socket says 172.21.15.19:47354) 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff773fff700 1 -- 172.21.15.19:0/2449017767 learned_addr learned my addr 172.21.15.19:0/2449017767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff7737fe700 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 0x55a8cee37690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8cee2e230 con 0x55a8cee37bd0 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff778de3700 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a8ceceaba0 con 0x55a8cee2d480 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.626+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a8cee3e4b0 con 0x55a8ced2ad80 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.627+0000 7ff78925cb80 10 client.0 _ll_register_callbacks cb 0x55a8cebc1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.627+0000 7ff778de3700 1 -- 172.21.15.19:0/2449017767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 msgr2=0x55a8cee37150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.627+0000 7ff778de3700 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee37150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.627+0000 7ff778de3700 1 -- 172.21.15.19:0/2449017767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 msgr2=0x55a8cee37690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.627+0000 7ff778de3700 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 0x55a8cee37690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.627+0000 7ff778de3700 1 -- 172.21.15.19:0/2449017767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8cebcb510 con 0x55a8cee37bd0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.627+0000 7ff778de3700 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee477a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b0d0 tx=0x7ff76c00baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf1996e5f5d4827 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.627+0000 7ff7537fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.628+0000 7ff7537fe700 1 -- 172.21.15.19:0/2449017767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0077c0 con 0x55a8cee37bd0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.628+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7680118d0 con 0x55a8cee37bd0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.628+0000 7ff7537fe700 1 -- 172.21.15.19:0/2449017767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff76c007960 con 0x55a8cee37bd0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.628+0000 7ff7537fe700 1 -- 172.21.15.19:0/2449017767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff76c006000 con 0x55a8cee37bd0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.628+0000 7ff78925cb80 10 client.15537 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.628+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8cee494f0 con 0x55a8cee37bd0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.628+0000 7ff7537fe700 1 -- 172.21.15.19:0/2449017767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7680118d0 con 0x55a8cee37bd0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.628+0000 7ff7537fe700 1 -- 172.21.15.19:0/2449017767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff76c011070 con 0x55a8cee37bd0 2024-05-09T01:27:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.629+0000 7ff78925cb80 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.629+0000 7ff78925cb80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.629+0000 7ff78925cb80 20 client.15537 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.629+0000 7ff78925cb80 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a8cee4bb30 0x55a8cee4df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.629+0000 7ff78925cb80 4 client.15537 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.629+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8ced5db00 con 0x55a8cee4bb30 2024-05-09T01:27:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.629+0000 7ff773fff700 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a8cee4bb30 0x55a8cee4df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.630+0000 7ff7537fe700 10 client.15537 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.630+0000 7ff773fff700 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a8cee4bb30 0x55a8cee4df90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84469d7c5f2eaa4 server_cookie=7041cbd49e546451 in_seq=0 out_seq=0 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.630+0000 7ff7537fe700 1 -- 172.21.15.19:0/2449017767 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8ced5db00 con 0x55a8cee4bb30 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.630+0000 7ff7537fe700 10 client.15537 handle_command_reply: tid=0 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 client.15537 shutdown 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 msgr2=0x55a8cee477a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee477a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b0d0 tx=0x7ff76c00baa0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 shutdown_connections 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a8cee4bb30 0x55a8cee4df90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee477a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8ced2ad80 0x55a8cee37150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff7537fe700 0 client.15537 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 --2- 172.21.15.19:0/2449017767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8cee2d480 0x55a8cee37690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 >> 172.21.15.19:0/2449017767 conn(0x55a8cecb3ff0 msgr2=0x55a8cee22070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 shutdown_connections 2024-05-09T01:27:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 1 -- 172.21.15.19:0/2449017767 wait complete. 2024-05-09T01:27:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.631+0000 7ff78925cb80 20 client.15537 trim_cache size 0 max 16384 2024-05-09T01:27:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.652+0000 7ff78925cb80 1 Processor -- start 2024-05-09T01:27:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.652+0000 7ff78925cb80 1 -- start start 2024-05-09T01:27:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.652+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 0x55a8cee22aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.652+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee2d480 0x55a8cee34d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.652+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee37bd0 0x55a8cee2cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.652+0000 7ff78925cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741aca60 con 0x55a8cee2d480 2024-05-09T01:27:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.652+0000 7ff78925cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741ae5a0 con 0x55a8ced2ad80 2024-05-09T01:27:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.652+0000 7ff78925cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741ad130 con 0x55a8cee37bd0 2024-05-09T01:27:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff7737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee2d480 0x55a8cee34d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff7737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee2d480 0x55a8cee34d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59602/0 (socket says 172.21.15.19:59602) 2024-05-09T01:27:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff7737fe700 1 -- 172.21.15.19:0/3449199246 learned_addr learned my addr 172.21.15.19:0/3449199246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff773fff700 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 0x55a8cee22aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff778de3700 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee37bd0 0x55a8cee2cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff7737fe700 1 -- 172.21.15.19:0/3449199246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee37bd0 msgr2=0x55a8cee2cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff7737fe700 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee37bd0 0x55a8cee2cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff7737fe700 1 -- 172.21.15.19:0/3449199246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 msgr2=0x55a8cee22aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff7737fe700 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 0x55a8cee22aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff7737fe700 1 -- 172.21.15.19:0/3449199246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8cee399f0 con 0x55a8cee2d480 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff778de3700 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee37bd0 0x55a8cee2cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.653+0000 7ff7737fe700 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee2d480 0x55a8cee34d70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55a8cee29d10 tx=0x7ff7680051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b6015a5d4fc8c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.654+0000 7ff759ffb700 1 -- 172.21.15.19:0/3449199246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768015070 con 0x55a8cee2d480 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.654+0000 7ff759ffb700 1 -- 172.21.15.19:0/3449199246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff768011d80 con 0x55a8cee2d480 2024-05-09T01:27:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.654+0000 7ff759ffb700 1 -- 172.21.15.19:0/3449199246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768022040 con 0x55a8cee2d480 2024-05-09T01:27:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.655+0000 7ff78925cb80 1 -- 172.21.15.19:0/3449199246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee2d480 msgr2=0x55a8cee34d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.655+0000 7ff78925cb80 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee2d480 0x55a8cee34d70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55a8cee29d10 tx=0x7ff7680051e0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.655+0000 7ff78925cb80 1 -- 172.21.15.19:0/3449199246 shutdown_connections 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.655+0000 7ff78925cb80 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee37bd0 0x55a8cee2cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.655+0000 7ff78925cb80 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 0x55a8cee22aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.655+0000 7ff78925cb80 1 --2- 172.21.15.19:0/3449199246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee2d480 0x55a8cee34d70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.655+0000 7ff78925cb80 1 -- 172.21.15.19:0/3449199246 >> 172.21.15.19:0/3449199246 conn(0x55a8cecb3ff0 msgr2=0x55a8cee2c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.655+0000 7ff78925cb80 1 -- 172.21.15.19:0/3449199246 shutdown_connections 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.655+0000 7ff78925cb80 1 -- 172.21.15.19:0/3449199246 wait complete. 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.656+0000 7ff78925cb80 1 -- start start 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.656+0000 7ff78925cb80 1 Processor -- start 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.656+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 0x55a8cee43700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.656+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee2d480 0x55a8cee43c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.656+0000 7ff78925cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee3c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.656+0000 7ff78925cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741aded0 con 0x55a8cee37bd0 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.656+0000 7ff78925cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741ad800 con 0x55a8ced2ad80 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.656+0000 7ff78925cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741aaa20 con 0x55a8cee2d480 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff78925cb80 10 client.0 _ll_register_callbacks cb 0x55a8cebc1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff778de3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee3c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7737fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee2d480 0x55a8cee43c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7737fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee2d480 0x55a8cee43c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:47378/0 (socket says 172.21.15.19:47378) 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7737fe700 1 -- 172.21.15.19:0/70099133 learned_addr learned my addr 172.21.15.19:0/70099133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff773fff700 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 0x55a8cee43700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7737fe700 1 -- 172.21.15.19:0/70099133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 msgr2=0x55a8cee43700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7737fe700 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 0x55a8cee43700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7737fe700 1 -- 172.21.15.19:0/70099133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 msgr2=0x55a8cee3c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7737fe700 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee3c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7737fe700 1 -- 172.21.15.19:0/70099133 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8cee54080 con 0x55a8cee2d480 2024-05-09T01:27:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff778de3700 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee3c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7737fe700 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee2d480 0x55a8cee43c40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7680056a0 tx=0x7ff7680056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e342c4cfdc06cdba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.657+0000 7ff7537fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.658+0000 7ff7537fe700 1 -- 172.21.15.19:0/70099133 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768022030 con 0x55a8cee2d480 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.658+0000 7ff7537fe700 1 -- 172.21.15.19:0/70099133 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff768011b60 con 0x55a8cee2d480 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.658+0000 7ff78925cb80 1 -- 172.21.15.19:0/70099133 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff768011d80 con 0x55a8cee2d480 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.658+0000 7ff7537fe700 1 -- 172.21.15.19:0/70099133 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff768015070 con 0x55a8cee2d480 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.658+0000 7ff78925cb80 10 client.25225 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.658+0000 7ff78925cb80 1 -- 172.21.15.19:0/70099133 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8ceebafb0 con 0x55a8cee2d480 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.659+0000 7ff7537fe700 1 -- 172.21.15.19:0/70099133 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff768011d80 con 0x55a8cee2d480 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.659+0000 7ff7537fe700 1 -- 172.21.15.19:0/70099133 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff76801e840 con 0x55a8cee2d480 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.659+0000 7ff78925cb80 10 client.25225 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.659+0000 7ff78925cb80 10 client.25225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.659+0000 7ff78925cb80 20 client.25225 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.659+0000 7ff78925cb80 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a8ceebb480 0x55a8ceebd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:07.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.659+0000 7ff78925cb80 4 client.25225 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:27:07.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.659+0000 7ff78925cb80 1 -- 172.21.15.19:0/70099133 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a8ced57ba0 con 0x55a8ceebb480 2024-05-09T01:27:07.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.659+0000 7ff773fff700 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a8ceebb480 0x55a8ceebd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.660+0000 7ff7537fe700 10 client.25225 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.660+0000 7ff773fff700 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a8ceebb480 0x55a8ceebd8a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a57259c5571686a server_cookie=2615c297a4de70a8 in_seq=0 out_seq=0 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.661+0000 7ff7537fe700 1 -- 172.21.15.19:0/70099133 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a8ced57ba0 con 0x55a8ceebb480 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.661+0000 7ff7537fe700 10 client.25225 handle_command_reply: tid=0 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.661+0000 7ff78925cb80 1 client.25225 shutdown 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.661+0000 7ff78925cb80 1 -- 172.21.15.19:0/70099133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee2d480 msgr2=0x55a8cee43c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.661+0000 7ff78925cb80 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee2d480 0x55a8cee43c40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7680056a0 tx=0x7ff7680056d0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.661+0000 7ff78925cb80 1 -- 172.21.15.19:0/70099133 shutdown_connections 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.661+0000 7ff78925cb80 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a8ceebb480 0x55a8ceebd8a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.662+0000 7ff78925cb80 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8cee37bd0 0x55a8cee3c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.662+0000 7ff78925cb80 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8ced2ad80 0x55a8cee43700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.662+0000 7ff78925cb80 1 --2- 172.21.15.19:0/70099133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8cee2d480 0x55a8cee43c40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.662+0000 7ff78925cb80 1 -- 172.21.15.19:0/70099133 >> 172.21.15.19:0/70099133 conn(0x55a8cecb3ff0 msgr2=0x55a8cee22800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.662+0000 7ff7537fe700 0 client.25225 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.662+0000 7ff78925cb80 1 -- 172.21.15.19:0/70099133 shutdown_connections 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.662+0000 7ff78925cb80 1 -- 172.21.15.19:0/70099133 wait complete. 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.662+0000 7ff78925cb80 20 client.25225 trim_cache size 0 max 16384 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "86072fe2-5f9a-43a9-999b-3fa3697e4107", 2024-05-09T01:27:07.661 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:27:07.662 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 -- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff75c0b2910 msgr2=0x7ff75c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff75c0b2910 0x7ff75c0b4e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff774109770 tx=0x7ff768009260 comp rx=0 tx=0).stop 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 -- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7741034e0 msgr2=0x7ff77419adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7741034e0 0x7ff77419adb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff76400d950 tx=0x7ff76400dd10 comp rx=0 tx=0).stop 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 -- 172.21.15.19:0/1045805123 shutdown_connections 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff774108760 0x7ff77419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff75c0b2910 0x7ff75c0b4e20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7741034e0 0x7ff77419adb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 --2- 172.21.15.19:0/1045805123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff774107e10 0x7ff77419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 -- 172.21.15.19:0/1045805123 >> 172.21.15.19:0/1045805123 conn(0x7ff7740fef50 msgr2=0x7ff774075c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.663+0000 7ff77a846700 1 -- 172.21.15.19:0/1045805123 shutdown_connections 2024-05-09T01:27:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:07.664+0000 7ff77a846700 1 -- 172.21.15.19:0/1045805123 wait complete. 2024-05-09T01:27:08.474 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:27:08.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:08 smithi059 ceph-mon[34351]: pgmap v426: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:27:08.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:08 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:27:08.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:08 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:27:08.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:08 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:27:08.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:08 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:27:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:08 smithi019 ceph-mon[30158]: pgmap v426: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:27:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:08 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:27:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:08 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:27:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:08 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:27:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:08 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:27:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:08 smithi196 ceph-mon[35041]: pgmap v426: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:27:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:08 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:27:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:08 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:27:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:08 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:27:08.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:08 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:27:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:27:10.516 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:10 smithi019 ceph-mon[30158]: pgmap v427: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:27:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.534+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/517156952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 msgr2=0x7f2a481049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.534+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/517156952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 0x7f2a481049e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2a440099c0 tx=0x7f2a44009cd0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.534+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/517156952 shutdown_connections 2024-05-09T01:27:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.534+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/517156952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48109850 0x7f2a481111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.534+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/517156952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 0x7f2a481049e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.534+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/517156952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48108f00 0x7f2a48109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.534+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/517156952 >> 172.21.15.19:0/517156952 conn(0x7f2a480ffea0 msgr2=0x7f2a481022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.535+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/517156952 shutdown_connections 2024-05-09T01:27:10.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.535+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/517156952 wait complete. 2024-05-09T01:27:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.535+0000 7f2a4f4e2700 1 Processor -- start 2024-05-09T01:27:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4f4e2700 1 -- start start 2024-05-09T01:27:10.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4f4e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 0x7f2a4819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4f4e2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48108f00 0x7f2a4819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4f4e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48109850 0x7f2a4819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4f4e2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48114480 con 0x7f2a481045d0 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4d27e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 0x7f2a4819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4f4e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48114300 con 0x7f2a48109850 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4f4e2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48114600 con 0x7f2a48108f00 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4d27e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 0x7f2a4819ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58828/0 (socket says 172.21.15.19:58828) 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4d27e700 1 -- 172.21.15.19:0/1734585692 learned_addr learned my addr 172.21.15.19:0/1734585692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48109850 0x7f2a4819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.536+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48108f00 0x7f2a4819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.537+0000 7f2a4d27e700 1 -- 172.21.15.19:0/1734585692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48108f00 msgr2=0x7f2a4819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.537+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48108f00 0x7f2a4819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.537+0000 7f2a4d27e700 1 -- 172.21.15.19:0/1734585692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48109850 msgr2=0x7f2a4819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.537+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48109850 0x7f2a4819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.537+0000 7f2a4d27e700 1 -- 172.21.15.19:0/1734585692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4819fc60 con 0x7f2a481045d0 2024-05-09T01:27:10.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.537+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 0x7f2a4819ad90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400b580 tx=0x7f2a440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:10.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.537+0000 7f2a3e7fc700 1 -- 172.21.15.19:0/1734585692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44027070 con 0x7f2a481045d0 2024-05-09T01:27:10.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.537+0000 7f2a3e7fc700 1 -- 172.21.15.19:0/1734585692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a44003f20 con 0x7f2a481045d0 2024-05-09T01:27:10.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.537+0000 7f2a3e7fc700 1 -- 172.21.15.19:0/1734585692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44022d10 con 0x7f2a481045d0 2024-05-09T01:27:10.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.538+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/1734585692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4819fef0 con 0x7f2a481045d0 2024-05-09T01:27:10.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.538+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/1734585692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a481a0380 con 0x7f2a481045d0 2024-05-09T01:27:10.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.541+0000 7f2a3e7fc700 1 -- 172.21.15.19:0/1734585692 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f2a44022470 con 0x7f2a481045d0 2024-05-09T01:27:10.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.542+0000 7f2a3e7fc700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2a340b6da0 0x7f2a340b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.542+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2a340b6da0 0x7f2a340b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.542+0000 7f2a3e7fc700 1 -- 172.21.15.19:0/1734585692 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2a440e30e0 con 0x7f2a481045d0 2024-05-09T01:27:10.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.542+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2a340b6da0 0x7f2a340b92b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2a48103520 tx=0x7f2a3800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:10.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a5def8b80 1 Processor -- start 2024-05-09T01:27:10.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a5def8b80 1 -- start start 2024-05-09T01:27:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b2609d00 0x55b0b2712240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 0x55b0b2712780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b270c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a5def8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b0b262f0a0 con 0x55b0b270c400 2024-05-09T01:27:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a5def8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b0b25d49a0 con 0x55b0b258f4a0 2024-05-09T01:27:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a5def8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b0b262b060 con 0x55b0b2609d00 2024-05-09T01:27:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a4da7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b270c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a4da7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b270c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58836/0 (socket says 172.21.15.19:58836) 2024-05-09T01:27:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a4da7f700 1 -- 172.21.15.19:0/1098917794 learned_addr learned my addr 172.21.15.19:0/1098917794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/1098917794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b2609d00 0x55b0b2712240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.544+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/1098917794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 0x55b0b2712780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.545+0000 7f2a4da7f700 1 -- 172.21.15.19:0/1098917794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b2609d00 msgr2=0x55b0b2712240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.545+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/1098917794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b2609d00 0x55b0b2712240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.545+0000 7f2a4da7f700 1 -- 172.21.15.19:0/1098917794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 msgr2=0x55b0b2712780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.545+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/1098917794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 0x55b0b2712780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.545+0000 7f2a4da7f700 1 -- 172.21.15.19:0/1098917794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b25c5500 con 0x55b0b270c400 2024-05-09T01:27:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.545+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/1098917794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b270c810 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2a40003f80 tx=0x7f2a40009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41aad36365cc1c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.545+0000 7f2a337fe700 1 -- 172.21.15.19:0/1098917794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40004bb0 con 0x55b0b270c400 2024-05-09T01:27:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.545+0000 7f2a337fe700 1 -- 172.21.15.19:0/1098917794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a40004d50 con 0x55b0b270c400 2024-05-09T01:27:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.545+0000 7f2a337fe700 1 -- 172.21.15.19:0/1098917794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40007d50 con 0x55b0b270c400 2024-05-09T01:27:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1098917794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 msgr2=0x55b0b270c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1098917794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b270c810 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2a40003f80 tx=0x7f2a40009ba0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1098917794 shutdown_connections 2024-05-09T01:27:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1098917794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b270c810 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1098917794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b2609d00 0x55b0b2712240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1098917794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 0x55b0b2712780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1098917794 >> 172.21.15.19:0/1098917794 conn(0x55b0b2592f70 msgr2=0x55b0b2707220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1098917794 shutdown_connections 2024-05-09T01:27:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1098917794 wait complete. 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.547+0000 7f2a5def8b80 1 -- start start 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a5def8b80 1 Processor -- start 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 0x55b0b2716040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b2609d00 0x55b0b2716580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a4ca7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b2609d00 0x55b0b2716580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a4ca7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b2609d00 0x55b0b2716580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58842/0 (socket says 172.21.15.19:58842) 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b270c400 0x55b0b2716ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a4ca7d700 1 -- 172.21.15.19:0/2722135718 learned_addr learned my addr 172.21.15.19:0/2722135718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b0b270d160 con 0x55b0b2609d00 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b0b25c9b20 con 0x55b0b258f4a0 2024-05-09T01:27:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b0b2719020 con 0x55b0b270c400 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.548+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 0x55b0b2716040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b270c400 0x55b0b2716ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a4d27e700 1 -- 172.21.15.19:0/2722135718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b270c400 msgr2=0x55b0b2716ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b270c400 0x55b0b2716ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a4d27e700 1 -- 172.21.15.19:0/2722135718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b2609d00 msgr2=0x55b0b2716580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b2609d00 0x55b0b2716580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a4d27e700 1 -- 172.21.15.19:0/2722135718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b24aa490 con 0x55b0b258f4a0 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b2609d00 0x55b0b2716580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a5def8b80 10 client.0 _ll_register_callbacks cb 0x55b0b24a06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 0x55b0b2716040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a440e0a70 tx=0x7f2a440e0ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85da280b53ccd30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.549+0000 7f2a30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.550+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a40004d50 con 0x55b0b258f4a0 2024-05-09T01:27:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.550+0000 7f2a30ff9700 1 -- 172.21.15.19:0/2722135718 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a440aa610 con 0x55b0b258f4a0 2024-05-09T01:27:10.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.550+0000 7f2a30ff9700 1 -- 172.21.15.19:0/2722135718 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2a44004380 con 0x55b0b258f4a0 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.550+0000 7f2a30ff9700 1 -- 172.21.15.19:0/2722135718 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a440ae9c0 con 0x55b0b258f4a0 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.550+0000 7f2a5def8b80 10 client.25223 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.550+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55b0b27236d0 con 0x55b0b258f4a0 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.551+0000 7f2a30ff9700 1 -- 172.21.15.19:0/2722135718 <== mon.2 v2:172.21.15.59:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2a440b2f50 con 0x55b0b258f4a0 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.551+0000 7f2a5def8b80 10 client.25223 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.551+0000 7f2a5def8b80 10 client.25223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.551+0000 7f2a5def8b80 20 client.25223 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.551+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b0b2725e50 0x55b0b2728360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.551+0000 7f2a5def8b80 4 client.25223 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.551+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0b263ca80 con 0x55b0b2725e50 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.551+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b0b2725e50 0x55b0b2728360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.552+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b0b2725e50 0x55b0b2728360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=944918fcc4ed13c8 server_cookie=efae1c0b60b1429 in_seq=0 out_seq=0 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.552+0000 7f2a30ff9700 10 client.25223 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.552+0000 7f2a30ff9700 1 -- 172.21.15.19:0/2722135718 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b0b263ca80 con 0x55b0b2725e50 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.552+0000 7f2a30ff9700 10 client.25223 handle_command_reply: tid=0 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.552+0000 7f2a5def8b80 1 client.25223 shutdown 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 msgr2=0x55b0b2716040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 0x55b0b2716040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a440e0a70 tx=0x7f2a440e0ef0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 shutdown_connections 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b0b2725e50 0x55b0b2728360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b270c400 0x55b0b2716ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b258f4a0 0x55b0b2716040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/2722135718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b2609d00 0x55b0b2716580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 >> 172.21.15.19:0/2722135718 conn(0x55b0b2592f70 msgr2=0x55b0b2700ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a30ff9700 0 client.25223 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 shutdown_connections 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 1 -- 172.21.15.19:0/2722135718 wait complete. 2024-05-09T01:27:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.553+0000 7f2a5def8b80 20 client.25223 trim_cache size 0 max 16384 2024-05-09T01:27:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a5def8b80 1 Processor -- start 2024-05-09T01:27:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a5def8b80 1 -- start start 2024-05-09T01:27:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b2714fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 0x55b0b2721ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b2701700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a5def8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481ac930 con 0x55b0b270c400 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a5def8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481ae470 con 0x55b0b2609d00 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a5def8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481ad000 con 0x55b0b258f4a0 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a4da7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b2701700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a4da7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b2701700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58852/0 (socket says 172.21.15.19:58852) 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a4da7f700 1 -- 172.21.15.19:0/1507415174 learned_addr learned my addr 172.21.15.19:0/1507415174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.585+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b2714fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 0x55b0b2721ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a4da7f700 1 -- 172.21.15.19:0/1507415174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 msgr2=0x55b0b2714fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b2714fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a4da7f700 1 -- 172.21.15.19:0/1507415174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 msgr2=0x55b0b2721ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 0x55b0b2721ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a4da7f700 1 -- 172.21.15.19:0/1507415174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b27188e0 con 0x55b0b270c400 2024-05-09T01:27:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 0x55b0b2721ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:10.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b2701700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2a400096d0 tx=0x7f2a40004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8137573c2ba53b6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:10.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a32ffd700 1 -- 172.21.15.19:0/1507415174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40004570 con 0x55b0b270c400 2024-05-09T01:27:10.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a32ffd700 1 -- 172.21.15.19:0/1507415174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a40007500 con 0x55b0b270c400 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.586+0000 7f2a32ffd700 1 -- 172.21.15.19:0/1507415174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400156a0 con 0x55b0b270c400 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1507415174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 msgr2=0x55b0b2701700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b2701700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2a400096d0 tx=0x7f2a40004f60 comp rx=0 tx=0).stop 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1507415174 shutdown_connections 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b2701700 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b2714fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1507415174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 0x55b0b2721ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1507415174 >> 172.21.15.19:0/1507415174 conn(0x55b0b2592f70 msgr2=0x55b0b2700ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1507415174 shutdown_connections 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1507415174 wait complete. 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 -- start start 2024-05-09T01:27:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.588+0000 7f2a5def8b80 1 Processor -- start 2024-05-09T01:27:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b271e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 0x55b0b272cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a5def8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b272d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a5def8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481adda0 con 0x55b0b270c400 2024-05-09T01:27:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a5def8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481ad6d0 con 0x55b0b2609d00 2024-05-09T01:27:10.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a5def8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481aa8f0 con 0x55b0b258f4a0 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a5def8b80 10 client.0 _ll_register_callbacks cb 0x55b0b24a06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a4d27e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b271e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a4d27e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b271e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36968/0 (socket says 172.21.15.19:36968) 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a4d27e700 1 -- 172.21.15.19:0/1499631084 learned_addr learned my addr 172.21.15.19:0/1499631084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 0x55b0b272cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.589+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b272d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a4d27e700 1 -- 172.21.15.19:0/1499631084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 msgr2=0x55b0b272cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 0x55b0b272cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a4d27e700 1 -- 172.21.15.19:0/1499631084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 msgr2=0x55b0b272d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b272d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a4d27e700 1 -- 172.21.15.19:0/1499631084 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b271ecb0 con 0x55b0b258f4a0 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a4da7f700 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b272d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:10.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a4d27e700 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b271e470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a440aa310 tx=0x7f2a44009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceba8157b21c8e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a30ff9700 1 -- 172.21.15.19:0/1499631084 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a440aa610 con 0x55b0b258f4a0 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a30ff9700 1 -- 172.21.15.19:0/1499631084 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2a440a6c00 con 0x55b0b258f4a0 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.590+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1499631084 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a40007500 con 0x55b0b258f4a0 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a30ff9700 1 -- 172.21.15.19:0/1499631084 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a440b33e0 con 0x55b0b258f4a0 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a5def8b80 10 client.25237 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1499631084 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b0b27901e0 con 0x55b0b258f4a0 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a30ff9700 1 -- 172.21.15.19:0/1499631084 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a40004d50 con 0x55b0b258f4a0 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a30ff9700 1 -- 172.21.15.19:0/1499631084 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2a440a5f80 con 0x55b0b258f4a0 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a5def8b80 10 client.25237 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a5def8b80 10 client.25237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a5def8b80 20 client.25237 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b0b27906f0 0x55b0b2792b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a5def8b80 4 client.25237 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.591+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1499631084 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2a40007500 con 0x55b0b27906f0 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b0b27906f0 0x55b0b2792b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a4ca7d700 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b0b27906f0 0x55b0b2792b10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ce0e3bced8beae8 server_cookie=74b3b4713149fd43 in_seq=0 out_seq=0 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a30ff9700 10 client.25237 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a30ff9700 1 -- 172.21.15.19:0/1499631084 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f2a40007500 con 0x55b0b27906f0 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a30ff9700 10 client.25237 handle_command_reply: tid=0 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a5def8b80 1 client.25237 shutdown 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1499631084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 msgr2=0x55b0b271e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b271e470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a440aa310 tx=0x7f2a44009280 comp rx=0 tx=0).stop 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1499631084 shutdown_connections 2024-05-09T01:27:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.592+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b0b27906f0 0x55b0b2792b10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.593+0000 7f2a30ff9700 0 client.25237 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.593+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0b270c400 0x55b0b272d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.593+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0b258f4a0 0x55b0b271e470 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.593+0000 7f2a5def8b80 1 --2- 172.21.15.19:0/1499631084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0b2609d00 0x55b0b272cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.593+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1499631084 >> 172.21.15.19:0/1499631084 conn(0x55b0b2592f70 msgr2=0x55b0b270b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.593+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1499631084 shutdown_connections 2024-05-09T01:27:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.593+0000 7f2a5def8b80 1 -- 172.21.15.19:0/1499631084 wait complete. 2024-05-09T01:27:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.593+0000 7f2a5def8b80 20 client.25237 trim_cache size 0 max 16384 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "86072fe2-5f9a-43a9-999b-3fa3697e4107": { 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "86072fe2-5f9a-43a9-999b-3fa3697e4107", 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:27:10.594 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:27:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2a340b6da0 msgr2=0x7f2a340b92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2a340b6da0 0x7f2a340b92b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2a48103520 tx=0x7f2a3800a300 comp rx=0 tx=0).stop 2024-05-09T01:27:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 msgr2=0x7f2a4819ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 0x7f2a4819ad90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400b580 tx=0x7f2a440047b0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/1734585692 shutdown_connections 2024-05-09T01:27:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48109850 0x7f2a4819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2a340b6da0 0x7f2a340b92b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a481045d0 0x7f2a4819ad90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 --2- 172.21.15.19:0/1734585692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48108f00 0x7f2a4819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:10.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/1734585692 >> 172.21.15.19:0/1734585692 conn(0x7f2a480ffea0 msgr2=0x7f2a4810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:10.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/1734585692 shutdown_connections 2024-05-09T01:27:10.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:10.594+0000 7f2a4f4e2700 1 -- 172.21.15.19:0/1734585692 wait complete. 2024-05-09T01:27:10.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:10 smithi059 ceph-mon[34351]: pgmap v427: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:27:10.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:10 smithi196 ceph-mon[35041]: pgmap v427: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:27:11.455 INFO:tasks.cephfs.filesystem:scrub status for tag:86072fe2-5f9a-43a9-999b-3fa3697e4107 - {'path': '/', 'tag': '86072fe2-5f9a-43a9-999b-3fa3697e4107', 'options': 'recursive,force'} 2024-05-09T01:27:12.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:12 smithi059 ceph-mon[34351]: pgmap v428: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T01:27:12.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:12 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:27:12.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:12 smithi019 ceph-mon[30158]: pgmap v428: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T01:27:12.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:12 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:27:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:12 smithi196 ceph-mon[35041]: pgmap v428: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T01:27:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:12 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:27:14.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:14 smithi019 ceph-mon[30158]: pgmap v429: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T01:27:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.454+0000 7fb206ad8640 1 -- 172.21.15.19:0/256860623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200109bb0 msgr2=0x7fb20010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.454+0000 7fb206ad8640 1 --2- 172.21.15.19:0/256860623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200109bb0 0x7fb20010bfd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ca0 tx=0x7fb1f802a8c0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.454+0000 7fb206ad8640 1 -- 172.21.15.19:0/256860623 shutdown_connections 2024-05-09T01:27:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.454+0000 7fb206ad8640 1 --2- 172.21.15.19:0/256860623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200109bb0 0x7fb20010bfd0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.454+0000 7fb206ad8640 1 --2- 172.21.15.19:0/256860623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb200104eb0 0x7fb2001052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.454+0000 7fb206ad8640 1 --2- 172.21.15.19:0/256860623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb200103c70 0x7fb200104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.454+0000 7fb206ad8640 1 -- 172.21.15.19:0/256860623 >> 172.21.15.19:0/256860623 conn(0x7fb2000fee60 msgr2=0x7fb200101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.454+0000 7fb206ad8640 1 -- 172.21.15.19:0/256860623 shutdown_connections 2024-05-09T01:27:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.454+0000 7fb206ad8640 1 -- 172.21.15.19:0/256860623 wait complete. 2024-05-09T01:27:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb206ad8640 1 Processor -- start 2024-05-09T01:27:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb206ad8640 1 -- start start 2024-05-09T01:27:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb206ad8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb200103c70 0x7fb200076980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb206ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200104eb0 0x7fb200075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb206ad8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb200109bb0 0x7fb200075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb206ad8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb20010f7a0 con 0x7fb200104eb0 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb206ad8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb20010f620 con 0x7fb200103c70 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb206ad8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb20010f920 con 0x7fb200109bb0 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200104eb0 0x7fb200075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200104eb0 0x7fb200075020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58886/0 (socket says 172.21.15.19:58886) 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb1fffff640 1 -- 172.21.15.19:0/2832692585 learned_addr learned my addr 172.21.15.19:0/2832692585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb205075640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb200109bb0 0x7fb200075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.455+0000 7fb204874640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb200103c70 0x7fb200076980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb1fffff640 1 -- 172.21.15.19:0/2832692585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb200109bb0 msgr2=0x7fb200075560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb1fffff640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb200109bb0 0x7fb200075560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb1fffff640 1 -- 172.21.15.19:0/2832692585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb200103c70 msgr2=0x7fb200076980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb1fffff640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb200103c70 0x7fb200076980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb1fffff640 1 -- 172.21.15.19:0/2832692585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb200075c90 con 0x7fb200104eb0 2024-05-09T01:27:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb1fffff640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200104eb0 0x7fb200075020 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e9f0 tx=0x7fb1ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb1fdffb640 1 -- 172.21.15.19:0/2832692585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00cd80 con 0x7fb200104eb0 2024-05-09T01:27:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb206ad8640 1 -- 172.21.15.19:0/2832692585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2001a43a0 con 0x7fb200104eb0 2024-05-09T01:27:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb1fdffb640 1 -- 172.21.15.19:0/2832692585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1ec0108e0 con 0x7fb200104eb0 2024-05-09T01:27:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb1fdffb640 1 -- 172.21.15.19:0/2832692585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec018980 con 0x7fb200104eb0 2024-05-09T01:27:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.456+0000 7fb206ad8640 1 -- 172.21.15.19:0/2832692585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2001a4890 con 0x7fb200104eb0 2024-05-09T01:27:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb20d988740 1 Processor -- start 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb20d988740 1 -- start start 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb20d988740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 0x559e5d650500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb20d988740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d650a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb20d988740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d657980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb20d988740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e5d5b47c0 con 0x559e5d28b4d0 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb20d988740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e5d5500e0 con 0x559e5d2832f0 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb20d988740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559e5d5b0780 con 0x559e5d2ac1b0 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb204874640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d650a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb204874640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d650a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56354/0 (socket says 172.21.15.19:56354) 2024-05-09T01:27:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb204874640 1 -- 172.21.15.19:0/3230654583 learned_addr learned my addr 172.21.15.19:0/3230654583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.460+0000 7fb205075640 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 0x559e5d650500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.461+0000 7fb1fdffb640 1 -- 172.21.15.19:0/2832692585 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fb1ec010430 con 0x7fb200104eb0 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.461+0000 7fb1fffff640 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d657980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.461+0000 7fb205075640 1 -- 172.21.15.19:0/3230654583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 msgr2=0x559e5d650a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.461+0000 7fb205075640 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d650a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.461+0000 7fb205075640 1 -- 172.21.15.19:0/3230654583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 msgr2=0x559e5d657980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.461+0000 7fb205075640 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d657980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.461+0000 7fb205075640 1 -- 172.21.15.19:0/3230654583 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5d5797c0 con 0x559e5d2ac1b0 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.461+0000 7fb1fffff640 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d657980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.462+0000 7fb205075640 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 0x559e5d650500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8031b50 tx=0x7fb1f802f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef786bfc58153443 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.462+0000 7fb1eaffd640 1 -- 172.21.15.19:0/3230654583 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8031650 con 0x559e5d2ac1b0 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.462+0000 7fb1eaffd640 1 -- 172.21.15.19:0/3230654583 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f8036030 con 0x559e5d2ac1b0 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.462+0000 7fb1fdffb640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1f00b2870 0x7fb1f00b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.462+0000 7fb1fdffb640 1 -- 172.21.15.19:0/2832692585 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb1ec0e2770 con 0x7fb200104eb0 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.462+0000 7fb204874640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1f00b2870 0x7fb1f00b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb1eaffd640 1 -- 172.21.15.19:0/3230654583 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8004670 con 0x559e5d2ac1b0 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb20d988740 1 -- 172.21.15.19:0/3230654583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 msgr2=0x559e5d650500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb204874640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1f00b2870 0x7fb1f00b4d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x559e5d6518e0 tx=0x7fb1f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb20d988740 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 0x559e5d650500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8031b50 tx=0x7fb1f802f410 comp rx=0 tx=0).stop 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb20d988740 1 -- 172.21.15.19:0/3230654583 shutdown_connections 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb20d988740 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d657980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb20d988740 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d650a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb20d988740 1 --2- 172.21.15.19:0/3230654583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 0x559e5d650500 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb20d988740 1 -- 172.21.15.19:0/3230654583 >> 172.21.15.19:0/3230654583 conn(0x559e5d2a0300 msgr2=0x559e5d64be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb20d988740 1 -- 172.21.15.19:0/3230654583 shutdown_connections 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.463+0000 7fb20d988740 1 -- 172.21.15.19:0/3230654583 wait complete. 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.464+0000 7fb20d988740 1 -- start start 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.464+0000 7fb20d988740 1 Processor -- start 2024-05-09T01:27:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.464+0000 7fb20d988740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.464+0000 7fb20d988740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d65a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.464+0000 7fb20d988740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 0x559e5d661580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.464+0000 7fb20d988740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e5d6581f0 con 0x559e5d28b4d0 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.464+0000 7fb20d988740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e5d53b9c0 con 0x559e5d2832f0 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.464+0000 7fb20d988740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559e5d658370 con 0x559e5d2ac1b0 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb20d988740 10 client.0 _ll_register_callbacks cb 0x559e5d568020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d65a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d65a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58902/0 (socket says 172.21.15.19:58902) 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1fffff640 1 -- 172.21.15.19:0/3815900574 learned_addr learned my addr 172.21.15.19:0/3815900574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb204874640 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb205075640 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 0x559e5d661580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1fffff640 1 -- 172.21.15.19:0/3815900574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 msgr2=0x559e5d661580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1fffff640 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 0x559e5d661580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1fffff640 1 -- 172.21.15.19:0/3815900574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 msgr2=0x559e5d65a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1fffff640 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1fffff640 1 -- 172.21.15.19:0/3815900574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5d5621f0 con 0x559e5d28b4d0 2024-05-09T01:27:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1fffff640 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d65a690 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0a9ba0 tx=0x7fb1ec010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fa0431226a9d41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.465+0000 7fb1dffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.466+0000 7fb1dffff640 1 -- 172.21.15.19:0/3815900574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec0a9c10 con 0x559e5d28b4d0 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.466+0000 7fb1dffff640 1 -- 172.21.15.19:0/3815900574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb1ec010a90 con 0x559e5d28b4d0 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.466+0000 7fb20d988740 1 -- 172.21.15.19:0/3815900574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f8036030 con 0x559e5d28b4d0 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.466+0000 7fb1dffff640 1 -- 172.21.15.19:0/3815900574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb1ec0adfc0 con 0x559e5d28b4d0 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.466+0000 7fb20d988740 10 client.15570 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.466+0000 7fb20d988740 1 -- 172.21.15.19:0/3815900574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e5d317790 con 0x559e5d28b4d0 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.466+0000 7fb1dffff640 1 -- 172.21.15.19:0/3815900574 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f8036030 con 0x559e5d28b4d0 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.466+0000 7fb1dffff640 1 -- 172.21.15.19:0/3815900574 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb1ec0a88e0 con 0x559e5d28b4d0 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.467+0000 7fb20d988740 10 client.15570 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.467+0000 7fb20d988740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.467+0000 7fb20d988740 20 client.15570 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.467+0000 7fb20d988740 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559e5d26ad80 0x559e5d667990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.467+0000 7fb20d988740 4 client.15570 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.467+0000 7fb20d988740 1 -- 172.21.15.19:0/3815900574 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e5d5c23d0 con 0x559e5d26ad80 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.467+0000 7fb204874640 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559e5d26ad80 0x559e5d667990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.467+0000 7fb204874640 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559e5d26ad80 0x559e5d667990 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bef154f8b0487d8 server_cookie=91a15adc2a253884 in_seq=0 out_seq=0 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.467+0000 7fb1dffff640 10 client.15570 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.468+0000 7fb1dffff640 1 -- 172.21.15.19:0/3815900574 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e5d5c23d0 con 0x559e5d26ad80 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.468+0000 7fb1dffff640 10 client.15570 handle_command_reply: tid=0 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.468+0000 7fb20d988740 1 client.15570 shutdown 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.468+0000 7fb20d988740 1 -- 172.21.15.19:0/3815900574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 msgr2=0x559e5d65a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d65a690 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0a9ba0 tx=0x7fb1ec010720 comp rx=0 tx=0).stop 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 1 -- 172.21.15.19:0/3815900574 shutdown_connections 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559e5d26ad80 0x559e5d667990 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2ac1b0 0x559e5d661580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d28b4d0 0x559e5d65a690 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 1 --2- 172.21.15.19:0/3815900574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 1 -- 172.21.15.19:0/3815900574 >> 172.21.15.19:0/3815900574 conn(0x559e5d2a0300 msgr2=0x559e5d64a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb1dffff640 0 client.15570 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 1 -- 172.21.15.19:0/3815900574 shutdown_connections 2024-05-09T01:27:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 1 -- 172.21.15.19:0/3815900574 wait complete. 2024-05-09T01:27:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.469+0000 7fb20d988740 20 client.15570 trim_cache size 0 max 16384 2024-05-09T01:27:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.484+0000 7fb20d988740 1 Processor -- start 2024-05-09T01:27:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.484+0000 7fb20d988740 1 -- start start 2024-05-09T01:27:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb20d988740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2a0300 0x559e5d654690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb20d988740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2832f0 0x559e5d654bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb20d988740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d64b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb20d988740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001ad5b0 con 0x559e5d2ac1b0 2024-05-09T01:27:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb205075640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d64b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb205075640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d64b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58918/0 (socket says 172.21.15.19:58918) 2024-05-09T01:27:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb205075640 1 -- 172.21.15.19:0/842339686 learned_addr learned my addr 172.21.15.19:0/842339686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb20d988740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001af0f0 con 0x559e5d2a0300 2024-05-09T01:27:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb20d988740 1 -- 172.21.15.19:0/842339686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001adc80 con 0x559e5d2832f0 2024-05-09T01:27:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb204874640 1 --2- 172.21.15.19:0/842339686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2a0300 0x559e5d654690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.485+0000 7fb1fffff640 1 --2- 172.21.15.19:0/842339686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2832f0 0x559e5d654bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.486+0000 7fb204874640 1 -- 172.21.15.19:0/842339686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2832f0 msgr2=0x559e5d654bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.486+0000 7fb204874640 1 --2- 172.21.15.19:0/842339686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2832f0 0x559e5d654bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.486+0000 7fb204874640 1 -- 172.21.15.19:0/842339686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 msgr2=0x559e5d64b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.486+0000 7fb204874640 1 --2- 172.21.15.19:0/842339686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d64b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.486+0000 7fb204874640 1 -- 172.21.15.19:0/842339686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5d38e5f0 con 0x559e5d2a0300 2024-05-09T01:27:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.486+0000 7fb204874640 1 --2- 172.21.15.19:0/842339686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2a0300 0x559e5d654690 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x559e5d668090 tx=0x7fb1f401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270dada749d152b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.486+0000 7fb1ea7fc640 1 -- 172.21.15.19:0/842339686 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f401fc30 con 0x559e5d2a0300 2024-05-09T01:27:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.486+0000 7fb1ea7fc640 1 -- 172.21.15.19:0/842339686 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f4039070 con 0x559e5d2a0300 2024-05-09T01:27:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.486+0000 7fb1ea7fc640 1 -- 172.21.15.19:0/842339686 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4034ad0 con 0x559e5d2a0300 2024-05-09T01:27:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.487+0000 7fb20d988740 1 -- 172.21.15.19:0/842339686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2a0300 msgr2=0x559e5d654690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.487+0000 7fb20d988740 1 --2- 172.21.15.19:0/842339686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2a0300 0x559e5d654690 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x559e5d668090 tx=0x7fb1f401a000 comp rx=0 tx=0).stop 2024-05-09T01:27:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.487+0000 7fb20d988740 1 -- 172.21.15.19:0/842339686 shutdown_connections 2024-05-09T01:27:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.487+0000 7fb20d988740 1 --2- 172.21.15.19:0/842339686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d64b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.487+0000 7fb20d988740 1 --2- 172.21.15.19:0/842339686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2832f0 0x559e5d654bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.487+0000 7fb20d988740 1 --2- 172.21.15.19:0/842339686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2a0300 0x559e5d654690 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.487+0000 7fb20d988740 1 -- 172.21.15.19:0/842339686 >> 172.21.15.19:0/842339686 conn(0x559e5d650b60 msgr2=0x559e5d658810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.488+0000 7fb20d988740 1 -- 172.21.15.19:0/842339686 shutdown_connections 2024-05-09T01:27:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.488+0000 7fb20d988740 1 -- 172.21.15.19:0/842339686 wait complete. 2024-05-09T01:27:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.488+0000 7fb20d988740 1 -- start start 2024-05-09T01:27:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.488+0000 7fb20d988740 1 Processor -- start 2024-05-09T01:27:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb20d988740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb20d988740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2a0300 0x559e5d65eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb20d988740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d65fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb20d988740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001aea20 con 0x559e5d2ac1b0 2024-05-09T01:27:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb20d988740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001ae350 con 0x559e5d2832f0 2024-05-09T01:27:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb20d988740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001ab590 con 0x559e5d2a0300 2024-05-09T01:27:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb20d988740 10 client.0 _ll_register_callbacks cb 0x7fb1ec0a5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb205075640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d65fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb204874640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb205075640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d65fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58932/0 (socket says 172.21.15.19:58932) 2024-05-09T01:27:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb204874640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56388/0 (socket says 172.21.15.19:56388) 2024-05-09T01:27:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb205075640 1 -- 172.21.15.19:0/556667730 learned_addr learned my addr 172.21.15.19:0/556667730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb1fffff640 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2a0300 0x559e5d65eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb204874640 1 -- 172.21.15.19:0/556667730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2a0300 msgr2=0x559e5d65eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb204874640 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2a0300 0x559e5d65eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb204874640 1 -- 172.21.15.19:0/556667730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 msgr2=0x559e5d65fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb204874640 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d65fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb204874640 1 -- 172.21.15.19:0/556667730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5d578ed0 con 0x559e5d2832f0 2024-05-09T01:27:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb205075640 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d65fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.489+0000 7fb1fffff640 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2a0300 0x559e5d65eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.490+0000 7fb204874640 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65e970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4019040 tx=0x7fb1f401fd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51c7846b3c70a40f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.490+0000 7fb1dffff640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:27:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.490+0000 7fb1dffff640 1 -- 172.21.15.19:0/556667730 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4005240 con 0x559e5d2832f0 2024-05-09T01:27:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.490+0000 7fb1dffff640 1 -- 172.21.15.19:0/556667730 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb1f40053e0 con 0x559e5d2832f0 2024-05-09T01:27:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.490+0000 7fb20d988740 1 -- 172.21.15.19:0/556667730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f4039070 con 0x559e5d2832f0 2024-05-09T01:27:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.490+0000 7fb1dffff640 1 -- 172.21.15.19:0/556667730 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb1f404d070 con 0x559e5d2832f0 2024-05-09T01:27:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.490+0000 7fb20d988740 10 client.25247 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.490+0000 7fb20d988740 1 -- 172.21.15.19:0/556667730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e5d6cffb0 con 0x559e5d2832f0 2024-05-09T01:27:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.490+0000 7fb1dffff640 1 -- 172.21.15.19:0/556667730 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f4039070 con 0x559e5d2832f0 2024-05-09T01:27:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb1dffff640 1 -- 172.21.15.19:0/556667730 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb1f4034a90 con 0x559e5d2832f0 2024-05-09T01:27:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb20d988740 10 client.25247 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb20d988740 10 client.25247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb20d988740 20 client.25247 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb20d988740 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559e5d2401c0 0x559e5d6d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb20d988740 4 client.25247 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:27:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb20d988740 1 -- 172.21.15.19:0/556667730 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e5d5bc240 con 0x559e5d2401c0 2024-05-09T01:27:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb1fffff640 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559e5d2401c0 0x559e5d6d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb1dffff640 10 client.25247 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.491+0000 7fb1fffff640 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559e5d2401c0 0x559e5d6d2940 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79fe15054835a978 server_cookie=cea2aeecfa8d3bf4 in_seq=0 out_seq=0 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb1dffff640 1 -- 172.21.15.19:0/556667730 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559e5d5bc240 con 0x559e5d2401c0 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb1dffff640 10 client.25247 handle_command_reply: tid=0 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb20d988740 1 client.25247 shutdown 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb20d988740 1 -- 172.21.15.19:0/556667730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 msgr2=0x559e5d65e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb20d988740 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65e970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4019040 tx=0x7fb1f401fd90 comp rx=0 tx=0).stop 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb20d988740 1 -- 172.21.15.19:0/556667730 shutdown_connections 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb20d988740 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559e5d2401c0 0x559e5d6d2940 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb20d988740 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e5d2ac1b0 0x559e5d65fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb1dffff640 0 client.25247 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb20d988740 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e5d2a0300 0x559e5d65eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb20d988740 1 --2- 172.21.15.19:0/556667730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e5d2832f0 0x559e5d65e970 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.496+0000 7fb20d988740 1 -- 172.21.15.19:0/556667730 >> 172.21.15.19:0/556667730 conn(0x559e5d650b60 msgr2=0x559e5d65a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.497+0000 7fb20d988740 1 -- 172.21.15.19:0/556667730 shutdown_connections 2024-05-09T01:27:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.497+0000 7fb20d988740 1 -- 172.21.15.19:0/556667730 wait complete. 2024-05-09T01:27:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.497+0000 7fb20d988740 20 client.25247 trim_cache size 0 max 16384 2024-05-09T01:27:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:27:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:27:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 -- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1f00b2870 msgr2=0x7fb1f00b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1f00b2870 0x7fb1f00b4d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x559e5d6518e0 tx=0x7fb1f4005f90 comp rx=0 tx=0).stop 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 -- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200104eb0 msgr2=0x7fb200075020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200104eb0 0x7fb200075020 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e9f0 tx=0x7fb1ec00eeb0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 -- 172.21.15.19:0/2832692585 shutdown_connections 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1f00b2870 0x7fb1f00b4d80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb200109bb0 0x7fb200075560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb200104eb0 0x7fb200075020 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 --2- 172.21.15.19:0/2832692585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb200103c70 0x7fb200076980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 -- 172.21.15.19:0/2832692585 >> 172.21.15.19:0/2832692585 conn(0x7fb2000fee60 msgr2=0x7fb200105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 -- 172.21.15.19:0/2832692585 shutdown_connections 2024-05-09T01:27:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:14.498+0000 7fb206ad8640 1 -- 172.21.15.19:0/2832692585 wait complete. 2024-05-09T01:27:14.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:14 smithi059 ceph-mon[34351]: pgmap v429: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T01:27:14.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:14 smithi196 ceph-mon[35041]: pgmap v429: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T01:27:16.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:16 smithi059 ceph-mon[34351]: pgmap v430: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:27:16.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:16 smithi019 ceph-mon[30158]: pgmap v430: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:27:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:16 smithi196 ceph-mon[35041]: pgmap v430: 97 pgs: 97 active+clean; 674 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:27:18.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:18 smithi059 ceph-mon[34351]: pgmap v431: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-09T01:27:18.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:18 smithi019 ceph-mon[30158]: pgmap v431: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-09T01:27:18.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:18 smithi196 ceph-mon[35041]: pgmap v431: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-09T01:27:20.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:20 smithi059 ceph-mon[34351]: pgmap v432: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:27:20.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:20 smithi019 ceph-mon[30158]: pgmap v432: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:27:20.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:20 smithi196 ceph-mon[35041]: pgmap v432: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:27:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:22 smithi059 ceph-mon[34351]: pgmap v433: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:27:22.598 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:22 smithi059 ceph-mon[34351]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-05-09T01:27:22.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:22 smithi019 ceph-mon[30158]: pgmap v433: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:27:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:22 smithi019 ceph-mon[30158]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-05-09T01:27:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:22 smithi196 ceph-mon[35041]: pgmap v433: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:27:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:22 smithi196 ceph-mon[35041]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-05-09T01:27:24.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:24 smithi059 ceph-mon[34351]: pgmap v434: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:27:24.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:24 smithi019 ceph-mon[30158]: pgmap v434: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:27:24.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:24 smithi196 ceph-mon[35041]: pgmap v434: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:27:26.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:26 smithi059 ceph-mon[34351]: pgmap v435: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T01:27:26.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:26 smithi019 ceph-mon[30158]: pgmap v435: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T01:27:26.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:26 smithi196 ceph-mon[35041]: pgmap v435: 97 pgs: 97 active+clean; 675 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T01:27:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.652+0000 7f03d51f3640 1 -- 172.21.15.19:0/1238296345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0108a90 msgr2=0x7f03d010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.652+0000 7f03d51f3640 1 --2- 172.21.15.19:0/1238296345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0108a90 0x7f03d010aeb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009ca0 tx=0x7f03c802a8c0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.652+0000 7f03d51f3640 1 -- 172.21.15.19:0/1238296345 shutdown_connections 2024-05-09T01:27:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.652+0000 7f03d51f3640 1 --2- 172.21.15.19:0/1238296345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0108a90 0x7f03d010aeb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.652+0000 7f03d51f3640 1 --2- 172.21.15.19:0/1238296345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0103d90 0x7f03d01041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.652+0000 7f03d51f3640 1 --2- 172.21.15.19:0/1238296345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0102b50 0x7f03d0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.652+0000 7f03d51f3640 1 -- 172.21.15.19:0/1238296345 >> 172.21.15.19:0/1238296345 conn(0x7f03d00fdd40 msgr2=0x7f03d01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.652+0000 7f03d51f3640 1 -- 172.21.15.19:0/1238296345 shutdown_connections 2024-05-09T01:27:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.652+0000 7f03d51f3640 1 -- 172.21.15.19:0/1238296345 wait complete. 2024-05-09T01:27:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.653+0000 7f03d51f3640 1 Processor -- start 2024-05-09T01:27:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.653+0000 7f03d51f3640 1 -- start start 2024-05-09T01:27:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.653+0000 7f03d51f3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0102b50 0x7f03d019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.653+0000 7f03d51f3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0103d90 0x7f03d019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.653+0000 7f03d51f3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0108a90 0x7f03d01a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.653+0000 7f03d51f3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010f770 con 0x7f03d0103d90 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.653+0000 7f03d51f3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010f5f0 con 0x7f03d0102b50 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03d51f3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010f8f0 con 0x7f03d0108a90 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ce59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0103d90 0x7f03d019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ce59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0103d90 0x7f03d019a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37642/0 (socket says 172.21.15.19:37642) 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ce59c640 1 -- 172.21.15.19:0/2631626071 learned_addr learned my addr 172.21.15.19:0/2631626071 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03cf59e640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0108a90 0x7f03d01a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ced9d640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0102b50 0x7f03d019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ce59c640 1 -- 172.21.15.19:0/2631626071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0108a90 msgr2=0x7f03d01a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ce59c640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0108a90 0x7f03d01a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ce59c640 1 -- 172.21.15.19:0/2631626071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0102b50 msgr2=0x7f03d019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ce59c640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0102b50 0x7f03d019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ce59c640 1 -- 172.21.15.19:0/2631626071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d010b6a0 con 0x7f03d0103d90 2024-05-09T01:27:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.654+0000 7f03ce59c640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0103d90 0x7f03d019a850 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f03c400d950 tx=0x7f03c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.655+0000 7f03b7fff640 1 -- 172.21.15.19:0/2631626071 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4017070 con 0x7f03d0103d90 2024-05-09T01:27:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.655+0000 7f03d51f3640 1 -- 172.21.15.19:0/2631626071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d010b990 con 0x7f03d0103d90 2024-05-09T01:27:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.655+0000 7f03b7fff640 1 -- 172.21.15.19:0/2631626071 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c4009c30 con 0x7f03d0103d90 2024-05-09T01:27:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.656+0000 7f03b7fff640 1 -- 172.21.15.19:0/2631626071 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c400f620 con 0x7f03d0103d90 2024-05-09T01:27:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.656+0000 7f03d51f3640 1 -- 172.21.15.19:0/2631626071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d010bf30 con 0x7f03d0103d90 2024-05-09T01:27:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.659+0000 7f03b7fff640 1 -- 172.21.15.19:0/2631626071 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f03c4005020 con 0x7f03d0103d90 2024-05-09T01:27:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.659+0000 7f03dc17e740 1 Processor -- start 2024-05-09T01:27:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.659+0000 7f03dc17e740 1 -- start start 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f51500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f51a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f58980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03dc17e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a33eb57c0 con 0x561a33b842f0 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03dc17e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561a33e510e0 con 0x561a33bad1b0 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03dc17e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a33eb1780 con 0x561a33b8c4d0 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03cf59e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f51500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03ced9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f51a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03cf59e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f51500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:59944/0 (socket says 172.21.15.19:59944) 2024-05-09T01:27:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03cf59e640 1 -- 172.21.15.19:0/265645571 learned_addr learned my addr 172.21.15.19:0/265645571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.660+0000 7f03ce59c640 1 --2- 172.21.15.19:0/265645571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f58980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03cf59e640 1 -- 172.21.15.19:0/265645571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 msgr2=0x561a33f58980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03cf59e640 1 --2- 172.21.15.19:0/265645571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f58980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03cf59e640 1 -- 172.21.15.19:0/265645571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 msgr2=0x561a33f51a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03cf59e640 1 --2- 172.21.15.19:0/265645571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f51a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03b7fff640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03b80b27e0 0x7f03b80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03cf59e640 1 -- 172.21.15.19:0/265645571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a33e7a7c0 con 0x561a33bad1b0 2024-05-09T01:27:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03ced9d640 1 --2- 172.21.15.19:0/265645571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f51a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03ce59c640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03b80b27e0 0x7f03b80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03cf59e640 1 --2- 172.21.15.19:0/265645571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f51500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03c8035000 tx=0x7f03c802f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f683fbc6cd8e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03b7fff640 1 -- 172.21.15.19:0/2631626071 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03c40e17a0 con 0x7f03d0103d90 2024-05-09T01:27:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03b57fa640 1 -- 172.21.15.19:0/265645571 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8031650 con 0x561a33bad1b0 2024-05-09T01:27:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03b57fa640 1 -- 172.21.15.19:0/265645571 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c8036030 con 0x561a33bad1b0 2024-05-09T01:27:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.661+0000 7f03b57fa640 1 -- 172.21.15.19:0/265645571 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8004800 con 0x561a33bad1b0 2024-05-09T01:27:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 -- 172.21.15.19:0/265645571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 msgr2=0x561a33f51500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 --2- 172.21.15.19:0/265645571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f51500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03c8035000 tx=0x7f03c802f4c0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03ce59c640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03b80b27e0 0x7f03b80b4cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f03c4087a70 tx=0x7f03c40a8bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 -- 172.21.15.19:0/265645571 shutdown_connections 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 --2- 172.21.15.19:0/265645571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f58980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 --2- 172.21.15.19:0/265645571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f51a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 --2- 172.21.15.19:0/265645571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f51500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 -- 172.21.15.19:0/265645571 >> 172.21.15.19:0/265645571 conn(0x561a33ba1300 msgr2=0x561a33f4ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 -- 172.21.15.19:0/265645571 shutdown_connections 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 -- 172.21.15.19:0/265645571 wait complete. 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 -- start start 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.663+0000 7f03dc17e740 1 Processor -- start 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b842f0 0x561a33f5b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b8c4d0 0x561a33f5b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03ced9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b8c4d0 0x561a33f5b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03ced9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b8c4d0 0x561a33f5b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37662/0 (socket says 172.21.15.19:37662) 2024-05-09T01:27:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03cf59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b842f0 0x561a33f5b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03ced9d640 1 -- 172.21.15.19:0/3241697979 learned_addr learned my addr 172.21.15.19:0/3241697979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03dc17e740 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f62580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a33f591f0 con 0x561a33b8c4d0 2024-05-09T01:27:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561a33e3c9c0 con 0x561a33bad1b0 2024-05-09T01:27:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.664+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a33f59370 con 0x561a33b842f0 2024-05-09T01:27:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03ced9d640 1 -- 172.21.15.19:0/3241697979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b842f0 msgr2=0x561a33f5b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03ced9d640 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b842f0 0x561a33f5b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03ced9d640 1 -- 172.21.15.19:0/3241697979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 msgr2=0x561a33f62580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03ced9d640 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f62580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03ced9d640 1 -- 172.21.15.19:0/3241697979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a33e631f0 con 0x561a33b8c4d0 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03cf59e640 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b842f0 0x561a33f5b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03dc17e740 10 client.0 _ll_register_callbacks cb 0x561a33e69020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c8036030 con 0x561a33b8c4d0 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03ced9d640 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b8c4d0 0x561a33f5b620 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f03c0012a50 tx=0x7f03c0012f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb3beb47c8ce256 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.665+0000 7f03b2ff5640 10 client.15588 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.666+0000 7f03b2ff5640 1 -- 172.21.15.19:0/3241697979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0012190 con 0x561a33b8c4d0 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.666+0000 7f03b2ff5640 1 -- 172.21.15.19:0/3241697979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03c0004030 con 0x561a33b8c4d0 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.666+0000 7f03b2ff5640 1 -- 172.21.15.19:0/3241697979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03c0004290 con 0x561a33b8c4d0 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.666+0000 7f03dc17e740 10 client.15588 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.666+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=28}) -- 0x561a33c18790 con 0x561a33b8c4d0 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.667+0000 7f03b2ff5640 1 -- 172.21.15.19:0/3241697979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f03c000f960 con 0x561a33b8c4d0 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.667+0000 7f03dc17e740 10 client.15588 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.667+0000 7f03dc17e740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.667+0000 7f03dc17e740 20 client.15588 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.667+0000 7f03dc17e740 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561a33b6bd80 0x561a33f686f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.667+0000 7f03dc17e740 4 client.15588 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.667+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a33ec33d0 con 0x561a33b6bd80 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.667+0000 7f03cf59e640 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561a33b6bd80 0x561a33f686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.668+0000 7f03cf59e640 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561a33b6bd80 0x561a33f686f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d477ad00a845cff server_cookie=4fbf0dd85151227b in_seq=0 out_seq=0 2024-05-09T01:27:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.668+0000 7f03b2ff5640 10 client.15588 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.668+0000 7f03b2ff5640 1 -- 172.21.15.19:0/3241697979 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a33ec33d0 con 0x561a33b6bd80 2024-05-09T01:27:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.668+0000 7f03b2ff5640 10 client.15588 handle_command_reply: tid=0 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.668+0000 7f03dc17e740 1 client.15588 shutdown 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b8c4d0 msgr2=0x561a33f5b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b8c4d0 0x561a33f5b620 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f03c0012a50 tx=0x7f03c0012f10 comp rx=0 tx=0).stop 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 shutdown_connections 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561a33b6bd80 0x561a33f686f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f62580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b8c4d0 0x561a33f5b620 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 --2- 172.21.15.19:0/3241697979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b842f0 0x561a33f5b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 >> 172.21.15.19:0/3241697979 conn(0x561a33ba1300 msgr2=0x561a33f4fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03b2ff5640 0 client.15588 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 shutdown_connections 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 1 -- 172.21.15.19:0/3241697979 wait complete. 2024-05-09T01:27:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.669+0000 7f03dc17e740 20 client.15588 trim_cache size 0 max 16384 2024-05-09T01:27:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.699+0000 7f03dc17e740 1 Processor -- start 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03dc17e740 1 -- start start 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33b842f0 0x561a33f59e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33bad1b0 0x561a33f629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03dc17e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01ad590 con 0x561a33bad1b0 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03dc17e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01af0d0 con 0x561a33b842f0 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03dc17e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01adc60 con 0x561a33b8c4d0 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03cf59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33bad1b0 0x561a33f629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03cf59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33bad1b0 0x561a33f629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37678/0 (socket says 172.21.15.19:37678) 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03cf59e640 1 -- 172.21.15.19:0/1844391616 learned_addr learned my addr 172.21.15.19:0/1844391616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03ced9d640 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.700+0000 7f03cf59e640 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33b842f0 0x561a33f59e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03ced9d640 1 -- 172.21.15.19:0/1844391616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33b842f0 msgr2=0x561a33f59e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03ced9d640 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33b842f0 0x561a33f59e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03ced9d640 1 -- 172.21.15.19:0/1844391616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33bad1b0 msgr2=0x561a33f629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03ced9d640 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33bad1b0 0x561a33f629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03ced9d640 1 -- 172.21.15.19:0/1844391616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a33c8f5f0 con 0x561a33b8c4d0 2024-05-09T01:27:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03cf59e640 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33b842f0 0x561a33f59e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03ced9d640 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f624a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f03c0001eb0 tx=0x7f03c0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=748cfe6e56f5d73d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03b4ff9640 1 -- 172.21.15.19:0/1844391616 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0004430 con 0x561a33b8c4d0 2024-05-09T01:27:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03b4ff9640 1 -- 172.21.15.19:0/1844391616 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c8036030 con 0x561a33b8c4d0 2024-05-09T01:27:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.701+0000 7f03b4ff9640 1 -- 172.21.15.19:0/1844391616 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0005030 con 0x561a33b8c4d0 2024-05-09T01:27:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 -- 172.21.15.19:0/1844391616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 msgr2=0x561a33f624a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f624a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f03c0001eb0 tx=0x7f03c0009f90 comp rx=0 tx=0).stop 2024-05-09T01:27:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 -- 172.21.15.19:0/1844391616 shutdown_connections 2024-05-09T01:27:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33bad1b0 0x561a33f629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f624a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 --2- 172.21.15.19:0/1844391616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33b842f0 0x561a33f59e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 -- 172.21.15.19:0/1844391616 >> 172.21.15.19:0/1844391616 conn(0x561a33ba1300 msgr2=0x561a33f59710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 -- 172.21.15.19:0/1844391616 shutdown_connections 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 -- 172.21.15.19:0/1844391616 wait complete. 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 -- start start 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.703+0000 7f03dc17e740 1 Processor -- start 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f62580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f5f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03dc17e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f5f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03dc17e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01aea00 con 0x561a33b842f0 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03dc17e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01ae330 con 0x561a33bad1b0 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03dc17e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01ab570 con 0x561a33b8c4d0 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03dc17e740 10 client.0 _ll_register_callbacks cb 0x7f03c0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f62580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f62580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37690/0 (socket says 172.21.15.19:37690) 2024-05-09T01:27:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 -- 172.21.15.19:0/4139099001 learned_addr learned my addr 172.21.15.19:0/4139099001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f5f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03ced9d640 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f5f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 -- 172.21.15.19:0/4139099001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 msgr2=0x561a33f5f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f5f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 -- 172.21.15.19:0/4139099001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 msgr2=0x561a33f5f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f5f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 -- 172.21.15.19:0/4139099001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a33e79ed0 con 0x561a33b842f0 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03cf59e640 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f62580 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f03c802ade0 tx=0x7f03c802f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e411917ceaa52b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.704+0000 7f03b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.705+0000 7f03b2ff5640 1 -- 172.21.15.19:0/4139099001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c805e040 con 0x561a33b842f0 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.705+0000 7f03b2ff5640 1 -- 172.21.15.19:0/4139099001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03c8003970 con 0x561a33b842f0 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.705+0000 7f03dc17e740 1 -- 172.21.15.19:0/4139099001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c8036030 con 0x561a33b842f0 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.705+0000 7f03b2ff5640 1 -- 172.21.15.19:0/4139099001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03c8003b70 con 0x561a33b842f0 2024-05-09T01:27:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.705+0000 7f03dc17e740 10 client.15594 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.705+0000 7f03dc17e740 1 -- 172.21.15.19:0/4139099001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a33fcf770 con 0x561a33b842f0 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.705+0000 7f03b2ff5640 1 -- 172.21.15.19:0/4139099001 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c8003d10 con 0x561a33b842f0 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.705+0000 7f03b2ff5640 1 -- 172.21.15.19:0/4139099001 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f03c8043070 con 0x561a33b842f0 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.706+0000 7f03dc17e740 10 client.15594 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.706+0000 7f03dc17e740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.706+0000 7f03dc17e740 20 client.15594 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.706+0000 7f03dc17e740 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561a33b411c0 0x561a33fd1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.706+0000 7f03dc17e740 4 client.15594 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.706+0000 7f03dc17e740 1 -- 172.21.15.19:0/4139099001 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f03c8036030 con 0x561a33b411c0 2024-05-09T01:27:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.706+0000 7f03ced9d640 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561a33b411c0 0x561a33fd1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.706+0000 7f03b2ff5640 10 client.15594 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.706+0000 7f03ced9d640 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561a33b411c0 0x561a33fd1df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef35289859278d16 server_cookie=89b4c56c4f49ae03 in_seq=0 out_seq=0 2024-05-09T01:27:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03b2ff5640 1 -- 172.21.15.19:0/4139099001 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f03c8036030 con 0x561a33b411c0 2024-05-09T01:27:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03b2ff5640 10 client.15594 handle_command_reply: tid=0 2024-05-09T01:27:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 client.15594 shutdown 2024-05-09T01:27:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 -- 172.21.15.19:0/4139099001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 msgr2=0x561a33f62580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f62580 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f03c802ade0 tx=0x7f03c802f4c0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 -- 172.21.15.19:0/4139099001 shutdown_connections 2024-05-09T01:27:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561a33b411c0 0x561a33fd1df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a33bad1b0 0x561a33f5f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a33b8c4d0 0x561a33f5f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 --2- 172.21.15.19:0/4139099001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a33b842f0 0x561a33f62580 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 -- 172.21.15.19:0/4139099001 >> 172.21.15.19:0/4139099001 conn(0x561a33ba1300 msgr2=0x561a33f4fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03b2ff5640 0 client.15594 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 -- 172.21.15.19:0/4139099001 shutdown_connections 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.713+0000 7f03dc17e740 1 -- 172.21.15.19:0/4139099001 wait complete. 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03dc17e740 20 client.15594 trim_cache size 0 max 16384 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:27:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:27:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 -- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03b80b27e0 msgr2=0x7f03b80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03b80b27e0 0x7f03b80b4cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f03c4087a70 tx=0x7f03c40a8bc0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 -- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0103d90 msgr2=0x7f03d019a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0103d90 0x7f03d019a850 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f03c400d950 tx=0x7f03c400de10 comp rx=0 tx=0).stop 2024-05-09T01:27:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 -- 172.21.15.19:0/2631626071 shutdown_connections 2024-05-09T01:27:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03b80b27e0 0x7f03b80b4cf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0108a90 0x7f03d01a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0103d90 0x7f03d019a850 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 --2- 172.21.15.19:0/2631626071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0102b50 0x7f03d019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.714+0000 7f03d51f3640 1 -- 172.21.15.19:0/2631626071 >> 172.21.15.19:0/2631626071 conn(0x7f03d00fdd40 msgr2=0x7f03d0104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.715+0000 7f03d51f3640 1 -- 172.21.15.19:0/2631626071 shutdown_connections 2024-05-09T01:27:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:27.715+0000 7f03d51f3640 1 -- 172.21.15.19:0/2631626071 wait complete. 2024-05-09T01:27:28.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:28 smithi059 ceph-mon[34351]: pgmap v436: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:27:28.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:28 smithi019 ceph-mon[30158]: pgmap v436: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:27:28.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:28 smithi196 ceph-mon[35041]: pgmap v436: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:27:30.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:30 smithi059 ceph-mon[34351]: pgmap v437: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T01:27:30.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:30 smithi019 ceph-mon[30158]: pgmap v437: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T01:27:30.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:30 smithi196 ceph-mon[35041]: pgmap v437: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T01:27:32.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:32 smithi059 ceph-mon[34351]: pgmap v438: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T01:27:32.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:32 smithi019 ceph-mon[30158]: pgmap v438: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T01:27:32.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:32 smithi196 ceph-mon[35041]: pgmap v438: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T01:27:34.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:34 smithi059 ceph-mon[34351]: pgmap v439: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T01:27:34.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:34 smithi019 ceph-mon[30158]: pgmap v439: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T01:27:34.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:34 smithi196 ceph-mon[35041]: pgmap v439: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T01:27:36.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:36 smithi019 ceph-mon[30158]: pgmap v440: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:27:36.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:36 smithi196 ceph-mon[35041]: pgmap v440: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:27:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:36 smithi059 ceph-mon[34351]: pgmap v440: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T01:27:37.881 INFO:tasks.workunit.client.0.smithi019.stderr:Updating files: 91% (11230/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-09T01:27:38.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:38 smithi019 ceph-mon[30158]: pgmap v441: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T01:27:38.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:38 smithi196 ceph-mon[35041]: pgmap v441: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T01:27:38.771 INFO:tasks.workunit.client.0.smithi019.stderr:Note: switching to 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'. 2024-05-09T01:27:38.771 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-09T01:27:38.771 INFO:tasks.workunit.client.0.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr: git switch -c 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr:Or undo this operation with: 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr: git switch - 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-09T01:27:38.772 INFO:tasks.workunit.client.0.smithi019.stderr:HEAD is now at c9da4049f63 Merge PR #57081 into wip-pdonnell-testing-20240508.183908-debug 2024-05-09T01:27:38.778 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T01:27:38.841 INFO:tasks.workunit.client.0.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T01:27:38.843 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T01:27:38.843 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T01:27:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:38 smithi059 ceph-mon[34351]: pgmap v441: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T01:27:39.498 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T01:27:39.559 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T01:27:39.601 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T01:27:39.602 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T01:27:39.602 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T01:27:39.642 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T01:27:39.646 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:27:39.646 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T01:27:39.703 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-05-09T01:27:39.704 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-05-09T01:27:39.704 DEBUG:teuthology.orchestra.run.smithi019:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c9da4049f638c0f84bdd9b4e33c42e86046ac554 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-05-09T01:27:39.766 INFO:tasks.workunit.client.0.smithi019.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-05-09T01:27:39.766 INFO:tasks.workunit.client.0.smithi019.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-05-09T01:27:39.766 INFO:tasks.workunit.client.0.smithi019.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-05-09T01:27:40.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:40 smithi019 ceph-mon[30158]: pgmap v442: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:27:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:40 smithi196 ceph-mon[35041]: pgmap v442: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:27:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 -- 172.21.15.19:0/3489785903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518102520 msgr2=0x7fa518102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 --2- 172.21.15.19:0/3489785903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518102520 0x7fa518102930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa514009930 tx=0x7fa51402a420 comp rx=0 tx=0).stop 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 -- 172.21.15.19:0/3489785903 shutdown_connections 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 --2- 172.21.15.19:0/3489785903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5181037c0 0x7fa518103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 --2- 172.21.15.19:0/3489785903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa518102e70 0x7fa518103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 --2- 172.21.15.19:0/3489785903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa518102520 0x7fa518102930 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 -- 172.21.15.19:0/3489785903 >> 172.21.15.19:0/3489785903 conn(0x7fa5180fdd30 msgr2=0x7fa5181001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 -- 172.21.15.19:0/3489785903 shutdown_connections 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 -- 172.21.15.19:0/3489785903 wait complete. 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 Processor -- start 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.834+0000 7fa51ff76640 1 -- start start 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51ff76640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa518102520 0x7fa51819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51ff76640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa518102e70 0x7fa51819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51ff76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5181037c0 0x7fa5181a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51ff76640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa518111950 con 0x7fa5181037c0 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51ff76640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181117d0 con 0x7fa518102e70 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51ff76640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa518111ad0 con 0x7fa518102520 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51e513640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5181037c0 0x7fa5181a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51e513640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5181037c0 0x7fa5181a3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58998/0 (socket says 172.21.15.19:58998) 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51e513640 1 -- 172.21.15.19:0/250803187 learned_addr learned my addr 172.21.15.19:0/250803187 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51dd12640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa518102520 0x7fa51819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51d511640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa518102e70 0x7fa51819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51e513640 1 -- 172.21.15.19:0/250803187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa518102520 msgr2=0x7fa51819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51e513640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa518102520 0x7fa51819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51e513640 1 -- 172.21.15.19:0/250803187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa518102e70 msgr2=0x7fa51819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51e513640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa518102e70 0x7fa51819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51e513640 1 -- 172.21.15.19:0/250803187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5181a4150 con 0x7fa5181037c0 2024-05-09T01:27:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.835+0000 7fa51e513640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5181037c0 0x7fa5181a3ae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa51000e9d0 tx=0x7fa51000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.836+0000 7fa50effd640 1 -- 172.21.15.19:0/250803187 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51000cd80 con 0x7fa5181037c0 2024-05-09T01:27:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.836+0000 7fa50effd640 1 -- 172.21.15.19:0/250803187 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa510005020 con 0x7fa5181037c0 2024-05-09T01:27:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.836+0000 7fa51ff76640 1 -- 172.21.15.19:0/250803187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5181a4440 con 0x7fa5181037c0 2024-05-09T01:27:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.836+0000 7fa50effd640 1 -- 172.21.15.19:0/250803187 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5100116d0 con 0x7fa5181037c0 2024-05-09T01:27:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.836+0000 7fa51ff76640 1 -- 172.21.15.19:0/250803187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5181a4990 con 0x7fa5181037c0 2024-05-09T01:27:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.839+0000 7fa526e2c740 1 Processor -- start 2024-05-09T01:27:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.839+0000 7fa50effd640 1 -- 172.21.15.19:0/250803187 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa5100051d0 con 0x7fa5181037c0 2024-05-09T01:27:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa526e2c740 1 -- start start 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec2e1b0 0x55836efd2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec052f0 0x55836efd2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efd9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa526e2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55836ef367c0 con 0x55836ec052f0 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa526e2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55836eed20e0 con 0x55836ec2e1b0 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa526e2c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55836ef32780 con 0x55836ec0d4d0 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51dd12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec052f0 0x55836efd2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51dd12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec052f0 0x55836efd2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59000/0 (socket says 172.21.15.19:59000) 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51dd12640 1 -- 172.21.15.19:0/2564540053 learned_addr learned my addr 172.21.15.19:0/2564540053 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51d511640 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efd9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51d511640 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec2e1b0 0x55836efd2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa50effd640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5040b27e0 0x7fa5040b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51dd12640 1 -- 172.21.15.19:0/2564540053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 msgr2=0x55836efd9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51dd12640 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efd9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51dd12640 1 -- 172.21.15.19:0/2564540053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec2e1b0 msgr2=0x55836efd2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51dd12640 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec2e1b0 0x55836efd2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51dd12640 1 -- 172.21.15.19:0/2564540053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836eefb7c0 con 0x55836ec052f0 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.840+0000 7fa51d511640 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec2e1b0 0x55836efd2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.841+0000 7fa51e513640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5040b27e0 0x7fa5040b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.841+0000 7fa50effd640 1 -- 172.21.15.19:0/250803187 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa5100e27b0 con 0x7fa5181037c0 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.841+0000 7fa51dd12640 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec052f0 0x55836efd2a80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa514031440 tx=0x7fa514032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b100f7c0a9251d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.841+0000 7fa503fff640 1 -- 172.21.15.19:0/2564540053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51403f070 con 0x55836ec052f0 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.841+0000 7fa503fff640 1 -- 172.21.15.19:0/2564540053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa514004040 con 0x55836ec052f0 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.841+0000 7fa503fff640 1 -- 172.21.15.19:0/2564540053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514035070 con 0x55836ec052f0 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.841+0000 7fa51e513640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5040b27e0 0x7fa5040b4cf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5100a9d40 tx=0x7fa5100a6480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 -- 172.21.15.19:0/2564540053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec052f0 msgr2=0x55836efd2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec052f0 0x55836efd2a80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa514031440 tx=0x7fa514032ba0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 -- 172.21.15.19:0/2564540053 shutdown_connections 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efd9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec052f0 0x55836efd2a80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 --2- 172.21.15.19:0/2564540053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec2e1b0 0x55836efd2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 -- 172.21.15.19:0/2564540053 >> 172.21.15.19:0/2564540053 conn(0x55836ec22300 msgr2=0x55836efcde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 -- 172.21.15.19:0/2564540053 shutdown_connections 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 -- 172.21.15.19:0/2564540053 wait complete. 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.842+0000 7fa526e2c740 1 -- start start 2024-05-09T01:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa526e2c740 1 Processor -- start 2024-05-09T01:27:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efdc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec0d4d0 0x55836efdc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec2e1b0 0x55836efe3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa526e2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55836efda370 con 0x55836ec0d4d0 2024-05-09T01:27:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa526e2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55836eebd9c0 con 0x55836ec052f0 2024-05-09T01:27:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa526e2c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55836efda4f0 con 0x55836ec2e1b0 2024-05-09T01:27:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa526e2c740 10 client.0 _ll_register_callbacks cb 0x55836eeea020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa51dd12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec0d4d0 0x55836efdc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa51dd12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec0d4d0 0x55836efdc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59006/0 (socket says 172.21.15.19:59006) 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa51dd12640 1 -- 172.21.15.19:0/14412416 learned_addr learned my addr 172.21.15.19:0/14412416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa51d511640 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efdc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.843+0000 7fa51d511640 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec2e1b0 0x55836efe3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa51dd12640 1 -- 172.21.15.19:0/14412416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec2e1b0 msgr2=0x55836efe3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa51dd12640 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec2e1b0 0x55836efe3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa51dd12640 1 -- 172.21.15.19:0/14412416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 msgr2=0x55836efdc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa51dd12640 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efdc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa51dd12640 1 -- 172.21.15.19:0/14412416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836eee41f0 con 0x55836ec0d4d0 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa51d511640 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec2e1b0 0x55836efe3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa51dd12640 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec0d4d0 0x55836efdc710 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa514009900 tx=0x7fa514004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18028943bcf9d3f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa5017fa640 1 -- 172.21.15.19:0/14412416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51403f070 con 0x55836ec0d4d0 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa526e2c740 1 -- 172.21.15.19:0/14412416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa514004040 con 0x55836ec0d4d0 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa5017fa640 1 -- 172.21.15.19:0/14412416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa5140035f0 con 0x55836ec0d4d0 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa5017fa640 1 -- 172.21.15.19:0/14412416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa514035040 con 0x55836ec0d4d0 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa526e2c740 10 client.15606 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.844+0000 7fa526e2c740 1 -- 172.21.15.19:0/14412416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55836ec99790 con 0x55836ec0d4d0 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.845+0000 7fa5017fa640 1 -- 172.21.15.19:0/14412416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa514004040 con 0x55836ec0d4d0 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.845+0000 7fa5017fa640 1 -- 172.21.15.19:0/14412416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa5140048f0 con 0x55836ec0d4d0 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.845+0000 7fa526e2c740 10 client.15606 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.845+0000 7fa526e2c740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.845+0000 7fa526e2c740 20 client.15606 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.845+0000 7fa526e2c740 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55836ebecd80 0x55836efe9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.845+0000 7fa526e2c740 4 client.15606 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.845+0000 7fa526e2c740 1 -- 172.21.15.19:0/14412416 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55836ef443d0 con 0x55836ebecd80 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.845+0000 7fa51d511640 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55836ebecd80 0x55836efe9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.846+0000 7fa5017fa640 10 client.15606 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.846+0000 7fa51d511640 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55836ebecd80 0x55836efe9730 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb20016d17fe915a server_cookie=c9a3f3f82f3fcd02 in_seq=0 out_seq=0 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.846+0000 7fa5017fa640 1 -- 172.21.15.19:0/14412416 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55836ef443d0 con 0x55836ebecd80 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.846+0000 7fa5017fa640 10 client.15606 handle_command_reply: tid=0 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.846+0000 7fa526e2c740 1 client.15606 shutdown 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.846+0000 7fa526e2c740 1 -- 172.21.15.19:0/14412416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec0d4d0 msgr2=0x55836efdc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.846+0000 7fa526e2c740 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec0d4d0 0x55836efdc710 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa514009900 tx=0x7fa514004270 comp rx=0 tx=0).stop 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa526e2c740 1 -- 172.21.15.19:0/14412416 shutdown_connections 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa526e2c740 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55836ebecd80 0x55836efe9730 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa526e2c740 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec2e1b0 0x55836efe3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa526e2c740 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ec0d4d0 0x55836efdc710 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa5017fa640 0 client.15606 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa526e2c740 1 --2- 172.21.15.19:0/14412416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efdc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa526e2c740 1 -- 172.21.15.19:0/14412416 >> 172.21.15.19:0/14412416 conn(0x55836ec22300 msgr2=0x55836efcc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa526e2c740 1 -- 172.21.15.19:0/14412416 shutdown_connections 2024-05-09T01:27:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa526e2c740 1 -- 172.21.15.19:0/14412416 wait complete. 2024-05-09T01:27:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.847+0000 7fa526e2c740 20 client.15606 trim_cache size 0 max 16384 2024-05-09T01:27:40.850 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:40 smithi059 ceph-mon[34351]: pgmap v442: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:27:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.862+0000 7fa526e2c740 1 Processor -- start 2024-05-09T01:27:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa526e2c740 1 -- start start 2024-05-09T01:27:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efcdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efe3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efe3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa526e2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181ad740 con 0x55836ebecd80 2024-05-09T01:27:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa526e2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181af280 con 0x55836ec052f0 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa526e2c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181ade10 con 0x55836ec0d4d0 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efcdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efcdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59016/0 (socket says 172.21.15.19:59016) 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 -- 172.21.15.19:0/933813848 learned_addr learned my addr 172.21.15.19:0/933813848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51dd12640 1 --2- 172.21.15.19:0/933813848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efe3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 --2- 172.21.15.19:0/933813848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efe3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 -- 172.21.15.19:0/933813848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 msgr2=0x55836efe3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 --2- 172.21.15.19:0/933813848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efe3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 -- 172.21.15.19:0/933813848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 msgr2=0x55836efe3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 --2- 172.21.15.19:0/933813848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efe3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 -- 172.21.15.19:0/933813848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836ed105f0 con 0x55836ebecd80 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.863+0000 7fa51d511640 1 --2- 172.21.15.19:0/933813848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efcdf20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa508000c00 tx=0x7fa5080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f62161043a5341d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa5037fe640 1 -- 172.21.15.19:0/933813848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508011dd0 con 0x55836ebecd80 2024-05-09T01:27:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa5037fe640 1 -- 172.21.15.19:0/933813848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa508028070 con 0x55836ebecd80 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa5037fe640 1 -- 172.21.15.19:0/933813848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508012ac0 con 0x55836ebecd80 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa526e2c740 1 -- 172.21.15.19:0/933813848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 msgr2=0x55836efcdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa526e2c740 1 --2- 172.21.15.19:0/933813848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efcdf20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa508000c00 tx=0x7fa5080046c0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa526e2c740 1 -- 172.21.15.19:0/933813848 shutdown_connections 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa526e2c740 1 --2- 172.21.15.19:0/933813848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efe3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa526e2c740 1 --2- 172.21.15.19:0/933813848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efe3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa526e2c740 1 --2- 172.21.15.19:0/933813848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efcdf20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.864+0000 7fa526e2c740 1 -- 172.21.15.19:0/933813848 >> 172.21.15.19:0/933813848 conn(0x55836ec22300 msgr2=0x55836efcd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.865+0000 7fa526e2c740 1 -- 172.21.15.19:0/933813848 shutdown_connections 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.865+0000 7fa526e2c740 1 -- 172.21.15.19:0/933813848 wait complete. 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.865+0000 7fa526e2c740 1 -- start start 2024-05-09T01:27:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.865+0000 7fa526e2c740 1 Processor -- start 2024-05-09T01:27:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.865+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efe1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.865+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efe1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.865+0000 7fa526e2c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efe25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa526e2c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181aebb0 con 0x55836ebecd80 2024-05-09T01:27:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa526e2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181ae4e0 con 0x55836ec052f0 2024-05-09T01:27:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa526e2c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181ab720 con 0x55836ec0d4d0 2024-05-09T01:27:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa526e2c740 10 client.0 _ll_register_callbacks cb 0x7fa50800ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efe1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efe1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59028/0 (socket says 172.21.15.19:59028) 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 -- 172.21.15.19:0/1953671690 learned_addr learned my addr 172.21.15.19:0/1953671690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51dd12640 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efe1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efe25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 -- 172.21.15.19:0/1953671690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 msgr2=0x55836efe25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efe25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 -- 172.21.15.19:0/1953671690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 msgr2=0x55836efe1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efe1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 -- 172.21.15.19:0/1953671690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836eefaed0 con 0x55836ebecd80 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa51d511640 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efe1210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa508014830 tx=0x7fa508014960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b0da53a6da9b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa5017fa640 1 -- 172.21.15.19:0/1953671690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508036020 con 0x55836ebecd80 2024-05-09T01:27:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa5017fa640 1 -- 172.21.15.19:0/1953671690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa508011dd0 con 0x55836ebecd80 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.866+0000 7fa526e2c740 1 -- 172.21.15.19:0/1953671690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa508028070 con 0x55836ebecd80 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa5017fa640 1 -- 172.21.15.19:0/1953671690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa508012790 con 0x55836ebecd80 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa526e2c740 10 client.15612 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa526e2c740 1 -- 172.21.15.19:0/1953671690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55836efe3c40 con 0x55836ebecd80 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa5017fa640 1 -- 172.21.15.19:0/1953671690 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa508028070 con 0x55836ebecd80 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa5017fa640 1 -- 172.21.15.19:0/1953671690 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa508021700 con 0x55836ebecd80 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa526e2c740 10 client.15612 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa526e2c740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa526e2c740 20 client.15612 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa526e2c740 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55836ec2e1b0 0x55836f053cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa526e2c740 4 client.15612 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa526e2c740 1 -- 172.21.15.19:0/1953671690 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55836ef3e240 con 0x55836ec2e1b0 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.867+0000 7fa51dd12640 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55836ec2e1b0 0x55836f053cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.868+0000 7fa51dd12640 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55836ec2e1b0 0x55836f053cd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ddff1743b4ac979 server_cookie=761f1d2190e85197 in_seq=0 out_seq=0 2024-05-09T01:27:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.868+0000 7fa5017fa640 10 client.15612 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa5017fa640 1 -- 172.21.15.19:0/1953671690 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55836ef3e240 con 0x55836ec2e1b0 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa5017fa640 10 client.15612 handle_command_reply: tid=0 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 client.15612 shutdown 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 -- 172.21.15.19:0/1953671690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 msgr2=0x55836efe1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efe1210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa508014830 tx=0x7fa508014960 comp rx=0 tx=0).stop 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 -- 172.21.15.19:0/1953671690 shutdown_connections 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55836ec2e1b0 0x55836f053cd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55836ec0d4d0 0x55836efe25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55836ec052f0 0x55836efe1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 --2- 172.21.15.19:0/1953671690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55836ebecd80 0x55836efe1210 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa5017fa640 0 client.15612 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 -- 172.21.15.19:0/1953671690 >> 172.21.15.19:0/1953671690 conn(0x55836ec22300 msgr2=0x55836efcc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 -- 172.21.15.19:0/1953671690 shutdown_connections 2024-05-09T01:27:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 1 -- 172.21.15.19:0/1953671690 wait complete. 2024-05-09T01:27:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.878+0000 7fa526e2c740 20 client.15612 trim_cache size 0 max 16384 2024-05-09T01:27:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:27:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:27:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:27:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 -- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5040b27e0 msgr2=0x7fa5040b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5040b27e0 0x7fa5040b4cf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5100a9d40 tx=0x7fa5100a6480 comp rx=0 tx=0).stop 2024-05-09T01:27:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 -- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5181037c0 msgr2=0x7fa5181a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5181037c0 0x7fa5181a3ae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa51000e9d0 tx=0x7fa51000ee90 comp rx=0 tx=0).stop 2024-05-09T01:27:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 -- 172.21.15.19:0/250803187 shutdown_connections 2024-05-09T01:27:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa5040b27e0 0x7fa5040b4cf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5181037c0 0x7fa5181a3ae0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa518102e70 0x7fa51819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 --2- 172.21.15.19:0/250803187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa518102520 0x7fa51819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 -- 172.21.15.19:0/250803187 >> 172.21.15.19:0/250803187 conn(0x7fa5180fdd30 msgr2=0x7fa51810a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 -- 172.21.15.19:0/250803187 shutdown_connections 2024-05-09T01:27:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:40.879+0000 7fa51ff76640 1 -- 172.21.15.19:0/250803187 wait complete. 2024-05-09T01:27:41.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:27:41.725 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:27:42.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:42 smithi019 ceph-mon[30158]: pgmap v443: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T01:27:42.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:42 smithi196 ceph-mon[35041]: pgmap v443: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T01:27:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:42 smithi059 ceph-mon[34351]: pgmap v443: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T01:27:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.475+0000 7f3ce7015700 1 -- 172.21.15.19:0/2016799521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0105500 msgr2=0x7f3ce0105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.475+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2016799521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0105500 0x7f3ce0105910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009990 tx=0x7f3cd4009ca0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.475+0000 7f3ce7015700 1 -- 172.21.15.19:0/2016799521 shutdown_connections 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.475+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2016799521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce010e8e0 0x7f3ce0110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.475+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2016799521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce0105500 0x7f3ce0105910 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.475+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2016799521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ce0105e50 0x7f3ce010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.475+0000 7f3ce7015700 1 -- 172.21.15.19:0/2016799521 >> 172.21.15.19:0/2016799521 conn(0x7f3ce00fedc0 msgr2=0x7f3ce01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.475+0000 7f3ce7015700 1 -- 172.21.15.19:0/2016799521 shutdown_connections 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.475+0000 7f3ce7015700 1 -- 172.21.15.19:0/2016799521 wait complete. 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce7015700 1 Processor -- start 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce7015700 1 -- start start 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce7015700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ce0105500 0x7f3ce01031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce7015700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce0105e50 0x7f3ce0103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce7015700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce010e8e0 0x7f3ce0103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce7015700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0114430 con 0x7f3ce010e8e0 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce7015700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01142b0 con 0x7f3ce0105e50 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce7015700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01145b0 con 0x7f3ce0105500 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce55b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce010e8e0 0x7f3ce0103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce55b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce010e8e0 0x7f3ce0103c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59046/0 (socket says 172.21.15.19:59046) 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.476+0000 7f3ce55b2700 1 -- 172.21.15.19:0/2632487683 learned_addr learned my addr 172.21.15.19:0/2632487683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3cdffff700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce0105e50 0x7f3ce0103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ce0105500 0x7f3ce01031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3ce55b2700 1 -- 172.21.15.19:0/2632487683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ce0105500 msgr2=0x7f3ce01031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ce0105500 0x7f3ce01031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3ce55b2700 1 -- 172.21.15.19:0/2632487683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce0105e50 msgr2=0x7f3ce0103730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce0105e50 0x7f3ce0103730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3ce55b2700 1 -- 172.21.15.19:0/2632487683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce01a9030 con 0x7f3ce010e8e0 2024-05-09T01:27:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ce0105500 0x7f3ce01031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce010e8e0 0x7f3ce0103c70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8007d70 tx=0x7f3cd800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3cddffb700 1 -- 172.21.15.19:0/2632487683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8016020 con 0x7f3ce010e8e0 2024-05-09T01:27:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3ce7015700 1 -- 172.21.15.19:0/2632487683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce01a9320 con 0x7f3ce010e8e0 2024-05-09T01:27:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3cddffb700 1 -- 172.21.15.19:0/2632487683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd800ad40 con 0x7f3ce010e8e0 2024-05-09T01:27:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.477+0000 7f3cddffb700 1 -- 172.21.15.19:0/2632487683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd80115e0 con 0x7f3ce010e8e0 2024-05-09T01:27:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.478+0000 7f3ce7015700 1 -- 172.21.15.19:0/2632487683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce01a9860 con 0x7f3ce010e8e0 2024-05-09T01:27:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.481+0000 7f3cddffb700 1 -- 172.21.15.19:0/2632487683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3cd8019020 con 0x7f3ce010e8e0 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cf5a2bb80 1 Processor -- start 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cf5a2bb80 1 -- start start 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cddffb700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cd00b2880 0x7f3cd00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cdffff700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cd00b2880 0x7f3cd00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 0x556372e4d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372cca4a0 0x556372e4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372e47400 0x556372e47810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cddffb700 1 -- 172.21.15.19:0/2632487683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3cd80e21a0 con 0x7f3ce010e8e0 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556372d6a0a0 con 0x556372e47400 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556372d0f9a0 con 0x556372d44d00 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3ce55b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372e47400 0x556372e47810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3ce55b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372e47400 0x556372e47810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59056/0 (socket says 172.21.15.19:59056) 2024-05-09T01:27:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3ce55b2700 1 -- 172.21.15.19:0/2856368685 learned_addr learned my addr 172.21.15.19:0/2856368685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2856368685 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556372d66060 con 0x556372cca4a0 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.482+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 0x556372e4d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3cdffff700 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372cca4a0 0x556372e4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3cdffff700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cd00b2880 0x7f3cd00b4d90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0102300 tx=0x7f3ccc00f3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ce55b2700 1 -- 172.21.15.19:0/2856368685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372cca4a0 msgr2=0x556372e4d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372cca4a0 0x556372e4d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ce55b2700 1 -- 172.21.15.19:0/2856368685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 msgr2=0x556372e4d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 0x556372e4d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ce55b2700 1 -- 172.21.15.19:0/2856368685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556372d00500 con 0x556372e47400 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3cdffff700 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372cca4a0 0x556372e4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 0x556372e4d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372e47400 0x556372e47810 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80a5820 tx=0x7f3cd8006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97addabacf31522f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ccaffd700 1 -- 172.21.15.19:0/2856368685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd80cc0f0 con 0x556372e47400 2024-05-09T01:27:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ccaffd700 1 -- 172.21.15.19:0/2856368685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd80a9ea0 con 0x556372e47400 2024-05-09T01:27:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.483+0000 7f3ccaffd700 1 -- 172.21.15.19:0/2856368685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd80b2170 con 0x556372e47400 2024-05-09T01:27:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.484+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2856368685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372e47400 msgr2=0x556372e47810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.484+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372e47400 0x556372e47810 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80a5820 tx=0x7f3cd8006040 comp rx=0 tx=0).stop 2024-05-09T01:27:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.484+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2856368685 shutdown_connections 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.484+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372e47400 0x556372e47810 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.484+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 0x556372e4d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.484+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2856368685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372cca4a0 0x556372e4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.484+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2856368685 >> 172.21.15.19:0/2856368685 conn(0x556372ccdf70 msgr2=0x556372e42220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.484+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2856368685 shutdown_connections 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.484+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2856368685 wait complete. 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3cf5a2bb80 1 -- start start 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3cf5a2bb80 1 Processor -- start 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372d44d00 0x556372e515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372e47400 0x556372e51b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556372e48160 con 0x556372cca4a0 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556372d04b20 con 0x556372e47400 2024-05-09T01:27:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556372e54020 con 0x556372d44d00 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3cf5a2bb80 10 client.0 _ll_register_callbacks cb 0x556372bdb6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.485+0000 7f3ce4db1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce4db1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59060/0 (socket says 172.21.15.19:59060) 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce4db1700 1 -- 172.21.15.19:0/2155206956 learned_addr learned my addr 172.21.15.19:0/2155206956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3cdffff700 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372d44d00 0x556372e515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372e47400 0x556372e51b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce4db1700 1 -- 172.21.15.19:0/2155206956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372d44d00 msgr2=0x556372e515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372d44d00 0x556372e515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce4db1700 1 -- 172.21.15.19:0/2155206956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372e47400 msgr2=0x556372e51b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372e47400 0x556372e51b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce4db1700 1 -- 172.21.15.19:0/2155206956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556372be5490 con 0x556372cca4a0 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372e47400 0x556372e51b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3cdffff700 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372d44d00 0x556372e515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e510b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4015000 tx=0x7f3cd4009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85649db30bca831d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3cbffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3cbffff700 1 -- 172.21.15.19:0/2155206956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4003c70 con 0x556372cca4a0 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3cbffff700 1 -- 172.21.15.19:0/2155206956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3cd4005070 con 0x556372cca4a0 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2155206956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd80a9ea0 con 0x556372cca4a0 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3cbffff700 1 -- 172.21.15.19:0/2155206956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3cd4003e10 con 0x556372cca4a0 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.486+0000 7f3cf5a2bb80 10 client.15624 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2155206956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556372e5ea90 con 0x556372cca4a0 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cbffff700 1 -- 172.21.15.19:0/2155206956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd80a9ea0 con 0x556372cca4a0 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cbffff700 1 -- 172.21.15.19:0/2155206956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3cd4022460 con 0x556372cca4a0 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cf5a2bb80 10 client.15624 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cf5a2bb80 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cf5a2bb80 20 client.15624 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556372e611c0 0x556372e63600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cf5a2bb80 4 client.15624 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2155206956 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556372d77a80 con 0x556372e611c0 2024-05-09T01:27:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.487+0000 7f3cdffff700 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556372e611c0 0x556372e63600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.488+0000 7f3cdffff700 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556372e611c0 0x556372e63600 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0e56ff9caa2b308 server_cookie=f5f0b955c6706345 in_seq=0 out_seq=0 2024-05-09T01:27:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.488+0000 7f3cbffff700 10 client.15624 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cbffff700 1 -- 172.21.15.19:0/2155206956 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556372d77a80 con 0x556372e611c0 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cbffff700 10 client.15624 handle_command_reply: tid=0 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 client.15624 shutdown 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2155206956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 msgr2=0x556372e510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e510b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4015000 tx=0x7f3cd4009fc0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2155206956 shutdown_connections 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556372e611c0 0x556372e63600 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372e47400 0x556372e51b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e510b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cbffff700 0 client.15624 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/2155206956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372d44d00 0x556372e515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2155206956 >> 172.21.15.19:0/2155206956 conn(0x556372ccdf70 msgr2=0x556372e3bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2155206956 shutdown_connections 2024-05-09T01:27:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/2155206956 wait complete. 2024-05-09T01:27:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.489+0000 7f3cf5a2bb80 20 client.15624 trim_cache size 0 max 16384 2024-05-09T01:27:43.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.509+0000 7f3cf5a2bb80 1 Processor -- start 2024-05-09T01:27:43.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.509+0000 7f3cf5a2bb80 1 -- start start 2024-05-09T01:27:43.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372cca4a0 0x556372e4c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372d44d00 0x556372e3ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 0x556372e51970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b0ca0 con 0x556372d44d00 2024-05-09T01:27:43.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b27e0 con 0x556372cca4a0 2024-05-09T01:27:43.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b1370 con 0x556372e47400 2024-05-09T01:27:43.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cdffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372d44d00 0x556372e3ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cdffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372d44d00 0x556372e3ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59072/0 (socket says 172.21.15.19:59072) 2024-05-09T01:27:43.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cdffff700 1 -- 172.21.15.19:0/3880250176 learned_addr learned my addr 172.21.15.19:0/3880250176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:43.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372cca4a0 0x556372e4c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 0x556372e51970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cdffff700 1 -- 172.21.15.19:0/3880250176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 msgr2=0x556372e51970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:43.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cdffff700 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 0x556372e51970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cdffff700 1 -- 172.21.15.19:0/3880250176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372cca4a0 msgr2=0x556372e4c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:43.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cdffff700 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372cca4a0 0x556372e4c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cdffff700 1 -- 172.21.15.19:0/3880250176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556372e48600 con 0x556372d44d00 2024-05-09T01:27:43.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372cca4a0 0x556372e4c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:43.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.510+0000 7f3cdffff700 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372d44d00 0x556372e3ca20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc019000 tx=0x7f3ccc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c656058059b172ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:43.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cca7fc700 1 -- 172.21.15.19:0/3880250176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc018070 con 0x556372d44d00 2024-05-09T01:27:43.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cca7fc700 1 -- 172.21.15.19:0/3880250176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ccc003c60 con 0x556372d44d00 2024-05-09T01:27:43.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cca7fc700 1 -- 172.21.15.19:0/3880250176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc010690 con 0x556372d44d00 2024-05-09T01:27:43.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/3880250176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372d44d00 msgr2=0x556372e3ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:43.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372d44d00 0x556372e3ca20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc019000 tx=0x7f3ccc014000 comp rx=0 tx=0).stop 2024-05-09T01:27:43.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/3880250176 shutdown_connections 2024-05-09T01:27:43.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 0x556372e51970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372cca4a0 0x556372e4c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/3880250176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372d44d00 0x556372e3ca20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/3880250176 >> 172.21.15.19:0/3880250176 conn(0x556372ccdf70 msgr2=0x556372e4acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/3880250176 shutdown_connections 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.511+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/3880250176 wait complete. 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3cf5a2bb80 1 -- start start 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3cf5a2bb80 1 Processor -- start 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 0x556372e68a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3cf5a2bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 0x556372e68f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b2110 con 0x556372cca4a0 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b1a40 con 0x556372d44d00 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3cf5a2bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01aec40 con 0x556372e47400 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3cf5a2bb80 10 client.0 _ll_register_callbacks cb 0x556372bdb6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:43.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.512+0000 7f3ce4db1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3ce4db1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e684d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59088/0 (socket says 172.21.15.19:59088) 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3ce4db1700 1 -- 172.21.15.19:0/835310603 learned_addr learned my addr 172.21.15.19:0/835310603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3cdffff700 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 0x556372e68a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3ce55b2700 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 0x556372e68f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3ce4db1700 1 -- 172.21.15.19:0/835310603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 msgr2=0x556372e68f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 0x556372e68f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3ce4db1700 1 -- 172.21.15.19:0/835310603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 msgr2=0x556372e68a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 0x556372e68a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3ce4db1700 1 -- 172.21.15.19:0/835310603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556372eca9b0 con 0x556372cca4a0 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3ce4db1700 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e684d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4022f80 tx=0x7f3cd4022ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa64c6c652db8334 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3cbffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3cbffff700 1 -- 172.21.15.19:0/835310603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd403c060 con 0x556372cca4a0 2024-05-09T01:27:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/835310603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ccc003c60 con 0x556372cca4a0 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3cbffff700 1 -- 172.21.15.19:0/835310603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3cd40054d0 con 0x556372cca4a0 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3cbffff700 1 -- 172.21.15.19:0/835310603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3cd40056d0 con 0x556372cca4a0 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3cf5a2bb80 10 client.15630 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.513+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/835310603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556372ecba90 con 0x556372cca4a0 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.514+0000 7f3cbffff700 1 -- 172.21.15.19:0/835310603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ccc003c60 con 0x556372cca4a0 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.514+0000 7f3cbffff700 1 -- 172.21.15.19:0/835310603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3cd400f460 con 0x556372cca4a0 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.514+0000 7f3cf5a2bb80 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.514+0000 7f3cf5a2bb80 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.514+0000 7f3cf5a2bb80 20 client.15630 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.514+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556372ecc1a0 0x556372ece5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.514+0000 7f3cf5a2bb80 4 client.15630 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.514+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/835310603 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556372d71b20 con 0x556372ecc1a0 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.514+0000 7f3cdffff700 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556372ecc1a0 0x556372ece5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:43.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cdffff700 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556372ecc1a0 0x556372ece5c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=958bdb004b80f496 server_cookie=bfd48dd53808122d in_seq=0 out_seq=0 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cbffff700 10 client.15630 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cbffff700 1 -- 172.21.15.19:0/835310603 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556372d71b20 con 0x556372ecc1a0 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cbffff700 10 client.15630 handle_command_reply: tid=0 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 client.15630 shutdown 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/835310603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 msgr2=0x556372e684d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e684d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4022f80 tx=0x7f3cd4022ea0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/835310603 shutdown_connections 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556372ecc1a0 0x556372ece5c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556372e47400 0x556372e68f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cbffff700 0 client.15630 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556372cca4a0 0x556372e684d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 --2- 172.21.15.19:0/835310603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556372d44d00 0x556372e68a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/835310603 >> 172.21.15.19:0/835310603 conn(0x556372ccdf70 msgr2=0x556372e3c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/835310603 shutdown_connections 2024-05-09T01:27:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 1 -- 172.21.15.19:0/835310603 wait complete. 2024-05-09T01:27:43.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.515+0000 7f3cf5a2bb80 20 client.15630 trim_cache size 0 max 16384 2024-05-09T01:27:43.519 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:27:43.519 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:27:43.519 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:27:43.519 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 -- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cd00b2880 msgr2=0x7f3cd00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cd00b2880 0x7f3cd00b4d90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0102300 tx=0x7f3ccc00f3c0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 -- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce010e8e0 msgr2=0x7f3ce0103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce010e8e0 0x7f3ce0103c70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8007d70 tx=0x7f3cd800a340 comp rx=0 tx=0).stop 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 -- 172.21.15.19:0/2632487683 shutdown_connections 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cd00b2880 0x7f3cd00b4d90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ce010e8e0 0x7f3ce0103c70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ce0105500 0x7f3ce01031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 --2- 172.21.15.19:0/2632487683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce0105e50 0x7f3ce0103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 -- 172.21.15.19:0/2632487683 >> 172.21.15.19:0/2632487683 conn(0x7f3ce00fedc0 msgr2=0x7f3ce00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 -- 172.21.15.19:0/2632487683 shutdown_connections 2024-05-09T01:27:43.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:43.516+0000 7f3ce7015700 1 -- 172.21.15.19:0/2632487683 wait complete. 2024-05-09T01:27:44.426 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:27:44.426 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:27:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:44 smithi019 ceph-mon[30158]: pgmap v444: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-09T01:27:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:27:44.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:44 smithi196 ceph-mon[35041]: pgmap v444: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-09T01:27:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:44 smithi059 ceph-mon[34351]: pgmap v444: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-09T01:27:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.369+0000 7fa34fdc1700 1 -- 172.21.15.19:0/1373111757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348108f10 msgr2=0x7fa348109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.369+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/1373111757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348108f10 0x7fa348109320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa3380099c0 tx=0x7fa338009cd0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.369+0000 7fa34fdc1700 1 -- 172.21.15.19:0/1373111757 shutdown_connections 2024-05-09T01:27:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.369+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/1373111757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348109860 0x7fa3481111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.369+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/1373111757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3481045e0 0x7fa3481049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.369+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/1373111757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa348108f10 0x7fa348109320 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.369+0000 7fa34fdc1700 1 -- 172.21.15.19:0/1373111757 >> 172.21.15.19:0/1373111757 conn(0x7fa3480ffe70 msgr2=0x7fa3481022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.369+0000 7fa34fdc1700 1 -- 172.21.15.19:0/1373111757 shutdown_connections 2024-05-09T01:27:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.369+0000 7fa34fdc1700 1 -- 172.21.15.19:0/1373111757 wait complete. 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34fdc1700 1 Processor -- start 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34fdc1700 1 -- start start 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34fdc1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3481045e0 0x7fa348075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34fdc1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa348108f10 0x7fa348076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34fdc1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348109860 0x7fa348076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34fdc1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa348114470 con 0x7fa3481045e0 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34fdc1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3481142f0 con 0x7fa348108f10 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34fdc1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3481145f0 con 0x7fa348109860 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34db5d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3481045e0 0x7fa348075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34db5d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3481045e0 0x7fa348075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59106/0 (socket says 172.21.15.19:59106) 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.370+0000 7fa34db5d700 1 -- 172.21.15.19:0/2676010786 learned_addr learned my addr 172.21.15.19:0/2676010786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:46.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34d35c700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa348108f10 0x7fa348076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34e35e700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348109860 0x7fa348076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34db5d700 1 -- 172.21.15.19:0/2676010786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348109860 msgr2=0x7fa348076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34db5d700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348109860 0x7fa348076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34db5d700 1 -- 172.21.15.19:0/2676010786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa348108f10 msgr2=0x7fa348076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34db5d700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa348108f10 0x7fa348076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34db5d700 1 -- 172.21.15.19:0/2676010786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3481a91f0 con 0x7fa3481045e0 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34db5d700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3481045e0 0x7fa348075ac0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa34400d950 tx=0x7fa34400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa33effd700 1 -- 172.21.15.19:0/2676010786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344016070 con 0x7fa3481045e0 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3481a94e0 con 0x7fa3481045e0 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa33effd700 1 -- 172.21.15.19:0/2676010786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34400fd40 con 0x7fa3481045e0 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa33effd700 1 -- 172.21.15.19:0/2676010786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34400e3b0 con 0x7fa3481045e0 2024-05-09T01:27:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.371+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3481a9a40 con 0x7fa3481045e0 2024-05-09T01:27:46.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.372+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3480512c0 con 0x7fa3481045e0 2024-05-09T01:27:46.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.374+0000 7fa33effd700 1 -- 172.21.15.19:0/2676010786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa344019070 con 0x7fa3481045e0 2024-05-09T01:27:46.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.375+0000 7fa33effd700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3340b2880 0x7fa3340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:46.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.375+0000 7fa33effd700 1 -- 172.21.15.19:0/2676010786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa3440e1c80 con 0x7fa3481045e0 2024-05-09T01:27:46.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.375+0000 7fa34d35c700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3340b2880 0x7fa3340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:46.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.375+0000 7fa34d35c700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3340b2880 0x7fa3340b4d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa338006010 tx=0x7fa338000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:46.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.378+0000 7fa33effd700 1 -- 172.21.15.19:0/2676010786 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3440a9980 con 0x7fa3481045e0 2024-05-09T01:27:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.643+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa34802e5f0 con 0x7fa3481045e0 2024-05-09T01:27:46.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:46 smithi019 ceph-mon[30158]: pgmap v445: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-09T01:27:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.643+0000 7fa33effd700 1 -- 172.21.15.19:0/2676010786 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fa3440a9280 con 0x7fa3481045e0 2024-05-09T01:27:46.641 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:27:46.641 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:27:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3340b2880 msgr2=0x7fa3340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3340b2880 0x7fa3340b4d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa338006010 tx=0x7fa338000bc0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3481045e0 msgr2=0x7fa348075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3481045e0 0x7fa348075ac0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa34400d950 tx=0x7fa34400dc60 comp rx=0 tx=0).stop 2024-05-09T01:27:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 shutdown_connections 2024-05-09T01:27:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa348109860 0x7fa348076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa3340b2880 0x7fa3340b4d90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3481045e0 0x7fa348075ac0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 --2- 172.21.15.19:0/2676010786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa348108f10 0x7fa348076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 >> 172.21.15.19:0/2676010786 conn(0x7fa3480ffe70 msgr2=0x7fa348107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 shutdown_connections 2024-05-09T01:27:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:46.646+0000 7fa34fdc1700 1 -- 172.21.15.19:0/2676010786 wait complete. 2024-05-09T01:27:46.644 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:27:46.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:46 smithi196 ceph-mon[35041]: pgmap v445: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-09T01:27:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:46 smithi059 ceph-mon[34351]: pgmap v445: 97 pgs: 97 active+clean; 676 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-09T01:27:47.405 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:27:47.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:47 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2676010786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:27:47.661 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:27:47.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:47 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2676010786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:27:47.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:47 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2676010786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:27:48.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:48 smithi019 ceph-mon[30158]: pgmap v446: 97 pgs: 97 active+clean; 743 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-05-09T01:27:48.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:48 smithi196 ceph-mon[35041]: pgmap v446: 97 pgs: 97 active+clean; 743 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-05-09T01:27:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:48 smithi059 ceph-mon[34351]: pgmap v446: 97 pgs: 97 active+clean; 743 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-05-09T01:27:49.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.401+0000 7f50c442f700 1 -- 172.21.15.19:0/3092082685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 msgr2=0x7f50bc108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:49.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.401+0000 7f50c442f700 1 --2- 172.21.15.19:0/3092082685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 0x7f50bc108220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f50b00099c0 tx=0x7f50b0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.401+0000 7f50c442f700 1 -- 172.21.15.19:0/3092082685 shutdown_connections 2024-05-09T01:27:49.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.401+0000 7f50c442f700 1 --2- 172.21.15.19:0/3092082685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50bc108760 0x7f50bc1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.401+0000 7f50c442f700 1 --2- 172.21.15.19:0/3092082685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50bc1034e0 0x7f50bc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.401+0000 7f50c442f700 1 --2- 172.21.15.19:0/3092082685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 0x7f50bc108220 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.401+0000 7f50c442f700 1 -- 172.21.15.19:0/3092082685 >> 172.21.15.19:0/3092082685 conn(0x7f50bc0fef30 msgr2=0x7f50bc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:49.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.401+0000 7f50c442f700 1 -- 172.21.15.19:0/3092082685 shutdown_connections 2024-05-09T01:27:49.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.401+0000 7f50c442f700 1 -- 172.21.15.19:0/3092082685 wait complete. 2024-05-09T01:27:49.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.402+0000 7f50c442f700 1 Processor -- start 2024-05-09T01:27:49.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.402+0000 7f50c442f700 1 -- start start 2024-05-09T01:27:49.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.402+0000 7f50c442f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50bc1034e0 0x7f50bc110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.402+0000 7f50c442f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 0x7f50bc110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.402+0000 7f50c442f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50bc108760 0x7f50bc111520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.402+0000 7f50c442f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1144e0 con 0x7f50bc107e10 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.402+0000 7f50c442f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc114360 con 0x7f50bc1034e0 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.402+0000 7f50c442f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc114660 con 0x7f50bc108760 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c19ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 0x7f50bc110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c19ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 0x7f50bc110fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59126/0 (socket says 172.21.15.19:59126) 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c19ca700 1 -- 172.21.15.19:0/828594906 learned_addr learned my addr 172.21.15.19:0/828594906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c21cb700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50bc1034e0 0x7f50bc110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c29cc700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50bc108760 0x7f50bc111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c19ca700 1 -- 172.21.15.19:0/828594906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50bc108760 msgr2=0x7f50bc111520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c19ca700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50bc108760 0x7f50bc111520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c19ca700 1 -- 172.21.15.19:0/828594906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50bc1034e0 msgr2=0x7f50bc110aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c19ca700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50bc1034e0 0x7f50bc110aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c19ca700 1 -- 172.21.15.19:0/828594906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc1a9220 con 0x7f50bc107e10 2024-05-09T01:27:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c19ca700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 0x7f50bc110fe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f50b00045e0 tx=0x7f50b0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50af7fe700 1 -- 172.21.15.19:0/828594906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0026070 con 0x7f50bc107e10 2024-05-09T01:27:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50af7fe700 1 -- 172.21.15.19:0/828594906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b000f460 con 0x7f50bc107e10 2024-05-09T01:27:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c442f700 1 -- 172.21.15.19:0/828594906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc1a94b0 con 0x7f50bc107e10 2024-05-09T01:27:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50af7fe700 1 -- 172.21.15.19:0/828594906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0009e60 con 0x7f50bc107e10 2024-05-09T01:27:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.403+0000 7f50c442f700 1 -- 172.21.15.19:0/828594906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc1a9900 con 0x7f50bc107e10 2024-05-09T01:27:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.406+0000 7f50af7fe700 1 -- 172.21.15.19:0/828594906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f50b0029070 con 0x7f50bc107e10 2024-05-09T01:27:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.407+0000 7f50af7fe700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50a80b2880 0x7f50a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.407+0000 7f50d2e45b80 1 Processor -- start 2024-05-09T01:27:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.407+0000 7f50c21cb700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50a80b2880 0x7f50a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.407+0000 7f50af7fe700 1 -- 172.21.15.19:0/828594906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f50b00e2a60 con 0x7f50bc107e10 2024-05-09T01:27:49.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.407+0000 7f50d2e45b80 1 -- start start 2024-05-09T01:27:49.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 0x5581b85cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c21cb700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50a80b2880 0x7f50a80b4d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009c60 tx=0x7f50b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:49.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b85c66a0 0x5581b85c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5581b84e8ff0 con 0x5581b85c66a0 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5581b84877f0 con 0x5581b84494a0 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581b84e4fb0 con 0x5581b8533160 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c29cc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b85c66a0 0x5581b85c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c29cc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b85c66a0 0x5581b85c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59128/0 (socket says 172.21.15.19:59128) 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c29cc700 1 -- 172.21.15.19:0/1046782334 learned_addr learned my addr 172.21.15.19:0/1046782334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c21cb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 0x5581b85cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c19ca700 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c29cc700 1 -- 172.21.15.19:0/1046782334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 msgr2=0x5581b85cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c29cc700 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 0x5581b85cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c29cc700 1 -- 172.21.15.19:0/1046782334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 msgr2=0x5581b85cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c29cc700 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.408+0000 7f50c29cc700 1 -- 172.21.15.19:0/1046782334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b847f370 con 0x5581b85c66a0 2024-05-09T01:27:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.409+0000 7f50c21cb700 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 0x5581b85cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.409+0000 7f50c19ca700 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.409+0000 7f50c29cc700 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b85c66a0 0x5581b85c6ad0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f50b4003f30 tx=0x7f50b4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c3bae4b539a7b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.409+0000 7f50acd47700 1 -- 172.21.15.19:0/1046782334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4004670 con 0x5581b85c66a0 2024-05-09T01:27:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.409+0000 7f50acd47700 1 -- 172.21.15.19:0/1046782334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b401a070 con 0x5581b85c66a0 2024-05-09T01:27:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.409+0000 7f50acd47700 1 -- 172.21.15.19:0/1046782334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b40156c0 con 0x5581b85c66a0 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.410+0000 7f50d2e45b80 1 -- 172.21.15.19:0/1046782334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b85c66a0 msgr2=0x5581b85c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.410+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b85c66a0 0x5581b85c6ad0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f50b4003f30 tx=0x7f50b4009c20 comp rx=0 tx=0).stop 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.410+0000 7f50d2e45b80 1 -- 172.21.15.19:0/1046782334 shutdown_connections 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.410+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b85c66a0 0x5581b85c6ad0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.410+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 0x5581b85cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.410+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/1046782334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.410+0000 7f50d2e45b80 1 -- 172.21.15.19:0/1046782334 >> 172.21.15.19:0/1046782334 conn(0x5581b84a96a0 msgr2=0x5581b85c1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.410+0000 7f50d2e45b80 1 -- 172.21.15.19:0/1046782334 shutdown_connections 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.410+0000 7f50d2e45b80 1 -- 172.21.15.19:0/1046782334 wait complete. 2024-05-09T01:27:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50d2e45b80 1 -- start start 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50d2e45b80 1 Processor -- start 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b84494a0 0x5581b85d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 0x5581b85d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 0x5581b85d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5581b8364ba0 con 0x5581b84494a0 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5581b84a3150 con 0x5581b85c66a0 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581b85d31d0 con 0x5581b8533160 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50d2e45b80 10 client.0 _ll_register_callbacks cb 0x5581b835a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50c21cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b84494a0 0x5581b85d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50c21cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b84494a0 0x5581b85d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59132/0 (socket says 172.21.15.19:59132) 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50c21cb700 1 -- 172.21.15.19:0/3089113023 learned_addr learned my addr 172.21.15.19:0/3089113023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.411+0000 7f50c19ca700 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 0x5581b85d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50c29cc700 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 0x5581b85d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50c21cb700 1 -- 172.21.15.19:0/3089113023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 msgr2=0x5581b85d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50c21cb700 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 0x5581b85d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50c21cb700 1 -- 172.21.15.19:0/3089113023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 msgr2=0x5581b85d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50c21cb700 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 0x5581b85d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50c21cb700 1 -- 172.21.15.19:0/3089113023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b8364490 con 0x5581b84494a0 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50c21cb700 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b84494a0 0x5581b85d02c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f50b801dc00 tx=0x7f50b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905553e7629c1703 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3089113023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8004240 con 0x5581b84494a0 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3089113023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f50b8009040 con 0x5581b84494a0 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3089113023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b401a070 con 0x5581b84494a0 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3089113023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50b80043e0 con 0x5581b84494a0 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50d2e45b80 10 client.15648 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.412+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3089113023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581b85ddd40 con 0x5581b84494a0 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.413+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3089113023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b401a070 con 0x5581b84494a0 2024-05-09T01:27:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.413+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3089113023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50b8003ab0 con 0x5581b84494a0 2024-05-09T01:27:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.413+0000 7f50d2e45b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.413+0000 7f50d2e45b80 10 client.15648 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:27:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.413+0000 7f50d2e45b80 20 client.15648 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.413+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5581b85e0380 0x5581b85e27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.413+0000 7f50d2e45b80 4 client.15648 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.413+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3089113023 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581b84f69d0 con 0x5581b85e0380 2024-05-09T01:27:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.413+0000 7f50c19ca700 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5581b85e0380 0x5581b85e27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.414+0000 7f50a5ffb700 10 client.15648 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:27:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.414+0000 7f50c19ca700 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5581b85e0380 0x5581b85e27e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f3373bdf91b693b server_cookie=d13e555761b3d8ea in_seq=0 out_seq=0 2024-05-09T01:27:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3089113023 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5581b84f69d0 con 0x5581b85e0380 2024-05-09T01:27:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50a5ffb700 10 client.15648 handle_command_reply: tid=0 2024-05-09T01:27:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 client.15648 shutdown 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3089113023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b84494a0 msgr2=0x5581b85d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b84494a0 0x5581b85d02c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f50b801dc00 tx=0x7f50b80047c0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3089113023 shutdown_connections 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5581b85e0380 0x5581b85e27e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50a5ffb700 0 client.15648 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 0x5581b85d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b84494a0 0x5581b85d02c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3089113023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b8533160 0x5581b85d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3089113023 >> 172.21.15.19:0/3089113023 conn(0x5581b84a96a0 msgr2=0x5581b85c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3089113023 shutdown_connections 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3089113023 wait complete. 2024-05-09T01:27:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.415+0000 7f50d2e45b80 20 client.15648 trim_cache size 0 max 16384 2024-05-09T01:27:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50d2e45b80 1 Processor -- start 2024-05-09T01:27:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50d2e45b80 1 -- start start 2024-05-09T01:27:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b84494a0 0x5581b85c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 0x5581b85ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1b0e30 con 0x5581b8533160 2024-05-09T01:27:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1b2970 con 0x5581b85c66a0 2024-05-09T01:27:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1b1500 con 0x5581b84494a0 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50c19ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50c19ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59142/0 (socket says 172.21.15.19:59142) 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50c19ca700 1 -- 172.21.15.19:0/3665128075 learned_addr learned my addr 172.21.15.19:0/3665128075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50c29cc700 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 0x5581b85ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50c21cb700 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b84494a0 0x5581b85c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.431+0000 7f50c19ca700 1 -- 172.21.15.19:0/3665128075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b84494a0 msgr2=0x5581b85c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.432+0000 7f50c19ca700 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b84494a0 0x5581b85c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.432+0000 7f50c19ca700 1 -- 172.21.15.19:0/3665128075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 msgr2=0x5581b85ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.432+0000 7f50c19ca700 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 0x5581b85ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.432+0000 7f50c19ca700 1 -- 172.21.15.19:0/3665128075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b85c6d00 con 0x5581b8533160 2024-05-09T01:27:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.432+0000 7f50c29cc700 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 0x5581b85ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.432+0000 7f50c19ca700 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85ca340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f50b00a7300 tx=0x7f50b0009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da091309ab61926 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.432+0000 7f50a7fff700 1 -- 172.21.15.19:0/3665128075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00ae440 con 0x5581b8533160 2024-05-09T01:27:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.432+0000 7f50a7fff700 1 -- 172.21.15.19:0/3665128075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b00b2e60 con 0x5581b8533160 2024-05-09T01:27:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.432+0000 7f50a7fff700 1 -- 172.21.15.19:0/3665128075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b000f910 con 0x5581b8533160 2024-05-09T01:27:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3665128075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 msgr2=0x5581b85ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85ca340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f50b00a7300 tx=0x7f50b0009230 comp rx=0 tx=0).stop 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3665128075 shutdown_connections 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b85c66a0 0x5581b85ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b84494a0 0x5581b85c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3665128075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85ca340 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3665128075 >> 172.21.15.19:0/3665128075 conn(0x5581b84a96a0 msgr2=0x5581b85cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3665128075 shutdown_connections 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3665128075 wait complete. 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 -- start start 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.433+0000 7f50d2e45b80 1 Processor -- start 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50d2e45b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b85c66a0 0x5581b85db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1b22a0 con 0x5581b8533160 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1b1bd0 con 0x5581b84494a0 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50d2e45b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1aedd0 con 0x5581b85c66a0 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50d2e45b80 10 client.0 _ll_register_callbacks cb 0x5581b835a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c19ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c19ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85db3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59146/0 (socket says 172.21.15.19:59146) 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c19ca700 1 -- 172.21.15.19:0/3666183349 learned_addr learned my addr 172.21.15.19:0/3666183349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c21cb700 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c19ca700 1 -- 172.21.15.19:0/3666183349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b85c66a0 msgr2=0x5581b85db900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c19ca700 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b85c66a0 0x5581b85db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c19ca700 1 -- 172.21.15.19:0/3666183349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 msgr2=0x5581b85d86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c19ca700 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85d86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c19ca700 1 -- 172.21.15.19:0/3666183349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b85dbe40 con 0x5581b8533160 2024-05-09T01:27:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c21cb700 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50c19ca700 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85db3c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f50b00aa020 tx=0x7f50b00ae440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273702dafc5d8fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3666183349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00a6320 con 0x5581b8533160 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3666183349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f50b000f910 con 0x5581b8533160 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.434+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3666183349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b00b2e60 con 0x5581b8533160 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3666183349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50b00a64c0 con 0x5581b8533160 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50d2e45b80 10 client.15654 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3666183349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581b864a800 con 0x5581b8533160 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3666183349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b00b2e60 con 0x5581b8533160 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3666183349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50b00a5a00 con 0x5581b8533160 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50d2e45b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50d2e45b80 10 client.15654 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50d2e45b80 20 client.15654 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5581b85e0380 0x5581b864ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50d2e45b80 4 client.15654 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3666183349 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581b84f0a70 con 0x5581b85e0380 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.435+0000 7f50c21cb700 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5581b85e0380 0x5581b864ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50c21cb700 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5581b85e0380 0x5581b864ced0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6c22203ab88a9d0 server_cookie=926b031715860b61 in_seq=0 out_seq=0 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50a5ffb700 10 client.15654 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50a5ffb700 1 -- 172.21.15.19:0/3666183349 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581b84f0a70 con 0x5581b85e0380 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50a5ffb700 10 client.15654 handle_command_reply: tid=0 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 client.15654 shutdown 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3666183349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 msgr2=0x5581b85db3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85db3c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f50b00aa020 tx=0x7f50b00ae440 comp rx=0 tx=0).stop 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3666183349 shutdown_connections 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5581b85e0380 0x5581b864ced0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b85c66a0 0x5581b85db900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581b84494a0 0x5581b85d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 --2- 172.21.15.19:0/3666183349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5581b8533160 0x5581b85db3c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3666183349 >> 172.21.15.19:0/3666183349 conn(0x5581b84a96a0 msgr2=0x5581b85c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50a5ffb700 0 client.15654 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:27:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3666183349 shutdown_connections 2024-05-09T01:27:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 1 -- 172.21.15.19:0/3666183349 wait complete. 2024-05-09T01:27:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.436+0000 7f50d2e45b80 20 client.15654 trim_cache size 0 max 16384 2024-05-09T01:27:49.439 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 -- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50a80b2880 msgr2=0x7f50a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50a80b2880 0x7f50a80b4d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009c60 tx=0x7f50b8009260 comp rx=0 tx=0).stop 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 -- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 msgr2=0x7f50bc110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 0x7f50bc110fe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f50b00045e0 tx=0x7f50b0004610 comp rx=0 tx=0).stop 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 -- 172.21.15.19:0/828594906 shutdown_connections 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50bc108760 0x7f50bc111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50bc1034e0 0x7f50bc110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50a80b2880 0x7f50a80b4d90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 --2- 172.21.15.19:0/828594906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50bc107e10 0x7f50bc110fe0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 -- 172.21.15.19:0/828594906 >> 172.21.15.19:0/828594906 conn(0x7f50bc0fef30 msgr2=0x7f50bc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 -- 172.21.15.19:0/828594906 shutdown_connections 2024-05-09T01:27:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:49.437+0000 7f50c442f700 1 -- 172.21.15.19:0/828594906 wait complete. 2024-05-09T01:27:50.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:27:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:27:50.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:50 smithi019 ceph-mon[30158]: pgmap v447: 97 pgs: 97 active+clean; 743 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-05-09T01:27:50.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:50 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:27:50.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:50 smithi196 ceph-mon[35041]: pgmap v447: 97 pgs: 97 active+clean; 743 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-05-09T01:27:50.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:50 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:27:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:50 smithi059 ceph-mon[34351]: pgmap v447: 97 pgs: 97 active+clean; 743 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-05-09T01:27:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:50 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:27:51.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:27:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:27:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:27:51.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:27:51.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:27:51.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:27:51.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:27:51.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:27:51.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:27:52.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 -- 172.21.15.19:0/64618957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194109850 msgr2=0x7f91941111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:52.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 --2- 172.21.15.19:0/64618957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194109850 0x7f91941111b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b0d0 tx=0x7f918c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 -- 172.21.15.19:0/64618957 shutdown_connections 2024-05-09T01:27:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 --2- 172.21.15.19:0/64618957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194109850 0x7f91941111b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 --2- 172.21.15.19:0/64618957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91941045d0 0x7f91941049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 --2- 172.21.15.19:0/64618957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9194108f00 0x7f9194109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 -- 172.21.15.19:0/64618957 >> 172.21.15.19:0/64618957 conn(0x7f91940ffea0 msgr2=0x7f91941022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 -- 172.21.15.19:0/64618957 shutdown_connections 2024-05-09T01:27:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 -- 172.21.15.19:0/64618957 wait complete. 2024-05-09T01:27:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.184+0000 7f919a1b8700 1 Processor -- start 2024-05-09T01:27:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f919a1b8700 1 -- start start 2024-05-09T01:27:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f919a1b8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91941045d0 0x7f919419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f919a1b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194108f00 0x7f919419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f919a1b8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9194109850 0x7f919419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f919a1b8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9194114480 con 0x7f9194108f00 2024-05-09T01:27:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f919a1b8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9194114300 con 0x7f9194109850 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f919a1b8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9194114600 con 0x7f91941045d0 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194108f00 0x7f919419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f91937fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91941045d0 0x7f919419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194108f00 0x7f919419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44434/0 (socket says 172.21.15.19:44434) 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f9192ffd700 1 -- 172.21.15.19:0/2149680127 learned_addr learned my addr 172.21.15.19:0/2149680127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.185+0000 7f9193fff700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9194109850 0x7f919419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f9192ffd700 1 -- 172.21.15.19:0/2149680127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91941045d0 msgr2=0x7f919419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f9192ffd700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91941045d0 0x7f919419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f9192ffd700 1 -- 172.21.15.19:0/2149680127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9194109850 msgr2=0x7f919419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f9192ffd700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9194109850 0x7f919419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f9192ffd700 1 -- 172.21.15.19:0/2149680127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919419fc40 con 0x7f9194108f00 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f9192ffd700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194108f00 0x7f919419b310 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f918800d980 tx=0x7f918800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f9190ff9700 1 -- 172.21.15.19:0/2149680127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188017070 con 0x7f9194108f00 2024-05-09T01:27:52.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f9190ff9700 1 -- 172.21.15.19:0/2149680127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918800fd40 con 0x7f9194108f00 2024-05-09T01:27:52.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f919a1b8700 1 -- 172.21.15.19:0/2149680127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919419ff30 con 0x7f9194108f00 2024-05-09T01:27:52.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f9190ff9700 1 -- 172.21.15.19:0/2149680127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918800e3b0 con 0x7f9194108f00 2024-05-09T01:27:52.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.186+0000 7f919a1b8700 1 -- 172.21.15.19:0/2149680127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91941a7850 con 0x7f9194108f00 2024-05-09T01:27:52.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.189+0000 7f9190ff9700 1 -- 172.21.15.19:0/2149680127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9188010610 con 0x7f9194108f00 2024-05-09T01:27:52.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f9190ff9700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f917c0b2880 0x7f917c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f9190ff9700 1 -- 172.21.15.19:0/2149680127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f91880e1c30 con 0x7f9194108f00 2024-05-09T01:27:52.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91937fe700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f917c0b2880 0x7f917c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91a8bceb80 1 Processor -- start 2024-05-09T01:27:52.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91a8bceb80 1 -- start start 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91937fe700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f917c0b2880 0x7f917c0b4d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9184005950 tx=0x7f918400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564349228d00 0x564349331240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349331780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x56434932b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56434924e0a0 con 0x5643491ae4a0 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5643491f39a0 con 0x56434932b400 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56434924a060 con 0x564349228d00 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349331780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349331780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44436/0 (socket says 172.21.15.19:44436) 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.190+0000 7f9192ffd700 1 -- 172.21.15.19:0/3757661551 learned_addr learned my addr 172.21.15.19:0/3757661551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:52.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9193fff700 1 --2- 172.21.15.19:0/3757661551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x56434932b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f91937fe700 1 --2- 172.21.15.19:0/3757661551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564349228d00 0x564349331240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9192ffd700 1 -- 172.21.15.19:0/3757661551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564349228d00 msgr2=0x564349331240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9192ffd700 1 --2- 172.21.15.19:0/3757661551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564349228d00 0x564349331240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9192ffd700 1 -- 172.21.15.19:0/3757661551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 msgr2=0x56434932b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9192ffd700 1 --2- 172.21.15.19:0/3757661551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x56434932b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9192ffd700 1 -- 172.21.15.19:0/3757661551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643491e4500 con 0x5643491ae4a0 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9192ffd700 1 --2- 172.21.15.19:0/3757661551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349331780 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f91880a52b0 tx=0x7f91880a8a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9944f42699abb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9179ffb700 1 -- 172.21.15.19:0/3757661551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188020040 con 0x5643491ae4a0 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9179ffb700 1 -- 172.21.15.19:0/3757661551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91880a9930 con 0x5643491ae4a0 2024-05-09T01:27:52.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.191+0000 7f9179ffb700 1 -- 172.21.15.19:0/3757661551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91880b1e60 con 0x5643491ae4a0 2024-05-09T01:27:52.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.192+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3757661551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 msgr2=0x564349331780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:52.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.192+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3757661551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349331780 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f91880a52b0 tx=0x7f91880a8a20 comp rx=0 tx=0).stop 2024-05-09T01:27:52.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.192+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3757661551 shutdown_connections 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.192+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3757661551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x56434932b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.192+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3757661551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564349228d00 0x564349331240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.192+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3757661551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349331780 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.192+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3757661551 >> 172.21.15.19:0/3757661551 conn(0x5643491b1f70 msgr2=0x564349326220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.192+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3757661551 shutdown_connections 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.192+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3757661551 wait complete. 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91a8bceb80 1 -- start start 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91a8bceb80 1 Processor -- start 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349334fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564349228d00 0x564349335510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56434932b400 0x564349335a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56434932c0a0 con 0x5643491ae4a0 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5643491e8b20 con 0x564349228d00 2024-05-09T01:27:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56434932c220 con 0x56434932b400 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349334fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91a8bceb80 10 client.0 _ll_register_callbacks cb 0x5643490bf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349334fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44440/0 (socket says 172.21.15.19:44440) 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f91937fe700 1 -- 172.21.15.19:0/3980485779 learned_addr learned my addr 172.21.15.19:0/3980485779 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.193+0000 7f9192ffd700 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564349228d00 0x564349335510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f9193fff700 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56434932b400 0x564349335a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f91937fe700 1 -- 172.21.15.19:0/3980485779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56434932b400 msgr2=0x564349335a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f91937fe700 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56434932b400 0x564349335a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f91937fe700 1 -- 172.21.15.19:0/3980485779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564349228d00 msgr2=0x564349335510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f91937fe700 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564349228d00 0x564349335510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f91937fe700 1 -- 172.21.15.19:0/3980485779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643490c9490 con 0x5643491ae4a0 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f91937fe700 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349334fd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f91840116c0 tx=0x7f91840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a0be63e1e54147 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f9172ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f9172ffd700 1 -- 172.21.15.19:0/3980485779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184004a30 con 0x5643491ae4a0 2024-05-09T01:27:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3980485779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91880a9930 con 0x5643491ae4a0 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f9172ffd700 1 -- 172.21.15.19:0/3980485779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9184004bd0 con 0x5643491ae4a0 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f9172ffd700 1 -- 172.21.15.19:0/3980485779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9184004dd0 con 0x5643491ae4a0 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f91a8bceb80 10 client.15666 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.194+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3980485779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564349342960 con 0x5643491ae4a0 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.195+0000 7f9172ffd700 1 -- 172.21.15.19:0/3980485779 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91880a9930 con 0x5643491ae4a0 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.195+0000 7f9172ffd700 1 -- 172.21.15.19:0/3980485779 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f918401f8b0 con 0x5643491ae4a0 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.195+0000 7f91a8bceb80 10 client.15666 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.195+0000 7f91a8bceb80 10 client.15666 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.195+0000 7f91a8bceb80 20 client.15666 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.195+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x564349344fa0 0x5643493473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.195+0000 7f91a8bceb80 4 client.15666 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.195+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3980485779 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56434925ba80 con 0x564349344fa0 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.195+0000 7f9192ffd700 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x564349344fa0 0x5643493473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.196+0000 7f9192ffd700 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x564349344fa0 0x5643493473e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d785a8f3b2312de server_cookie=ed5f3c8e59883250 in_seq=0 out_seq=0 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.196+0000 7f9172ffd700 10 client.15666 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.196+0000 7f9172ffd700 1 -- 172.21.15.19:0/3980485779 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56434925ba80 con 0x564349344fa0 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.196+0000 7f9172ffd700 10 client.15666 handle_command_reply: tid=0 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.196+0000 7f91a8bceb80 1 client.15666 shutdown 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3980485779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 msgr2=0x564349334fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349334fd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f91840116c0 tx=0x7f91840047c0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3980485779 shutdown_connections 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x564349344fa0 0x5643493473e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56434932b400 0x564349335a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643491ae4a0 0x564349334fd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f9172ffd700 0 client.15666 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/3980485779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564349228d00 0x564349335510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3980485779 >> 172.21.15.19:0/3980485779 conn(0x5643491b1f70 msgr2=0x56434931fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:52.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3980485779 shutdown_connections 2024-05-09T01:27:52.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 1 -- 172.21.15.19:0/3980485779 wait complete. 2024-05-09T01:27:52.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.197+0000 7f91a8bceb80 20 client.15666 trim_cache size 0 max 16384 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f91a8bceb80 1 Processor -- start 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f91a8bceb80 1 -- start start 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 0x564349320a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349333ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x564349340d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91941aed20 con 0x564349228d00 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91941b0860 con 0x56434932b400 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91941af3f0 con 0x5643491ae4a0 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349333ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349333ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44454/0 (socket says 172.21.15.19:44454) 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.212+0000 7f9192ffd700 1 -- 172.21.15.19:0/2243578786 learned_addr learned my addr 172.21.15.19:0/2243578786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f9193fff700 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x564349340d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f91937fe700 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 0x564349320a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f9192ffd700 1 -- 172.21.15.19:0/2243578786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 msgr2=0x564349320a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f9192ffd700 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 0x564349320a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f9192ffd700 1 -- 172.21.15.19:0/2243578786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 msgr2=0x564349340d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f9192ffd700 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x564349340d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f9192ffd700 1 -- 172.21.15.19:0/2243578786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564349341b30 con 0x564349228d00 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f91937fe700 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 0x564349320a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f9192ffd700 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349333ec0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f91880a8f30 tx=0x7f9188005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6487d939136ea4d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f91797fa700 1 -- 172.21.15.19:0/2243578786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188020040 con 0x564349228d00 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f91797fa700 1 -- 172.21.15.19:0/2243578786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91880a9930 con 0x564349228d00 2024-05-09T01:27:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.213+0000 7f91797fa700 1 -- 172.21.15.19:0/2243578786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91880ad120 con 0x564349228d00 2024-05-09T01:27:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2243578786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 msgr2=0x564349333ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349333ec0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f91880a8f30 tx=0x7f9188005da0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2243578786 shutdown_connections 2024-05-09T01:27:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x564349340d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 0x564349320a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2243578786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349333ec0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2243578786 >> 172.21.15.19:0/2243578786 conn(0x5643491b1f70 msgr2=0x5643493337b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2243578786 shutdown_connections 2024-05-09T01:27:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2243578786 wait complete. 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.214+0000 7f91a8bceb80 1 -- start start 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f91a8bceb80 1 Processor -- start 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 0x56434933d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349338800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f91a8bceb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x564349338d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91941b0190 con 0x564349228d00 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91941afac0 con 0x56434932b400 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f91a8bceb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91941accc0 con 0x5643491ae4a0 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349338800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f9192ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349338800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44468/0 (socket says 172.21.15.19:44468) 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f9192ffd700 1 -- 172.21.15.19:0/2525943819 learned_addr learned my addr 172.21.15.19:0/2525943819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f91a8bceb80 10 client.0 _ll_register_callbacks cb 0x5643490bf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f9193fff700 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x564349338d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.215+0000 7f91937fe700 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 0x56434933d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9192ffd700 1 -- 172.21.15.19:0/2525943819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 msgr2=0x56434933d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9192ffd700 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 0x56434933d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9192ffd700 1 -- 172.21.15.19:0/2525943819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 msgr2=0x564349338d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9192ffd700 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x564349338d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9192ffd700 1 -- 172.21.15.19:0/2525943819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564349339300 con 0x564349228d00 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9192ffd700 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349338800 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f91880a5260 tx=0x7f9188010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28867b3287eb20f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9172ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9172ffd700 1 -- 172.21.15.19:0/2525943819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188020070 con 0x564349228d00 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9172ffd700 1 -- 172.21.15.19:0/2525943819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f91880b1c00 con 0x564349228d00 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2525943819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91880a9930 con 0x564349228d00 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f9172ffd700 1 -- 172.21.15.19:0/2525943819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91880b1e00 con 0x564349228d00 2024-05-09T01:27:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f91a8bceb80 10 client.15672 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.216+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2525943819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643493b3870 con 0x564349228d00 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.217+0000 7f9172ffd700 1 -- 172.21.15.19:0/2525943819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91880a9930 con 0x564349228d00 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.217+0000 7f9172ffd700 1 -- 172.21.15.19:0/2525943819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f91880a4ba0 con 0x564349228d00 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.217+0000 7f91a8bceb80 10 client.15672 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.217+0000 7f91a8bceb80 10 client.15672 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.217+0000 7f91a8bceb80 20 client.15672 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.217+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5643493b3a40 0x5643493b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.217+0000 7f91a8bceb80 4 client.15672 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.217+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2525943819 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564349255b20 con 0x5643493b3a40 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.217+0000 7f91937fe700 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5643493b3a40 0x5643493b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91937fe700 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5643493b3a40 0x5643493b5e60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6757127a6da1b928 server_cookie=7395d594159c3796 in_seq=0 out_seq=0 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f9172ffd700 10 client.15672 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f9172ffd700 1 -- 172.21.15.19:0/2525943819 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564349255b20 con 0x5643493b3a40 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f9172ffd700 10 client.15672 handle_command_reply: tid=0 2024-05-09T01:27:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 client.15672 shutdown 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2525943819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 msgr2=0x564349338800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349338800 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f91880a5260 tx=0x7f9188010000 comp rx=0 tx=0).stop 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2525943819 shutdown_connections 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5643493b3a40 0x5643493b5e60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56434932b400 0x564349338d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f9172ffd700 0 client.15672 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643491ae4a0 0x56434933d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 --2- 172.21.15.19:0/2525943819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564349228d00 0x564349338800 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2525943819 >> 172.21.15.19:0/2525943819 conn(0x5643491b1f70 msgr2=0x564349320780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2525943819 shutdown_connections 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 1 -- 172.21.15.19:0/2525943819 wait complete. 2024-05-09T01:27:52.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.218+0000 7f91a8bceb80 20 client.15672 trim_cache size 0 max 16384 2024-05-09T01:27:52.221 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:27:52.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 -- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f917c0b2880 msgr2=0x7f917c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:52.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f917c0b2880 0x7f917c0b4d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9184005950 tx=0x7f918400b410 comp rx=0 tx=0).stop 2024-05-09T01:27:52.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 -- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194108f00 msgr2=0x7f919419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:52.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194108f00 0x7f919419b310 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f918800d980 tx=0x7f918800dc90 comp rx=0 tx=0).stop 2024-05-09T01:27:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 -- 172.21.15.19:0/2149680127 shutdown_connections 2024-05-09T01:27:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9194109850 0x7f919419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91941045d0 0x7f919419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f917c0b2880 0x7f917c0b4d90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 --2- 172.21.15.19:0/2149680127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9194108f00 0x7f919419b310 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 -- 172.21.15.19:0/2149680127 >> 172.21.15.19:0/2149680127 conn(0x7f91940ffea0 msgr2=0x7f9194108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 -- 172.21.15.19:0/2149680127 shutdown_connections 2024-05-09T01:27:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:52.219+0000 7f919a1b8700 1 -- 172.21.15.19:0/2149680127 wait complete. 2024-05-09T01:27:52.808 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:52 smithi019 ceph-mon[30158]: pgmap v448: 97 pgs: 97 active+clean; 806 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:27:52.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:52 smithi059 ceph-mon[34351]: pgmap v448: 97 pgs: 97 active+clean; 806 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:27:52.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:52 smithi196 ceph-mon[35041]: pgmap v448: 97 pgs: 97 active+clean; 806 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:27:53.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:27:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:27:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 -- 172.21.15.19:0/2525171966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec1047f0 msgr2=0x7f31ec10af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/2525171966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec1047f0 0x7f31ec10af40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009ca0 tx=0x7f31e402a8c0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 -- 172.21.15.19:0/2525171966 shutdown_connections 2024-05-09T01:27:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/2525171966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec1047f0 0x7f31ec10af40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/2525171966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec103c90 0x7f31ec104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/2525171966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31ec103130 0x7f31ec103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 -- 172.21.15.19:0/2525171966 >> 172.21.15.19:0/2525171966 conn(0x7f31ec0fd4b0 msgr2=0x7f31ec0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 -- 172.21.15.19:0/2525171966 shutdown_connections 2024-05-09T01:27:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 -- 172.21.15.19:0/2525171966 wait complete. 2024-05-09T01:27:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.982+0000 7f31f1a0c640 1 Processor -- start 2024-05-09T01:27:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31f1a0c640 1 -- start start 2024-05-09T01:27:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31f1a0c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31ec103130 0x7f31ec19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31f1a0c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec103c90 0x7f31ec19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31f1a0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec19f5d0 0x7f31ec1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31f1a0c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec10e8a0 con 0x7f31ec19f5d0 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31f1a0c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec10e720 con 0x7f31ec103c90 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31f1a0c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec10ea20 con 0x7f31ec103130 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31eb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec19f5d0 0x7f31ec1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31eb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec19f5d0 0x7f31ec1a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44478/0 (socket says 172.21.15.19:44478) 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31eb7fe640 1 -- 172.21.15.19:0/3078769058 learned_addr learned my addr 172.21.15.19:0/3078769058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31eaffd640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31ec103130 0x7f31ec19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec103c90 0x7f31ec19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31eb7fe640 1 -- 172.21.15.19:0/3078769058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31ec103130 msgr2=0x7f31ec19a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31eb7fe640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31ec103130 0x7f31ec19a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31eb7fe640 1 -- 172.21.15.19:0/3078769058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec103c90 msgr2=0x7f31ec19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31eb7fe640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec103c90 0x7f31ec19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.983+0000 7f31eb7fe640 1 -- 172.21.15.19:0/3078769058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec1a2060 con 0x7f31ec19f5d0 2024-05-09T01:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.984+0000 7f31eb7fe640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec19f5d0 0x7f31ec1a19f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f31e40097b0 tx=0x7f31e4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.984+0000 7f31f0a0a640 1 -- 172.21.15.19:0/3078769058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4004240 con 0x7f31ec19f5d0 2024-05-09T01:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.984+0000 7f31f0a0a640 1 -- 172.21.15.19:0/3078769058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31e402f380 con 0x7f31ec19f5d0 2024-05-09T01:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.984+0000 7f31f1a0c640 1 -- 172.21.15.19:0/3078769058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ec1a22f0 con 0x7f31ec19f5d0 2024-05-09T01:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.984+0000 7f31f0a0a640 1 -- 172.21.15.19:0/3078769058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e402fc30 con 0x7f31ec19f5d0 2024-05-09T01:27:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.984+0000 7f31f1a0c640 1 -- 172.21.15.19:0/3078769058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ec1a2840 con 0x7f31ec19f5d0 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.986+0000 7f31f0a0a640 1 -- 172.21.15.19:0/3078769058 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f31e40043e0 con 0x7f31ec19f5d0 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31f8997740 1 Processor -- start 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31f8997740 1 -- start start 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31f8997740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c67500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31f8997740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31f8997740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d68a24d0 0x55c7d6c6e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31f8997740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c7d6bcb7c0 con 0x55c7d689a2f0 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31f8997740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7d6b670e0 con 0x55c7d68a24d0 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31f8997740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c7d6bc7780 con 0x55c7d68c31b0 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31eaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31eaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c67a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44482/0 (socket says 172.21.15.19:44482) 2024-05-09T01:27:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31eaffd640 1 -- 172.21.15.19:0/3186232758 learned_addr learned my addr 172.21.15.19:0/3186232758 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c67500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d68a24d0 0x55c7d6c6e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31f0a0a640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f31d40b27e0 0x7f31d40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31eaffd640 1 -- 172.21.15.19:0/3186232758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 msgr2=0x55c7d6c67500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31eaffd640 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c67500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31eaffd640 1 -- 172.21.15.19:0/3186232758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d68a24d0 msgr2=0x55c7d6c6e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31eaffd640 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d68a24d0 0x55c7d6c6e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31eaffd640 1 -- 172.21.15.19:0/3186232758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7d6b907c0 con 0x55c7d689a2f0 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31eb7fe640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f31d40b27e0 0x7f31d40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.987+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c67500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.988+0000 7f31f0a0a640 1 -- 172.21.15.19:0/3078769058 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f31e4103aa0 con 0x7f31ec19f5d0 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.988+0000 7f31eaffd640 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c67a80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00bd10 tx=0x7f31dc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd1bed5c74a7737 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.988+0000 7f31d1ffb640 1 -- 172.21.15.19:0/3186232758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc005220 con 0x55c7d689a2f0 2024-05-09T01:27:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.988+0000 7f31eb7fe640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f31d40b27e0 0x7f31d40b4cf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31e40c8280 tx=0x7f31e402f6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.988+0000 7f31d1ffb640 1 -- 172.21.15.19:0/3186232758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31dc004c50 con 0x55c7d689a2f0 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.988+0000 7f31d1ffb640 1 -- 172.21.15.19:0/3186232758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc011820 con 0x55c7d689a2f0 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 -- 172.21.15.19:0/3186232758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 msgr2=0x55c7d6c67a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c67a80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00bd10 tx=0x7f31dc0107a0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 -- 172.21.15.19:0/3186232758 shutdown_connections 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d68a24d0 0x55c7d6c6e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c67a80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 --2- 172.21.15.19:0/3186232758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c67500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 -- 172.21.15.19:0/3186232758 >> 172.21.15.19:0/3186232758 conn(0x55c7d68b7300 msgr2=0x55c7d6c62e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 -- 172.21.15.19:0/3186232758 shutdown_connections 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 -- 172.21.15.19:0/3186232758 wait complete. 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 -- start start 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.989+0000 7f31f8997740 1 Processor -- start 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31f8997740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 0x55c7d6c75630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31f8997740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c75b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31f8997740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c7cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31f8997740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c7d6c736c0 con 0x55c7d68a24d0 2024-05-09T01:27:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31f8997740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7d6b529c0 con 0x55c7d689a2f0 2024-05-09T01:27:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31f8997740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c7d6c73840 con 0x55c7d68c31b0 2024-05-09T01:27:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31f8997740 10 client.0 _ll_register_callbacks cb 0x55c7d6b7f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31eaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c75b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31eaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c75b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44492/0 (socket says 172.21.15.19:44492) 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31eaffd640 1 -- 172.21.15.19:0/923725848 learned_addr learned my addr 172.21.15.19:0/923725848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 0x55c7d6c75630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c7cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31eaffd640 1 -- 172.21.15.19:0/923725848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 msgr2=0x55c7d6c7cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31eaffd640 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c7cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31eaffd640 1 -- 172.21.15.19:0/923725848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 msgr2=0x55c7d6c75630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31eaffd640 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 0x55c7d6c75630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31eaffd640 1 -- 172.21.15.19:0/923725848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7d6b791f0 con 0x55c7d68a24d0 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c7cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31eaffd640 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c75b70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00b670 tx=0x7f31dc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b411ac171838083 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31caffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.990+0000 7f31caffd640 1 -- 172.21.15.19:0/923725848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc005210 con 0x55c7d68a24d0 2024-05-09T01:27:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31caffd640 1 -- 172.21.15.19:0/923725848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f31dc004020 con 0x55c7d68a24d0 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31f8997740 1 -- 172.21.15.19:0/923725848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31dc004c50 con 0x55c7d68a24d0 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31caffd640 1 -- 172.21.15.19:0/923725848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f31dc004a60 con 0x55c7d68a24d0 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31f8997740 10 client.15684 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31f8997740 1 -- 172.21.15.19:0/923725848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7d692e790 con 0x55c7d68a24d0 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31caffd640 1 -- 172.21.15.19:0/923725848 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31dc004c50 con 0x55c7d68a24d0 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31caffd640 1 -- 172.21.15.19:0/923725848 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f31dc011520 con 0x55c7d68a24d0 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31f8997740 10 client.15684 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31f8997740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31f8997740 20 client.15684 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31f8997740 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c7d6881d80 0x55c7d6c82c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31f8997740 4 client.15684 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.991+0000 7f31f8997740 1 -- 172.21.15.19:0/923725848 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7d6bd93d0 con 0x55c7d6881d80 2024-05-09T01:27:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.992+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c7d6881d80 0x55c7d6c82c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.992+0000 7f31caffd640 10 client.15684 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.992+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c7d6881d80 0x55c7d6c82c50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f31a29baa909fcf server_cookie=31e78eadeaf2fec9 in_seq=0 out_seq=0 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31caffd640 1 -- 172.21.15.19:0/923725848 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c7d6bd93d0 con 0x55c7d6881d80 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31caffd640 10 client.15684 handle_command_reply: tid=0 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 client.15684 shutdown 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 -- 172.21.15.19:0/923725848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 msgr2=0x55c7d6c75b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c75b70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00b670 tx=0x7f31dc0048f0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 -- 172.21.15.19:0/923725848 shutdown_connections 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c7d6881d80 0x55c7d6c82c50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68c31b0 0x55c7d6c7cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c75b70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31caffd640 0 client.15684 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 --2- 172.21.15.19:0/923725848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 0x55c7d6c75630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 -- 172.21.15.19:0/923725848 >> 172.21.15.19:0/923725848 conn(0x55c7d68b7300 msgr2=0x55c7d6c61700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 -- 172.21.15.19:0/923725848 shutdown_connections 2024-05-09T01:27:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 1 -- 172.21.15.19:0/923725848 wait complete. 2024-05-09T01:27:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:53.993+0000 7f31f8997740 20 client.15684 trim_cache size 0 max 16384 2024-05-09T01:27:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.008+0000 7f31f8997740 1 Processor -- start 2024-05-09T01:27:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.008+0000 7f31f8997740 1 -- start start 2024-05-09T01:27:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.008+0000 7f31f8997740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d6881d80 0x55c7d6c6ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31f8997740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31f8997740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31f8997740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec1a9150 con 0x55c7d689a2f0 2024-05-09T01:27:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31f8997740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec1aac90 con 0x55c7d6881d80 2024-05-09T01:27:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31f8997740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec1a9820 con 0x55c7d68a24d0 2024-05-09T01:27:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31eaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31eaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44500/0 (socket says 172.21.15.19:44500) 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31eaffd640 1 -- 172.21.15.19:0/3192684664 learned_addr learned my addr 172.21.15.19:0/3192684664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/3192684664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d6881d80 0x55c7d6c6ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/3192684664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31eaffd640 1 -- 172.21.15.19:0/3192684664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68a24d0 msgr2=0x55c7d6c7cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31eaffd640 1 --2- 172.21.15.19:0/3192684664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31eaffd640 1 -- 172.21.15.19:0/3192684664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d6881d80 msgr2=0x55c7d6c6ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31eaffd640 1 --2- 172.21.15.19:0/3192684664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d6881d80 0x55c7d6c6ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31eaffd640 1 -- 172.21.15.19:0/3192684664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7d69a55f0 con 0x55c7d689a2f0 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31eaffd640 1 --2- 172.21.15.19:0/3192684664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7c9f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f31dc005d00 tx=0x7f31dc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2ef32229fd6186 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31d17fa640 1 -- 172.21.15.19:0/3192684664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc011c90 con 0x55c7d689a2f0 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31d17fa640 1 -- 172.21.15.19:0/3192684664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31dc00b810 con 0x55c7d689a2f0 2024-05-09T01:27:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.009+0000 7f31d17fa640 1 -- 172.21.15.19:0/3192684664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc011680 con 0x55c7d689a2f0 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.010+0000 7f31f8997740 1 -- 172.21.15.19:0/3192684664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 msgr2=0x55c7d6c7c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.010+0000 7f31f8997740 1 --2- 172.21.15.19:0/3192684664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7c9f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f31dc005d00 tx=0x7f31dc0107a0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.010+0000 7f31f8997740 1 -- 172.21.15.19:0/3192684664 shutdown_connections 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.010+0000 7f31f8997740 1 --2- 172.21.15.19:0/3192684664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.010+0000 7f31f8997740 1 --2- 172.21.15.19:0/3192684664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7c9f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.010+0000 7f31f8997740 1 --2- 172.21.15.19:0/3192684664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d6881d80 0x55c7d6c6ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.010+0000 7f31f8997740 1 -- 172.21.15.19:0/3192684664 >> 172.21.15.19:0/3192684664 conn(0x55c7d68b7300 msgr2=0x55c7d6c6a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.010+0000 7f31f8997740 1 -- 172.21.15.19:0/3192684664 shutdown_connections 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.010+0000 7f31f8997740 1 -- 172.21.15.19:0/3192684664 wait complete. 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31f8997740 1 -- start start 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31f8997740 1 Processor -- start 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31f8997740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d6881d80 0x55c7d6c7a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31f8997740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31f8997740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31f8997740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec1aa5c0 con 0x55c7d68a24d0 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31f8997740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec1a9ef0 con 0x55c7d689a2f0 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31f8997740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec1a7130 con 0x55c7d6881d80 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31f8997740 10 client.0 _ll_register_callbacks cb 0x7f31e000ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31ea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31ea7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44506/0 (socket says 172.21.15.19:44506) 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31ea7fc640 1 -- 172.21.15.19:0/4098984499 learned_addr learned my addr 172.21.15.19:0/4098984499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d6881d80 0x55c7d6c7a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.011+0000 7f31eaffd640 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31ea7fc640 1 -- 172.21.15.19:0/4098984499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d6881d80 msgr2=0x55c7d6c7a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d6881d80 0x55c7d6c7a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31ea7fc640 1 -- 172.21.15.19:0/4098984499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 msgr2=0x55c7d6c7a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31ea7fc640 1 -- 172.21.15.19:0/4098984499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7d6b8fed0 con 0x55c7d68a24d0 2024-05-09T01:27:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31ea7fc640 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7b920 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f31ec19b1b0 tx=0x7f31e0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beed1e810729e6f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31caffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31caffd640 1 -- 172.21.15.19:0/4098984499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e001fe60 con 0x55c7d68a24d0 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31caffd640 1 -- 172.21.15.19:0/4098984499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f31e001b040 con 0x55c7d68a24d0 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31f8997740 1 -- 172.21.15.19:0/4098984499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31dc00b810 con 0x55c7d68a24d0 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31caffd640 1 -- 172.21.15.19:0/4098984499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f31e0005020 con 0x55c7d68a24d0 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31f8997740 10 client.15690 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31f8997740 1 -- 172.21.15.19:0/4098984499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7d6c7d0a0 con 0x55c7d68a24d0 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31caffd640 1 -- 172.21.15.19:0/4098984499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31dc00b810 con 0x55c7d68a24d0 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.012+0000 7f31caffd640 1 -- 172.21.15.19:0/4098984499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f31e00115c0 con 0x55c7d68a24d0 2024-05-09T01:27:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.013+0000 7f31f8997740 10 client.15690 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.013+0000 7f31f8997740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:27:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.013+0000 7f31f8997740 20 client.15690 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.013+0000 7f31f8997740 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c7d68c31b0 0x55c7d6ced1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.013+0000 7f31f8997740 4 client.15690 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:27:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.013+0000 7f31f8997740 1 -- 172.21.15.19:0/4098984499 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7d6bd3240 con 0x55c7d68c31b0 2024-05-09T01:27:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.013+0000 7f31eaffd640 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c7d68c31b0 0x55c7d6ced1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.013+0000 7f31eaffd640 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c7d68c31b0 0x55c7d6ced1b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bd387db49aaf625 server_cookie=e5f3cdc53ac907f4 in_seq=0 out_seq=0 2024-05-09T01:27:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.013+0000 7f31caffd640 10 client.15690 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31caffd640 1 -- 172.21.15.19:0/4098984499 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c7d6bd3240 con 0x55c7d68c31b0 2024-05-09T01:27:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31caffd640 10 client.15690 handle_command_reply: tid=0 2024-05-09T01:27:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 client.15690 shutdown 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 -- 172.21.15.19:0/4098984499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 msgr2=0x55c7d6c7b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7b920 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f31ec19b1b0 tx=0x7f31e0015040 comp rx=0 tx=0).stop 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 -- 172.21.15.19:0/4098984499 shutdown_connections 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c7d68c31b0 0x55c7d6ced1b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c7d68a24d0 0x55c7d6c7b920 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7d689a2f0 0x55c7d6c7a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31caffd640 0 client.15690 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 --2- 172.21.15.19:0/4098984499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7d6881d80 0x55c7d6c7a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 -- 172.21.15.19:0/4098984499 >> 172.21.15.19:0/4098984499 conn(0x55c7d68b7300 msgr2=0x55c7d6c62760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 -- 172.21.15.19:0/4098984499 shutdown_connections 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.028+0000 7f31f8997740 1 -- 172.21.15.19:0/4098984499 wait complete. 2024-05-09T01:27:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f8997740 20 client.15690 trim_cache size 0 max 16384 2024-05-09T01:27:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:27:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:27:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 -- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f31d40b27e0 msgr2=0x7f31d40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f31d40b27e0 0x7f31d40b4cf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31e40c8280 tx=0x7f31e402f6f0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 -- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec19f5d0 msgr2=0x7f31ec1a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec19f5d0 0x7f31ec1a19f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f31e40097b0 tx=0x7f31e4003c20 comp rx=0 tx=0).stop 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 -- 172.21.15.19:0/3078769058 shutdown_connections 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f31d40b27e0 0x7f31d40b4cf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31ec19f5d0 0x7f31ec1a19f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec103c90 0x7f31ec19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 --2- 172.21.15.19:0/3078769058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31ec103130 0x7f31ec19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 -- 172.21.15.19:0/3078769058 >> 172.21.15.19:0/3078769058 conn(0x7f31ec0fd4b0 msgr2=0x7f31ec0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 -- 172.21.15.19:0/3078769058 shutdown_connections 2024-05-09T01:27:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:27:54.029+0000 7f31f1a0c640 1 -- 172.21.15.19:0/3078769058 wait complete. 2024-05-09T01:27:54.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:54 smithi019 ceph-mon[30158]: pgmap v449: 97 pgs: 97 active+clean; 806 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:27:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:54 smithi059 ceph-mon[34351]: pgmap v449: 97 pgs: 97 active+clean; 806 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:27:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:54 smithi196 ceph-mon[35041]: pgmap v449: 97 pgs: 97 active+clean; 806 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:27:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.165+0000 7fd913cb9700 1 -- 172.21.15.19:0/3781896164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 msgr2=0x7fd90c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.165+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3781896164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 0x7fd90c1100a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd90400b0d0 tx=0x7fd90400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.165+0000 7fd913cb9700 1 -- 172.21.15.19:0/3781896164 shutdown_connections 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.165+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3781896164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 0x7fd90c1100a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.165+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3781896164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd90c1034c0 0x7fd90c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.165+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3781896164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd90c107df0 0x7fd90c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.165+0000 7fd913cb9700 1 -- 172.21.15.19:0/3781896164 >> 172.21.15.19:0/3781896164 conn(0x7fd90c0fedc0 msgr2=0x7fd90c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.165+0000 7fd913cb9700 1 -- 172.21.15.19:0/3781896164 shutdown_connections 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.165+0000 7fd913cb9700 1 -- 172.21.15.19:0/3781896164 wait complete. 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.166+0000 7fd913cb9700 1 Processor -- start 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.166+0000 7fd913cb9700 1 -- start start 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.166+0000 7fd913cb9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd90c1034c0 0x7fd90c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.166+0000 7fd913cb9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd90c107df0 0x7fd90c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd913cb9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 0x7fd90c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd913cb9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c06c180 con 0x7fd90c108740 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd913cb9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c06c000 con 0x7fd90c1034c0 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd913cb9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c06c300 con 0x7fd90c107df0 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd912256700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 0x7fd90c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd911a55700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd90c1034c0 0x7fd90c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd912256700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 0x7fd90c19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44520/0 (socket says 172.21.15.19:44520) 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd912256700 1 -- 172.21.15.19:0/3140775677 learned_addr learned my addr 172.21.15.19:0/3140775677 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd911a55700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd90c1034c0 0x7fd90c19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35322/0 (socket says 172.21.15.19:35322) 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd911254700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd90c107df0 0x7fd90c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd912256700 1 -- 172.21.15.19:0/3140775677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd90c107df0 msgr2=0x7fd90c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd912256700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd90c107df0 0x7fd90c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd912256700 1 -- 172.21.15.19:0/3140775677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd90c1034c0 msgr2=0x7fd90c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd912256700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd90c1034c0 0x7fd90c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd912256700 1 -- 172.21.15.19:0/3140775677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c19fc10 con 0x7fd90c108740 2024-05-09T01:27:55.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.167+0000 7fd912256700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 0x7fd90c19f570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd9040095c0 tx=0x7fd904007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:55.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.168+0000 7fd902ffd700 1 -- 172.21.15.19:0/3140775677 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90401f070 con 0x7fd90c108740 2024-05-09T01:27:55.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.168+0000 7fd902ffd700 1 -- 172.21.15.19:0/3140775677 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd904007e20 con 0x7fd90c108740 2024-05-09T01:27:55.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.168+0000 7fd913cb9700 1 -- 172.21.15.19:0/3140775677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c19fea0 con 0x7fd90c108740 2024-05-09T01:27:55.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.168+0000 7fd902ffd700 1 -- 172.21.15.19:0/3140775677 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90401b4a0 con 0x7fd90c108740 2024-05-09T01:27:55.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.168+0000 7fd913cb9700 1 -- 172.21.15.19:0/3140775677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c1a0380 con 0x7fd90c108740 2024-05-09T01:27:55.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.171+0000 7fd902ffd700 1 -- 172.21.15.19:0/3140775677 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd904022070 con 0x7fd90c108740 2024-05-09T01:27:55.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.172+0000 7fd902ffd700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8f80b2880 0x7fd8f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.172+0000 7fd9226cfb80 1 Processor -- start 2024-05-09T01:27:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.172+0000 7fd902ffd700 1 -- 172.21.15.19:0/3140775677 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd90400f020 con 0x7fd90c108740 2024-05-09T01:27:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.172+0000 7fd911a55700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8f80b2880 0x7fd8f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd9226cfb80 1 -- start start 2024-05-09T01:27:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a582240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a3ff4a0 0x556a1a582780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a57c400 0x556a1a57c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556a1a49f0a0 con 0x556a1a57c400 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556a1a4449a0 con 0x556a1a479d00 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556a1a49b060 con 0x556a1a3ff4a0 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd911a55700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8f80b2880 0x7fd8f80b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd908005950 tx=0x7fd90800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd911a55700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a582240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd911254700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a3ff4a0 0x556a1a582780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd911a55700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a582240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35336/0 (socket says 172.21.15.19:35336) 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd911a55700 1 -- 172.21.15.19:0/3662106014 learned_addr learned my addr 172.21.15.19:0/3662106014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.173+0000 7fd912256700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a57c400 0x556a1a57c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.174+0000 7fd911254700 1 -- 172.21.15.19:0/3662106014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 msgr2=0x556a1a582240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.174+0000 7fd911254700 1 --2- 172.21.15.19:0/3662106014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a582240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.174+0000 7fd911254700 1 -- 172.21.15.19:0/3662106014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a57c400 msgr2=0x556a1a57c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.174+0000 7fd911254700 1 --2- 172.21.15.19:0/3662106014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a57c400 0x556a1a57c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.174+0000 7fd911254700 1 -- 172.21.15.19:0/3662106014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1a435500 con 0x556a1a3ff4a0 2024-05-09T01:27:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.174+0000 7fd911254700 1 --2- 172.21.15.19:0/3662106014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a3ff4a0 0x556a1a582780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc005950 tx=0x7fd8fc00ae30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=538ed1e0bcb02a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.174+0000 7fd8f7fff700 1 -- 172.21.15.19:0/3662106014 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc00fc90 con 0x556a1a3ff4a0 2024-05-09T01:27:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.174+0000 7fd8f7fff700 1 -- 172.21.15.19:0/3662106014 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8fc0044d0 con 0x556a1a3ff4a0 2024-05-09T01:27:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.174+0000 7fd8f7fff700 1 -- 172.21.15.19:0/3662106014 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc018440 con 0x556a1a3ff4a0 2024-05-09T01:27:55.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.175+0000 7fd9226cfb80 1 -- 172.21.15.19:0/3662106014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a3ff4a0 msgr2=0x556a1a582780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:55.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.175+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/3662106014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a3ff4a0 0x556a1a582780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc005950 tx=0x7fd8fc00ae30 comp rx=0 tx=0).stop 2024-05-09T01:27:55.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.175+0000 7fd9226cfb80 1 -- 172.21.15.19:0/3662106014 shutdown_connections 2024-05-09T01:27:55.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.175+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/3662106014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a57c400 0x556a1a57c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.175+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/3662106014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a582240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.175+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/3662106014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a3ff4a0 0x556a1a582780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.175+0000 7fd9226cfb80 1 -- 172.21.15.19:0/3662106014 >> 172.21.15.19:0/3662106014 conn(0x556a1a402f70 msgr2=0x556a1a577220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.175+0000 7fd9226cfb80 1 -- 172.21.15.19:0/3662106014 shutdown_connections 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.175+0000 7fd9226cfb80 1 -- 172.21.15.19:0/3662106014 wait complete. 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd9226cfb80 1 -- start start 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd9226cfb80 1 Processor -- start 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a585f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a586480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a5869c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556a1a57d160 con 0x556a1a3ff4a0 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556a1a439b20 con 0x556a1a479d00 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556a1a589020 con 0x556a1a57c400 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd9226cfb80 10 client.0 _ll_register_callbacks cb 0x556a1a3106c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.176+0000 7fd911a55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a585f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911254700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a586480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911a55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a585f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44538/0 (socket says 172.21.15.19:44538) 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911254700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a586480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35350/0 (socket says 172.21.15.19:35350) 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911a55700 1 -- 172.21.15.19:0/1188793254 learned_addr learned my addr 172.21.15.19:0/1188793254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd912256700 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a5869c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911a55700 1 -- 172.21.15.19:0/1188793254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 msgr2=0x556a1a5869c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911a55700 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a5869c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911a55700 1 -- 172.21.15.19:0/1188793254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 msgr2=0x556a1a586480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911a55700 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a586480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911a55700 1 -- 172.21.15.19:0/1188793254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1a31a490 con 0x556a1a3ff4a0 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911254700 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a586480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd911a55700 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a585f40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd9080116a0 tx=0x7fd9080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f06cb34921bd2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd8f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd8f57fa700 1 -- 172.21.15.19:0/1188793254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90801e830 con 0x556a1a3ff4a0 2024-05-09T01:27:55.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd8f57fa700 1 -- 172.21.15.19:0/1188793254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd90801e9d0 con 0x556a1a3ff4a0 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.177+0000 7fd9226cfb80 1 -- 172.21.15.19:0/1188793254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8fc0044d0 con 0x556a1a3ff4a0 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd8f57fa700 1 -- 172.21.15.19:0/1188793254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd908011e40 con 0x556a1a3ff4a0 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd9226cfb80 10 client.15702 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd9226cfb80 1 -- 172.21.15.19:0/1188793254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a1a593970 con 0x556a1a3ff4a0 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd8f57fa700 1 -- 172.21.15.19:0/1188793254 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8fc0044d0 con 0x556a1a3ff4a0 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd8f57fa700 1 -- 172.21.15.19:0/1188793254 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd908026020 con 0x556a1a3ff4a0 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd9226cfb80 10 client.15702 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd9226cfb80 10 client.15702 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd9226cfb80 20 client.15702 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556a1a595fb0 0x556a1a598410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd9226cfb80 4 client.15702 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.178+0000 7fd9226cfb80 1 -- 172.21.15.19:0/1188793254 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a1a4aca80 con 0x556a1a595fb0 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.179+0000 7fd911254700 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556a1a595fb0 0x556a1a598410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.179+0000 7fd911254700 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556a1a595fb0 0x556a1a598410 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7ddb340f8d3d3c3 server_cookie=4adca09f89f95ffe in_seq=0 out_seq=0 2024-05-09T01:27:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.179+0000 7fd8f57fa700 10 client.15702 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:27:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd8f57fa700 1 -- 172.21.15.19:0/1188793254 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a1a4aca80 con 0x556a1a595fb0 2024-05-09T01:27:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd8f57fa700 10 client.15702 handle_command_reply: tid=0 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 client.15702 shutdown 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 -- 172.21.15.19:0/1188793254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 msgr2=0x556a1a585f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a585f40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd9080116a0 tx=0x7fd9080047c0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 -- 172.21.15.19:0/1188793254 shutdown_connections 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a5869c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a585f40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556a1a595fb0 0x556a1a598410 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/1188793254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a586480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd8f57fa700 0 client.15702 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 -- 172.21.15.19:0/1188793254 >> 172.21.15.19:0/1188793254 conn(0x556a1a402f70 msgr2=0x556a1a570ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 -- 172.21.15.19:0/1188793254 shutdown_connections 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.180+0000 7fd9226cfb80 1 -- 172.21.15.19:0/1188793254 wait complete. 2024-05-09T01:27:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.181+0000 7fd9226cfb80 20 client.15702 trim_cache size 0 max 16384 2024-05-09T01:27:55.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.196+0000 7fd9226cfb80 1 Processor -- start 2024-05-09T01:27:55.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.196+0000 7fd9226cfb80 1 -- start start 2024-05-09T01:27:55.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.196+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a581410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.196+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a5772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.196+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a571700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.196+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1aca00 con 0x556a1a3ff4a0 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.196+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1ae540 con 0x556a1a479d00 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.196+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1ad0d0 con 0x556a1a57c400 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911a55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a581410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911a55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a581410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44552/0 (socket says 172.21.15.19:44552) 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911a55700 1 -- 172.21.15.19:0/955922300 learned_addr learned my addr 172.21.15.19:0/955922300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911254700 1 --2- 172.21.15.19:0/955922300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a5772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd912256700 1 --2- 172.21.15.19:0/955922300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a571700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911a55700 1 -- 172.21.15.19:0/955922300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 msgr2=0x556a1a571700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911a55700 1 --2- 172.21.15.19:0/955922300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a571700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911a55700 1 -- 172.21.15.19:0/955922300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 msgr2=0x556a1a5772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:55.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911a55700 1 --2- 172.21.15.19:0/955922300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a5772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911a55700 1 -- 172.21.15.19:0/955922300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1a5739d0 con 0x556a1a3ff4a0 2024-05-09T01:27:55.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd911a55700 1 --2- 172.21.15.19:0/955922300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a581410 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd908005d90 tx=0x7fd908004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd404d54dc2c4d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:55.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.197+0000 7fd8f77fe700 1 -- 172.21.15.19:0/955922300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90801e830 con 0x556a1a3ff4a0 2024-05-09T01:27:55.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.198+0000 7fd8f77fe700 1 -- 172.21.15.19:0/955922300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd908037070 con 0x556a1a3ff4a0 2024-05-09T01:27:55.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.198+0000 7fd8f77fe700 1 -- 172.21.15.19:0/955922300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90803c470 con 0x556a1a3ff4a0 2024-05-09T01:27:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 -- 172.21.15.19:0/955922300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 msgr2=0x556a1a581410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/955922300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a581410 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd908005d90 tx=0x7fd908004710 comp rx=0 tx=0).stop 2024-05-09T01:27:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 -- 172.21.15.19:0/955922300 shutdown_connections 2024-05-09T01:27:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/955922300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a571700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/955922300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a581410 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/955922300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a5772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 -- 172.21.15.19:0/955922300 >> 172.21.15.19:0/955922300 conn(0x556a1a402f70 msgr2=0x556a1a570ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 -- 172.21.15.19:0/955922300 shutdown_connections 2024-05-09T01:27:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 -- 172.21.15.19:0/955922300 wait complete. 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 -- start start 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.199+0000 7fd9226cfb80 1 Processor -- start 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a58e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a58e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd9226cfb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a59cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1ade70 con 0x556a1a3ff4a0 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1ad7a0 con 0x556a1a479d00 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd9226cfb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1aa9a0 con 0x556a1a57c400 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd9226cfb80 10 client.0 _ll_register_callbacks cb 0x556a1a3106c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd911a55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a58e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd911a55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a58e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44560/0 (socket says 172.21.15.19:44560) 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd911a55700 1 -- 172.21.15.19:0/4190353240 learned_addr learned my addr 172.21.15.19:0/4190353240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd912256700 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a59cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.200+0000 7fd911254700 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a58e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd911a55700 1 -- 172.21.15.19:0/4190353240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 msgr2=0x556a1a59cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd911a55700 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a59cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd911a55700 1 -- 172.21.15.19:0/4190353240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 msgr2=0x556a1a58e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd911a55700 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a58e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd911a55700 1 -- 172.21.15.19:0/4190353240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1a59d1d0 con 0x556a1a3ff4a0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd911a55700 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a58e1a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd9080116a0 tx=0x7fd9080057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a0395389a9643 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd8f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd8f57fa700 1 -- 172.21.15.19:0/4190353240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908004e40 con 0x556a1a3ff4a0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd8f57fa700 1 -- 172.21.15.19:0/4190353240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd90801e630 con 0x556a1a3ff4a0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd9226cfb80 1 -- 172.21.15.19:0/4190353240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd908037070 con 0x556a1a3ff4a0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd8f57fa700 1 -- 172.21.15.19:0/4190353240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd90801e870 con 0x556a1a3ff4a0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd9226cfb80 10 client.15708 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.201+0000 7fd9226cfb80 1 -- 172.21.15.19:0/4190353240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a1a5fffe0 con 0x556a1a3ff4a0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.202+0000 7fd8f57fa700 1 -- 172.21.15.19:0/4190353240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd908037070 con 0x556a1a3ff4a0 2024-05-09T01:27:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.202+0000 7fd8f57fa700 1 -- 172.21.15.19:0/4190353240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd908043020 con 0x556a1a3ff4a0 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.202+0000 7fd9226cfb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.202+0000 7fd9226cfb80 10 client.15708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.202+0000 7fd9226cfb80 20 client.15708 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.202+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556a1a600550 0x556a1a602970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.202+0000 7fd9226cfb80 4 client.15708 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.202+0000 7fd9226cfb80 1 -- 172.21.15.19:0/4190353240 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a1a4a6b20 con 0x556a1a600550 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.202+0000 7fd911254700 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556a1a600550 0x556a1a602970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd8f57fa700 10 client.15708 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd911254700 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556a1a600550 0x556a1a602970 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd8f1afca63065d4 server_cookie=8cf40fdfa2ae513b in_seq=0 out_seq=0 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd8f57fa700 1 -- 172.21.15.19:0/4190353240 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a1a4a6b20 con 0x556a1a600550 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd8f57fa700 10 client.15708 handle_command_reply: tid=0 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 client.15708 shutdown 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 -- 172.21.15.19:0/4190353240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 msgr2=0x556a1a58e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a58e1a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd9080116a0 tx=0x7fd9080057e0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 -- 172.21.15.19:0/4190353240 shutdown_connections 2024-05-09T01:27:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a1a57c400 0x556a1a59cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a1a3ff4a0 0x556a1a58e1a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556a1a600550 0x556a1a602970 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 --2- 172.21.15.19:0/4190353240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1a479d00 0x556a1a58e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd8f57fa700 0 client.15708 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:27:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 -- 172.21.15.19:0/4190353240 >> 172.21.15.19:0/4190353240 conn(0x556a1a402f70 msgr2=0x556a1a580460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 -- 172.21.15.19:0/4190353240 shutdown_connections 2024-05-09T01:27:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 1 -- 172.21.15.19:0/4190353240 wait complete. 2024-05-09T01:27:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.203+0000 7fd9226cfb80 20 client.15708 trim_cache size 0 max 16384 2024-05-09T01:27:55.207 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:27:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 -- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8f80b2880 msgr2=0x7fd8f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8f80b2880 0x7fd8f80b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd908005950 tx=0x7fd90800b410 comp rx=0 tx=0).stop 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 -- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 msgr2=0x7fd90c19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 0x7fd90c19f570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd9040095c0 tx=0x7fd904007a20 comp rx=0 tx=0).stop 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 -- 172.21.15.19:0/3140775677 shutdown_connections 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8f80b2880 0x7fd8f80b4d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90c108740 0x7fd90c19f570 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd90c1034c0 0x7fd90c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 --2- 172.21.15.19:0/3140775677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd90c107df0 0x7fd90c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 -- 172.21.15.19:0/3140775677 >> 172.21.15.19:0/3140775677 conn(0x7fd90c0fedc0 msgr2=0x7fd90c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 -- 172.21.15.19:0/3140775677 shutdown_connections 2024-05-09T01:27:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:55.204+0000 7fd913cb9700 1 -- 172.21.15.19:0/3140775677 wait complete. 2024-05-09T01:27:55.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:27:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:27:56.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:56 smithi019 ceph-mon[30158]: pgmap v450: 97 pgs: 97 active+clean; 806 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:27:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:56 smithi059 ceph-mon[34351]: pgmap v450: 97 pgs: 97 active+clean; 806 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:27:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:56 smithi196 ceph-mon[35041]: pgmap v450: 97 pgs: 97 active+clean; 806 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:27:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.968+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2452116564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c1034c0 msgr2=0x7f3a4c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.968+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2452116564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c1034c0 0x7f3a4c1038d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0099c0 tx=0x7f3a3c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.968+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2452116564 shutdown_connections 2024-05-09T01:27:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.968+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2452116564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a4c108740 0x7f3a4c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.968+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2452116564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c1034c0 0x7f3a4c1038d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.968+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2452116564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a4c107df0 0x7f3a4c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.968+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2452116564 >> 172.21.15.19:0/2452116564 conn(0x7f3a4c0fedc0 msgr2=0x7f3a4c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.968+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2452116564 shutdown_connections 2024-05-09T01:27:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.968+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2452116564 wait complete. 2024-05-09T01:27:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.969+0000 7f3a51bc0700 1 Processor -- start 2024-05-09T01:27:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a51bc0700 1 -- start start 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a51bc0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a4c1034c0 0x7f3a4c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a51bc0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a4c107df0 0x7f3a4c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a51bc0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108740 0x7f3a4c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a51bc0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c113370 con 0x7f3a4c108740 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a51bc0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1131f0 con 0x7f3a4c1034c0 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a51bc0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1134f0 con 0x7f3a4c107df0 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108740 0x7f3a4c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a4c107df0 0x7f3a4c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108740 0x7f3a4c19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44576/0 (socket says 172.21.15.19:44576) 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a4bfff700 1 -- 172.21.15.19:0/2183129577 learned_addr learned my addr 172.21.15.19:0/2183129577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.970+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a4c1034c0 0x7f3a4c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a4bfff700 1 -- 172.21.15.19:0/2183129577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a4c107df0 msgr2=0x7f3a4c19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a4bfff700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a4c107df0 0x7f3a4c19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a4bfff700 1 -- 172.21.15.19:0/2183129577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a4c1034c0 msgr2=0x7f3a4c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a4bfff700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a4c1034c0 0x7f3a4c19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a4bfff700 1 -- 172.21.15.19:0/2183129577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c1a6fc0 con 0x7f3a4c108740 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a4c1034c0 0x7f3a4c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a4affd700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a4c107df0 0x7f3a4c19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a4bfff700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108740 0x7f3a4c19c710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3a44007ee0 tx=0x7f3a4400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a48ff9700 1 -- 172.21.15.19:0/2183129577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44004bb0 con 0x7f3a4c108740 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2183129577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c1a72b0 con 0x7f3a4c108740 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a48ff9700 1 -- 172.21.15.19:0/2183129577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a44004d50 con 0x7f3a4c108740 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.971+0000 7f3a48ff9700 1 -- 172.21.15.19:0/2183129577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44005920 con 0x7f3a4c108740 2024-05-09T01:27:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.972+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2183129577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c1a77a0 con 0x7f3a4c108740 2024-05-09T01:27:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.974+0000 7f3a48ff9700 1 -- 172.21.15.19:0/2183129577 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3a4401a020 con 0x7f3a4c108740 2024-05-09T01:27:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.975+0000 7f3a48ff9700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a340b2880 0x7f3a340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.975+0000 7f3a48ff9700 1 -- 172.21.15.19:0/2183129577 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3a440e18b0 con 0x7f3a4c108740 2024-05-09T01:27:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.975+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a340b2880 0x7f3a340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.975+0000 7f3a605d6b80 1 Processor -- start 2024-05-09T01:27:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.975+0000 7f3a605d6b80 1 -- start start 2024-05-09T01:27:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b02ed00 0x55bb4b135ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b136030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 0x55bb4b13a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b0540a0 con 0x55bb4afb44a0 2024-05-09T01:27:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bb4aff99a0 con 0x55bb4b02ed00 2024-05-09T01:27:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b050060 con 0x55bb4b13a1e0 2024-05-09T01:27:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a340b2880 0x7f3a340b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c005b40 tx=0x7f3a3c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b136030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b136030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44590/0 (socket says 172.21.15.19:44590) 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4affd700 1 -- 172.21.15.19:0/4198641175 learned_addr learned my addr 172.21.15.19:0/4198641175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4bfff700 1 --2- 172.21.15.19:0/4198641175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 0x55bb4b13a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/4198641175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b02ed00 0x55bb4b135ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4affd700 1 -- 172.21.15.19:0/4198641175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 msgr2=0x55bb4b13a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4affd700 1 --2- 172.21.15.19:0/4198641175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 0x55bb4b13a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4affd700 1 -- 172.21.15.19:0/4198641175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b02ed00 msgr2=0x55bb4b135ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4affd700 1 --2- 172.21.15.19:0/4198641175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b02ed00 0x55bb4b135ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4affd700 1 -- 172.21.15.19:0/4198641175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4afea500 con 0x55bb4afb44a0 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.976+0000 7f3a4affd700 1 --2- 172.21.15.19:0/4198641175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b136030 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a40011540 tx=0x7f3a40011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86903656b1804e13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.977+0000 7f3a31ffb700 1 -- 172.21.15.19:0/4198641175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40004a30 con 0x55bb4afb44a0 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.977+0000 7f3a31ffb700 1 -- 172.21.15.19:0/4198641175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a400054d0 con 0x55bb4afb44a0 2024-05-09T01:27:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.977+0000 7f3a31ffb700 1 -- 172.21.15.19:0/4198641175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000e8a0 con 0x55bb4afb44a0 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.977+0000 7f3a605d6b80 1 -- 172.21.15.19:0/4198641175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 msgr2=0x55bb4b136030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/4198641175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b136030 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a40011540 tx=0x7f3a40011850 comp rx=0 tx=0).stop 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 -- 172.21.15.19:0/4198641175 shutdown_connections 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/4198641175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 0x55bb4b13a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/4198641175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b02ed00 0x55bb4b135ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/4198641175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b136030 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 -- 172.21.15.19:0/4198641175 >> 172.21.15.19:0/4198641175 conn(0x55bb4afb7f70 msgr2=0x55bb4b12c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 -- 172.21.15.19:0/4198641175 shutdown_connections 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 -- 172.21.15.19:0/4198641175 wait complete. 2024-05-09T01:27:57.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 -- start start 2024-05-09T01:27:57.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.978+0000 7f3a605d6b80 1 Processor -- start 2024-05-09T01:27:57.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4afb44a0 0x55bb4b13f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b13f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 0x55bb4b13fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b13af00 con 0x55bb4b02ed00 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bb4afeeb20 con 0x55bb4afb44a0 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b142470 con 0x55bb4b13a1e0 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b13f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a605d6b80 10 client.0 _ll_register_callbacks cb 0x55bb4aec56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b13f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44602/0 (socket says 172.21.15.19:44602) 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4affd700 1 -- 172.21.15.19:0/1658641884 learned_addr learned my addr 172.21.15.19:0/1658641884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4bfff700 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 0x55bb4b13fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4afb44a0 0x55bb4b13f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4affd700 1 -- 172.21.15.19:0/1658641884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 msgr2=0x55bb4b13fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4affd700 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 0x55bb4b13fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4affd700 1 -- 172.21.15.19:0/1658641884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4afb44a0 msgr2=0x55bb4b13f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4affd700 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4afb44a0 0x55bb4b13f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4affd700 1 -- 172.21.15.19:0/1658641884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4aecf490 con 0x55bb4b02ed00 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4afb44a0 0x55bb4b13f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a4affd700 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b13f880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a4001b040 tx=0x7f3a4000eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa840e0bfd8535a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.979+0000 7f3a2affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a2affd700 1 -- 172.21.15.19:0/1658641884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40028070 con 0x55bb4b02ed00 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a2affd700 1 -- 172.21.15.19:0/1658641884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3a4000ee00 con 0x55bb4b02ed00 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1658641884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a400054d0 con 0x55bb4b02ed00 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a2affd700 1 -- 172.21.15.19:0/1658641884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a40033050 con 0x55bb4b02ed00 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a605d6b80 10 client.15720 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1658641884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb4b14cd10 con 0x55bb4b02ed00 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a2affd700 1 -- 172.21.15.19:0/1658641884 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a400054d0 con 0x55bb4b02ed00 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a2affd700 1 -- 172.21.15.19:0/1658641884 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3a40019790 con 0x55bb4b02ed00 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a605d6b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a605d6b80 10 client.15720 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:27:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a605d6b80 20 client.15720 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55bb4b14f240 0x55bb4b151660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a605d6b80 4 client.15720 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.980+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1658641884 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb4b061a80 con 0x55bb4b14f240 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.981+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55bb4b14f240 0x55bb4b151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.981+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55bb4b14f240 0x55bb4b151660 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e72c09ab5bb10bb0 server_cookie=5d3c2b27bd7b57cc in_seq=0 out_seq=0 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.981+0000 7f3a2affd700 10 client.15720 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a2affd700 1 -- 172.21.15.19:0/1658641884 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb4b061a80 con 0x55bb4b14f240 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a2affd700 10 client.15720 handle_command_reply: tid=0 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 client.15720 shutdown 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1658641884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 msgr2=0x55bb4b13f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b13f880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a4001b040 tx=0x7f3a4000eb60 comp rx=0 tx=0).stop 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1658641884 shutdown_connections 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55bb4b14f240 0x55bb4b151660 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b13a1e0 0x55bb4b13fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4afb44a0 0x55bb4b13f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a2affd700 0 client.15720 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:27:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1658641884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b13f880 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1658641884 >> 172.21.15.19:0/1658641884 conn(0x55bb4afb7f70 msgr2=0x55bb4b125ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1658641884 shutdown_connections 2024-05-09T01:27:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1658641884 wait complete. 2024-05-09T01:27:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.982+0000 7f3a605d6b80 20 client.15720 trim_cache size 0 max 16384 2024-05-09T01:27:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a605d6b80 1 Processor -- start 2024-05-09T01:27:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a605d6b80 1 -- start start 2024-05-09T01:27:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4afb44a0 0x55bb4b126a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b14b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 0x55bb4b156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1acb60 con 0x55bb4b02ed00 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1ae6a0 con 0x55bb4b13a1e0 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1ad230 con 0x55bb4afb44a0 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b14b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b14b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44616/0 (socket says 172.21.15.19:44616) 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a4affd700 1 -- 172.21.15.19:0/1309844147 learned_addr learned my addr 172.21.15.19:0/1309844147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.998+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/1309844147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4afb44a0 0x55bb4b126a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a4bfff700 1 --2- 172.21.15.19:0/1309844147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 0x55bb4b156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a4affd700 1 -- 172.21.15.19:0/1309844147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4afb44a0 msgr2=0x55bb4b126a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a4affd700 1 --2- 172.21.15.19:0/1309844147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4afb44a0 0x55bb4b126a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a4affd700 1 -- 172.21.15.19:0/1309844147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 msgr2=0x55bb4b156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a4affd700 1 --2- 172.21.15.19:0/1309844147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 0x55bb4b156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a4affd700 1 -- 172.21.15.19:0/1309844147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4b1289d0 con 0x55bb4b02ed00 2024-05-09T01:27:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a4affd700 1 --2- 172.21.15.19:0/1309844147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b14b220 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3a40000c00 tx=0x7f3a4000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2905a27c3dc71938 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a317fa700 1 -- 172.21.15.19:0/1309844147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40030070 con 0x55bb4b02ed00 2024-05-09T01:27:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a317fa700 1 -- 172.21.15.19:0/1309844147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a400198e0 con 0x55bb4b02ed00 2024-05-09T01:27:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:57.999+0000 7f3a317fa700 1 -- 172.21.15.19:0/1309844147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40028070 con 0x55bb4b02ed00 2024-05-09T01:27:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1309844147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 msgr2=0x55bb4b14b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1309844147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b14b220 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3a40000c00 tx=0x7f3a4000ee90 comp rx=0 tx=0).stop 2024-05-09T01:27:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1309844147 shutdown_connections 2024-05-09T01:27:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1309844147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 0x55bb4b156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1309844147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4afb44a0 0x55bb4b126a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/1309844147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4b02ed00 0x55bb4b14b220 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1309844147 >> 172.21.15.19:0/1309844147 conn(0x55bb4afb7f70 msgr2=0x55bb4b139750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1309844147 shutdown_connections 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 -- 172.21.15.19:0/1309844147 wait complete. 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 -- start start 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.000+0000 7f3a605d6b80 1 Processor -- start 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b147b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b02ed00 0x55bb4b144f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a605d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 0x55bb4b145440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1adfd0 con 0x55bb4afb44a0 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1ad900 con 0x55bb4b13a1e0 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a605d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1aab00 con 0x55bb4b02ed00 2024-05-09T01:27:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a605d6b80 10 client.0 _ll_register_callbacks cb 0x55bb4aec56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b147b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44630/0 (socket says 172.21.15.19:44630) 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2449718257 learned_addr learned my addr 172.21.15.19:0/2449718257 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4affd700 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b02ed00 0x55bb4b144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4bfff700 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 0x55bb4b145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2449718257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b02ed00 msgr2=0x55bb4b144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b02ed00 0x55bb4b144f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2449718257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 msgr2=0x55bb4b145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 0x55bb4b145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2449718257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4b145be0 con 0x55bb4afb44a0 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4bfff700 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 0x55bb4b145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:27:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b147b30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0099c0 tx=0x7f3a3c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5568e230dc11d35f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.001+0000 7f3a2affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a2affd700 1 -- 172.21.15.19:0/2449718257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c036060 con 0x55bb4afb44a0 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a2affd700 1 -- 172.21.15.19:0/2449718257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3a3c031850 con 0x55bb4afb44a0 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a605d6b80 1 -- 172.21.15.19:0/2449718257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a400198e0 con 0x55bb4afb44a0 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a2affd700 1 -- 172.21.15.19:0/2449718257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a3c031a50 con 0x55bb4afb44a0 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a605d6b80 10 client.15726 fetch_fsmap learned FSMap version 28 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a605d6b80 1 -- 172.21.15.19:0/2449718257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb4b1b97b0 con 0x55bb4afb44a0 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a2affd700 1 -- 172.21.15.19:0/2449718257 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a400198e0 con 0x55bb4afb44a0 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a2affd700 1 -- 172.21.15.19:0/2449718257 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3a3c00f800 con 0x55bb4afb44a0 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a605d6b80 10 client.15726 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:27:58.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a605d6b80 10 client.15726 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a605d6b80 20 client.15726 populate_metadata read hostname 'smithi019' 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55bb4b1b9e70 0x55bb4b1bc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a605d6b80 4 client.15726 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.002+0000 7f3a605d6b80 1 -- 172.21.15.19:0/2449718257 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb4b05bb20 con 0x55bb4b1b9e70 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.003+0000 7f3a4affd700 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55bb4b1b9e70 0x55bb4b1bc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.003+0000 7f3a2affd700 10 client.15726 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.003+0000 7f3a4affd700 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55bb4b1b9e70 0x55bb4b1bc290 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59f7cd77cfc72f92 server_cookie=d679b8dfc8cd7d3d in_seq=0 out_seq=0 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.003+0000 7f3a2affd700 1 -- 172.21.15.19:0/2449718257 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb4b05bb20 con 0x55bb4b1b9e70 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.003+0000 7f3a2affd700 10 client.15726 handle_command_reply: tid=0 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.003+0000 7f3a605d6b80 1 client.15726 shutdown 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 -- 172.21.15.19:0/2449718257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 msgr2=0x55bb4b147b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:58.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b147b30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0099c0 tx=0x7f3a3c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 -- 172.21.15.19:0/2449718257 shutdown_connections 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55bb4b1b9e70 0x55bb4b1bc290 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb4b13a1e0 0x55bb4b145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb4afb44a0 0x55bb4b147b30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 --2- 172.21.15.19:0/2449718257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b02ed00 0x55bb4b144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a2affd700 0 client.15726 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 -- 172.21.15.19:0/2449718257 >> 172.21.15.19:0/2449718257 conn(0x55bb4afb7f70 msgr2=0x55bb4b1303f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 -- 172.21.15.19:0/2449718257 shutdown_connections 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 1 -- 172.21.15.19:0/2449718257 wait complete. 2024-05-09T01:27:58.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a605d6b80 20 client.15726 trim_cache size 0 max 16384 2024-05-09T01:27:58.008 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:27:58.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a340b2880 msgr2=0x7f3a340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a340b2880 0x7f3a340b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c005b40 tx=0x7f3a3c01a040 comp rx=0 tx=0).stop 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108740 msgr2=0x7f3a4c19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108740 0x7f3a4c19c710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3a44007ee0 tx=0x7f3a4400d3b0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2183129577 shutdown_connections 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3a340b2880 0x7f3a340b4d90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a4c108740 0x7f3a4c19c710 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a4c1034c0 0x7f3a4c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.004+0000 7f3a51bc0700 1 --2- 172.21.15.19:0/2183129577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a4c107df0 0x7f3a4c19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.005+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2183129577 >> 172.21.15.19:0/2183129577 conn(0x7f3a4c0fedc0 msgr2=0x7f3a4c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.005+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2183129577 shutdown_connections 2024-05-09T01:27:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:27:58.005+0000 7f3a51bc0700 1 -- 172.21.15.19:0/2183129577 wait complete. 2024-05-09T01:27:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:27:58 smithi019 ceph-mon[30158]: pgmap v451: 97 pgs: 97 active+clean; 875 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-05-09T01:27:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:27:58 smithi059 ceph-mon[34351]: pgmap v451: 97 pgs: 97 active+clean; 875 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-05-09T01:27:58.923 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:27:58.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:27:58 smithi196 ceph-mon[35041]: pgmap v451: 97 pgs: 97 active+clean; 875 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-05-09T01:27:59.178 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:00.749 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:00 smithi019 ceph-mon[30158]: pgmap v452: 97 pgs: 97 active+clean; 875 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:00 smithi059 ceph-mon[34351]: pgmap v452: 97 pgs: 97 active+clean; 875 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:00.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:00 smithi196 ceph-mon[35041]: pgmap v452: 97 pgs: 97 active+clean; 875 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.993+0000 7f08c94f7700 1 -- 172.21.15.19:0/935249059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 msgr2=0x7f08c4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.993+0000 7f08c94f7700 1 --2- 172.21.15.19:0/935249059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 0x7f08c4109330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f08b80099c0 tx=0x7f08b8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.993+0000 7f08c94f7700 1 -- 172.21.15.19:0/935249059 shutdown_connections 2024-05-09T01:28:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.993+0000 7f08c94f7700 1 --2- 172.21.15.19:0/935249059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08c4109870 0x7f08c41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.993+0000 7f08c94f7700 1 --2- 172.21.15.19:0/935249059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08c41045f0 0x7f08c4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.993+0000 7f08c94f7700 1 --2- 172.21.15.19:0/935249059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 0x7f08c4109330 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:00.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.993+0000 7f08c94f7700 1 -- 172.21.15.19:0/935249059 >> 172.21.15.19:0/935249059 conn(0x7f08c4100040 msgr2=0x7f08c4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.994+0000 7f08c94f7700 1 -- 172.21.15.19:0/935249059 shutdown_connections 2024-05-09T01:28:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.994+0000 7f08c94f7700 1 -- 172.21.15.19:0/935249059 wait complete. 2024-05-09T01:28:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.994+0000 7f08c94f7700 1 Processor -- start 2024-05-09T01:28:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.994+0000 7f08c94f7700 1 -- start start 2024-05-09T01:28:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c94f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08c41045f0 0x7f08c419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:00.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c94f7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 0x7f08c419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c94f7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08c4109870 0x7f08c419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c94f7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41144f0 con 0x7f08c4108f20 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c94f7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4114370 con 0x7f08c41045f0 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c94f7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4114670 con 0x7f08c4109870 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 0x7f08c419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 0x7f08c419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42084/0 (socket says 172.21.15.19:42084) 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c27fc700 1 -- 172.21.15.19:0/4817706 learned_addr learned my addr 172.21.15.19:0/4817706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08c41045f0 0x7f08c419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c37fe700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08c4109870 0x7f08c419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c27fc700 1 -- 172.21.15.19:0/4817706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08c4109870 msgr2=0x7f08c419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c27fc700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08c4109870 0x7f08c419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c27fc700 1 -- 172.21.15.19:0/4817706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08c41045f0 msgr2=0x7f08c419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:00.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c27fc700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08c41045f0 0x7f08c419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c27fc700 1 -- 172.21.15.19:0/4817706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c419fd70 con 0x7f08c4108f20 2024-05-09T01:28:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08c41045f0 0x7f08c419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.995+0000 7f08c27fc700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 0x7f08c419b440 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009f50 tx=0x7f08b80037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.996+0000 7f08abfff700 1 -- 172.21.15.19:0/4817706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8027070 con 0x7f08c4108f20 2024-05-09T01:28:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.996+0000 7f08c94f7700 1 -- 172.21.15.19:0/4817706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c41a0000 con 0x7f08c4108f20 2024-05-09T01:28:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.996+0000 7f08abfff700 1 -- 172.21.15.19:0/4817706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b8004140 con 0x7f08c4108f20 2024-05-09T01:28:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.996+0000 7f08abfff700 1 -- 172.21.15.19:0/4817706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b800f460 con 0x7f08c4108f20 2024-05-09T01:28:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.996+0000 7f08c94f7700 1 -- 172.21.15.19:0/4817706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c41a04e0 con 0x7f08c4108f20 2024-05-09T01:28:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.998+0000 7f08abfff700 1 -- 172.21.15.19:0/4817706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f08b802a070 con 0x7f08c4108f20 2024-05-09T01:28:00.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.999+0000 7f08abfff700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f08ac0b2880 0x7f08ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:00.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.999+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f08ac0b2880 0x7f08ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:00.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.999+0000 7f08abfff700 1 -- 172.21.15.19:0/4817706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f08b80e3230 con 0x7f08c4108f20 2024-05-09T01:28:00.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.999+0000 7f08d7f0db80 1 Processor -- start 2024-05-09T01:28:00.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:00.999+0000 7f08d7f0db80 1 -- start start 2024-05-09T01:28:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29cc3160 0x55fa29d5c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f08ac0b2880 0x7f08ac0b4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f08b4007fd0 tx=0x7f08b4007830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d5ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d56ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fa29c78ff0 con 0x55fa29bd94a0 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa29c177f0 con 0x55fa29cc3160 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa29c74fb0 con 0x55fa29d566a0 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d5ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d5ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42092/0 (socket says 172.21.15.19:42092) 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c27fc700 1 -- 172.21.15.19:0/3777758811 learned_addr learned my addr 172.21.15.19:0/3777758811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c37fe700 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d56ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29cc3160 0x55fa29d5c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c27fc700 1 -- 172.21.15.19:0/3777758811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 msgr2=0x55fa29d56ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c27fc700 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d56ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c27fc700 1 -- 172.21.15.19:0/3777758811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29cc3160 msgr2=0x55fa29d5c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c27fc700 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29cc3160 0x55fa29d5c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c27fc700 1 -- 172.21.15.19:0/3777758811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa29c0f370 con 0x55fa29bd94a0 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.000+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29cc3160 0x55fa29d5c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.001+0000 7f08c27fc700 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d5ca30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f08b80a6a20 tx=0x7f08b8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1d6e26efe4e95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.001+0000 7f08a97fa700 1 -- 172.21.15.19:0/3777758811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80aa4b0 con 0x55fa29bd94a0 2024-05-09T01:28:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.001+0000 7f08a97fa700 1 -- 172.21.15.19:0/3777758811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b80aec70 con 0x55fa29bd94a0 2024-05-09T01:28:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.001+0000 7f08a97fa700 1 -- 172.21.15.19:0/3777758811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80b3200 con 0x55fa29bd94a0 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3777758811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 msgr2=0x55fa29d5ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d5ca30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f08b80a6a20 tx=0x7f08b8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3777758811 shutdown_connections 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d56ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29cc3160 0x55fa29d5c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3777758811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d5ca30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3777758811 >> 172.21.15.19:0/3777758811 conn(0x55fa29c396a0 msgr2=0x55fa29d51400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3777758811 shutdown_connections 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3777758811 wait complete. 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 -- start start 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.002+0000 7f08d7f0db80 1 Processor -- start 2024-05-09T01:28:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d60160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29cc3160 0x55fa29d606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29d566a0 0x55fa29d60be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fa29af4ba0 con 0x55fa29bd94a0 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa29c33150 con 0x55fa29d566a0 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa29d57360 con 0x55fa29cc3160 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08d7f0db80 10 client.0 _ll_register_callbacks cb 0x55fa29aea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d60160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d60160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42108/0 (socket says 172.21.15.19:42108) 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c2ffd700 1 -- 172.21.15.19:0/3303601545 learned_addr learned my addr 172.21.15.19:0/3303601545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c27fc700 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29cc3160 0x55fa29d606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c37fe700 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29d566a0 0x55fa29d60be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c2ffd700 1 -- 172.21.15.19:0/3303601545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29cc3160 msgr2=0x55fa29d606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29cc3160 0x55fa29d606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c2ffd700 1 -- 172.21.15.19:0/3303601545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29d566a0 msgr2=0x55fa29d60be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29d566a0 0x55fa29d60be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c2ffd700 1 -- 172.21.15.19:0/3303601545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa29af4490 con 0x55fa29bd94a0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c37fe700 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29d566a0 0x55fa29d60be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d60160 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f08b4016f50 tx=0x7f08b4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2593f24591f043f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.003+0000 7f08a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.004+0000 7f08a6ff5700 1 -- 172.21.15.19:0/3303601545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4004db0 con 0x55fa29bd94a0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.004+0000 7f08a6ff5700 1 -- 172.21.15.19:0/3303601545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f08b4015a70 con 0x55fa29bd94a0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.004+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3303601545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b80aec70 con 0x55fa29bd94a0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.004+0000 7f08a6ff5700 1 -- 172.21.15.19:0/3303601545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08b4015c70 con 0x55fa29bd94a0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.004+0000 7f08d7f0db80 10 client.15738 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.004+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3303601545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa29d71df0 con 0x55fa29bd94a0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.004+0000 7f08a6ff5700 1 -- 172.21.15.19:0/3303601545 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b80aec70 con 0x55fa29bd94a0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.004+0000 7f08a6ff5700 1 -- 172.21.15.19:0/3303601545 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f08b4021070 con 0x55fa29bd94a0 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.005+0000 7f08d7f0db80 10 client.15738 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.005+0000 7f08d7f0db80 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.005+0000 7f08d7f0db80 20 client.15738 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.005+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fa29d74480 0x55fa29d768a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.005+0000 7f08d7f0db80 4 client.15738 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.005+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3303601545 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa29c869d0 con 0x55fa29d74480 2024-05-09T01:28:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.005+0000 7f08c27fc700 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fa29d74480 0x55fa29d768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.006+0000 7f08c27fc700 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fa29d74480 0x55fa29d768a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7200aa2d4f48f2 server_cookie=336fd21416d3add7 in_seq=0 out_seq=0 2024-05-09T01:28:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.006+0000 7f08a6ff5700 10 client.15738 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.006+0000 7f08a6ff5700 1 -- 172.21.15.19:0/3303601545 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa29c869d0 con 0x55fa29d74480 2024-05-09T01:28:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.006+0000 7f08a6ff5700 10 client.15738 handle_command_reply: tid=0 2024-05-09T01:28:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.006+0000 7f08d7f0db80 1 client.15738 shutdown 2024-05-09T01:28:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3303601545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 msgr2=0x55fa29d60160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d60160 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f08b4016f50 tx=0x7f08b4004b40 comp rx=0 tx=0).stop 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3303601545 shutdown_connections 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fa29d74480 0x55fa29d768a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29d566a0 0x55fa29d60be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29bd94a0 0x55fa29d60160 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3303601545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29cc3160 0x55fa29d606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3303601545 >> 172.21.15.19:0/3303601545 conn(0x55fa29c396a0 msgr2=0x55fa29d511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08a6ff5700 0 client.15738 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3303601545 shutdown_connections 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3303601545 wait complete. 2024-05-09T01:28:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.007+0000 7f08d7f0db80 20 client.15738 trim_cache size 0 max 16384 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.022+0000 7f08d7f0db80 1 Processor -- start 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.022+0000 7f08d7f0db80 1 -- start start 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.022+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 0x55fa29d5a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d5b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d60a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41aca70 con 0x55fa29cc3160 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41ae5b0 con 0x55fa29bd94a0 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41ad140 con 0x55fa29d566a0 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d5b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d5b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42110/0 (socket says 172.21.15.19:42110) 2024-05-09T01:28:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c27fc700 1 -- 172.21.15.19:0/3273833040 learned_addr learned my addr 172.21.15.19:0/3273833040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/3273833040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 0x55fa29d5a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c37fe700 1 --2- 172.21.15.19:0/3273833040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d60a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c27fc700 1 -- 172.21.15.19:0/3273833040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 msgr2=0x55fa29d60a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c27fc700 1 --2- 172.21.15.19:0/3273833040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d60a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c27fc700 1 -- 172.21.15.19:0/3273833040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 msgr2=0x55fa29d5a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c27fc700 1 --2- 172.21.15.19:0/3273833040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 0x55fa29d5a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c27fc700 1 -- 172.21.15.19:0/3273833040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa29d5c050 con 0x55fa29cc3160 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08c27fc700 1 --2- 172.21.15.19:0/3273833040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d5b960 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b80aa190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b50797613929b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08a8ff9700 1 -- 172.21.15.19:0/3273833040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8090070 con 0x55fa29cc3160 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08a8ff9700 1 -- 172.21.15.19:0/3273833040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b80aec70 con 0x55fa29cc3160 2024-05-09T01:28:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.023+0000 7f08a8ff9700 1 -- 172.21.15.19:0/3273833040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80c3030 con 0x55fa29cc3160 2024-05-09T01:28:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.024+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3273833040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 msgr2=0x55fa29d5b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.024+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3273833040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d5b960 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b80aa190 comp rx=0 tx=0).stop 2024-05-09T01:28:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.024+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3273833040 shutdown_connections 2024-05-09T01:28:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.024+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3273833040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d60a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.024+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3273833040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 0x55fa29d5a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.024+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/3273833040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d5b960 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.024+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3273833040 >> 172.21.15.19:0/3273833040 conn(0x55fa29c396a0 msgr2=0x55fa29d5b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.024+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3273833040 shutdown_connections 2024-05-09T01:28:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.024+0000 7f08d7f0db80 1 -- 172.21.15.19:0/3273833040 wait complete. 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08d7f0db80 1 -- start start 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08d7f0db80 1 Processor -- start 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 0x55fa29d7c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d7b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08d7f0db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d7b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41adee0 con 0x55fa29cc3160 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41ad810 con 0x55fa29bd94a0 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08d7f0db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41aaa10 con 0x55fa29d566a0 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08d7f0db80 10 client.0 _ll_register_callbacks cb 0x55fa29aea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d7b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d7b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42120/0 (socket says 172.21.15.19:42120) 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08c27fc700 1 -- 172.21.15.19:0/1530528462 learned_addr learned my addr 172.21.15.19:0/1530528462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08c37fe700 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d7b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.025+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 0x55fa29d7c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08c27fc700 1 -- 172.21.15.19:0/1530528462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 msgr2=0x55fa29d7b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08c27fc700 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d7b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08c27fc700 1 -- 172.21.15.19:0/1530528462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 msgr2=0x55fa29d7c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08c27fc700 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 0x55fa29d7c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08c27fc700 1 -- 172.21.15.19:0/1530528462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa29d7bbf0 con 0x55fa29cc3160 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08c27fc700 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d7b070 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f08b8004370 tx=0x7f08b8009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4ffa5f1c40b095 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08a6ff5700 1 -- 172.21.15.19:0/1530528462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80c3070 con 0x55fa29cc3160 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08a6ff5700 1 -- 172.21.15.19:0/1530528462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f08b80afe00 con 0x55fa29cc3160 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08d7f0db80 1 -- 172.21.15.19:0/1530528462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b80aec70 con 0x55fa29cc3160 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08a6ff5700 1 -- 172.21.15.19:0/1530528462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08b8090050 con 0x55fa29cc3160 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08d7f0db80 10 client.15744 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08d7f0db80 1 -- 172.21.15.19:0/1530528462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa29d7bbf0 con 0x55fa29cc3160 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.026+0000 7f08a6ff5700 1 -- 172.21.15.19:0/1530528462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b80aec70 con 0x55fa29cc3160 2024-05-09T01:28:01.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08a6ff5700 1 -- 172.21.15.19:0/1530528462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f08b806d070 con 0x55fa29cc3160 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08d7f0db80 10 client.15744 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08d7f0db80 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08d7f0db80 20 client.15744 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fa29d6c4b0 0x55fa29d6c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08d7f0db80 4 client.15744 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08d7f0db80 1 -- 172.21.15.19:0/1530528462 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa29c80a70 con 0x55fa29d6c4b0 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fa29d6c4b0 0x55fa29d6c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08c2ffd700 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fa29d6c4b0 0x55fa29d6c8c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3de4425a31f04072 server_cookie=736213ce8a7e23ca in_seq=0 out_seq=0 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.027+0000 7f08a6ff5700 10 client.15744 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08a6ff5700 1 -- 172.21.15.19:0/1530528462 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa29c80a70 con 0x55fa29d6c4b0 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08a6ff5700 10 client.15744 handle_command_reply: tid=0 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 client.15744 shutdown 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 -- 172.21.15.19:0/1530528462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 msgr2=0x55fa29d7b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d7b070 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f08b8004370 tx=0x7f08b8009700 comp rx=0 tx=0).stop 2024-05-09T01:28:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 -- 172.21.15.19:0/1530528462 shutdown_connections 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fa29d6c4b0 0x55fa29d6c8c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa29d566a0 0x55fa29d7b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa29bd94a0 0x55fa29d7c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 --2- 172.21.15.19:0/1530528462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa29cc3160 0x55fa29d7b070 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08a6ff5700 0 client.15744 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 -- 172.21.15.19:0/1530528462 >> 172.21.15.19:0/1530528462 conn(0x55fa29c396a0 msgr2=0x55fa29d59e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 -- 172.21.15.19:0/1530528462 shutdown_connections 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 1 -- 172.21.15.19:0/1530528462 wait complete. 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.028+0000 7f08d7f0db80 20 client.15744 trim_cache size 0 max 16384 2024-05-09T01:28:01.030 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 -- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f08ac0b2880 msgr2=0x7f08ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f08ac0b2880 0x7f08ac0b4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f08b4007fd0 tx=0x7f08b4007830 comp rx=0 tx=0).stop 2024-05-09T01:28:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 -- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 msgr2=0x7f08c419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 0x7f08c419b440 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009f50 tx=0x7f08b80037c0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 -- 172.21.15.19:0/4817706 shutdown_connections 2024-05-09T01:28:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08c4109870 0x7f08c419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08c41045f0 0x7f08c419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f08ac0b2880 0x7f08ac0b4d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 --2- 172.21.15.19:0/4817706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08c4108f20 0x7f08c419b440 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 -- 172.21.15.19:0/4817706 >> 172.21.15.19:0/4817706 conn(0x7f08c4100040 msgr2=0x7f08c410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 -- 172.21.15.19:0/4817706 shutdown_connections 2024-05-09T01:28:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:01.029+0000 7f08c94f7700 1 -- 172.21.15.19:0/4817706 wait complete. 2024-05-09T01:28:02.734 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:28:02.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:28:02.746 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:02 smithi196 ceph-mon[35041]: pgmap v453: 97 pgs: 97 active+clean; 933 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:02.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:02 smithi019 ceph-mon[30158]: pgmap v453: 97 pgs: 97 active+clean; 933 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:02 smithi059 ceph-mon[34351]: pgmap v453: 97 pgs: 97 active+clean; 933 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:04.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.734+0000 7f3de90ae700 1 -- 172.21.15.19:0/2401102228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4109880 msgr2=0x7f3de41111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:04.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.734+0000 7f3de90ae700 1 --2- 172.21.15.19:0/2401102228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4109880 0x7f3de41111e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b0d0 tx=0x7f3ddc00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.734+0000 7f3de90ae700 1 -- 172.21.15.19:0/2401102228 shutdown_connections 2024-05-09T01:28:04.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.734+0000 7f3de90ae700 1 --2- 172.21.15.19:0/2401102228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4109880 0x7f3de41111e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.734+0000 7f3de90ae700 1 --2- 172.21.15.19:0/2401102228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3de4104650 0x7f3de4104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.734+0000 7f3de90ae700 1 --2- 172.21.15.19:0/2401102228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de4108f30 0x7f3de4109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.734+0000 7f3de90ae700 1 -- 172.21.15.19:0/2401102228 >> 172.21.15.19:0/2401102228 conn(0x7f3de4079f00 msgr2=0x7f3de407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:04.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.734+0000 7f3de90ae700 1 -- 172.21.15.19:0/2401102228 shutdown_connections 2024-05-09T01:28:04.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.734+0000 7f3de90ae700 1 -- 172.21.15.19:0/2401102228 wait complete. 2024-05-09T01:28:04.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:04 smithi019 ceph-mon[30158]: pgmap v454: 97 pgs: 97 active+clean; 933 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:28:04.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.735+0000 7f3de90ae700 1 Processor -- start 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.735+0000 7f3de90ae700 1 -- start start 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de90ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de4104650 0x7f3de419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de90ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4108f30 0x7f3de419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de90ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3de4109880 0x7f3de419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de90ae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4114520 con 0x7f3de4108f30 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de90ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41143a0 con 0x7f3de4109880 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de90ae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41146a0 con 0x7f3de4104650 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4108f30 0x7f3de419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4108f30 0x7f3de419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42132/0 (socket says 172.21.15.19:42132) 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de259c700 1 -- 172.21.15.19:0/56680280 learned_addr learned my addr 172.21.15.19:0/56680280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de4104650 0x7f3de419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de359e700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3de4109880 0x7f3de419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de259c700 1 -- 172.21.15.19:0/56680280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de4104650 msgr2=0x7f3de419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de259c700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de4104650 0x7f3de419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de259c700 1 -- 172.21.15.19:0/56680280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3de4109880 msgr2=0x7f3de419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de259c700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3de4109880 0x7f3de419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de259c700 1 -- 172.21.15.19:0/56680280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de419fd60 con 0x7f3de4108f30 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.736+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de4104650 0x7f3de419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.737+0000 7f3de259c700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4108f30 0x7f3de419b430 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800b500 tx=0x7f3dd800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.737+0000 7f3dcbfff700 1 -- 172.21.15.19:0/56680280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8015020 con 0x7f3de4108f30 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.737+0000 7f3dcbfff700 1 -- 172.21.15.19:0/56680280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dd8011ce0 con 0x7f3de4108f30 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.737+0000 7f3dcbfff700 1 -- 172.21.15.19:0/56680280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8010d40 con 0x7f3de4108f30 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.737+0000 7f3de90ae700 1 -- 172.21.15.19:0/56680280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de41a0050 con 0x7f3de4108f30 2024-05-09T01:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.737+0000 7f3de90ae700 1 -- 172.21.15.19:0/56680280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de41a7970 con 0x7f3de4108f30 2024-05-09T01:28:04.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.740+0000 7f3dcbfff700 1 -- 172.21.15.19:0/56680280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3dd8018020 con 0x7f3de4108f30 2024-05-09T01:28:04.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.741+0000 7f3dcbfff700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dcc0b2900 0x7f3dcc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.741+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dcc0b2900 0x7f3dcc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.741+0000 7f3dcbfff700 1 -- 172.21.15.19:0/56680280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3dd80e1990 con 0x7f3de4108f30 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.741+0000 7f3df7ac4b80 1 Processor -- start 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.741+0000 7f3df7ac4b80 1 -- start start 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 0x557b5e278570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dcc0b2900 0x7f3dcc0b4e10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009cb0 tx=0x7f3dd4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 0x557b5e278ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e272b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557b5e195070 con 0x557b5e272720 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557b5e133870 con 0x557b5e1df1e0 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b5e191030 con 0x557b5e0f5520 2024-05-09T01:28:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e272b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e272b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42142/0 (socket says 172.21.15.19:42142) 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de359e700 1 -- 172.21.15.19:0/2822069579 learned_addr learned my addr 172.21.15.19:0/2822069579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/2822069579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 0x557b5e278570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de259c700 1 --2- 172.21.15.19:0/2822069579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 0x557b5e278ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de359e700 1 -- 172.21.15.19:0/2822069579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 msgr2=0x557b5e278ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de359e700 1 --2- 172.21.15.19:0/2822069579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 0x557b5e278ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de359e700 1 -- 172.21.15.19:0/2822069579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 msgr2=0x557b5e278570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de359e700 1 --2- 172.21.15.19:0/2822069579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 0x557b5e278570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de359e700 1 -- 172.21.15.19:0/2822069579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b5e12b3f0 con 0x557b5e272720 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.742+0000 7f3de359e700 1 --2- 172.21.15.19:0/2822069579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e272b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009cb0 tx=0x7f3ddc009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807bd0e6ffb2e71c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.743+0000 7f3dc97fa700 1 -- 172.21.15.19:0/2822069579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00bca0 con 0x557b5e272720 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.743+0000 7f3dc97fa700 1 -- 172.21.15.19:0/2822069579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc020070 con 0x557b5e272720 2024-05-09T01:28:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.743+0000 7f3dc97fa700 1 -- 172.21.15.19:0/2822069579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc005680 con 0x557b5e272720 2024-05-09T01:28:04.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.743+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2822069579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 msgr2=0x557b5e272b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:04.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.743+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2822069579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e272b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009cb0 tx=0x7f3ddc009ce0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2822069579 shutdown_connections 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2822069579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e272b50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2822069579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 0x557b5e278570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2822069579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 0x557b5e278ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2822069579 >> 172.21.15.19:0/2822069579 conn(0x557b5e155720 msgr2=0x557b5e26d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2822069579 shutdown_connections 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2822069579 wait complete. 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 -- start start 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 Processor -- start 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 0x557b5e27c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 0x557b5e27c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e27cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557b5e010c20 con 0x557b5e272720 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557b5e14f1d0 con 0x557b5e1df1e0 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.744+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b5e27f250 con 0x557b5e0f5520 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e27cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3df7ac4b80 10 client.0 _ll_register_callbacks cb 0x557b5e006740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e27cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42154/0 (socket says 172.21.15.19:42154) 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de359e700 1 -- 172.21.15.19:0/2545709223 learned_addr learned my addr 172.21.15.19:0/2545709223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 0x557b5e27c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de259c700 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 0x557b5e27c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de359e700 1 -- 172.21.15.19:0/2545709223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 msgr2=0x557b5e27c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de359e700 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 0x557b5e27c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de359e700 1 -- 172.21.15.19:0/2545709223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 msgr2=0x557b5e27c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de359e700 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 0x557b5e27c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de359e700 1 -- 172.21.15.19:0/2545709223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b5e010510 con 0x557b5e272720 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de259c700 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 0x557b5e27c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3de359e700 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e27cca0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc005bd0 tx=0x7f3ddc004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3843df4f7ee4f87e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3dc6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/2545709223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0056d0 con 0x557b5e272720 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2545709223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ddc020070 con 0x557b5e272720 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.745+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/2545709223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3ddc0075c0 con 0x557b5e272720 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/2545709223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ddc009780 con 0x557b5e272720 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3df7ac4b80 10 client.15756 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2545709223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b5e289bb0 con 0x557b5e272720 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/2545709223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc020070 con 0x557b5e272720 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/2545709223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3ddc011070 con 0x557b5e272720 2024-05-09T01:28:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3df7ac4b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3df7ac4b80 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3df7ac4b80 20 client.15756 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557b5e28c080 0x557b5e28e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3df7ac4b80 4 client.15756 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2545709223 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b5e1a2a50 con 0x557b5e28c080 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.746+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557b5e28c080 0x557b5e28e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.747+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557b5e28c080 0x557b5e28e4a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd890e49abf1731f server_cookie=1509590d5d45fda2 in_seq=0 out_seq=0 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.747+0000 7f3dc6ff5700 10 client.15756 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/2545709223 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b5e1a2a50 con 0x557b5e28c080 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3dc6ff5700 10 client.15756 handle_command_reply: tid=0 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 client.15756 shutdown 2024-05-09T01:28:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2545709223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 msgr2=0x557b5e27cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e27cca0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc005bd0 tx=0x7f3ddc004cc0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2545709223 shutdown_connections 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557b5e28c080 0x557b5e28e4a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e272720 0x557b5e27cca0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e0f5520 0x557b5e27c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/2545709223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e1df1e0 0x557b5e27c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2545709223 >> 172.21.15.19:0/2545709223 conn(0x557b5e155720 msgr2=0x557b5e26d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3dc6ff5700 0 client.15756 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2545709223 shutdown_connections 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/2545709223 wait complete. 2024-05-09T01:28:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.748+0000 7f3df7ac4b80 20 client.15756 trim_cache size 0 max 16384 2024-05-09T01:28:04.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.769+0000 7f3df7ac4b80 1 Processor -- start 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.769+0000 7f3df7ac4b80 1 -- start start 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.769+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e27e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.769+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 0x557b5e2910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.769+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 0x557b5e291600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.769+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41aecf0 con 0x557b5e0f5520 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.769+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41b0830 con 0x557b5e272720 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.769+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41af3c0 con 0x557b5e1df1e0 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.769+0000 7f3de2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e27e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e27e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42162/0 (socket says 172.21.15.19:42162) 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de2d9d700 1 -- 172.21.15.19:0/795740274 learned_addr learned my addr 172.21.15.19:0/795740274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de259c700 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 0x557b5e2910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de359e700 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 0x557b5e291600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de2d9d700 1 -- 172.21.15.19:0/795740274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 msgr2=0x557b5e2910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 0x557b5e2910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de2d9d700 1 -- 172.21.15.19:0/795740274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 msgr2=0x557b5e291600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 0x557b5e291600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de2d9d700 1 -- 172.21.15.19:0/795740274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b5e291b40 con 0x557b5e0f5520 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de259c700 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 0x557b5e2910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e27e300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4011a50 tx=0x7f3dd4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39e4be66c9cb028 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3dc8ff9700 1 -- 172.21.15.19:0/795740274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4003a60 con 0x557b5e0f5520 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3dc8ff9700 1 -- 172.21.15.19:0/795740274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dd402e070 con 0x557b5e0f5520 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.770+0000 7f3dc8ff9700 1 -- 172.21.15.19:0/795740274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40126b0 con 0x557b5e0f5520 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/795740274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 msgr2=0x557b5e27e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e27e300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4011a50 tx=0x7f3dd4014000 comp rx=0 tx=0).stop 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/795740274 shutdown_connections 2024-05-09T01:28:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 0x557b5e291600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e27e300 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/795740274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 0x557b5e2910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/795740274 >> 172.21.15.19:0/795740274 conn(0x557b5e155720 msgr2=0x557b5e2764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/795740274 shutdown_connections 2024-05-09T01:28:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/795740274 wait complete. 2024-05-09T01:28:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.771+0000 7f3df7ac4b80 1 -- start start 2024-05-09T01:28:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3df7ac4b80 1 Processor -- start 2024-05-09T01:28:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e2f4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 0x557b5e2f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3df7ac4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 0x557b5e2ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41b0160 con 0x557b5e0f5520 2024-05-09T01:28:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41afa90 con 0x557b5e272720 2024-05-09T01:28:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3df7ac4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41acc90 con 0x557b5e1df1e0 2024-05-09T01:28:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3df7ac4b80 10 client.0 _ll_register_callbacks cb 0x557b5e277f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e2f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e2f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42166/0 (socket says 172.21.15.19:42166) 2024-05-09T01:28:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de2d9d700 1 -- 172.21.15.19:0/3357119964 learned_addr learned my addr 172.21.15.19:0/3357119964 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de359e700 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 0x557b5e2ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de259c700 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 0x557b5e2f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de2d9d700 1 -- 172.21.15.19:0/3357119964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 msgr2=0x557b5e2f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 0x557b5e2f4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de2d9d700 1 -- 172.21.15.19:0/3357119964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 msgr2=0x557b5e2ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 0x557b5e2ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de2d9d700 1 -- 172.21.15.19:0/3357119964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b5e2eefe0 con 0x557b5e0f5520 2024-05-09T01:28:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.772+0000 7f3de359e700 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 0x557b5e2ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3de2d9d700 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e2f4570 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4029b30 tx=0x7f3dd4029c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d31f4caa0b61a49 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3dc6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/3357119964 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4036070 con 0x557b5e0f5520 2024-05-09T01:28:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/3357119964 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3dd4029310 con 0x557b5e0f5520 2024-05-09T01:28:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/3357119964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dd402e070 con 0x557b5e0f5520 2024-05-09T01:28:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/3357119964 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3dd4029510 con 0x557b5e0f5520 2024-05-09T01:28:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3df7ac4b80 10 client.15762 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/3357119964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b5e2fbc80 con 0x557b5e0f5520 2024-05-09T01:28:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/3357119964 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dd402e070 con 0x557b5e0f5520 2024-05-09T01:28:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/3357119964 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3dd4029ce0 con 0x557b5e0f5520 2024-05-09T01:28:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3df7ac4b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3df7ac4b80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3df7ac4b80 20 client.15762 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557b5e2fc300 0x557b5e2fe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3df7ac4b80 4 client.15762 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:28:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.773+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/3357119964 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557b5e19caf0 con 0x557b5e2fc300 2024-05-09T01:28:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.774+0000 7f3de259c700 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557b5e2fc300 0x557b5e2fe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.774+0000 7f3dc6ff5700 10 client.15762 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.774+0000 7f3de259c700 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557b5e2fc300 0x557b5e2fe720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a32da54406c6463a server_cookie=15a9176d49de2606 in_seq=0 out_seq=0 2024-05-09T01:28:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.775+0000 7f3dc6ff5700 1 -- 172.21.15.19:0/3357119964 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557b5e19caf0 con 0x557b5e2fc300 2024-05-09T01:28:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.775+0000 7f3dc6ff5700 10 client.15762 handle_command_reply: tid=0 2024-05-09T01:28:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.775+0000 7f3df7ac4b80 1 client.15762 shutdown 2024-05-09T01:28:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/3357119964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 msgr2=0x557b5e2f4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e2f4570 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4029b30 tx=0x7f3dd4029c10 comp rx=0 tx=0).stop 2024-05-09T01:28:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/3357119964 shutdown_connections 2024-05-09T01:28:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557b5e2fc300 0x557b5e2fe720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b5e272720 0x557b5e2ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557b5e0f5520 0x557b5e2f4570 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3dc6ff5700 0 client.15762 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 --2- 172.21.15.19:0/3357119964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b5e1df1e0 0x557b5e2f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/3357119964 >> 172.21.15.19:0/3357119964 conn(0x557b5e155720 msgr2=0x557b5e27e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/3357119964 shutdown_connections 2024-05-09T01:28:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 1 -- 172.21.15.19:0/3357119964 wait complete. 2024-05-09T01:28:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3df7ac4b80 20 client.15762 trim_cache size 0 max 16384 2024-05-09T01:28:04.783 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:28:04.783 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:28:04.783 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "6b3a671a-cb0a-44a1-8761-7355b2c8b065", 2024-05-09T01:28:04.783 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:28:04.783 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:28:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3de90ae700 1 -- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dcc0b2900 msgr2=0x7f3dcc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3de90ae700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dcc0b2900 0x7f3dcc0b4e10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009cb0 tx=0x7f3dd4009260 comp rx=0 tx=0).stop 2024-05-09T01:28:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3de90ae700 1 -- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4108f30 msgr2=0x7f3de419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.776+0000 7f3de90ae700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4108f30 0x7f3de419b430 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800b500 tx=0x7f3dd800b810 comp rx=0 tx=0).stop 2024-05-09T01:28:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.777+0000 7f3de90ae700 1 -- 172.21.15.19:0/56680280 shutdown_connections 2024-05-09T01:28:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.777+0000 7f3de90ae700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3de4109880 0x7f3de419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.777+0000 7f3de90ae700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de4104650 0x7f3de419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.777+0000 7f3de90ae700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dcc0b2900 0x7f3dcc0b4e10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.777+0000 7f3de90ae700 1 --2- 172.21.15.19:0/56680280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3de4108f30 0x7f3de419b430 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.777+0000 7f3de90ae700 1 -- 172.21.15.19:0/56680280 >> 172.21.15.19:0/56680280 conn(0x7f3de4079f00 msgr2=0x7f3de4102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.777+0000 7f3de90ae700 1 -- 172.21.15.19:0/56680280 shutdown_connections 2024-05-09T01:28:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:04.777+0000 7f3de90ae700 1 -- 172.21.15.19:0/56680280 wait complete. 2024-05-09T01:28:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:04 smithi059 ceph-mon[34351]: pgmap v454: 97 pgs: 97 active+clean; 933 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:28:04.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:04 smithi196 ceph-mon[35041]: pgmap v454: 97 pgs: 97 active+clean; 933 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:28:05.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:28:05.828 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:05 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:28:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:05 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:05 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:28:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:05 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:05.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:05 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:28:05.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:05 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:05.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:05 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:28:05.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:05 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:05 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:28:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:05 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:05 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:28:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:05 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:06.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:06 smithi019 ceph-mon[30158]: pgmap v455: 97 pgs: 97 active+clean; 933 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:06 smithi059 ceph-mon[34351]: pgmap v455: 97 pgs: 97 active+clean; 933 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:06.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:06 smithi196 ceph-mon[35041]: pgmap v455: 97 pgs: 97 active+clean; 933 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.144+0000 7fac58601640 1 -- 172.21.15.19:0/3177244099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50104eb0 msgr2=0x7fac501052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.144+0000 7fac58601640 1 --2- 172.21.15.19:0/3177244099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50104eb0 0x7fac501052c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fac40009900 tx=0x7fac4002a410 comp rx=0 tx=0).stop 2024-05-09T01:28:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.144+0000 7fac58601640 1 -- 172.21.15.19:0/3177244099 shutdown_connections 2024-05-09T01:28:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.144+0000 7fac58601640 1 --2- 172.21.15.19:0/3177244099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac50109bb0 0x7fac5010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.144+0000 7fac58601640 1 --2- 172.21.15.19:0/3177244099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50104eb0 0x7fac501052c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.144+0000 7fac58601640 1 --2- 172.21.15.19:0/3177244099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50103c70 0x7fac50104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.144+0000 7fac58601640 1 -- 172.21.15.19:0/3177244099 >> 172.21.15.19:0/3177244099 conn(0x7fac500fee60 msgr2=0x7fac50101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.144+0000 7fac58601640 1 -- 172.21.15.19:0/3177244099 shutdown_connections 2024-05-09T01:28:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.144+0000 7fac58601640 1 -- 172.21.15.19:0/3177244099 wait complete. 2024-05-09T01:28:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.145+0000 7fac58601640 1 Processor -- start 2024-05-09T01:28:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.145+0000 7fac58601640 1 -- start start 2024-05-09T01:28:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.145+0000 7fac58601640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac50103c70 0x7fac5019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.145+0000 7fac58601640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50104eb0 0x7fac5019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.145+0000 7fac58601640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50109bb0 0x7fac501a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.145+0000 7fac58601640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac5010f790 con 0x7fac50109bb0 2024-05-09T01:28:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.145+0000 7fac58601640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac5010f610 con 0x7fac50103c70 2024-05-09T01:28:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.145+0000 7fac58601640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac5010f910 con 0x7fac50104eb0 2024-05-09T01:28:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.145+0000 7fac56b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50109bb0 0x7fac501a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac56b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50109bb0 0x7fac501a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42190/0 (socket says 172.21.15.19:42190) 2024-05-09T01:28:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac56b9e640 1 -- 172.21.15.19:0/3973590106 learned_addr learned my addr 172.21.15.19:0/3973590106 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac5639d640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac50103c70 0x7fac5019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac55b9c640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50104eb0 0x7fac5019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac56b9e640 1 -- 172.21.15.19:0/3973590106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50104eb0 msgr2=0x7fac5019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac56b9e640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50104eb0 0x7fac5019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac56b9e640 1 -- 172.21.15.19:0/3973590106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac50103c70 msgr2=0x7fac5019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac56b9e640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac50103c70 0x7fac5019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac56b9e640 1 -- 172.21.15.19:0/3973590106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac501a1ed0 con 0x7fac50109bb0 2024-05-09T01:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac56b9e640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50109bb0 0x7fac501a1860 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fac48009fd0 tx=0x7fac4800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac477fe640 1 -- 172.21.15.19:0/3973590106 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48017070 con 0x7fac50109bb0 2024-05-09T01:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac477fe640 1 -- 172.21.15.19:0/3973590106 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac48003e20 con 0x7fac50109bb0 2024-05-09T01:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac58601640 1 -- 172.21.15.19:0/3973590106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac501a21c0 con 0x7fac50109bb0 2024-05-09T01:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac477fe640 1 -- 172.21.15.19:0/3973590106 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48010460 con 0x7fac50109bb0 2024-05-09T01:28:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.146+0000 7fac58601640 1 -- 172.21.15.19:0/3973590106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac501a26c0 con 0x7fac50109bb0 2024-05-09T01:28:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.149+0000 7fac5f58f740 1 Processor -- start 2024-05-09T01:28:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac477fe640 1 -- 172.21.15.19:0/3973590106 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fac48003970 con 0x7fac50109bb0 2024-05-09T01:28:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac5f58f740 1 -- start start 2024-05-09T01:28:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db7f7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 0x5592db7f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db7fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac5f58f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592db75b7c0 con 0x5592db4531b0 2024-05-09T01:28:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac5f58f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5592db6f70e0 con 0x5592db4324d0 2024-05-09T01:28:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac5f58f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592db757780 con 0x5592db42a2f0 2024-05-09T01:28:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac55b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db7f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac55b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db7f7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42194/0 (socket says 172.21.15.19:42194) 2024-05-09T01:28:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac55b9c640 1 -- 172.21.15.19:0/1168843179 learned_addr learned my addr 172.21.15.19:0/1168843179 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.150+0000 7fac5639d640 1 --2- 172.21.15.19:0/1168843179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 0x5592db7f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac55b9c640 1 --2- 172.21.15.19:0/1168843179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db7fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac5639d640 1 -- 172.21.15.19:0/1168843179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 msgr2=0x5592db7fe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac5639d640 1 --2- 172.21.15.19:0/1168843179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db7fe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac5639d640 1 -- 172.21.15.19:0/1168843179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 msgr2=0x5592db7f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac5639d640 1 --2- 172.21.15.19:0/1168843179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db7f7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac5639d640 1 -- 172.21.15.19:0/1168843179 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592db7207c0 con 0x5592db42a2f0 2024-05-09T01:28:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac477fe640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac3c0b27e0 0x7fac3c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac55b9c640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac3c0b27e0 0x7fac3c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac5639d640 1 --2- 172.21.15.19:0/1168843179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 0x5592db7f7a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac50104870 tx=0x7fac4c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38dd3c51a5d4b3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac477fe640 1 -- 172.21.15.19:0/3973590106 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fac480e1f40 con 0x7fac50109bb0 2024-05-09T01:28:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac44bfe640 1 -- 172.21.15.19:0/1168843179 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c01c480 con 0x5592db42a2f0 2024-05-09T01:28:07.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.151+0000 7fac44bfe640 1 -- 172.21.15.19:0/1168843179 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4c0118f0 con 0x5592db42a2f0 2024-05-09T01:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.152+0000 7fac44bfe640 1 -- 172.21.15.19:0/1168843179 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c01a7a0 con 0x5592db42a2f0 2024-05-09T01:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.152+0000 7fac5f58f740 1 -- 172.21.15.19:0/1168843179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 msgr2=0x5592db7f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.152+0000 7fac5f58f740 1 --2- 172.21.15.19:0/1168843179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 0x5592db7f7a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac50104870 tx=0x7fac4c0107a0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.152+0000 7fac55b9c640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac3c0b27e0 0x7fac3c0b4cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x5592db7f86d0 tx=0x7fac4003f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.152+0000 7fac5f58f740 1 -- 172.21.15.19:0/1168843179 shutdown_connections 2024-05-09T01:28:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.152+0000 7fac5f58f740 1 --2- 172.21.15.19:0/1168843179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db7fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.152+0000 7fac5f58f740 1 --2- 172.21.15.19:0/1168843179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 0x5592db7f7a80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.152+0000 7fac5f58f740 1 --2- 172.21.15.19:0/1168843179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db7f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.152+0000 7fac5f58f740 1 -- 172.21.15.19:0/1168843179 >> 172.21.15.19:0/1168843179 conn(0x5592db447300 msgr2=0x5592db7f2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 -- 172.21.15.19:0/1168843179 shutdown_connections 2024-05-09T01:28:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 -- 172.21.15.19:0/1168843179 wait complete. 2024-05-09T01:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 -- start start 2024-05-09T01:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 Processor -- start 2024-05-09T01:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 0x5592db801200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db801740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db8086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592db7ff3d0 con 0x5592db4531b0 2024-05-09T01:28:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5592db6e29c0 con 0x5592db4324d0 2024-05-09T01:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592db7ff550 con 0x5592db42a2f0 2024-05-09T01:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac5f58f740 10 client.0 _ll_register_callbacks cb 0x5592db70f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac56b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db8086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac56b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db8086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42208/0 (socket says 172.21.15.19:42208) 2024-05-09T01:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.153+0000 7fac56b9e640 1 -- 172.21.15.19:0/3681897101 learned_addr learned my addr 172.21.15.19:0/3681897101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac5639d640 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 0x5592db801200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac55b9c640 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db801740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac56b9e640 1 -- 172.21.15.19:0/3681897101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 msgr2=0x5592db801200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac56b9e640 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 0x5592db801200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac56b9e640 1 -- 172.21.15.19:0/3681897101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 msgr2=0x5592db801740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac56b9e640 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db801740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac56b9e640 1 -- 172.21.15.19:0/3681897101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592db7091f0 con 0x5592db4531b0 2024-05-09T01:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac56b9e640 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db8086a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fac480a5670 tx=0x7fac48003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa66726bae2cd18d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac39ffb640 1 -- 172.21.15.19:0/3681897101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48020040 con 0x5592db4531b0 2024-05-09T01:28:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac39ffb640 1 -- 172.21.15.19:0/3681897101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fac480910d0 con 0x5592db4531b0 2024-05-09T01:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac5f58f740 1 -- 172.21.15.19:0/3681897101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac4c0118f0 con 0x5592db4531b0 2024-05-09T01:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac39ffb640 1 -- 172.21.15.19:0/3681897101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac480b23a0 con 0x5592db4531b0 2024-05-09T01:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac5f58f740 10 client.15774 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.154+0000 7fac5f58f740 1 -- 172.21.15.19:0/3681897101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592db4be790 con 0x5592db4531b0 2024-05-09T01:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.155+0000 7fac39ffb640 1 -- 172.21.15.19:0/3681897101 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4c0118f0 con 0x5592db4531b0 2024-05-09T01:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.155+0000 7fac39ffb640 1 -- 172.21.15.19:0/3681897101 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac480a4f60 con 0x5592db4531b0 2024-05-09T01:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.155+0000 7fac5f58f740 10 client.15774 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.155+0000 7fac5f58f740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.155+0000 7fac5f58f740 20 client.15774 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.155+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5592db411d80 0x5592db80ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.155+0000 7fac5f58f740 4 client.15774 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.155+0000 7fac5f58f740 1 -- 172.21.15.19:0/3681897101 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592db7693d0 con 0x5592db411d80 2024-05-09T01:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.155+0000 7fac5639d640 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5592db411d80 0x5592db80ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.156+0000 7fac5639d640 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5592db411d80 0x5592db80ec20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8bd8516421b0d3 server_cookie=c812aac962cdd876 in_seq=0 out_seq=0 2024-05-09T01:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.156+0000 7fac39ffb640 10 client.15774 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.156+0000 7fac39ffb640 1 -- 172.21.15.19:0/3681897101 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592db7693d0 con 0x5592db411d80 2024-05-09T01:28:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.156+0000 7fac39ffb640 10 client.15774 handle_command_reply: tid=0 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.156+0000 7fac5f58f740 1 client.15774 shutdown 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 -- 172.21.15.19:0/3681897101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 msgr2=0x5592db8086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db8086a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fac480a5670 tx=0x7fac48003da0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 -- 172.21.15.19:0/3681897101 shutdown_connections 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5592db411d80 0x5592db80ec20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4531b0 0x5592db8086a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db801740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac39ffb640 0 client.15774 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3681897101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db42a2f0 0x5592db801200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 -- 172.21.15.19:0/3681897101 >> 172.21.15.19:0/3681897101 conn(0x5592db447300 msgr2=0x5592db7f1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 -- 172.21.15.19:0/3681897101 shutdown_connections 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 1 -- 172.21.15.19:0/3681897101 wait complete. 2024-05-09T01:28:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.157+0000 7fac5f58f740 20 client.15774 trim_cache size 0 max 16384 2024-05-09T01:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.172+0000 7fac5f58f740 1 Processor -- start 2024-05-09T01:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac5f58f740 1 -- start start 2024-05-09T01:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db42a2f0 0x5592db7fab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4324d0 0x5592db7fb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 0x5592db7fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac5f58f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac501ab430 con 0x5592db4324d0 2024-05-09T01:28:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac5f58f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac501acf70 con 0x5592db42a2f0 2024-05-09T01:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac5f58f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac501abb00 con 0x5592db4531b0 2024-05-09T01:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac55b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4324d0 0x5592db7fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac55b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4324d0 0x5592db7fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42220/0 (socket says 172.21.15.19:42220) 2024-05-09T01:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac55b9c640 1 -- 172.21.15.19:0/49924342 learned_addr learned my addr 172.21.15.19:0/49924342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac56b9e640 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 0x5592db7fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac5639d640 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db42a2f0 0x5592db7fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac55b9c640 1 -- 172.21.15.19:0/49924342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 msgr2=0x5592db7fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac55b9c640 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 0x5592db7fbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac55b9c640 1 -- 172.21.15.19:0/49924342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db42a2f0 msgr2=0x5592db7fab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac55b9c640 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db42a2f0 0x5592db7fab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac55b9c640 1 -- 172.21.15.19:0/49924342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592db5355f0 con 0x5592db4324d0 2024-05-09T01:28:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac56b9e640 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 0x5592db7fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.173+0000 7fac55b9c640 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4324d0 0x5592db7fb660 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fac40009230 tx=0x7fac40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff178d0b49ff7f9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.174+0000 7fac3bfff640 1 -- 172.21.15.19:0/49924342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40035040 con 0x5592db4324d0 2024-05-09T01:28:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.174+0000 7fac3bfff640 1 -- 172.21.15.19:0/49924342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac40004240 con 0x5592db4324d0 2024-05-09T01:28:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.174+0000 7fac3bfff640 1 -- 172.21.15.19:0/49924342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4003b7e0 con 0x5592db4324d0 2024-05-09T01:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.174+0000 7fac5f58f740 1 -- 172.21.15.19:0/49924342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4324d0 msgr2=0x5592db7fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.174+0000 7fac5f58f740 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4324d0 0x5592db7fb660 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fac40009230 tx=0x7fac40009260 comp rx=0 tx=0).stop 2024-05-09T01:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 -- 172.21.15.19:0/49924342 shutdown_connections 2024-05-09T01:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 0x5592db7fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db4324d0 0x5592db7fb660 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 --2- 172.21.15.19:0/49924342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db42a2f0 0x5592db7fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 -- 172.21.15.19:0/49924342 >> 172.21.15.19:0/49924342 conn(0x5592db447300 msgr2=0x5592db7ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 -- 172.21.15.19:0/49924342 shutdown_connections 2024-05-09T01:28:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 -- 172.21.15.19:0/49924342 wait complete. 2024-05-09T01:28:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 -- start start 2024-05-09T01:28:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 Processor -- start 2024-05-09T01:28:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db42a2f0 0x5592db805730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db805c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 0x5592db806ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac501ac8a0 con 0x5592db42a2f0 2024-05-09T01:28:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac501ac1d0 con 0x5592db4324d0 2024-05-09T01:28:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac501a9410 con 0x5592db4531b0 2024-05-09T01:28:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5f58f740 10 client.0 _ll_register_callbacks cb 0x7fac48004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5639d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db42a2f0 0x5592db805730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5639d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db42a2f0 0x5592db805730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42234/0 (socket says 172.21.15.19:42234) 2024-05-09T01:28:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.175+0000 7fac5639d640 1 -- 172.21.15.19:0/3697617095 learned_addr learned my addr 172.21.15.19:0/3697617095 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac55b9c640 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db805c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac5639d640 1 -- 172.21.15.19:0/3697617095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 msgr2=0x5592db806ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac5639d640 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 0x5592db806ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac5639d640 1 -- 172.21.15.19:0/3697617095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 msgr2=0x5592db805c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac5639d640 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db805c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac5639d640 1 -- 172.21.15.19:0/3697617095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592db71fed0 con 0x5592db42a2f0 2024-05-09T01:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac5639d640 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db42a2f0 0x5592db805730 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00b620 tx=0x7fac4c00bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3c2ed4155dbc8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac39ffb640 1 -- 172.21.15.19:0/3697617095 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c005240 con 0x5592db42a2f0 2024-05-09T01:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac39ffb640 1 -- 172.21.15.19:0/3697617095 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fac4c0053e0 con 0x5592db42a2f0 2024-05-09T01:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac5f58f740 1 -- 172.21.15.19:0/3697617095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac40004240 con 0x5592db42a2f0 2024-05-09T01:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac39ffb640 1 -- 172.21.15.19:0/3697617095 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac4c039050 con 0x5592db42a2f0 2024-05-09T01:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac5f58f740 10 client.15780 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.176+0000 7fac5f58f740 1 -- 172.21.15.19:0/3697617095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592db877030 con 0x5592db42a2f0 2024-05-09T01:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.177+0000 7fac39ffb640 1 -- 172.21.15.19:0/3697617095 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac40004240 con 0x5592db42a2f0 2024-05-09T01:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.177+0000 7fac39ffb640 1 -- 172.21.15.19:0/3697617095 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac4c0115e0 con 0x5592db42a2f0 2024-05-09T01:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.177+0000 7fac5f58f740 10 client.15780 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.177+0000 7fac5f58f740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.177+0000 7fac5f58f740 20 client.15780 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.177+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5592db3e71c0 0x5592db879790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.177+0000 7fac5f58f740 4 client.15780 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.177+0000 7fac5f58f740 1 -- 172.21.15.19:0/3697617095 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592db763240 con 0x5592db3e71c0 2024-05-09T01:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.177+0000 7fac55b9c640 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5592db3e71c0 0x5592db879790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.178+0000 7fac55b9c640 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5592db3e71c0 0x5592db879790 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac77f3a13c94823 server_cookie=83601ffeaa2acd5e in_seq=0 out_seq=0 2024-05-09T01:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.178+0000 7fac39ffb640 10 client.15780 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac39ffb640 1 -- 172.21.15.19:0/3697617095 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5592db763240 con 0x5592db3e71c0 2024-05-09T01:28:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac39ffb640 10 client.15780 handle_command_reply: tid=0 2024-05-09T01:28:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac5f58f740 1 client.15780 shutdown 2024-05-09T01:28:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac5f58f740 1 -- 172.21.15.19:0/3697617095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db42a2f0 msgr2=0x5592db805730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db42a2f0 0x5592db805730 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00b620 tx=0x7fac4c00bfa0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac5f58f740 1 -- 172.21.15.19:0/3697617095 shutdown_connections 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5592db3e71c0 0x5592db879790 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592db4531b0 0x5592db806ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592db4324d0 0x5592db805c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac5f58f740 1 --2- 172.21.15.19:0/3697617095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592db42a2f0 0x5592db805730 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac5f58f740 1 -- 172.21.15.19:0/3697617095 >> 172.21.15.19:0/3697617095 conn(0x5592db447300 msgr2=0x5592db7ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.191+0000 7fac39ffb640 0 client.15780 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac5f58f740 1 -- 172.21.15.19:0/3697617095 shutdown_connections 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac5f58f740 1 -- 172.21.15.19:0/3697617095 wait complete. 2024-05-09T01:28:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac5f58f740 20 client.15780 trim_cache size 0 max 16384 2024-05-09T01:28:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:28:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:28:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:28:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 -- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac3c0b27e0 msgr2=0x7fac3c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac3c0b27e0 0x7fac3c0b4cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x5592db7f86d0 tx=0x7fac4003f040 comp rx=0 tx=0).stop 2024-05-09T01:28:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 -- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50109bb0 msgr2=0x7fac501a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50109bb0 0x7fac501a1860 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fac48009fd0 tx=0x7fac4800b260 comp rx=0 tx=0).stop 2024-05-09T01:28:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 -- 172.21.15.19:0/3973590106 shutdown_connections 2024-05-09T01:28:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac3c0b27e0 0x7fac3c0b4cf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac50109bb0 0x7fac501a1860 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50104eb0 0x7fac5019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 --2- 172.21.15.19:0/3973590106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac50103c70 0x7fac5019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 -- 172.21.15.19:0/3973590106 >> 172.21.15.19:0/3973590106 conn(0x7fac500fee60 msgr2=0x7fac50105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 -- 172.21.15.19:0/3973590106 shutdown_connections 2024-05-09T01:28:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:07.192+0000 7fac58601640 1 -- 172.21.15.19:0/3973590106 wait complete. 2024-05-09T01:28:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.603+0000 7f18055ed700 1 -- 172.21.15.19:0/4084986121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800109870 msgr2=0x7f18001111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.603+0000 7f18055ed700 1 --2- 172.21.15.19:0/4084986121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800109870 0x7f18001111d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f17f80099c0 tx=0x7f17f8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.603+0000 7f18055ed700 1 -- 172.21.15.19:0/4084986121 shutdown_connections 2024-05-09T01:28:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.603+0000 7f18055ed700 1 --2- 172.21.15.19:0/4084986121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800109870 0x7f18001111d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.603+0000 7f18055ed700 1 --2- 172.21.15.19:0/4084986121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18001045f0 0x7f1800104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.603+0000 7f18055ed700 1 --2- 172.21.15.19:0/4084986121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1800108f20 0x7f1800109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.603+0000 7f18055ed700 1 -- 172.21.15.19:0/4084986121 >> 172.21.15.19:0/4084986121 conn(0x7f1800100040 msgr2=0x7f1800102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.604+0000 7f18055ed700 1 -- 172.21.15.19:0/4084986121 shutdown_connections 2024-05-09T01:28:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.604+0000 7f18055ed700 1 -- 172.21.15.19:0/4084986121 wait complete. 2024-05-09T01:28:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.604+0000 7f18055ed700 1 Processor -- start 2024-05-09T01:28:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.604+0000 7f18055ed700 1 -- start start 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f18055ed700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18001045f0 0x7f1800075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f18055ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800108f20 0x7f1800076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f18055ed700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800109870 0x7f1800076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f18055ed700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f18001144a0 con 0x7f1800108f20 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f18055ed700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1800114320 con 0x7f1800109870 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f18055ed700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1800114620 con 0x7f18001045f0 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17fe7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800108f20 0x7f1800076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17fe7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800108f20 0x7f1800076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42244/0 (socket says 172.21.15.19:42244) 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17fe7fc700 1 -- 172.21.15.19:0/1204125568 learned_addr learned my addr 172.21.15.19:0/1204125568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800109870 0x7f1800076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17feffd700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18001045f0 0x7f1800075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17fe7fc700 1 -- 172.21.15.19:0/1204125568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18001045f0 msgr2=0x7f1800075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18001045f0 0x7f1800075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17fe7fc700 1 -- 172.21.15.19:0/1204125568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800109870 msgr2=0x7f1800076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800109870 0x7f1800076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17fe7fc700 1 -- 172.21.15.19:0/1204125568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18001a91c0 con 0x7f1800108f20 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.605+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800109870 0x7f1800076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.606+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800108f20 0x7f1800076000 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17e800d950 tx=0x7f17e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.606+0000 7f17f7fff700 1 -- 172.21.15.19:0/1204125568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8016070 con 0x7f1800108f20 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.606+0000 7f18055ed700 1 -- 172.21.15.19:0/1204125568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18001a94b0 con 0x7f1800108f20 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.606+0000 7f17f7fff700 1 -- 172.21.15.19:0/1204125568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e8004830 con 0x7f1800108f20 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.606+0000 7f17f7fff700 1 -- 172.21.15.19:0/1204125568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8005520 con 0x7f1800108f20 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.606+0000 7f18055ed700 1 -- 172.21.15.19:0/1204125568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18001a99f0 con 0x7f1800108f20 2024-05-09T01:28:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.608+0000 7f17f7fff700 1 -- 172.21.15.19:0/1204125568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f17e800f690 con 0x7f1800108f20 2024-05-09T01:28:07.606 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:07 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:28:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.609+0000 7f17f7fff700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17ec0b2880 0x7f17ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.610+0000 7f17feffd700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17ec0b2880 0x7f17ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.610+0000 7f17f7fff700 1 -- 172.21.15.19:0/1204125568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f17e80e1f70 con 0x7f1800108f20 2024-05-09T01:28:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.610+0000 7f1814003b80 1 Processor -- start 2024-05-09T01:28:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.610+0000 7f17feffd700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17ec0b2880 0x7f17ec0b4d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f17f000f540 tx=0x7f17f000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.610+0000 7f1814003b80 1 -- start start 2024-05-09T01:28:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.610+0000 7f1814003b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 0x564f4a39c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.610+0000 7f1814003b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a39c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f1814003b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a396810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f1814003b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564f4a2b90a0 con 0x564f4a396400 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f1814003b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f4a25e9a0 con 0x564f4a293d00 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f1814003b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f4a2b5060 con 0x564f4a2194a0 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a396810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a396810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42250/0 (socket says 172.21.15.19:42250) 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17ff7fe700 1 -- 172.21.15.19:0/3114931699 learned_addr learned my addr 172.21.15.19:0/3114931699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17feffd700 1 --2- 172.21.15.19:0/3114931699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 0x564f4a39c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/3114931699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a39c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17ff7fe700 1 -- 172.21.15.19:0/3114931699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 msgr2=0x564f4a39c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/3114931699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a39c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17ff7fe700 1 -- 172.21.15.19:0/3114931699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 msgr2=0x564f4a39c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/3114931699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 0x564f4a39c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17ff7fe700 1 -- 172.21.15.19:0/3114931699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4a24f500 con 0x564f4a396400 2024-05-09T01:28:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.611+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/3114931699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a396810 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f17f80099c0 tx=0x7f17f8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a9b30dc5afaaef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.612+0000 7f17f57fa700 1 -- 172.21.15.19:0/3114931699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8027070 con 0x564f4a396400 2024-05-09T01:28:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.612+0000 7f17f57fa700 1 -- 172.21.15.19:0/3114931699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f8005560 con 0x564f4a396400 2024-05-09T01:28:07.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.612+0000 7f17f57fa700 1 -- 172.21.15.19:0/3114931699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8003d90 con 0x564f4a396400 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 -- 172.21.15.19:0/3114931699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 msgr2=0x564f4a396810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 --2- 172.21.15.19:0/3114931699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a396810 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f17f80099c0 tx=0x7f17f8005040 comp rx=0 tx=0).stop 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 -- 172.21.15.19:0/3114931699 shutdown_connections 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 --2- 172.21.15.19:0/3114931699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a396810 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 --2- 172.21.15.19:0/3114931699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 0x564f4a39c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 --2- 172.21.15.19:0/3114931699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a39c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 -- 172.21.15.19:0/3114931699 >> 172.21.15.19:0/3114931699 conn(0x564f4a21cf70 msgr2=0x564f4a391220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 -- 172.21.15.19:0/3114931699 shutdown_connections 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 -- 172.21.15.19:0/3114931699 wait complete. 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 -- start start 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.613+0000 7f1814003b80 1 Processor -- start 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f1814003b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a3a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f1814003b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a293d00 0x564f4a3a0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f1814003b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 0x564f4a3a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f1814003b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564f4a397100 con 0x564f4a293d00 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f1814003b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f4a253b20 con 0x564f4a396400 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f1814003b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f4a3ac200 con 0x564f4a2194a0 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f1814003b80 10 client.0 _ll_register_callbacks cb 0x564f4a12a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17fe7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a293d00 0x564f4a3a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17fe7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a293d00 0x564f4a3a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42252/0 (socket says 172.21.15.19:42252) 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17fe7fc700 1 -- 172.21.15.19:0/3660036765 learned_addr learned my addr 172.21.15.19:0/3660036765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17feffd700 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a3a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17fe7fc700 1 -- 172.21.15.19:0/3660036765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 msgr2=0x564f4a3a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 0x564f4a3a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a3a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17fe7fc700 1 -- 172.21.15.19:0/3660036765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 msgr2=0x564f4a3a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 0x564f4a3a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17fe7fc700 1 -- 172.21.15.19:0/3660036765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4a134490 con 0x564f4a293d00 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17feffd700 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a3a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a293d00 0x564f4a3a0650 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a94a0 tx=0x7f17e80a8f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f881ff785eaff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.614+0000 7f17e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f17e67fc700 1 -- 172.21.15.19:0/3660036765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80c3060 con 0x564f4a293d00 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f17e67fc700 1 -- 172.21.15.19:0/3660036765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f17e80049e0 con 0x564f4a293d00 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f1814003b80 1 -- 172.21.15.19:0/3660036765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f8005560 con 0x564f4a293d00 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f17e67fc700 1 -- 172.21.15.19:0/3660036765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17e80ad8c0 con 0x564f4a293d00 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f1814003b80 10 client.15792 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f1814003b80 1 -- 172.21.15.19:0/3660036765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f4a3adc50 con 0x564f4a293d00 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f17e67fc700 1 -- 172.21.15.19:0/3660036765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f8005560 con 0x564f4a293d00 2024-05-09T01:28:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f17e67fc700 1 -- 172.21.15.19:0/3660036765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f17e80a8770 con 0x564f4a293d00 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f1814003b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f1814003b80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.615+0000 7f1814003b80 20 client.15792 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.616+0000 7f1814003b80 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f4a3b01e0 0x564f4a3b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.616+0000 7f1814003b80 4 client.15792 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.616+0000 7f1814003b80 1 -- 172.21.15.19:0/3660036765 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f4a2c6a80 con 0x564f4a3b01e0 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.616+0000 7f17feffd700 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f4a3b01e0 0x564f4a3b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.616+0000 7f17feffd700 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f4a3b01e0 0x564f4a3b2600 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ee1d4243d0eab4 server_cookie=f6af168267251865 in_seq=0 out_seq=0 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.616+0000 7f17e67fc700 10 client.15792 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f17e67fc700 1 -- 172.21.15.19:0/3660036765 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f4a2c6a80 con 0x564f4a3b01e0 2024-05-09T01:28:07.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f17e67fc700 10 client.15792 handle_command_reply: tid=0 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 client.15792 shutdown 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 -- 172.21.15.19:0/3660036765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a293d00 msgr2=0x564f4a3a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a293d00 0x564f4a3a0650 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a94a0 tx=0x7f17e80a8f90 comp rx=0 tx=0).stop 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 -- 172.21.15.19:0/3660036765 shutdown_connections 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f4a3b01e0 0x564f4a3b2600 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f17e67fc700 0 client.15792 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 0x564f4a3a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a3a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 --2- 172.21.15.19:0/3660036765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a293d00 0x564f4a3a0650 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 -- 172.21.15.19:0/3660036765 >> 172.21.15.19:0/3660036765 conn(0x564f4a21cf70 msgr2=0x564f4a38aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 -- 172.21.15.19:0/3660036765 shutdown_connections 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.617+0000 7f1814003b80 1 -- 172.21.15.19:0/3660036765 wait complete. 2024-05-09T01:28:07.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.618+0000 7f1814003b80 20 client.15792 trim_cache size 0 max 16384 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.638+0000 7f1814003b80 1 Processor -- start 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.638+0000 7f1814003b80 1 -- start start 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f1814003b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a38b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f1814003b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 0x564f4a3a0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f1814003b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a399c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f1814003b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f18001b0da0 con 0x564f4a396400 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f1814003b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18001b28e0 con 0x564f4a293d00 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f1814003b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18001b1470 con 0x564f4a2194a0 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a399c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a399c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42256/0 (socket says 172.21.15.19:42256) 2024-05-09T01:28:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17ff7fe700 1 -- 172.21.15.19:0/2139383872 learned_addr learned my addr 172.21.15.19:0/2139383872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17feffd700 1 --2- 172.21.15.19:0/2139383872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a38b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/2139383872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 0x564f4a3a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17ff7fe700 1 -- 172.21.15.19:0/2139383872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 msgr2=0x564f4a38b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/2139383872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a38b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17ff7fe700 1 -- 172.21.15.19:0/2139383872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 msgr2=0x564f4a3a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/2139383872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 0x564f4a3a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17ff7fe700 1 -- 172.21.15.19:0/2139383872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4a39e680 con 0x564f4a396400 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.639+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/2139383872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a399c00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f17f800b6d0 tx=0x7f17f8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c642ae5d726342 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.640+0000 7f17f4ff9700 1 -- 172.21.15.19:0/2139383872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8027040 con 0x564f4a396400 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.640+0000 7f17f4ff9700 1 -- 172.21.15.19:0/2139383872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f8005560 con 0x564f4a396400 2024-05-09T01:28:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.640+0000 7f17f4ff9700 1 -- 172.21.15.19:0/2139383872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f800f460 con 0x564f4a396400 2024-05-09T01:28:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.640+0000 7f1814003b80 1 -- 172.21.15.19:0/2139383872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 msgr2=0x564f4a399c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.640+0000 7f1814003b80 1 --2- 172.21.15.19:0/2139383872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a399c00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f17f800b6d0 tx=0x7f17f8009650 comp rx=0 tx=0).stop 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 -- 172.21.15.19:0/2139383872 shutdown_connections 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 --2- 172.21.15.19:0/2139383872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a396400 0x564f4a399c00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 --2- 172.21.15.19:0/2139383872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a2194a0 0x564f4a38b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 --2- 172.21.15.19:0/2139383872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a293d00 0x564f4a3a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 -- 172.21.15.19:0/2139383872 >> 172.21.15.19:0/2139383872 conn(0x564f4a21cf70 msgr2=0x564f4a38aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 -- 172.21.15.19:0/2139383872 shutdown_connections 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 -- 172.21.15.19:0/2139383872 wait complete. 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 -- start start 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 Processor -- start 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a2194a0 0x564f4a3b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a293d00 0x564f4a3b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.641+0000 7f1814003b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 0x564f4a3b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f1814003b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f18001b2210 con 0x564f4a2194a0 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f1814003b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18001b1b40 con 0x564f4a396400 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f1814003b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18001aed40 con 0x564f4a293d00 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17feffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a2194a0 0x564f4a3b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f1814003b80 10 client.0 _ll_register_callbacks cb 0x564f4a12a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17feffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a2194a0 0x564f4a3b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42268/0 (socket says 172.21.15.19:42268) 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17feffd700 1 -- 172.21.15.19:0/789813983 learned_addr learned my addr 172.21.15.19:0/789813983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17ff7fe700 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 0x564f4a3b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a293d00 0x564f4a3b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17feffd700 1 -- 172.21.15.19:0/789813983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a293d00 msgr2=0x564f4a3b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17feffd700 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a293d00 0x564f4a3b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17feffd700 1 -- 172.21.15.19:0/789813983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 msgr2=0x564f4a3b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17feffd700 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 0x564f4a3b7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17feffd700 1 -- 172.21.15.19:0/789813983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4a3a8b60 con 0x564f4a2194a0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17feffd700 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a2194a0 0x564f4a3b7460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f17f0010010 tx=0x7f17f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ea6978fdbf2800 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17e67fc700 1 -- 172.21.15.19:0/789813983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f001d040 con 0x564f4a2194a0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f17e67fc700 1 -- 172.21.15.19:0/789813983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f17f002c310 con 0x564f4a2194a0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.642+0000 7f1814003b80 1 -- 172.21.15.19:0/789813983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f8005560 con 0x564f4a2194a0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f17e67fc700 1 -- 172.21.15.19:0/789813983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17f002c510 con 0x564f4a2194a0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f1814003b80 10 client.15798 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f1814003b80 1 -- 172.21.15.19:0/789813983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f4a41aa00 con 0x564f4a2194a0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f17e67fc700 1 -- 172.21.15.19:0/789813983 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f8005560 con 0x564f4a2194a0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f17e67fc700 1 -- 172.21.15.19:0/789813983 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f17f002cce0 con 0x564f4a2194a0 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f1814003b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f1814003b80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f1814003b80 20 client.15798 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f1814003b80 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f4a3b01e0 0x564f4a41d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f1814003b80 4 client.15798 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f1814003b80 1 -- 172.21.15.19:0/789813983 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f4a2c0b20 con 0x564f4a3b01e0 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.643+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f4a3b01e0 0x564f4a41d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f17e67fc700 10 client.15798 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f17fe7fc700 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f4a3b01e0 0x564f4a41d140 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b15210a2403eb98d server_cookie=932ac59a6d7d90a9 in_seq=0 out_seq=0 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f17e67fc700 1 -- 172.21.15.19:0/789813983 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564f4a2c0b20 con 0x564f4a3b01e0 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f17e67fc700 10 client.15798 handle_command_reply: tid=0 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 client.15798 shutdown 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 -- 172.21.15.19:0/789813983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a2194a0 msgr2=0x564f4a3b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a2194a0 0x564f4a3b7460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f17f0010010 tx=0x7f17f0004ad0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 -- 172.21.15.19:0/789813983 shutdown_connections 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f4a3b01e0 0x564f4a41d140 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f4a396400 0x564f4a3b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f4a2194a0 0x564f4a3b7460 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 --2- 172.21.15.19:0/789813983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f4a293d00 0x564f4a3b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 -- 172.21.15.19:0/789813983 >> 172.21.15.19:0/789813983 conn(0x564f4a21cf70 msgr2=0x564f4a399c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f17e67fc700 0 client.15798 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 -- 172.21.15.19:0/789813983 shutdown_connections 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 1 -- 172.21.15.19:0/789813983 wait complete. 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.644+0000 7f1814003b80 20 client.15798 trim_cache size 0 max 16384 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:28:07.647 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:28:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 -- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17ec0b2880 msgr2=0x7f17ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17ec0b2880 0x7f17ec0b4d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f17f000f540 tx=0x7f17f000b040 comp rx=0 tx=0).stop 2024-05-09T01:28:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 -- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800108f20 msgr2=0x7f1800076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800108f20 0x7f1800076000 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17e800d950 tx=0x7f17e800dc60 comp rx=0 tx=0).stop 2024-05-09T01:28:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 -- 172.21.15.19:0/1204125568 shutdown_connections 2024-05-09T01:28:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800109870 0x7f1800076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18001045f0 0x7f1800075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f17ec0b2880 0x7f17ec0b4d90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 --2- 172.21.15.19:0/1204125568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1800108f20 0x7f1800076000 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:07.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 -- 172.21.15.19:0/1204125568 >> 172.21.15.19:0/1204125568 conn(0x7f1800100040 msgr2=0x7f1800108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:07.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 -- 172.21.15.19:0/1204125568 shutdown_connections 2024-05-09T01:28:07.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:07.645+0000 7f18055ed700 1 -- 172.21.15.19:0/1204125568 wait complete. 2024-05-09T01:28:07.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:07 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:28:07.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:07 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:28:08.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:08 smithi019 ceph-mon[30158]: pgmap v456: 97 pgs: 97 active+clean; 997 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:08 smithi059 ceph-mon[34351]: pgmap v456: 97 pgs: 97 active+clean; 997 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:08.868 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:28:08.868 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:28:08.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:08 smithi196 ceph-mon[35041]: pgmap v456: 97 pgs: 97 active+clean; 997 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:10.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:10 smithi059 ceph-mon[34351]: pgmap v457: 97 pgs: 97 active+clean; 997 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:28:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:10 smithi019 ceph-mon[30158]: pgmap v457: 97 pgs: 97 active+clean; 997 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:28:10.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:10 smithi196 ceph-mon[35041]: pgmap v457: 97 pgs: 97 active+clean; 997 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:28:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.998+0000 7f8540f80700 1 -- 172.21.15.19:0/503436265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 msgr2=0x7f853c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.998+0000 7f8540f80700 1 --2- 172.21.15.19:0/503436265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 0x7f853c1038d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f852c0099c0 tx=0x7f852c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.998+0000 7f8540f80700 1 -- 172.21.15.19:0/503436265 shutdown_connections 2024-05-09T01:28:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.998+0000 7f8540f80700 1 --2- 172.21.15.19:0/503436265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f853c108740 0x7f853c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.998+0000 7f8540f80700 1 --2- 172.21.15.19:0/503436265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 0x7f853c1038d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.998+0000 7f8540f80700 1 --2- 172.21.15.19:0/503436265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c107df0 0x7f853c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.998+0000 7f8540f80700 1 -- 172.21.15.19:0/503436265 >> 172.21.15.19:0/503436265 conn(0x7f853c0fedf0 msgr2=0x7f853c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.998+0000 7f8540f80700 1 -- 172.21.15.19:0/503436265 shutdown_connections 2024-05-09T01:28:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.998+0000 7f8540f80700 1 -- 172.21.15.19:0/503436265 wait complete. 2024-05-09T01:28:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f8540f80700 1 Processor -- start 2024-05-09T01:28:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f8540f80700 1 -- start start 2024-05-09T01:28:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f8540f80700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 0x7f853c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f8540f80700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f853c107df0 0x7f853c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f8540f80700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c108740 0x7f853c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f8540f80700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f853c113370 con 0x7f853c1034c0 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f8540f80700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1131f0 con 0x7f853c107df0 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f8540f80700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1134f0 con 0x7f853c108740 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f853a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 0x7f853c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f853a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 0x7f853c0ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34556/0 (socket says 172.21.15.19:34556) 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f853a59c700 1 -- 172.21.15.19:0/900957943 learned_addr learned my addr 172.21.15.19:0/900957943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:11.999+0000 7f8539d9b700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f853c107df0 0x7f853c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f853ad9d700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c108740 0x7f853c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f853a59c700 1 -- 172.21.15.19:0/900957943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c108740 msgr2=0x7f853c0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f853a59c700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c108740 0x7f853c0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f853a59c700 1 -- 172.21.15.19:0/900957943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f853c107df0 msgr2=0x7f853c0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f853a59c700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f853c107df0 0x7f853c0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f853a59c700 1 -- 172.21.15.19:0/900957943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c1a91d0 con 0x7f853c1034c0 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f853a59c700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 0x7f853c0ff2d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f852c00b580 tx=0x7f852c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f852b7fe700 1 -- 172.21.15.19:0/900957943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c027070 con 0x7f853c1034c0 2024-05-09T01:28:12.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f852b7fe700 1 -- 172.21.15.19:0/900957943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852c003f20 con 0x7f853c1034c0 2024-05-09T01:28:12.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c1a9460 con 0x7f853c1034c0 2024-05-09T01:28:12.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f852b7fe700 1 -- 172.21.15.19:0/900957943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c022d70 con 0x7f853c1034c0 2024-05-09T01:28:12.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.000+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c1a98b0 con 0x7f853c1034c0 2024-05-09T01:28:12.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.002+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f853c0512c0 con 0x7f853c1034c0 2024-05-09T01:28:12.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.003+0000 7f852b7fe700 1 -- 172.21.15.19:0/900957943 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f852c022500 con 0x7f853c1034c0 2024-05-09T01:28:12.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.004+0000 7f852b7fe700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85240b2880 0x7f85240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:12.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.004+0000 7f8539d9b700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85240b2880 0x7f85240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:12.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.004+0000 7f852b7fe700 1 -- 172.21.15.19:0/900957943 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f852c0e2f30 con 0x7f853c1034c0 2024-05-09T01:28:12.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.004+0000 7f8539d9b700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85240b2880 0x7f85240b4d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f853c102290 tx=0x7f853000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.009+0000 7f852b7fe700 1 -- 172.21.15.19:0/900957943 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f852c013070 con 0x7f853c1034c0 2024-05-09T01:28:12.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.274+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f853c02e7b0 con 0x7f853c1034c0 2024-05-09T01:28:12.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.275+0000 7f852b7fe700 1 -- 172.21.15.19:0/900957943 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f852c0aac60 con 0x7f853c1034c0 2024-05-09T01:28:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:28:12.272 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:28:12.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85240b2880 msgr2=0x7f85240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:12.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85240b2880 0x7f85240b4d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f853c102290 tx=0x7f853000b410 comp rx=0 tx=0).stop 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 msgr2=0x7f853c0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 0x7f853c0ff2d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f852c00b580 tx=0x7f852c0047b0 comp rx=0 tx=0).stop 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 shutdown_connections 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c108740 0x7f853c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85240b2880 0x7f85240b4d90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f853c1034c0 0x7f853c0ff2d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 --2- 172.21.15.19:0/900957943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f853c107df0 0x7f853c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 >> 172.21.15.19:0/900957943 conn(0x7f853c0fedf0 msgr2=0x7f853c1063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 shutdown_connections 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:12.277+0000 7f8540f80700 1 -- 172.21.15.19:0/900957943 wait complete. 2024-05-09T01:28:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:28:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:12 smithi059 ceph-mon[34351]: pgmap v458: 97 pgs: 97 active+clean; 1.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:28:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:12 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/900957943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:28:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:12 smithi019 ceph-mon[30158]: pgmap v458: 97 pgs: 97 active+clean; 1.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:28:12.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:12 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/900957943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:28:12.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:12 smithi196 ceph-mon[35041]: pgmap v458: 97 pgs: 97 active+clean; 1.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:28:12.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:12 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/900957943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:28:12.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:28:13.243 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:14.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:14 smithi019 ceph-mon[30158]: pgmap v459: 97 pgs: 97 active+clean; 1.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:28:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:14 smithi059 ceph-mon[34351]: pgmap v459: 97 pgs: 97 active+clean; 1.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:28:14.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:14 smithi196 ceph-mon[35041]: pgmap v459: 97 pgs: 97 active+clean; 1.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:28:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.957+0000 7fbcccdf9700 1 -- 172.21.15.19:0/1000355971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc81034e0 msgr2=0x7fbcc81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.957+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/1000355971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc81034e0 0x7fbcc81038f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800b0d0 tx=0x7fbcb800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.957+0000 7fbcccdf9700 1 -- 172.21.15.19:0/1000355971 shutdown_connections 2024-05-09T01:28:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.957+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/1000355971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcc8108760 0x7fbcc81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.957+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/1000355971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc81034e0 0x7fbcc81038f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.957+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/1000355971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc8107e10 0x7fbcc8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.957+0000 7fbcccdf9700 1 -- 172.21.15.19:0/1000355971 >> 172.21.15.19:0/1000355971 conn(0x7fbcc80fef30 msgr2=0x7fbcc8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.957+0000 7fbcccdf9700 1 -- 172.21.15.19:0/1000355971 shutdown_connections 2024-05-09T01:28:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.957+0000 7fbcccdf9700 1 -- 172.21.15.19:0/1000355971 wait complete. 2024-05-09T01:28:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.958+0000 7fbcccdf9700 1 Processor -- start 2024-05-09T01:28:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.958+0000 7fbcccdf9700 1 -- start start 2024-05-09T01:28:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcccdf9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81034e0 0x7fbcc8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcccdf9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcc8107e10 0x7fbcc819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcccdf9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc8108760 0x7fbcc819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcccdf9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8113390 con 0x7fbcc8108760 2024-05-09T01:28:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcccdf9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8113210 con 0x7fbcc81034e0 2024-05-09T01:28:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcccdf9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8113510 con 0x7fbcc8107e10 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc8108760 0x7fbcc819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc8108760 0x7fbcc819e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34582/0 (socket says 172.21.15.19:34582) 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/46331958 learned_addr learned my addr 172.21.15.19:0/46331958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc659c700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81034e0 0x7fbcc8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc5d9b700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcc8107e10 0x7fbcc819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/46331958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcc8107e10 msgr2=0x7fbcc819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcc8107e10 0x7fbcc819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/46331958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81034e0 msgr2=0x7fbcc8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81034e0 0x7fbcc8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.959+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/46331958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc819ec10 con 0x7fbcc8108760 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.960+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc8108760 0x7fbcc819e570 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000d980 tx=0x7fbcc000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.960+0000 7fbcbf7fe700 1 -- 172.21.15.19:0/46331958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0017070 con 0x7fbcc8108760 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.960+0000 7fbcccdf9700 1 -- 172.21.15.19:0/46331958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc819ef00 con 0x7fbcc8108760 2024-05-09T01:28:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.960+0000 7fbcbf7fe700 1 -- 172.21.15.19:0/46331958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc0005020 con 0x7fbcc8108760 2024-05-09T01:28:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.960+0000 7fbcbf7fe700 1 -- 172.21.15.19:0/46331958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000fa10 con 0x7fbcc8108760 2024-05-09T01:28:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.960+0000 7fbcccdf9700 1 -- 172.21.15.19:0/46331958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc81a6820 con 0x7fbcc8108760 2024-05-09T01:28:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.963+0000 7fbcbf7fe700 1 -- 172.21.15.19:0/46331958 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbcc000fbb0 con 0x7fbcc8108760 2024-05-09T01:28:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.963+0000 7fbcbf7fe700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbcb40b2880 0x7fbcb40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.964+0000 7fbcc659c700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbcb40b2880 0x7fbcb40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.964+0000 7fbcbf7fe700 1 -- 172.21.15.19:0/46331958 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbcc00e1fd0 con 0x7fbcc8108760 2024-05-09T01:28:14.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.964+0000 7fbcdb80fb80 1 Processor -- start 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.964+0000 7fbcc659c700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbcb40b2880 0x7fbcb40b4d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009fd0 tx=0x7fbcb800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.964+0000 7fbcdb80fb80 1 -- start start 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.964+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cace76d00 0x558cacf7b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.964+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cacdfc4a0 0x558cacf7be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.964+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558cace9c0a0 con 0x558cacf7ffe0 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558cace419a0 con 0x558cacdfc4a0 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558cace98060 con 0x558cace76d00 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf803f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34586/0 (socket says 172.21.15.19:34586) 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/158468915 learned_addr learned my addr 172.21.15.19:0/158468915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc5d9b700 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cacdfc4a0 0x558cacf7be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc659c700 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cace76d00 0x558cacf7b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/158468915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cace76d00 msgr2=0x558cacf7b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cace76d00 0x558cacf7b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/158468915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cacdfc4a0 msgr2=0x558cacf7be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cacdfc4a0 0x558cacf7be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/158468915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cace32500 con 0x558cacf7ffe0 2024-05-09T01:28:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc5d9b700 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cacdfc4a0 0x558cacf7be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf803f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00a6840 tx=0x7fbcc000ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f92994704d0fbbe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.965+0000 7fbcbcff9700 1 -- 172.21.15.19:0/158468915 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00a95d0 con 0x558cacf7ffe0 2024-05-09T01:28:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.966+0000 7fbcbcff9700 1 -- 172.21.15.19:0/158468915 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc00ad980 con 0x558cacf7ffe0 2024-05-09T01:28:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.966+0000 7fbcbcff9700 1 -- 172.21.15.19:0/158468915 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00b2200 con 0x558cacf7ffe0 2024-05-09T01:28:14.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.966+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/158468915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 msgr2=0x558cacf803f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:14.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.966+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf803f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00a6840 tx=0x7fbcc000ffd0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.966+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/158468915 shutdown_connections 2024-05-09T01:28:14.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.966+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf803f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.966+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cace76d00 0x558cacf7b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.966+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/158468915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cacdfc4a0 0x558cacf7be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.966+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/158468915 >> 172.21.15.19:0/158468915 conn(0x558cacdfff70 msgr2=0x558cacf74220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:14.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/158468915 shutdown_connections 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/158468915 wait complete. 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 -- start start 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 Processor -- start 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 0x558cacf852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf85810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf85d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558cacf80d00 con 0x558cacf7ffe0 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558cace36b20 con 0x558cace76d00 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.967+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558cacf88270 con 0x558cacdfc4a0 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcdb80fb80 10 client.0 _ll_register_callbacks cb 0x558cacd0d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf85d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf85d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34592/0 (socket says 172.21.15.19:34592) 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/1666499785 learned_addr learned my addr 172.21.15.19:0/1666499785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:14.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc659c700 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 0x558cacf852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc5d9b700 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf85810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/1666499785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 msgr2=0x558cacf852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 0x558cacf852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/1666499785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 msgr2=0x558cacf85810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf85810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/1666499785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cacd17490 con 0x558cacf7ffe0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf85d50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00a9560 tx=0x7fbcc0005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66323f6e40fd50de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcadffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcadffb700 1 -- 172.21.15.19:0/1666499785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0014040 con 0x558cacf7ffe0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcadffb700 1 -- 172.21.15.19:0/1666499785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbcc00a9a40 con 0x558cacf7ffe0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/1666499785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc00ad980 con 0x558cacf7ffe0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.968+0000 7fbcadffb700 1 -- 172.21.15.19:0/1666499785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbcc00ca050 con 0x558cacf7ffe0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcdb80fb80 10 client.15816 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/1666499785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558cacf92c70 con 0x558cacf7ffe0 2024-05-09T01:28:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcadffb700 1 -- 172.21.15.19:0/1666499785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc00ad980 con 0x558cacf7ffe0 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcadffb700 1 -- 172.21.15.19:0/1666499785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbcc00a8830 con 0x558cacf7ffe0 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcdb80fb80 10 client.15816 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcdb80fb80 10 client.15816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcdb80fb80 20 client.15816 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558cacf95140 0x558cacf97560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcdb80fb80 4 client.15816 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/1666499785 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cacea9a80 con 0x558cacf95140 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.969+0000 7fbcc659c700 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558cacf95140 0x558cacf97560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.970+0000 7fbcc659c700 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558cacf95140 0x558cacf97560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=765175b7e2caf0cc server_cookie=d95f4c7b19b27c03 in_seq=0 out_seq=0 2024-05-09T01:28:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.970+0000 7fbcadffb700 10 client.15816 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:28:14.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.970+0000 7fbcadffb700 1 -- 172.21.15.19:0/1666499785 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558cacea9a80 con 0x558cacf95140 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.970+0000 7fbcadffb700 10 client.15816 handle_command_reply: tid=0 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.970+0000 7fbcdb80fb80 1 client.15816 shutdown 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/1666499785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 msgr2=0x558cacf85d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf85d50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00a9560 tx=0x7fbcc0005700 comp rx=0 tx=0).stop 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/1666499785 shutdown_connections 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558cacf95140 0x558cacf97560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf85d50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 0x558cacf852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/1666499785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf85810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/1666499785 >> 172.21.15.19:0/1666499785 conn(0x558cacdfff70 msgr2=0x558cacf783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcadffb700 0 client.15816 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/1666499785 shutdown_connections 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/1666499785 wait complete. 2024-05-09T01:28:14.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.971+0000 7fbcdb80fb80 20 client.15816 trim_cache size 0 max 16384 2024-05-09T01:28:14.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.986+0000 7fbcdb80fb80 1 Processor -- start 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.986+0000 7fbcdb80fb80 1 -- start start 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.986+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacdfc4a0 0x558cacf85b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.986+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf6e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.986+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacf7ffe0 0x558cacf783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.986+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81acbc0 con 0x558cacdfc4a0 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.986+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81ae700 con 0x558cace76d00 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.986+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81ad290 con 0x558cacf7ffe0 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacdfc4a0 0x558cacf85b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacdfc4a0 0x558cacf85b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34602/0 (socket says 172.21.15.19:34602) 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc659c700 1 -- 172.21.15.19:0/3685216448 learned_addr learned my addr 172.21.15.19:0/3685216448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc5d9b700 1 --2- 172.21.15.19:0/3685216448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf6e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/3685216448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacf7ffe0 0x558cacf783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc659c700 1 -- 172.21.15.19:0/3685216448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacf7ffe0 msgr2=0x558cacf783f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc659c700 1 --2- 172.21.15.19:0/3685216448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacf7ffe0 0x558cacf783f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc659c700 1 -- 172.21.15.19:0/3685216448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 msgr2=0x558cacf6e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc659c700 1 --2- 172.21.15.19:0/3685216448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf6e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc659c700 1 -- 172.21.15.19:0/3685216448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cacf746b0 con 0x558cacdfc4a0 2024-05-09T01:28:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcc659c700 1 --2- 172.21.15.19:0/3685216448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacdfc4a0 0x558cacf85b90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800b010 tx=0x7fbcb8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e8aae018b4be2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcaffff700 1 -- 172.21.15.19:0/3685216448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8022070 con 0x558cacdfc4a0 2024-05-09T01:28:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcaffff700 1 -- 172.21.15.19:0/3685216448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb8007b70 con 0x558cacdfc4a0 2024-05-09T01:28:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.987+0000 7fbcaffff700 1 -- 172.21.15.19:0/3685216448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8029430 con 0x558cacdfc4a0 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/3685216448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacdfc4a0 msgr2=0x558cacf85b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/3685216448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacdfc4a0 0x558cacf85b90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800b010 tx=0x7fbcb8022000 comp rx=0 tx=0).stop 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/3685216448 shutdown_connections 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/3685216448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacf7ffe0 0x558cacf783f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/3685216448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacdfc4a0 0x558cacf85b90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/3685216448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf6e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/3685216448 >> 172.21.15.19:0/3685216448 conn(0x558cacdfff70 msgr2=0x558cacf6dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/3685216448 shutdown_connections 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/3685216448 wait complete. 2024-05-09T01:28:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.988+0000 7fbcdb80fb80 1 -- start start 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcdb80fb80 1 Processor -- start 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 0x558cacf95140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf95680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcdb80fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf95bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81ae030 con 0x558cacf7ffe0 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81ad960 con 0x558cace76d00 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcdb80fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81aab60 con 0x558cacdfc4a0 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcdb80fb80 10 client.0 _ll_register_callbacks cb 0x558cacf7fef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf95bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf95bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34616/0 (socket says 172.21.15.19:34616) 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/607007613 learned_addr learned my addr 172.21.15.19:0/607007613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc5d9b700 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf95680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc659c700 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 0x558cacf95140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/607007613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 msgr2=0x558cacf95140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 0x558cacf95140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/607007613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 msgr2=0x558cacf95680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf95680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.989+0000 7fbcc6d9d700 1 -- 172.21.15.19:0/607007613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cacffd720 con 0x558cacf7ffe0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcc6d9d700 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf95bc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00a8c30 tx=0x7fbcc00a5050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153281a178ef7494 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcadffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcadffb700 1 -- 172.21.15.19:0/607007613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0014040 con 0x558cacf7ffe0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcadffb700 1 -- 172.21.15.19:0/607007613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbcc00a8830 con 0x558cacf7ffe0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/607007613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb8007b70 con 0x558cacf7ffe0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcadffb700 1 -- 172.21.15.19:0/607007613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbcc00a5490 con 0x558cacf7ffe0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcdb80fb80 10 client.15822 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/607007613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558cacffe9a0 con 0x558cacf7ffe0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcadffb700 1 -- 172.21.15.19:0/607007613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb8007b70 con 0x558cacf7ffe0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcadffb700 1 -- 172.21.15.19:0/607007613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbcc00b1fa0 con 0x558cacf7ffe0 2024-05-09T01:28:14.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcdb80fb80 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcdb80fb80 10 client.15822 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcdb80fb80 20 client.15822 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.990+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558cacffee00 0x558cad001220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.991+0000 7fbcdb80fb80 4 client.15822 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.991+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/607007613 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cacea3b20 con 0x558cacffee00 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.991+0000 7fbcc659c700 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558cacffee00 0x558cad001220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.991+0000 7fbcadffb700 10 client.15822 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.991+0000 7fbcc659c700 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558cacffee00 0x558cad001220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6a86d2cfc916527 server_cookie=1ced3da843216b88 in_seq=0 out_seq=0 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.991+0000 7fbcadffb700 1 -- 172.21.15.19:0/607007613 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cacea3b20 con 0x558cacffee00 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.991+0000 7fbcadffb700 10 client.15822 handle_command_reply: tid=0 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.991+0000 7fbcdb80fb80 1 client.15822 shutdown 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/607007613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 msgr2=0x558cacf95bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf95bc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00a8c30 tx=0x7fbcc00a5050 comp rx=0 tx=0).stop 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/607007613 shutdown_connections 2024-05-09T01:28:14.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558cacffee00 0x558cad001220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cacf7ffe0 0x558cacf95bc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cacdfc4a0 0x558cacf95140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 --2- 172.21.15.19:0/607007613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558cace76d00 0x558cacf95680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/607007613 >> 172.21.15.19:0/607007613 conn(0x558cacdfff70 msgr2=0x558cacf783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcadffb700 0 client.15822 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:28:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/607007613 shutdown_connections 2024-05-09T01:28:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 1 -- 172.21.15.19:0/607007613 wait complete. 2024-05-09T01:28:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcdb80fb80 20 client.15822 trim_cache size 0 max 16384 2024-05-09T01:28:14.994 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcccdf9700 1 -- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbcb40b2880 msgr2=0x7fbcb40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbcb40b2880 0x7fbcb40b4d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009fd0 tx=0x7fbcb800b040 comp rx=0 tx=0).stop 2024-05-09T01:28:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcccdf9700 1 -- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc8108760 msgr2=0x7fbcc819e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.992+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc8108760 0x7fbcc819e570 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000d980 tx=0x7fbcc000dc90 comp rx=0 tx=0).stop 2024-05-09T01:28:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.993+0000 7fbcccdf9700 1 -- 172.21.15.19:0/46331958 shutdown_connections 2024-05-09T01:28:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.993+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbcb40b2880 0x7fbcb40b4d90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.993+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcc8108760 0x7fbcc819e570 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.993+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81034e0 0x7fbcc8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.993+0000 7fbcccdf9700 1 --2- 172.21.15.19:0/46331958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcc8107e10 0x7fbcc819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.993+0000 7fbcccdf9700 1 -- 172.21.15.19:0/46331958 >> 172.21.15.19:0/46331958 conn(0x7fbcc80fef30 msgr2=0x7fbcc81096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.993+0000 7fbcccdf9700 1 -- 172.21.15.19:0/46331958 shutdown_connections 2024-05-09T01:28:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:14.993+0000 7fbcccdf9700 1 -- 172.21.15.19:0/46331958 wait complete. 2024-05-09T01:28:15.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:28:16.077 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:16 smithi059 ceph-mon[34351]: pgmap v460: 97 pgs: 97 active+clean; 1.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:28:16.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:16 smithi019 ceph-mon[30158]: pgmap v460: 97 pgs: 97 active+clean; 1.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:28:16.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:16 smithi196 ceph-mon[35041]: pgmap v460: 97 pgs: 97 active+clean; 1.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:28:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.750+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/1351043346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4108760 msgr2=0x7fbfa41100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.750+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/1351043346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4108760 0x7fbfa41100c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00b0d0 tx=0x7fbf9c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.750+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/1351043346 shutdown_connections 2024-05-09T01:28:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.750+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/1351043346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4108760 0x7fbfa41100c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.750+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/1351043346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfa41034e0 0x7fbfa41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.750+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/1351043346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4107e10 0x7fbfa4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.750+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/1351043346 >> 172.21.15.19:0/1351043346 conn(0x7fbfa40fef30 msgr2=0x7fbfa4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.750+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/1351043346 shutdown_connections 2024-05-09T01:28:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.750+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/1351043346 wait complete. 2024-05-09T01:28:17.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.751+0000 7fbfa9bdd700 1 Processor -- start 2024-05-09T01:28:17.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.751+0000 7fbfa9bdd700 1 -- start start 2024-05-09T01:28:17.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.751+0000 7fbfa9bdd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfa41034e0 0x7fbfa419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.751+0000 7fbfa9bdd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4107e10 0x7fbfa419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.751+0000 7fbfa9bdd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4108760 0x7fbfa41a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.751+0000 7fbfa9bdd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4113390 con 0x7fbfa4107e10 2024-05-09T01:28:17.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.751+0000 7fbfa9bdd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4113210 con 0x7fbfa4108760 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.751+0000 7fbfa9bdd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4113510 con 0x7fbfa41034e0 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4107e10 0x7fbfa419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4107e10 0x7fbfa419c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34636/0 (socket says 172.21.15.19:34636) 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa2ffd700 1 -- 172.21.15.19:0/424627389 learned_addr learned my addr 172.21.15.19:0/424627389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfa41034e0 0x7fbfa419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa3fff700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4108760 0x7fbfa41a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa2ffd700 1 -- 172.21.15.19:0/424627389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfa41034e0 msgr2=0x7fbfa419bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfa41034e0 0x7fbfa419bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa2ffd700 1 -- 172.21.15.19:0/424627389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4108760 msgr2=0x7fbfa41a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4108760 0x7fbfa41a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa2ffd700 1 -- 172.21.15.19:0/424627389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa419caa0 con 0x7fbfa4107e10 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa3fff700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4108760 0x7fbfa41a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.752+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4107e10 0x7fbfa419c1f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800d980 tx=0x7fbf9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:17.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.753+0000 7fbfa0ff9700 1 -- 172.21.15.19:0/424627389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98017070 con 0x7fbfa4107e10 2024-05-09T01:28:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.753+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/424627389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa41a7390 con 0x7fbfa4107e10 2024-05-09T01:28:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.753+0000 7fbfa0ff9700 1 -- 172.21.15.19:0/424627389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf98009de0 con 0x7fbfa4107e10 2024-05-09T01:28:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.753+0000 7fbfa0ff9700 1 -- 172.21.15.19:0/424627389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800f3c0 con 0x7fbfa4107e10 2024-05-09T01:28:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.753+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/424627389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa41a78d0 con 0x7fbfa4107e10 2024-05-09T01:28:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.755+0000 7fbfa0ff9700 1 -- 172.21.15.19:0/424627389 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbf98009930 con 0x7fbfa4107e10 2024-05-09T01:28:17.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.756+0000 7fbfa0ff9700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbf8c0b2880 0x7fbf8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.756+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbf8c0b2880 0x7fbf8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.756+0000 7fbfa0ff9700 1 -- 172.21.15.19:0/424627389 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbf980e20c0 con 0x7fbfa4107e10 2024-05-09T01:28:17.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfb85f3b80 1 Processor -- start 2024-05-09T01:28:17.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfb85f3b80 1 -- start start 2024-05-09T01:28:17.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbf8c0b2880 0x7fbf8c0b4d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:17.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984afcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56298497b4a0 0x562984afd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b015f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562984a1b0a0 con 0x5629849f5d00 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5629849c09a0 con 0x56298497b4a0 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562984a17060 con 0x562984b011e0 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984afcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984afcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34638/0 (socket says 172.21.15.19:34638) 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfa37fe700 1 -- 172.21.15.19:0/3018911070 learned_addr learned my addr 172.21.15.19:0/3018911070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfa3fff700 1 --2- 172.21.15.19:0/3018911070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.757+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/3018911070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56298497b4a0 0x562984afd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.758+0000 7fbfa37fe700 1 -- 172.21.15.19:0/3018911070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 msgr2=0x562984b015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.758+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/3018911070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.758+0000 7fbfa37fe700 1 -- 172.21.15.19:0/3018911070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56298497b4a0 msgr2=0x562984afd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.758+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/3018911070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56298497b4a0 0x562984afd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.758+0000 7fbfa37fe700 1 -- 172.21.15.19:0/3018911070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629849b1500 con 0x5629849f5d00 2024-05-09T01:28:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.758+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/3018911070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984afcad0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbf94010fa0 tx=0x7fbf94013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20961388b76123ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:17.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.758+0000 7fbf89ffb700 1 -- 172.21.15.19:0/3018911070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400be60 con 0x5629849f5d00 2024-05-09T01:28:17.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.758+0000 7fbf89ffb700 1 -- 172.21.15.19:0/3018911070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf94011db0 con 0x5629849f5d00 2024-05-09T01:28:17.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.758+0000 7fbf89ffb700 1 -- 172.21.15.19:0/3018911070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9401c770 con 0x5629849f5d00 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/3018911070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 msgr2=0x562984afcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/3018911070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984afcad0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbf94010fa0 tx=0x7fbf94013000 comp rx=0 tx=0).stop 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/3018911070 shutdown_connections 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/3018911070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/3018911070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984afcad0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/3018911070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56298497b4a0 0x562984afd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/3018911070 >> 172.21.15.19:0/3018911070 conn(0x56298497ef70 msgr2=0x562984af3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/3018911070 shutdown_connections 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/3018911070 wait complete. 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.759+0000 7fbfb85f3b80 1 -- start start 2024-05-09T01:28:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfb85f3b80 1 Processor -- start 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984b06440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 0x562984b06980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b06ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562984b01f00 con 0x56298497b4a0 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5629849b5b20 con 0x5629849f5d00 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562984b09470 con 0x562984b011e0 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfb85f3b80 10 client.0 _ll_register_callbacks cb 0x56298488c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984b06440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984b06440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34648/0 (socket says 172.21.15.19:34648) 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfa37fe700 1 -- 172.21.15.19:0/2988057457 learned_addr learned my addr 172.21.15.19:0/2988057457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.760+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 0x562984b06980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfa3fff700 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b06ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfa37fe700 1 -- 172.21.15.19:0/2988057457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 msgr2=0x562984b06ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b06ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfa37fe700 1 -- 172.21.15.19:0/2988057457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 msgr2=0x562984b06980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 0x562984b06980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfa37fe700 1 -- 172.21.15.19:0/2988057457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562984896490 con 0x56298497b4a0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984b06440 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbf9401eec0 tx=0x7fbf9401e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83ded671265b2a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbf82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2988057457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400be60 con 0x56298497b4a0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2988057457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbf94011bb0 con 0x56298497b4a0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2988057457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf94011db0 con 0x56298497b4a0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2988057457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf9401e8b0 con 0x56298497b4a0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfb85f3b80 10 client.15834 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2988057457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562984b13d80 con 0x56298497b4a0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.761+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2988057457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf94011db0 con 0x56298497b4a0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.762+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2988057457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbf9403c4b0 con 0x56298497b4a0 2024-05-09T01:28:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.762+0000 7fbfb85f3b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:17.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.762+0000 7fbfb85f3b80 10 client.15834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:28:17.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.762+0000 7fbfb85f3b80 20 client.15834 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:17.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.762+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x562984b16250 0x562984b18670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.762+0000 7fbfb85f3b80 4 client.15834 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:17.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.762+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2988057457 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562984a28a80 con 0x562984b16250 2024-05-09T01:28:17.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.762+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x562984b16250 0x562984b18670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.763+0000 7fbf82ffd700 10 client.15834 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:28:17.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.763+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x562984b16250 0x562984b18670 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fbce9fdb85abac2 server_cookie=1b6ee70fae4c586b in_seq=0 out_seq=0 2024-05-09T01:28:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.763+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2988057457 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562984a28a80 con 0x562984b16250 2024-05-09T01:28:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.763+0000 7fbf82ffd700 10 client.15834 handle_command_reply: tid=0 2024-05-09T01:28:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.763+0000 7fbfb85f3b80 1 client.15834 shutdown 2024-05-09T01:28:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2988057457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 msgr2=0x562984b06440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984b06440 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbf9401eec0 tx=0x7fbf9401e650 comp rx=0 tx=0).stop 2024-05-09T01:28:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2988057457 shutdown_connections 2024-05-09T01:28:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x562984b16250 0x562984b18670 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b06ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984b06440 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2988057457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 0x562984b06980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbf82ffd700 0 client.15834 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:28:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2988057457 >> 172.21.15.19:0/2988057457 conn(0x56298497ef70 msgr2=0x562984af73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2988057457 shutdown_connections 2024-05-09T01:28:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2988057457 wait complete. 2024-05-09T01:28:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.764+0000 7fbfb85f3b80 20 client.15834 trim_cache size 0 max 16384 2024-05-09T01:28:17.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.779+0000 7fbfb85f3b80 1 Processor -- start 2024-05-09T01:28:17.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.779+0000 7fbfb85f3b80 1 -- start start 2024-05-09T01:28:17.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.779+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984aeda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.779+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 0x562984b06d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.779+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b05310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.779+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41acba0 con 0x56298497b4a0 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.779+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41ae6e0 con 0x5629849f5d00 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.779+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41ad270 con 0x562984b011e0 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.779+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984aeda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984aeda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34662/0 (socket says 172.21.15.19:34662) 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa37fe700 1 -- 172.21.15.19:0/2576286345 learned_addr learned my addr 172.21.15.19:0/2576286345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 0x562984b06d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa3fff700 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b05310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa37fe700 1 -- 172.21.15.19:0/2576286345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 msgr2=0x562984b05310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b05310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa37fe700 1 -- 172.21.15.19:0/2576286345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 msgr2=0x562984b06d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:17.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 0x562984b06d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa37fe700 1 -- 172.21.15.19:0/2576286345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562984aef9d0 con 0x56298497b4a0 2024-05-09T01:28:17.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 0x562984b06d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:17.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984aeda20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005a80 tx=0x7fbf9400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38efb977a134c264 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:17.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbf897fa700 1 -- 172.21.15.19:0/2576286345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9401ea50 con 0x56298497b4a0 2024-05-09T01:28:17.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbf897fa700 1 -- 172.21.15.19:0/2576286345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf940054f0 con 0x56298497b4a0 2024-05-09T01:28:17.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.780+0000 7fbf897fa700 1 -- 172.21.15.19:0/2576286345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9403cc80 con 0x56298497b4a0 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2576286345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 msgr2=0x562984aeda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984aeda20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005a80 tx=0x7fbf9400bf80 comp rx=0 tx=0).stop 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2576286345 shutdown_connections 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562984b011e0 0x562984b05310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298497b4a0 0x562984aeda20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2576286345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629849f5d00 0x562984b06d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2576286345 >> 172.21.15.19:0/2576286345 conn(0x56298497ef70 msgr2=0x562984b04c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2576286345 shutdown_connections 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2576286345 wait complete. 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.781+0000 7fbfb85f3b80 1 -- start start 2024-05-09T01:28:17.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfb85f3b80 1 Processor -- start 2024-05-09T01:28:17.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56298497b4a0 0x562984b0e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984b0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfb85f3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562984b011e0 0x562984aff010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41ae010 con 0x5629849f5d00 2024-05-09T01:28:17.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41ad940 con 0x562984b011e0 2024-05-09T01:28:17.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfb85f3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41aab60 con 0x56298497b4a0 2024-05-09T01:28:17.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfb85f3b80 10 client.0 _ll_register_callbacks cb 0x56298488c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:17.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984b0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984b0ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34678/0 (socket says 172.21.15.19:34678) 2024-05-09T01:28:17.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfa2ffd700 1 -- 172.21.15.19:0/2581092864 learned_addr learned my addr 172.21.15.19:0/2581092864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56298497b4a0 0x562984b0e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.782+0000 7fbfa3fff700 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562984b011e0 0x562984aff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbfa2ffd700 1 -- 172.21.15.19:0/2581092864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56298497b4a0 msgr2=0x562984b0e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56298497b4a0 0x562984b0e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbfa2ffd700 1 -- 172.21.15.19:0/2581092864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562984b011e0 msgr2=0x562984aff010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562984b011e0 0x562984aff010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbfa2ffd700 1 -- 172.21.15.19:0/2581092864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562984aff710 con 0x5629849f5d00 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbfa2ffd700 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984b0ea70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009520 tx=0x7fbf98009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25ab07cd6896d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbf82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2581092864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980b24c0 con 0x5629849f5d00 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2581092864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbf980a5c50 con 0x5629849f5d00 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2581092864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf940054f0 con 0x5629849f5d00 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2581092864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf980106a0 con 0x5629849f5d00 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbfb85f3b80 10 client.15840 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2581092864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562984b1db50 con 0x5629849f5d00 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.783+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2581092864 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf940054f0 con 0x5629849f5d00 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2581092864 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbf9808fbd0 con 0x5629849f5d00 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbfb85f3b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbfb85f3b80 10 client.15840 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbfb85f3b80 20 client.15840 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x562984b809c0 0x562984b82de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbfb85f3b80 4 client.15840 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2581092864 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562984a22b20 con 0x562984b809c0 2024-05-09T01:28:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x562984b809c0 0x562984b82de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbf82ffd700 10 client.15840 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:28:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.784+0000 7fbfa37fe700 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x562984b809c0 0x562984b82de0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=876da87442ecc1 server_cookie=f5c6fddb5f708231 in_seq=0 out_seq=0 2024-05-09T01:28:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbf82ffd700 1 -- 172.21.15.19:0/2581092864 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562984a22b20 con 0x562984b809c0 2024-05-09T01:28:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbf82ffd700 10 client.15840 handle_command_reply: tid=0 2024-05-09T01:28:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 client.15840 shutdown 2024-05-09T01:28:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2581092864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 msgr2=0x562984b0ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984b0ea70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009520 tx=0x7fbf98009c20 comp rx=0 tx=0).stop 2024-05-09T01:28:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2581092864 shutdown_connections 2024-05-09T01:28:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x562984b809c0 0x562984b82de0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562984b011e0 0x562984aff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56298497b4a0 0x562984b0e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 --2- 172.21.15.19:0/2581092864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629849f5d00 0x562984b0ea70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbf82ffd700 0 client.15840 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:28:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2581092864 >> 172.21.15.19:0/2581092864 conn(0x56298497ef70 msgr2=0x562984aed780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2581092864 shutdown_connections 2024-05-09T01:28:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 1 -- 172.21.15.19:0/2581092864 wait complete. 2024-05-09T01:28:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.785+0000 7fbfb85f3b80 20 client.15840 trim_cache size 0 max 16384 2024-05-09T01:28:17.789 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:17.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbf8c0b2880 msgr2=0x7fbf8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:17.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbf8c0b2880 0x7fbf8c0b4d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400b410 comp rx=0 tx=0).stop 2024-05-09T01:28:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4107e10 msgr2=0x7fbfa419c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4107e10 0x7fbfa419c1f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800d980 tx=0x7fbf9800dc90 comp rx=0 tx=0).stop 2024-05-09T01:28:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/424627389 shutdown_connections 2024-05-09T01:28:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4108760 0x7fbfa41a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfa41034e0 0x7fbfa419bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbf8c0b2880 0x7fbf8c0b4d90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 --2- 172.21.15.19:0/424627389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa4107e10 0x7fbfa419c1f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/424627389 >> 172.21.15.19:0/424627389 conn(0x7fbfa40fef30 msgr2=0x7fbfa4107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/424627389 shutdown_connections 2024-05-09T01:28:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:17.786+0000 7fbfa9bdd700 1 -- 172.21.15.19:0/424627389 wait complete. 2024-05-09T01:28:18.595 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:28:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:18 smithi059 ceph-mon[34351]: pgmap v461: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:18.848 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:18 smithi019 ceph-mon[30158]: pgmap v461: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:18.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:18 smithi196 ceph-mon[35041]: pgmap v461: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.300+0000 7fb949557640 1 -- 172.21.15.19:0/3862026179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944108a90 msgr2=0x7fb94410aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.300+0000 7fb949557640 1 --2- 172.21.15.19:0/3862026179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944108a90 0x7fb94410aeb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb934009900 tx=0x7fb93402a410 comp rx=0 tx=0).stop 2024-05-09T01:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.300+0000 7fb949557640 1 -- 172.21.15.19:0/3862026179 shutdown_connections 2024-05-09T01:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.300+0000 7fb949557640 1 --2- 172.21.15.19:0/3862026179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944108a90 0x7fb94410aeb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.300+0000 7fb949557640 1 --2- 172.21.15.19:0/3862026179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb944103d90 0x7fb9441041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.300+0000 7fb949557640 1 --2- 172.21.15.19:0/3862026179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb944102b50 0x7fb944103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.300+0000 7fb949557640 1 -- 172.21.15.19:0/3862026179 >> 172.21.15.19:0/3862026179 conn(0x7fb9440fdd40 msgr2=0x7fb9441001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.300+0000 7fb949557640 1 -- 172.21.15.19:0/3862026179 shutdown_connections 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.300+0000 7fb949557640 1 -- 172.21.15.19:0/3862026179 wait complete. 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.301+0000 7fb949557640 1 Processor -- start 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.301+0000 7fb949557640 1 -- start start 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.301+0000 7fb949557640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb944102b50 0x7fb9440ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.301+0000 7fb949557640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944103d90 0x7fb9440fe270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.301+0000 7fb949557640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb944108a90 0x7fb9440fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.301+0000 7fb949557640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb94410e660 con 0x7fb944103d90 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.301+0000 7fb949557640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb94410e4e0 con 0x7fb944108a90 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.301+0000 7fb949557640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb94410e7e0 con 0x7fb944102b50 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.301+0000 7fb9427fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944103d90 0x7fb9440fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb9427fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944103d90 0x7fb9440fe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51330/0 (socket says 172.21.15.19:51330) 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb9427fc640 1 -- 172.21.15.19:0/2762795233 learned_addr learned my addr 172.21.15.19:0/2762795233 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb942ffd640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb944102b50 0x7fb9440ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb9437fe640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb944108a90 0x7fb9440fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb9427fc640 1 -- 172.21.15.19:0/2762795233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb944102b50 msgr2=0x7fb9440ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb9427fc640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb944102b50 0x7fb9440ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb9427fc640 1 -- 172.21.15.19:0/2762795233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb944108a90 msgr2=0x7fb9440fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb9427fc640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb944108a90 0x7fb9440fe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb9427fc640 1 -- 172.21.15.19:0/2762795233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9440feee0 con 0x7fb944103d90 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb9427fc640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944103d90 0x7fb9440fe270 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00d950 tx=0x7fb92c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb93bfff640 1 -- 172.21.15.19:0/2762795233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c017070 con 0x7fb944103d90 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb949557640 1 -- 172.21.15.19:0/2762795233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9441a43b0 con 0x7fb944103d90 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb93bfff640 1 -- 172.21.15.19:0/2762795233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92c009c30 con 0x7fb944103d90 2024-05-09T01:28:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb93bfff640 1 -- 172.21.15.19:0/2762795233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00f620 con 0x7fb944103d90 2024-05-09T01:28:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.302+0000 7fb949557640 1 -- 172.21.15.19:0/2762795233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9441a48a0 con 0x7fb944103d90 2024-05-09T01:28:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.305+0000 7fb950409740 1 Processor -- start 2024-05-09T01:28:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.305+0000 7fb950409740 1 -- start start 2024-05-09T01:28:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb950409740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3de500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb950409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb950409740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0194d0 0x55a6ad3e5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb950409740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a6ad3427c0 con 0x55a6ad03a1b0 2024-05-09T01:28:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb950409740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6ad2de0e0 con 0x55a6ad0194d0 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb950409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a6ad33e780 con 0x55a6ad0112f0 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb9437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb9437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3de500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51338/0 (socket says 172.21.15.19:51338) 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb9437fe640 1 -- 172.21.15.19:0/180060425 learned_addr learned my addr 172.21.15.19:0/180060425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb942ffd640 1 --2- 172.21.15.19:0/180060425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb93bfff640 1 -- 172.21.15.19:0/2762795233 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb92c010460 con 0x7fb944103d90 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb9437fe640 1 -- 172.21.15.19:0/180060425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0112f0 msgr2=0x55a6ad3dea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb9437fe640 1 --2- 172.21.15.19:0/180060425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb9437fe640 1 -- 172.21.15.19:0/180060425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0194d0 msgr2=0x55a6ad3e5980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb9437fe640 1 --2- 172.21.15.19:0/180060425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0194d0 0x55a6ad3e5980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb9437fe640 1 -- 172.21.15.19:0/180060425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6ad3077c0 con 0x55a6ad03a1b0 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.306+0000 7fb942ffd640 1 --2- 172.21.15.19:0/180060425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.307+0000 7fb9437fe640 1 --2- 172.21.15.19:0/180060425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3de500 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb934009900 tx=0x7fb934032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bde5edd38d4dde5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.307+0000 7fb9397fa640 1 -- 172.21.15.19:0/180060425 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93403f070 con 0x55a6ad03a1b0 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.307+0000 7fb9397fa640 1 -- 172.21.15.19:0/180060425 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb934004040 con 0x55a6ad03a1b0 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.307+0000 7fb9397fa640 1 -- 172.21.15.19:0/180060425 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934035070 con 0x55a6ad03a1b0 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.307+0000 7fb93bfff640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9300b27e0 0x7fb9300b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.307+0000 7fb942ffd640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9300b27e0 0x7fb9300b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.307+0000 7fb93bfff640 1 -- 172.21.15.19:0/2762795233 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb92c0e1e80 con 0x7fb944103d90 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 -- 172.21.15.19:0/180060425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 msgr2=0x55a6ad3de500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 --2- 172.21.15.19:0/180060425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3de500 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb934009900 tx=0x7fb934032ba0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb942ffd640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9300b27e0 0x7fb9300b4cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009fd0 tx=0x7fb93c009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 -- 172.21.15.19:0/180060425 shutdown_connections 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 --2- 172.21.15.19:0/180060425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0194d0 0x55a6ad3e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 --2- 172.21.15.19:0/180060425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 --2- 172.21.15.19:0/180060425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3de500 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 -- 172.21.15.19:0/180060425 >> 172.21.15.19:0/180060425 conn(0x55a6ad02e300 msgr2=0x55a6ad3d9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 -- 172.21.15.19:0/180060425 shutdown_connections 2024-05-09T01:28:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 -- 172.21.15.19:0/180060425 wait complete. 2024-05-09T01:28:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.308+0000 7fb950409740 1 -- start start 2024-05-09T01:28:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb950409740 1 Processor -- start 2024-05-09T01:28:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb950409740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 0x55a6ad3e8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb950409740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0194d0 0x55a6ad3e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb950409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb950409740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a6ad3e6370 con 0x55a6ad0194d0 2024-05-09T01:28:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb950409740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6ad2c99c0 con 0x55a6ad0112f0 2024-05-09T01:28:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb950409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a6ad3e64f0 con 0x55a6ad03a1b0 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb950409740 10 client.0 _ll_register_callbacks cb 0x55a6ad2f6020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb9427fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0194d0 0x55a6ad3e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb9427fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0194d0 0x55a6ad3e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51340/0 (socket says 172.21.15.19:51340) 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb9427fc640 1 -- 172.21.15.19:0/1435964350 learned_addr learned my addr 172.21.15.19:0/1435964350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb942ffd640 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 0x55a6ad3e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb9437fe640 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb9427fc640 1 -- 172.21.15.19:0/1435964350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad03a1b0 msgr2=0x55a6ad3ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb9427fc640 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb9427fc640 1 -- 172.21.15.19:0/1435964350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 msgr2=0x55a6ad3e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb9427fc640 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 0x55a6ad3e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.309+0000 7fb9427fc640 1 -- 172.21.15.19:0/1435964350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6ad2f01f0 con 0x55a6ad0194d0 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb9427fc640 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0194d0 0x55a6ad3e88a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0a9180 tx=0x7fb92c0a8bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31417c5bbf91bc6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb92a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb92a7fc640 1 -- 172.21.15.19:0/1435964350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c020070 con 0x55a6ad0194d0 2024-05-09T01:28:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb950409740 1 -- 172.21.15.19:0/1435964350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb934004040 con 0x55a6ad0194d0 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb92a7fc640 1 -- 172.21.15.19:0/1435964350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb92c0b0dc0 con 0x55a6ad0194d0 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb92a7fc640 1 -- 172.21.15.19:0/1435964350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb92c0a9b80 con 0x55a6ad0194d0 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb950409740 10 client.15852 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb950409740 1 -- 172.21.15.19:0/1435964350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6ad0a5790 con 0x55a6ad0194d0 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb92a7fc640 1 -- 172.21.15.19:0/1435964350 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb934004040 con 0x55a6ad0194d0 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb92a7fc640 1 -- 172.21.15.19:0/1435964350 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb92c0a3cc0 con 0x55a6ad0194d0 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb950409740 10 client.15852 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb950409740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb950409740 20 client.15852 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb950409740 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a6acff8d80 0x55a6ad3f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb950409740 4 client.15852 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.310+0000 7fb950409740 1 -- 172.21.15.19:0/1435964350 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6ad3503d0 con 0x55a6acff8d80 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.311+0000 7fb942ffd640 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a6acff8d80 0x55a6ad3f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.311+0000 7fb942ffd640 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a6acff8d80 0x55a6ad3f5c60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d0299bf5793d3b server_cookie=af2ced01a04707cf in_seq=0 out_seq=0 2024-05-09T01:28:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.311+0000 7fb92a7fc640 10 client.15852 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb92a7fc640 1 -- 172.21.15.19:0/1435964350 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6ad3503d0 con 0x55a6acff8d80 2024-05-09T01:28:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb92a7fc640 10 client.15852 handle_command_reply: tid=0 2024-05-09T01:28:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 client.15852 shutdown 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 -- 172.21.15.19:0/1435964350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0194d0 msgr2=0x55a6ad3e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0194d0 0x55a6ad3e88a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0a9180 tx=0x7fb92c0a8bf0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 -- 172.21.15.19:0/1435964350 shutdown_connections 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a6acff8d80 0x55a6ad3f5c60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0194d0 0x55a6ad3e88a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb92a7fc640 0 client.15852 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 --2- 172.21.15.19:0/1435964350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 0x55a6ad3e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 -- 172.21.15.19:0/1435964350 >> 172.21.15.19:0/1435964350 conn(0x55a6ad02e300 msgr2=0x55a6ad3dcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.312+0000 7fb950409740 1 -- 172.21.15.19:0/1435964350 shutdown_connections 2024-05-09T01:28:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.313+0000 7fb950409740 1 -- 172.21.15.19:0/1435964350 wait complete. 2024-05-09T01:28:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.313+0000 7fb950409740 20 client.15852 trim_cache size 0 max 16384 2024-05-09T01:28:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb950409740 1 Processor -- start 2024-05-09T01:28:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb950409740 1 -- start start 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb950409740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb950409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 0x55a6ad3dd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb950409740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb950409740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441ad6a0 con 0x55a6ad0112f0 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb950409740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441af1e0 con 0x55a6ad03a1b0 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb950409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441add70 con 0x55a6ad0194d0 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb942ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb942ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51344/0 (socket says 172.21.15.19:51344) 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb942ffd640 1 -- 172.21.15.19:0/2261502601 learned_addr learned my addr 172.21.15.19:0/2261502601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.328+0000 7fb9437fe640 1 --2- 172.21.15.19:0/2261502601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb9427fc640 1 --2- 172.21.15.19:0/2261502601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 0x55a6ad3dd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb942ffd640 1 -- 172.21.15.19:0/2261502601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 msgr2=0x55a6ad3dd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb942ffd640 1 --2- 172.21.15.19:0/2261502601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 0x55a6ad3dd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb942ffd640 1 -- 172.21.15.19:0/2261502601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad03a1b0 msgr2=0x55a6ad3ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb942ffd640 1 --2- 172.21.15.19:0/2261502601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb942ffd640 1 -- 172.21.15.19:0/2261502601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6ad11c5f0 con 0x55a6ad0112f0 2024-05-09T01:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb942ffd640 1 --2- 172.21.15.19:0/2261502601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dcc90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb944102420 tx=0x7fb93c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f057f791fd210edf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb938ff9640 1 -- 172.21.15.19:0/2261502601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c02e070 con 0x55a6ad0112f0 2024-05-09T01:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb938ff9640 1 -- 172.21.15.19:0/2261502601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb93c0298d0 con 0x55a6ad0112f0 2024-05-09T01:28:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.329+0000 7fb938ff9640 1 -- 172.21.15.19:0/2261502601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c01caa0 con 0x55a6ad0112f0 2024-05-09T01:28:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 -- 172.21.15.19:0/2261502601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0112f0 msgr2=0x55a6ad3dcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 --2- 172.21.15.19:0/2261502601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dcc90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb944102420 tx=0x7fb93c016000 comp rx=0 tx=0).stop 2024-05-09T01:28:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 -- 172.21.15.19:0/2261502601 shutdown_connections 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 --2- 172.21.15.19:0/2261502601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 --2- 172.21.15.19:0/2261502601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 0x55a6ad3dd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 --2- 172.21.15.19:0/2261502601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad0112f0 0x55a6ad3dcc90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 -- 172.21.15.19:0/2261502601 >> 172.21.15.19:0/2261502601 conn(0x55a6ad02e300 msgr2=0x55a6ad3e6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 -- 172.21.15.19:0/2261502601 shutdown_connections 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 -- 172.21.15.19:0/2261502601 wait complete. 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.330+0000 7fb950409740 1 -- start start 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb950409740 1 Processor -- start 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb950409740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 0x55a6ad3eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb950409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 0x55a6ad3ee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb950409740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb950409740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441aeb10 con 0x55a6ad03a1b0 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb950409740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441ae440 con 0x55a6ad0112f0 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb950409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441ab680 con 0x55a6ad0194d0 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb950409740 10 client.0 _ll_register_callbacks cb 0x7fb92c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb9437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb9437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51350/0 (socket says 172.21.15.19:51350) 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb9437fe640 1 -- 172.21.15.19:0/299425454 learned_addr learned my addr 172.21.15.19:0/299425454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb9427fc640 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 0x55a6ad3ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb942ffd640 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 0x55a6ad3eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb9437fe640 1 -- 172.21.15.19:0/299425454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 msgr2=0x55a6ad3ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb9437fe640 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 0x55a6ad3ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb9437fe640 1 -- 172.21.15.19:0/299425454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 msgr2=0x55a6ad3eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb9437fe640 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 0x55a6ad3eddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.331+0000 7fb9437fe640 1 -- 172.21.15.19:0/299425454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6ad306ed0 con 0x55a6ad03a1b0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.332+0000 7fb9437fe640 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef230 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb934009900 tx=0x7fb93402aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cc728db3bb475b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.332+0000 7fb92a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.332+0000 7fb92a7fc640 1 -- 172.21.15.19:0/299425454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934053060 con 0x55a6ad03a1b0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.332+0000 7fb92a7fc640 1 -- 172.21.15.19:0/299425454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb934004030 con 0x55a6ad03a1b0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.332+0000 7fb950409740 1 -- 172.21.15.19:0/299425454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb93c0298d0 con 0x55a6ad03a1b0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.332+0000 7fb92a7fc640 1 -- 172.21.15.19:0/299425454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb93403f040 con 0x55a6ad03a1b0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.332+0000 7fb950409740 10 client.15858 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.332+0000 7fb950409740 1 -- 172.21.15.19:0/299425454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6ad45e3c0 con 0x55a6ad03a1b0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.333+0000 7fb92a7fc640 1 -- 172.21.15.19:0/299425454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb93c0298d0 con 0x55a6ad03a1b0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.333+0000 7fb92a7fc640 1 -- 172.21.15.19:0/299425454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb934032000 con 0x55a6ad03a1b0 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.333+0000 7fb950409740 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.333+0000 7fb950409740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.333+0000 7fb950409740 20 client.15858 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.333+0000 7fb950409740 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a6acfce1c0 0x55a6ad460a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.333+0000 7fb950409740 4 client.15858 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:28:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.333+0000 7fb950409740 1 -- 172.21.15.19:0/299425454 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6ad34a240 con 0x55a6acfce1c0 2024-05-09T01:28:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.333+0000 7fb942ffd640 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a6acfce1c0 0x55a6ad460a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.334+0000 7fb92a7fc640 10 client.15858 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.334+0000 7fb942ffd640 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a6acfce1c0 0x55a6ad460a10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41bd8e7000afc668 server_cookie=efa02c2fbaecdc29 in_seq=0 out_seq=0 2024-05-09T01:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb92a7fc640 1 -- 172.21.15.19:0/299425454 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a6ad34a240 con 0x55a6acfce1c0 2024-05-09T01:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb92a7fc640 10 client.15858 handle_command_reply: tid=0 2024-05-09T01:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 client.15858 shutdown 2024-05-09T01:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 -- 172.21.15.19:0/299425454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 msgr2=0x55a6ad3ef230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef230 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb934009900 tx=0x7fb93402aa90 comp rx=0 tx=0).stop 2024-05-09T01:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 -- 172.21.15.19:0/299425454 shutdown_connections 2024-05-09T01:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a6acfce1c0 0x55a6ad460a10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6ad03a1b0 0x55a6ad3ef230 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6ad0194d0 0x55a6ad3ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 --2- 172.21.15.19:0/299425454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6ad0112f0 0x55a6ad3eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 -- 172.21.15.19:0/299425454 >> 172.21.15.19:0/299425454 conn(0x55a6ad02e300 msgr2=0x55a6ad3dcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb92a7fc640 0 client.15858 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.350+0000 7fb950409740 1 -- 172.21.15.19:0/299425454 shutdown_connections 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb950409740 1 -- 172.21.15.19:0/299425454 wait complete. 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb950409740 20 client.15858 trim_cache size 0 max 16384 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:28:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:28:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 -- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9300b27e0 msgr2=0x7fb9300b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9300b27e0 0x7fb9300b4cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009fd0 tx=0x7fb93c009670 comp rx=0 tx=0).stop 2024-05-09T01:28:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 -- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944103d90 msgr2=0x7fb9440fe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944103d90 0x7fb9440fe270 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00d950 tx=0x7fb92c00de10 comp rx=0 tx=0).stop 2024-05-09T01:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 -- 172.21.15.19:0/2762795233 shutdown_connections 2024-05-09T01:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9300b27e0 0x7fb9300b4cf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb944108a90 0x7fb9440fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb944103d90 0x7fb9440fe270 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 --2- 172.21.15.19:0/2762795233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb944102b50 0x7fb9440ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.351+0000 7fb949557640 1 -- 172.21.15.19:0/2762795233 >> 172.21.15.19:0/2762795233 conn(0x7fb9440fdd40 msgr2=0x7fb944109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.352+0000 7fb949557640 1 -- 172.21.15.19:0/2762795233 shutdown_connections 2024-05-09T01:28:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:20.352+0000 7fb949557640 1 -- 172.21.15.19:0/2762795233 wait complete. 2024-05-09T01:28:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.571+0000 7f20a4a49700 1 -- 172.21.15.19:0/1272040429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 msgr2=0x7f20a0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.571+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1272040429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 0x7f20a0108220 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f20880099c0 tx=0x7f2088009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.571+0000 7f20a4a49700 1 -- 172.21.15.19:0/1272040429 shutdown_connections 2024-05-09T01:28:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.571+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1272040429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20a0108760 0x7f20a01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.571+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1272040429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a01034e0 0x7f20a01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.571+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1272040429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 0x7f20a0108220 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.571+0000 7f20a4a49700 1 -- 172.21.15.19:0/1272040429 >> 172.21.15.19:0/1272040429 conn(0x7f20a00fef30 msgr2=0x7f20a0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.571+0000 7f20a4a49700 1 -- 172.21.15.19:0/1272040429 shutdown_connections 2024-05-09T01:28:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.571+0000 7f20a4a49700 1 -- 172.21.15.19:0/1272040429 wait complete. 2024-05-09T01:28:20.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f20a4a49700 1 Processor -- start 2024-05-09T01:28:20.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f20a4a49700 1 -- start start 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f20a4a49700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a01034e0 0x7f20a019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f20a4a49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 0x7f20a019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f20a4a49700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20a0108760 0x7f20a019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f20a4a49700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20a0113370 con 0x7f20a0107e10 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f20a4a49700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20a01131f0 con 0x7f20a0108760 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f20a4a49700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20a01134f0 con 0x7f20a01034e0 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f2095d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 0x7f20a019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.572+0000 7f2095d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 0x7f20a019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51360/0 (socket says 172.21.15.19:51360) 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2095d9b700 1 -- 172.21.15.19:0/1955103555 learned_addr learned my addr 172.21.15.19:0/1955103555 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f209ed9d700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20a0108760 0x7f20a019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f209e59c700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a01034e0 0x7f20a019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2095d9b700 1 -- 172.21.15.19:0/1955103555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a01034e0 msgr2=0x7f20a019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2095d9b700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a01034e0 0x7f20a019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2095d9b700 1 -- 172.21.15.19:0/1955103555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20a0108760 msgr2=0x7f20a019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2095d9b700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20a0108760 0x7f20a019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2095d9b700 1 -- 172.21.15.19:0/1955103555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a019fc30 con 0x7f20a0107e10 2024-05-09T01:28:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2095d9b700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 0x7f20a019b3b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f20880098e0 tx=0x7f2088003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2097fff700 1 -- 172.21.15.19:0/1955103555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088027070 con 0x7f20a0107e10 2024-05-09T01:28:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2097fff700 1 -- 172.21.15.19:0/1955103555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20880040a0 con 0x7f20a0107e10 2024-05-09T01:28:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f2097fff700 1 -- 172.21.15.19:0/1955103555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208800f460 con 0x7f20a0107e10 2024-05-09T01:28:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f20a4a49700 1 -- 172.21.15.19:0/1955103555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a019fec0 con 0x7f20a0107e10 2024-05-09T01:28:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.573+0000 7f20a4a49700 1 -- 172.21.15.19:0/1955103555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a01a03f0 con 0x7f20a0107e10 2024-05-09T01:28:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.576+0000 7f2097fff700 1 -- 172.21.15.19:0/1955103555 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f208802a070 con 0x7f20a0107e10 2024-05-09T01:28:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.577+0000 7f2097fff700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f208c0b2900 0x7f208c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.577+0000 7f209e59c700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f208c0b2900 0x7f208c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.577+0000 7f2097fff700 1 -- 172.21.15.19:0/1955103555 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f20880e3200 con 0x7f20a0107e10 2024-05-09T01:28:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.577+0000 7f20b345fb80 1 Processor -- start 2024-05-09T01:28:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.577+0000 7f209e59c700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f208c0b2900 0x7f208c0b4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2090005ea0 tx=0x7f2090005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.577+0000 7f20b345fb80 1 -- start start 2024-05-09T01:28:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 0x555f297734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29773a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 0x555f2976dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f20b345fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555f2968fff0 con 0x555f295f04a0 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f20b345fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f2962e7f0 con 0x555f2976d6a0 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f20b345fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f2968bfb0 con 0x555f296da160 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f2095d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29773a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f2095d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29773a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51364/0 (socket says 172.21.15.19:51364) 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f2095d9b700 1 -- 172.21.15.19:0/2794615877 learned_addr learned my addr 172.21.15.19:0/2794615877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f209ed9d700 1 --2- 172.21.15.19:0/2794615877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 0x555f2976dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f209e59c700 1 --2- 172.21.15.19:0/2794615877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 0x555f297734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f2095d9b700 1 -- 172.21.15.19:0/2794615877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 msgr2=0x555f297734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f2095d9b700 1 --2- 172.21.15.19:0/2794615877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 0x555f297734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f2095d9b700 1 -- 172.21.15.19:0/2794615877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 msgr2=0x555f2976dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f2095d9b700 1 --2- 172.21.15.19:0/2794615877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 0x555f2976dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.578+0000 7f2095d9b700 1 -- 172.21.15.19:0/2794615877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f29626370 con 0x555f295f04a0 2024-05-09T01:28:20.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.579+0000 7f2095d9b700 1 --2- 172.21.15.19:0/2794615877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29773a30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f20880aa530 tx=0x7f2088009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4852404afee7357 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.579+0000 7f2094d99700 1 -- 172.21.15.19:0/2794615877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880aa340 con 0x555f295f04a0 2024-05-09T01:28:20.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.579+0000 7f2094d99700 1 -- 172.21.15.19:0/2794615877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20880b1110 con 0x555f295f04a0 2024-05-09T01:28:20.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.579+0000 7f2094d99700 1 -- 172.21.15.19:0/2794615877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880aacf0 con 0x555f295f04a0 2024-05-09T01:28:20.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 -- 172.21.15.19:0/2794615877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 msgr2=0x555f29773a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 --2- 172.21.15.19:0/2794615877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29773a30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f20880aa530 tx=0x7f2088009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 -- 172.21.15.19:0/2794615877 shutdown_connections 2024-05-09T01:28:20.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 --2- 172.21.15.19:0/2794615877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 0x555f2976dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 --2- 172.21.15.19:0/2794615877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 0x555f297734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 --2- 172.21.15.19:0/2794615877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29773a30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 -- 172.21.15.19:0/2794615877 >> 172.21.15.19:0/2794615877 conn(0x555f296506a0 msgr2=0x555f29768400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 -- 172.21.15.19:0/2794615877 shutdown_connections 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 -- 172.21.15.19:0/2794615877 wait complete. 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 -- start start 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.580+0000 7f20b345fb80 1 Processor -- start 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29777140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 0x555f29777680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 0x555f29777bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f20b345fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555f2950bba0 con 0x555f295f04a0 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f20b345fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f2964a150 con 0x555f2976d6a0 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f20b345fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f2976e360 con 0x555f296da160 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f20b345fb80 10 client.0 _ll_register_callbacks cb 0x555f295016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:20.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29777140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29777140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51374/0 (socket says 172.21.15.19:51374) 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209e59c700 1 -- 172.21.15.19:0/4000124440 learned_addr learned my addr 172.21.15.19:0/4000124440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209ed9d700 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 0x555f29777bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f2095d9b700 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 0x555f29777680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209e59c700 1 -- 172.21.15.19:0/4000124440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 msgr2=0x555f29777680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209e59c700 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 0x555f29777680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209e59c700 1 -- 172.21.15.19:0/4000124440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 msgr2=0x555f29777bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209e59c700 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 0x555f29777bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209e59c700 1 -- 172.21.15.19:0/4000124440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2950b490 con 0x555f295f04a0 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209ed9d700 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 0x555f29777bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f209e59c700 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29777140 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f20900109b0 tx=0x7f20900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a684202e3eab8f4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f2085ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.581+0000 7f2085ffb700 1 -- 172.21.15.19:0/4000124440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209000fc60 con 0x555f295f04a0 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f20b345fb80 1 -- 172.21.15.19:0/4000124440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20880b1110 con 0x555f295f04a0 2024-05-09T01:28:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f2085ffb700 1 -- 172.21.15.19:0/4000124440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f209000fe00 con 0x555f295f04a0 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f2085ffb700 1 -- 172.21.15.19:0/4000124440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2090010e20 con 0x555f295f04a0 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f20b345fb80 10 client.15870 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f20b345fb80 1 -- 172.21.15.19:0/4000124440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f29788dc0 con 0x555f295f04a0 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f2085ffb700 1 -- 172.21.15.19:0/4000124440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20880b1110 con 0x555f295f04a0 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f2085ffb700 1 -- 172.21.15.19:0/4000124440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f209001d4e0 con 0x555f295f04a0 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f20b345fb80 10 client.15870 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f20b345fb80 10 client.15870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f20b345fb80 20 client.15870 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f20b345fb80 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f2978b400 0x555f2978d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f20b345fb80 4 client.15870 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.582+0000 7f20b345fb80 1 -- 172.21.15.19:0/4000124440 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f2969d9d0 con 0x555f2978b400 2024-05-09T01:28:20.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.583+0000 7f2095d9b700 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f2978b400 0x555f2978d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.583+0000 7f2095d9b700 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f2978b400 0x555f2978d840 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8e40a6be343b62f server_cookie=1b544a9b088b3f7d in_seq=0 out_seq=0 2024-05-09T01:28:20.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.583+0000 7f2085ffb700 10 client.15870 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:28:20.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f2085ffb700 1 -- 172.21.15.19:0/4000124440 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f2969d9d0 con 0x555f2978b400 2024-05-09T01:28:20.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f2085ffb700 10 client.15870 handle_command_reply: tid=0 2024-05-09T01:28:20.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 client.15870 shutdown 2024-05-09T01:28:20.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 -- 172.21.15.19:0/4000124440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 msgr2=0x555f29777140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29777140 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f20900109b0 tx=0x7f20900047c0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 -- 172.21.15.19:0/4000124440 shutdown_connections 2024-05-09T01:28:20.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f2978b400 0x555f2978d840 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f2976d6a0 0x555f29777bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f2085ffb700 0 client.15870 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:28:20.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f295f04a0 0x555f29777140 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 --2- 172.21.15.19:0/4000124440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f296da160 0x555f29777680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 -- 172.21.15.19:0/4000124440 >> 172.21.15.19:0/4000124440 conn(0x555f296506a0 msgr2=0x555f297681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 -- 172.21.15.19:0/4000124440 shutdown_connections 2024-05-09T01:28:20.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 1 -- 172.21.15.19:0/4000124440 wait complete. 2024-05-09T01:28:20.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.584+0000 7f20b345fb80 20 client.15870 trim_cache size 0 max 16384 2024-05-09T01:28:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.599+0000 7f20b345fb80 1 Processor -- start 2024-05-09T01:28:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.599+0000 7f20b345fb80 1 -- start start 2024-05-09T01:28:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296506a0 0x555f29771860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 0x555f29772930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:20 smithi019 ceph-mon[30158]: pgmap v462: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296da160 0x555f29777a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f20b345fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20a01ac940 con 0x555f296506a0 2024-05-09T01:28:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f20b345fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20a01ae480 con 0x555f296da160 2024-05-09T01:28:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f20b345fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20a01ad010 con 0x555f295f04a0 2024-05-09T01:28:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296506a0 0x555f29771860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296506a0 0x555f29771860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51382/0 (socket says 172.21.15.19:51382) 2024-05-09T01:28:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209e59c700 1 -- 172.21.15.19:0/1229424636 learned_addr learned my addr 172.21.15.19:0/1229424636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209ed9d700 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296da160 0x555f29777a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f2095d9b700 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 0x555f29772930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209e59c700 1 -- 172.21.15.19:0/1229424636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 msgr2=0x555f29772930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209e59c700 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 0x555f29772930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209e59c700 1 -- 172.21.15.19:0/1229424636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296da160 msgr2=0x555f29777a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209e59c700 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296da160 0x555f29777a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209e59c700 1 -- 172.21.15.19:0/1229424636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f29769150 con 0x555f296506a0 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.600+0000 7f209ed9d700 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296da160 0x555f29777a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.601+0000 7f209e59c700 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296506a0 0x555f29771860 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f209001d950 tx=0x7f209000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38458ce85b3671ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.601+0000 7f2087fff700 1 -- 172.21.15.19:0/1229424636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209000f6a0 con 0x555f296506a0 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.601+0000 7f2087fff700 1 -- 172.21.15.19:0/1229424636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f209001db40 con 0x555f296506a0 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.601+0000 7f2087fff700 1 -- 172.21.15.19:0/1229424636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090031690 con 0x555f296506a0 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 -- 172.21.15.19:0/1229424636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296506a0 msgr2=0x555f29771860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296506a0 0x555f29771860 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f209001d950 tx=0x7f209000a2d0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 -- 172.21.15.19:0/1229424636 shutdown_connections 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296da160 0x555f29777a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296506a0 0x555f29771860 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 --2- 172.21.15.19:0/1229424636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 0x555f29772930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 -- 172.21.15.19:0/1229424636 >> 172.21.15.19:0/1229424636 conn(0x555f2976de10 msgr2=0x555f29772220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 -- 172.21.15.19:0/1229424636 shutdown_connections 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 -- 172.21.15.19:0/1229424636 wait complete. 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 -- start start 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 Processor -- start 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 0x555f29783bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296506a0 0x555f297925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.602+0000 7f20b345fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296da160 0x555f29792bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f20b345fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20a01addb0 con 0x555f296da160 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f20b345fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20a01ad6e0 con 0x555f296506a0 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f20b345fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20a01aa900 con 0x555f295f04a0 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f20b345fb80 10 client.0 _ll_register_callbacks cb 0x555f295016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296da160 0x555f29792bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296da160 0x555f29792bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51396/0 (socket says 172.21.15.19:51396) 2024-05-09T01:28:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209ed9d700 1 -- 172.21.15.19:0/3576976580 learned_addr learned my addr 172.21.15.19:0/3576976580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f2095d9b700 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296506a0 0x555f297925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209e59c700 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 0x555f29783bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209ed9d700 1 -- 172.21.15.19:0/3576976580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 msgr2=0x555f29783bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209ed9d700 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 0x555f29783bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209ed9d700 1 -- 172.21.15.19:0/3576976580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296506a0 msgr2=0x555f297925d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209ed9d700 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296506a0 0x555f297925d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209ed9d700 1 -- 172.21.15.19:0/3576976580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f29793340 con 0x555f296da160 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f209ed9d700 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296da160 0x555f29792bd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2098015e30 tx=0x7f20980135b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5298331fc62c4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f2085ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f2085ffb700 1 -- 172.21.15.19:0/3576976580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209801b070 con 0x555f296da160 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f2085ffb700 1 -- 172.21.15.19:0/3576976580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2098013bc0 con 0x555f296da160 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.603+0000 7f20b345fb80 1 -- 172.21.15.19:0/3576976580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f209001db40 con 0x555f296da160 2024-05-09T01:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f2085ffb700 1 -- 172.21.15.19:0/3576976580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2098013dc0 con 0x555f296da160 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f20b345fb80 10 client.15876 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f20b345fb80 1 -- 172.21.15.19:0/3576976580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f297f5c50 con 0x555f296da160 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f2085ffb700 1 -- 172.21.15.19:0/3576976580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f209001db40 con 0x555f296da160 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f2085ffb700 1 -- 172.21.15.19:0/3576976580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2098020460 con 0x555f296da160 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f20b345fb80 10 client.15876 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f20b345fb80 10 client.15876 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f20b345fb80 20 client.15876 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f20b345fb80 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f2976d6a0 0x555f297f8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f20b345fb80 4 client.15876 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f20b345fb80 1 -- 172.21.15.19:0/3576976580 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f29697a70 con 0x555f2976d6a0 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.604+0000 7f209e59c700 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f2976d6a0 0x555f297f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f2085ffb700 10 client.15876 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f209e59c700 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f2976d6a0 0x555f297f8380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a310bd043392639c server_cookie=c662a7285411258b in_seq=0 out_seq=0 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f2085ffb700 1 -- 172.21.15.19:0/3576976580 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f29697a70 con 0x555f2976d6a0 2024-05-09T01:28:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f2085ffb700 10 client.15876 handle_command_reply: tid=0 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 client.15876 shutdown 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 -- 172.21.15.19:0/3576976580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296da160 msgr2=0x555f29792bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296da160 0x555f29792bd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2098015e30 tx=0x7f20980135b0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 -- 172.21.15.19:0/3576976580 shutdown_connections 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f296da160 0x555f29792bd0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f295f04a0 0x555f29783bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f2976d6a0 0x555f297f8380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 --2- 172.21.15.19:0/3576976580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f296506a0 0x555f297925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f2085ffb700 0 client.15876 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 -- 172.21.15.19:0/3576976580 >> 172.21.15.19:0/3576976580 conn(0x555f2976de10 msgr2=0x555f29770e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 -- 172.21.15.19:0/3576976580 shutdown_connections 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.605+0000 7f20b345fb80 1 -- 172.21.15.19:0/3576976580 wait complete. 2024-05-09T01:28:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20b345fb80 20 client.15876 trim_cache size 0 max 16384 2024-05-09T01:28:20.610 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:20.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 -- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f208c0b2900 msgr2=0x7f208c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f208c0b2900 0x7f208c0b4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2090005ea0 tx=0x7f2090005e30 comp rx=0 tx=0).stop 2024-05-09T01:28:20.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 -- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 msgr2=0x7f20a019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 0x7f20a019b3b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f20880098e0 tx=0x7f2088003680 comp rx=0 tx=0).stop 2024-05-09T01:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 -- 172.21.15.19:0/1955103555 shutdown_connections 2024-05-09T01:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20a0108760 0x7f20a019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a01034e0 0x7f20a019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f208c0b2900 0x7f208c0b4e10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 --2- 172.21.15.19:0/1955103555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0107e10 0x7f20a019b3b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 -- 172.21.15.19:0/1955103555 >> 172.21.15.19:0/1955103555 conn(0x7f20a00fef30 msgr2=0x7f20a006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 -- 172.21.15.19:0/1955103555 shutdown_connections 2024-05-09T01:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:20.606+0000 7f20a4a49700 1 -- 172.21.15.19:0/1955103555 wait complete. 2024-05-09T01:28:20.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:20 smithi059 ceph-mon[34351]: pgmap v462: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:20.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:20 smithi196 ceph-mon[35041]: pgmap v462: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:21.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:28:21.719 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:22 smithi059 ceph-mon[34351]: pgmap v463: 97 pgs: 97 active+clean; 1.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:28:22.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:22 smithi019 ceph-mon[30158]: pgmap v463: 97 pgs: 97 active+clean; 1.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:28:22.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:22 smithi196 ceph-mon[35041]: pgmap v463: 97 pgs: 97 active+clean; 1.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:28:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.342+0000 7f235867b700 1 -- 172.21.15.19:0/425821530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2350107df0 msgr2=0x7f2350108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.342+0000 7f235867b700 1 --2- 172.21.15.19:0/425821530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2350107df0 0x7f2350108200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f234c0099c0 tx=0x7f234c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.342+0000 7f235867b700 1 -- 172.21.15.19:0/425821530 shutdown_connections 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.342+0000 7f235867b700 1 --2- 172.21.15.19:0/425821530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2350108740 0x7f23501100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.342+0000 7f235867b700 1 --2- 172.21.15.19:0/425821530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23501034c0 0x7f23501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.342+0000 7f235867b700 1 --2- 172.21.15.19:0/425821530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2350107df0 0x7f2350108200 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.342+0000 7f235867b700 1 -- 172.21.15.19:0/425821530 >> 172.21.15.19:0/425821530 conn(0x7f23500fedf0 msgr2=0x7f2350101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.342+0000 7f235867b700 1 -- 172.21.15.19:0/425821530 shutdown_connections 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.342+0000 7f235867b700 1 -- 172.21.15.19:0/425821530 wait complete. 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f235867b700 1 Processor -- start 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f235867b700 1 -- start start 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f235867b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23501034c0 0x7f235019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f235867b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2350107df0 0x7f235019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f235867b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2350108740 0x7f235019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f235867b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2350113370 con 0x7f23501034c0 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f235867b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23501131f0 con 0x7f2350108740 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f235867b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23501134f0 con 0x7f2350107df0 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f2356417700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23501034c0 0x7f235019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f2356417700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23501034c0 0x7f235019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51426/0 (socket says 172.21.15.19:51426) 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.343+0000 7f2356417700 1 -- 172.21.15.19:0/67245767 learned_addr learned my addr 172.21.15.19:0/67245767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f2355c16700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2350107df0 0x7f235019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f2356c18700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2350108740 0x7f235019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f2356417700 1 -- 172.21.15.19:0/67245767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2350107df0 msgr2=0x7f235019c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f2356417700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2350107df0 0x7f235019c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f2356417700 1 -- 172.21.15.19:0/67245767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2350108740 msgr2=0x7f235019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f2356417700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2350108740 0x7f235019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f2356417700 1 -- 172.21.15.19:0/67245767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23501a6fe0 con 0x7f23501034c0 2024-05-09T01:28:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f2355c16700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2350107df0 0x7f235019c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f2356417700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23501034c0 0x7f235019bc90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f234400e9a0 tx=0x7f234400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f23437fe700 1 -- 172.21.15.19:0/67245767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234400cbd0 con 0x7f23501034c0 2024-05-09T01:28:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f23437fe700 1 -- 172.21.15.19:0/67245767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f234400cd70 con 0x7f23501034c0 2024-05-09T01:28:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f23437fe700 1 -- 172.21.15.19:0/67245767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344018680 con 0x7f23501034c0 2024-05-09T01:28:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f235867b700 1 -- 172.21.15.19:0/67245767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23501a72d0 con 0x7f23501034c0 2024-05-09T01:28:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.344+0000 7f235867b700 1 -- 172.21.15.19:0/67245767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23501a77e0 con 0x7f23501034c0 2024-05-09T01:28:23.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.347+0000 7f23437fe700 1 -- 172.21.15.19:0/67245767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2344010970 con 0x7f23501034c0 2024-05-09T01:28:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f23437fe700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f233c0b2880 0x7f233c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2355c16700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f233c0b2880 0x7f233c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2367091b80 1 Processor -- start 2024-05-09T01:28:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f23437fe700 1 -- 172.21.15.19:0/67245767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f23440e25f0 con 0x7f23501034c0 2024-05-09T01:28:23.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2367091b80 1 -- start start 2024-05-09T01:28:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2367091b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a45864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2355c16700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f233c0b2880 0x7f233c0b4d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f234c000c00 tx=0x7f234c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2367091b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44034a0 0x55f9a4586a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2367091b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a45806a0 0x55f9a4580ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2367091b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f9a44a2ff0 con 0x55f9a44ed160 2024-05-09T01:28:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2367091b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f9a44417f0 con 0x55f9a45806a0 2024-05-09T01:28:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.348+0000 7f2367091b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f9a449efb0 con 0x55f9a44034a0 2024-05-09T01:28:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356417700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a45864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356417700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a45864f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51432/0 (socket says 172.21.15.19:51432) 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356417700 1 -- 172.21.15.19:0/1283067366 learned_addr learned my addr 172.21.15.19:0/1283067366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356c18700 1 --2- 172.21.15.19:0/1283067366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a45806a0 0x55f9a4580ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2355c16700 1 --2- 172.21.15.19:0/1283067366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44034a0 0x55f9a4586a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356417700 1 -- 172.21.15.19:0/1283067366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44034a0 msgr2=0x55f9a4586a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356417700 1 --2- 172.21.15.19:0/1283067366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44034a0 0x55f9a4586a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356417700 1 -- 172.21.15.19:0/1283067366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a45806a0 msgr2=0x55f9a4580ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356417700 1 --2- 172.21.15.19:0/1283067366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a45806a0 0x55f9a4580ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356417700 1 -- 172.21.15.19:0/1283067366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a4439370 con 0x55f9a44ed160 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2356417700 1 --2- 172.21.15.19:0/1283067366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a45864f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f23440a6e60 tx=0x7f23440a9430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82734598b225c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2340ff9700 1 -- 172.21.15.19:0/1283067366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440aa2f0 con 0x55f9a44ed160 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2340ff9700 1 -- 172.21.15.19:0/1283067366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23440a9bf0 con 0x55f9a44ed160 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.349+0000 7f2340ff9700 1 -- 172.21.15.19:0/1283067366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440e20e0 con 0x55f9a44ed160 2024-05-09T01:28:23.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.350+0000 7f2367091b80 1 -- 172.21.15.19:0/1283067366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 msgr2=0x55f9a45864f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.350+0000 7f2367091b80 1 --2- 172.21.15.19:0/1283067366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a45864f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f23440a6e60 tx=0x7f23440a9430 comp rx=0 tx=0).stop 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.350+0000 7f2367091b80 1 -- 172.21.15.19:0/1283067366 shutdown_connections 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.350+0000 7f2367091b80 1 --2- 172.21.15.19:0/1283067366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a45806a0 0x55f9a4580ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.350+0000 7f2367091b80 1 --2- 172.21.15.19:0/1283067366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a45864f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.350+0000 7f2367091b80 1 --2- 172.21.15.19:0/1283067366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44034a0 0x55f9a4586a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.350+0000 7f2367091b80 1 -- 172.21.15.19:0/1283067366 >> 172.21.15.19:0/1283067366 conn(0x55f9a44636a0 msgr2=0x55f9a457b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.350+0000 7f2367091b80 1 -- 172.21.15.19:0/1283067366 shutdown_connections 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.350+0000 7f2367091b80 1 -- 172.21.15.19:0/1283067366 wait complete. 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2367091b80 1 -- start start 2024-05-09T01:28:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2367091b80 1 Processor -- start 2024-05-09T01:28:23.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2367091b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 0x55f9a458a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2367091b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44ed160 0x55f9a458a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2367091b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a45806a0 0x55f9a458acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2367091b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f9a431eba0 con 0x55f9a45806a0 2024-05-09T01:28:23.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2367091b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f9a445d150 con 0x55f9a44034a0 2024-05-09T01:28:23.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2367091b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f9a458d1d0 con 0x55f9a44ed160 2024-05-09T01:28:23.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2367091b80 10 client.0 _ll_register_callbacks cb 0x55f9a43146c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:23.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2356c18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a45806a0 0x55f9a458acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2356417700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 0x55f9a458a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2356c18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a45806a0 0x55f9a458acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51440/0 (socket says 172.21.15.19:51440) 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.351+0000 7f2356c18700 1 -- 172.21.15.19:0/1945379906 learned_addr learned my addr 172.21.15.19:0/1945379906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2355c16700 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44ed160 0x55f9a458a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2356c18700 1 -- 172.21.15.19:0/1945379906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44ed160 msgr2=0x55f9a458a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2356c18700 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44ed160 0x55f9a458a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2356c18700 1 -- 172.21.15.19:0/1945379906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 msgr2=0x55f9a458a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2356c18700 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 0x55f9a458a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2356c18700 1 -- 172.21.15.19:0/1945379906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a431e490 con 0x55f9a45806a0 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2356417700 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 0x55f9a458a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2355c16700 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44ed160 0x55f9a458a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2356c18700 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a45806a0 0x55f9a458acb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2348009fd0 tx=0x7f2348007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4182a0cec8e9b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2339ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2339ffb700 1 -- 172.21.15.19:0/1945379906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234801a020 con 0x55f9a45806a0 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2339ffb700 1 -- 172.21.15.19:0/1945379906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f23480118d0 con 0x55f9a45806a0 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2367091b80 1 -- 172.21.15.19:0/1945379906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23440a9bf0 con 0x55f9a45806a0 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2339ffb700 1 -- 172.21.15.19:0/1945379906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2348025050 con 0x55f9a45806a0 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2367091b80 10 client.15888 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.352+0000 7f2367091b80 1 -- 172.21.15.19:0/1945379906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9a4597c70 con 0x55f9a45806a0 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.353+0000 7f2339ffb700 1 -- 172.21.15.19:0/1945379906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23440a9bf0 con 0x55f9a45806a0 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.353+0000 7f2339ffb700 1 -- 172.21.15.19:0/1945379906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2348016b50 con 0x55f9a45806a0 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.353+0000 7f2367091b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.353+0000 7f2367091b80 10 client.15888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.353+0000 7f2367091b80 20 client.15888 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.353+0000 7f2367091b80 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f9a459a2b0 0x55f9a459c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.353+0000 7f2367091b80 4 client.15888 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.353+0000 7f2367091b80 1 -- 172.21.15.19:0/1945379906 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a44b09d0 con 0x55f9a459a2b0 2024-05-09T01:28:23.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.353+0000 7f2356417700 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f9a459a2b0 0x55f9a459c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.354+0000 7f2339ffb700 10 client.15888 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.354+0000 7f2356417700 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f9a459a2b0 0x55f9a459c700 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80c4898fcb36819b server_cookie=97148f94316ce4f1 in_seq=0 out_seq=0 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.354+0000 7f2339ffb700 1 -- 172.21.15.19:0/1945379906 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9a44b09d0 con 0x55f9a459a2b0 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.354+0000 7f2339ffb700 10 client.15888 handle_command_reply: tid=0 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.354+0000 7f2367091b80 1 client.15888 shutdown 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 -- 172.21.15.19:0/1945379906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a45806a0 msgr2=0x55f9a458acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a45806a0 0x55f9a458acb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2348009fd0 tx=0x7f2348007c20 comp rx=0 tx=0).stop 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 -- 172.21.15.19:0/1945379906 shutdown_connections 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f9a459a2b0 0x55f9a459c700 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a45806a0 0x55f9a458acb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 0x55f9a458a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 --2- 172.21.15.19:0/1945379906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a44ed160 0x55f9a458a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2339ffb700 0 client.15888 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:28:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 -- 172.21.15.19:0/1945379906 >> 172.21.15.19:0/1945379906 conn(0x55f9a44636a0 msgr2=0x55f9a457b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 -- 172.21.15.19:0/1945379906 shutdown_connections 2024-05-09T01:28:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 1 -- 172.21.15.19:0/1945379906 wait complete. 2024-05-09T01:28:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.355+0000 7f2367091b80 20 client.15888 trim_cache size 0 max 16384 2024-05-09T01:28:23.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2367091b80 1 Processor -- start 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2367091b80 1 -- start start 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2367091b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44034a0 0x55f9a45847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2367091b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44ed160 0x55f9a458aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2367091b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 0x55f9a4589230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2367091b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23501aee20 con 0x55f9a44034a0 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2367091b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b0960 con 0x55f9a44ed160 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2367091b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23501af4f0 con 0x55f9a45806a0 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2356417700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44034a0 0x55f9a45847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2356417700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44034a0 0x55f9a45847c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51446/0 (socket says 172.21.15.19:51446) 2024-05-09T01:28:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2356417700 1 -- 172.21.15.19:0/3002305601 learned_addr learned my addr 172.21.15.19:0/3002305601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.370+0000 7f2356c18700 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 0x55f9a4589230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f2355c16700 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44ed160 0x55f9a458aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f2356417700 1 -- 172.21.15.19:0/3002305601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 msgr2=0x55f9a4589230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f2356417700 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 0x55f9a4589230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f2356417700 1 -- 172.21.15.19:0/3002305601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44ed160 msgr2=0x55f9a458aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f2356417700 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44ed160 0x55f9a458aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f2356417700 1 -- 172.21.15.19:0/3002305601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a457cd00 con 0x55f9a44034a0 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f2355c16700 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44ed160 0x55f9a458aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f2356417700 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44034a0 0x55f9a45847c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f23440a5c70 tx=0x7f23440e2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20568c3da2c8a6bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f233bfff700 1 -- 172.21.15.19:0/3002305601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234406d020 con 0x55f9a44034a0 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f233bfff700 1 -- 172.21.15.19:0/3002305601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23440aa2f0 con 0x55f9a44034a0 2024-05-09T01:28:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.371+0000 7f233bfff700 1 -- 172.21.15.19:0/3002305601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440a57d0 con 0x55f9a44034a0 2024-05-09T01:28:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 -- 172.21.15.19:0/3002305601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44034a0 msgr2=0x55f9a45847c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44034a0 0x55f9a45847c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f23440a5c70 tx=0x7f23440e2040 comp rx=0 tx=0).stop 2024-05-09T01:28:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 -- 172.21.15.19:0/3002305601 shutdown_connections 2024-05-09T01:28:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 0x55f9a4589230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44034a0 0x55f9a45847c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 --2- 172.21.15.19:0/3002305601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44ed160 0x55f9a458aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 -- 172.21.15.19:0/3002305601 >> 172.21.15.19:0/3002305601 conn(0x55f9a44636a0 msgr2=0x55f9a4588b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 -- 172.21.15.19:0/3002305601 shutdown_connections 2024-05-09T01:28:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 -- 172.21.15.19:0/3002305601 wait complete. 2024-05-09T01:28:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.372+0000 7f2367091b80 1 -- start start 2024-05-09T01:28:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2367091b80 1 Processor -- start 2024-05-09T01:28:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2367091b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 0x55f9a45845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2367091b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a4582930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2367091b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 0x55f9a4582e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2367091b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b0290 con 0x55f9a44ed160 2024-05-09T01:28:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2367091b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23501afbc0 con 0x55f9a44034a0 2024-05-09T01:28:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2367091b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23501acdc0 con 0x55f9a45806a0 2024-05-09T01:28:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2367091b80 10 client.0 _ll_register_callbacks cb 0x55f9a43146c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2355c16700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a4582930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2355c16700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a4582930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51458/0 (socket says 172.21.15.19:51458) 2024-05-09T01:28:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2355c16700 1 -- 172.21.15.19:0/1417123031 learned_addr learned my addr 172.21.15.19:0/1417123031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2356c18700 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 0x55f9a4582e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2356417700 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 0x55f9a45845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2355c16700 1 -- 172.21.15.19:0/1417123031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 msgr2=0x55f9a4582e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2355c16700 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 0x55f9a4582e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2355c16700 1 -- 172.21.15.19:0/1417123031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 msgr2=0x55f9a45845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2355c16700 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 0x55f9a45845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2355c16700 1 -- 172.21.15.19:0/1417123031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a4574f60 con 0x55f9a44ed160 2024-05-09T01:28:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.373+0000 7f2356c18700 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 0x55f9a4582e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.374+0000 7f2355c16700 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a4582930 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f234c0099c0 tx=0x7f234c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93787f074a12eca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.374+0000 7f2339ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.374+0000 7f2339ffb700 1 -- 172.21.15.19:0/1417123031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c035060 con 0x55f9a44ed160 2024-05-09T01:28:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.374+0000 7f2339ffb700 1 -- 172.21.15.19:0/1417123031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f234c00bdf0 con 0x55f9a44ed160 2024-05-09T01:28:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.374+0000 7f2367091b80 1 -- 172.21.15.19:0/1417123031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23440aa2f0 con 0x55f9a44ed160 2024-05-09T01:28:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.374+0000 7f2339ffb700 1 -- 172.21.15.19:0/1417123031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f234c00f460 con 0x55f9a44ed160 2024-05-09T01:28:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.374+0000 7f2367091b80 10 client.15894 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.374+0000 7f2367091b80 1 -- 172.21.15.19:0/1417123031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9a4591f90 con 0x55f9a44ed160 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.374+0000 7f2339ffb700 1 -- 172.21.15.19:0/1417123031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23440aa2f0 con 0x55f9a44ed160 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.375+0000 7f2339ffb700 1 -- 172.21.15.19:0/1417123031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f234c022070 con 0x55f9a44ed160 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.375+0000 7f2367091b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.375+0000 7f2367091b80 10 client.15894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.375+0000 7f2367091b80 20 client.15894 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.375+0000 7f2367091b80 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f9a45926b0 0x55f9a460aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.375+0000 7f2367091b80 4 client.15894 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.375+0000 7f2367091b80 1 -- 172.21.15.19:0/1417123031 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9a44aaa70 con 0x55f9a45926b0 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.375+0000 7f2356417700 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f9a45926b0 0x55f9a460aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2339ffb700 10 client.15894 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:28:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2356417700 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f9a45926b0 0x55f9a460aee0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57fd0f25ede833d6 server_cookie=c612fe5bf9a9f642 in_seq=0 out_seq=0 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2339ffb700 1 -- 172.21.15.19:0/1417123031 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9a44aaa70 con 0x55f9a45926b0 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2339ffb700 10 client.15894 handle_command_reply: tid=0 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 client.15894 shutdown 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 -- 172.21.15.19:0/1417123031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 msgr2=0x55f9a4582930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a4582930 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f234c0099c0 tx=0x7f234c01e000 comp rx=0 tx=0).stop 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 -- 172.21.15.19:0/1417123031 shutdown_connections 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f9a45926b0 0x55f9a460aee0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9a45806a0 0x55f9a4582e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f9a44034a0 0x55f9a45845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 --2- 172.21.15.19:0/1417123031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f9a44ed160 0x55f9a4582930 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 -- 172.21.15.19:0/1417123031 >> 172.21.15.19:0/1417123031 conn(0x55f9a44636a0 msgr2=0x55f9a45859c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2339ffb700 0 client.15894 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:28:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 -- 172.21.15.19:0/1417123031 shutdown_connections 2024-05-09T01:28:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 1 -- 172.21.15.19:0/1417123031 wait complete. 2024-05-09T01:28:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.376+0000 7f2367091b80 20 client.15894 trim_cache size 0 max 16384 2024-05-09T01:28:23.381 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 -- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f233c0b2880 msgr2=0x7f233c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f233c0b2880 0x7f233c0b4d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f234c000c00 tx=0x7f234c01a040 comp rx=0 tx=0).stop 2024-05-09T01:28:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 -- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23501034c0 msgr2=0x7f235019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23501034c0 0x7f235019bc90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f234400e9a0 tx=0x7f234400ed60 comp rx=0 tx=0).stop 2024-05-09T01:28:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 -- 172.21.15.19:0/67245767 shutdown_connections 2024-05-09T01:28:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2350108740 0x7f235019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f233c0b2880 0x7f233c0b4d90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23501034c0 0x7f235019bc90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 --2- 172.21.15.19:0/67245767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2350107df0 0x7f235019c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 -- 172.21.15.19:0/67245767 >> 172.21.15.19:0/67245767 conn(0x7f23500fedf0 msgr2=0x7f23501096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 -- 172.21.15.19:0/67245767 shutdown_connections 2024-05-09T01:28:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:23.377+0000 7f235867b700 1 -- 172.21.15.19:0/67245767 wait complete. 2024-05-09T01:28:24.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:28:24.441 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:24.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:24 smithi019 ceph-mon[30158]: pgmap v464: 97 pgs: 97 active+clean; 1.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:28:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:24 smithi059 ceph-mon[34351]: pgmap v464: 97 pgs: 97 active+clean; 1.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:28:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:24 smithi196 ceph-mon[35041]: pgmap v464: 97 pgs: 97 active+clean; 1.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.346+0000 7fb95f950700 1 -- 172.21.15.19:0/3606870779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 msgr2=0x7fb9581111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.346+0000 7fb95f950700 1 --2- 172.21.15.19:0/3606870779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 0x7fb9581111b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb9500099c0 tx=0x7fb950009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.346+0000 7fb95f950700 1 -- 172.21.15.19:0/3606870779 shutdown_connections 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.346+0000 7fb95f950700 1 --2- 172.21.15.19:0/3606870779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 0x7fb9581111b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.346+0000 7fb95f950700 1 --2- 172.21.15.19:0/3606870779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9581045d0 0x7fb9581049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.346+0000 7fb95f950700 1 --2- 172.21.15.19:0/3606870779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb958108f00 0x7fb958109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.346+0000 7fb95f950700 1 -- 172.21.15.19:0/3606870779 >> 172.21.15.19:0/3606870779 conn(0x7fb9580ffe80 msgr2=0x7fb9581022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.346+0000 7fb95f950700 1 -- 172.21.15.19:0/3606870779 shutdown_connections 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.346+0000 7fb95f950700 1 -- 172.21.15.19:0/3606870779 wait complete. 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.347+0000 7fb95f950700 1 Processor -- start 2024-05-09T01:28:26.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.347+0000 7fb95f950700 1 -- start start 2024-05-09T01:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95f950700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9581045d0 0x7fb95819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95f950700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb958108f00 0x7fb95819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95f950700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 0x7fb95819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95f950700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb958114480 con 0x7fb958109850 2024-05-09T01:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95f950700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb958114300 con 0x7fb958108f00 2024-05-09T01:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95f950700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb958114600 con 0x7fb9581045d0 2024-05-09T01:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95deed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 0x7fb95819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95deed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 0x7fb95819f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51466/0 (socket says 172.21.15.19:51466) 2024-05-09T01:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95deed700 1 -- 172.21.15.19:0/2227591048 learned_addr learned my addr 172.21.15.19:0/2227591048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:26.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9581045d0 0x7fb95819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95ceeb700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb958108f00 0x7fb95819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95deed700 1 -- 172.21.15.19:0/2227591048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9581045d0 msgr2=0x7fb95819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:26.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95deed700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9581045d0 0x7fb95819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95deed700 1 -- 172.21.15.19:0/2227591048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb958108f00 msgr2=0x7fb95819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95deed700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb958108f00 0x7fb95819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95deed700 1 -- 172.21.15.19:0/2227591048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95819fb90 con 0x7fb958109850 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.348+0000 7fb95deed700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 0x7fb95819f5a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb950009990 tx=0x7fb9500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.349+0000 7fb9567fc700 1 -- 172.21.15.19:0/2227591048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950027070 con 0x7fb958109850 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.349+0000 7fb95f950700 1 -- 172.21.15.19:0/2227591048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95819fe20 con 0x7fb958109850 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.349+0000 7fb9567fc700 1 -- 172.21.15.19:0/2227591048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb950004020 con 0x7fb958109850 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.349+0000 7fb95f950700 1 -- 172.21.15.19:0/2227591048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9581a0350 con 0x7fb958109850 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.349+0000 7fb9567fc700 1 -- 172.21.15.19:0/2227591048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950022ce0 con 0x7fb958109850 2024-05-09T01:28:26.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.351+0000 7fb9567fc700 1 -- 172.21.15.19:0/2227591048 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb95002a070 con 0x7fb958109850 2024-05-09T01:28:26.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.352+0000 7fb9567fc700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9400b2880 0x7fb9400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.352+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9400b2880 0x7fb9400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.352+0000 7fb9567fc700 1 -- 172.21.15.19:0/2227591048 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb9500e3010 con 0x7fb958109850 2024-05-09T01:28:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.352+0000 7fb96e366b80 1 Processor -- start 2024-05-09T01:28:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb96e366b80 1 -- start start 2024-05-09T01:28:26.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc32f1160 0x559cc338a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc338aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc33846a0 0x559cc3384ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb96e366b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559cc32a6ff0 con 0x559cc33846a0 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb96e366b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559cc32457f0 con 0x559cc32074a0 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb96e366b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559cc32a2fb0 con 0x559cc32f1160 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9400b2880 0x7fb9400b4d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb9480060b0 tx=0x7fb94800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95deed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc33846a0 0x559cc3384ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95deed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc33846a0 0x559cc3384ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51478/0 (socket says 172.21.15.19:51478) 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95deed700 1 -- 172.21.15.19:0/237396918 learned_addr learned my addr 172.21.15.19:0/237396918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/237396918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc32f1160 0x559cc338a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95ceeb700 1 --2- 172.21.15.19:0/237396918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc338aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95deed700 1 -- 172.21.15.19:0/237396918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc32f1160 msgr2=0x559cc338a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95deed700 1 --2- 172.21.15.19:0/237396918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc32f1160 0x559cc338a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95deed700 1 -- 172.21.15.19:0/237396918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 msgr2=0x559cc338aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95deed700 1 --2- 172.21.15.19:0/237396918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc338aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.353+0000 7fb95deed700 1 -- 172.21.15.19:0/237396918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cc323d370 con 0x559cc33846a0 2024-05-09T01:28:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.354+0000 7fb95deed700 1 --2- 172.21.15.19:0/237396918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc33846a0 0x559cc3384ad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb9500aa340 tx=0x7fb9500a9ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375bca91f9320049 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.354+0000 7fb93f7fe700 1 -- 172.21.15.19:0/237396918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9500aa640 con 0x559cc33846a0 2024-05-09T01:28:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.354+0000 7fb93f7fe700 1 -- 172.21.15.19:0/237396918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9500ae9f0 con 0x559cc33846a0 2024-05-09T01:28:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.354+0000 7fb93f7fe700 1 -- 172.21.15.19:0/237396918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9500b2f80 con 0x559cc33846a0 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 -- 172.21.15.19:0/237396918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc33846a0 msgr2=0x559cc3384ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 --2- 172.21.15.19:0/237396918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc33846a0 0x559cc3384ad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb9500aa340 tx=0x7fb9500a9ee0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 -- 172.21.15.19:0/237396918 shutdown_connections 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 --2- 172.21.15.19:0/237396918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc33846a0 0x559cc3384ad0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 --2- 172.21.15.19:0/237396918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc32f1160 0x559cc338a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 --2- 172.21.15.19:0/237396918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc338aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 -- 172.21.15.19:0/237396918 >> 172.21.15.19:0/237396918 conn(0x559cc32676a0 msgr2=0x559cc337f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 -- 172.21.15.19:0/237396918 shutdown_connections 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 -- 172.21.15.19:0/237396918 wait complete. 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 -- start start 2024-05-09T01:28:26.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.355+0000 7fb96e366b80 1 Processor -- start 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc3393d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc3394250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 0x559cc338e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb96e366b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559cc3122ba0 con 0x559cc32f1160 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb96e366b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559cc3261150 con 0x559cc32074a0 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb96e366b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559cc3385360 con 0x559cc33846a0 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb96e366b80 10 client.0 _ll_register_callbacks cb 0x559cc31186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95ceeb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc3394250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95ceeb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc3394250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51488/0 (socket says 172.21.15.19:51488) 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95ceeb700 1 -- 172.21.15.19:0/338109844 learned_addr learned my addr 172.21.15.19:0/338109844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc3393d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95deed700 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 0x559cc338e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95ceeb700 1 -- 172.21.15.19:0/338109844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 msgr2=0x559cc338e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95ceeb700 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 0x559cc338e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95ceeb700 1 -- 172.21.15.19:0/338109844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 msgr2=0x559cc3393d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95ceeb700 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc3393d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95ceeb700 1 -- 172.21.15.19:0/338109844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cc3122490 con 0x559cc32f1160 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.356+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc3393d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb95ceeb700 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc3394250 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0106d0 tx=0x7fb94c010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552f9016767c87fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb93cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb93cff9700 1 -- 172.21.15.19:0/338109844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c004230 con 0x559cc32f1160 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb93cff9700 1 -- 172.21.15.19:0/338109844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb94c0043d0 con 0x559cc32f1160 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb96e366b80 1 -- 172.21.15.19:0/338109844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9500ae9f0 con 0x559cc32f1160 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb93cff9700 1 -- 172.21.15.19:0/338109844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb94c011770 con 0x559cc32f1160 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb96e366b80 10 client.15906 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb96e366b80 1 -- 172.21.15.19:0/338109844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559cc339b9c0 con 0x559cc32f1160 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb93cff9700 1 -- 172.21.15.19:0/338109844 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9500ae9f0 con 0x559cc32f1160 2024-05-09T01:28:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.357+0000 7fb93cff9700 1 -- 172.21.15.19:0/338109844 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb94c003c60 con 0x559cc32f1160 2024-05-09T01:28:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.358+0000 7fb96e366b80 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.358+0000 7fb96e366b80 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.358+0000 7fb96e366b80 20 client.15906 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.358+0000 7fb96e366b80 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559cc339e000 0x559cc33a0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.358+0000 7fb96e366b80 4 client.15906 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.358+0000 7fb96e366b80 1 -- 172.21.15.19:0/338109844 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cc32b49d0 con 0x559cc339e000 2024-05-09T01:28:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.358+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559cc339e000 0x559cc33a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.358+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559cc339e000 0x559cc33a0440 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe89d3cd9b67892 server_cookie=2b6c10a959c43d56 in_seq=0 out_seq=0 2024-05-09T01:28:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.358+0000 7fb93cff9700 10 client.15906 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:26.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb93cff9700 1 -- 172.21.15.19:0/338109844 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559cc32b49d0 con 0x559cc339e000 2024-05-09T01:28:26.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb93cff9700 10 client.15906 handle_command_reply: tid=0 2024-05-09T01:28:26.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb96e366b80 1 client.15906 shutdown 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb96e366b80 1 -- 172.21.15.19:0/338109844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 msgr2=0x559cc3394250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb96e366b80 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc3394250 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0106d0 tx=0x7fb94c010a90 comp rx=0 tx=0).stop 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb96e366b80 1 -- 172.21.15.19:0/338109844 shutdown_connections 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb96e366b80 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559cc339e000 0x559cc33a0440 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb96e366b80 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 0x559cc338e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb96e366b80 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc3393d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb93cff9700 0 client.15906 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb96e366b80 1 --2- 172.21.15.19:0/338109844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc3394250 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.359+0000 7fb96e366b80 1 -- 172.21.15.19:0/338109844 >> 172.21.15.19:0/338109844 conn(0x559cc32676a0 msgr2=0x559cc337f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.360+0000 7fb96e366b80 1 -- 172.21.15.19:0/338109844 shutdown_connections 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.360+0000 7fb96e366b80 1 -- 172.21.15.19:0/338109844 wait complete. 2024-05-09T01:28:26.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.360+0000 7fb96e366b80 20 client.15906 trim_cache size 0 max 16384 2024-05-09T01:28:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb96e366b80 1 Processor -- start 2024-05-09T01:28:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb96e366b80 1 -- start start 2024-05-09T01:28:26.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32074a0 0x559cc3389810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc32f1160 0x559cc33a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc33846a0 0x559cc33a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb96e366b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb94400d420 con 0x559cc32074a0 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb96e366b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb94400ef60 con 0x559cc33846a0 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb96e366b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb94400daf0 con 0x559cc32f1160 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb95d6ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32074a0 0x559cc3389810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb95d6ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32074a0 0x559cc3389810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51490/0 (socket says 172.21.15.19:51490) 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.375+0000 7fb95d6ec700 1 -- 172.21.15.19:0/1578044791 learned_addr learned my addr 172.21.15.19:0/1578044791 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb95deed700 1 --2- 172.21.15.19:0/1578044791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc33846a0 0x559cc33a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb95d6ec700 1 -- 172.21.15.19:0/1578044791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc32f1160 msgr2=0x559cc33a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/1578044791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc32f1160 0x559cc33a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb95d6ec700 1 -- 172.21.15.19:0/1578044791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc33846a0 msgr2=0x559cc33a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/1578044791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc33846a0 0x559cc33a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb95d6ec700 1 -- 172.21.15.19:0/1578044791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cc338c680 con 0x559cc32074a0 2024-05-09T01:28:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/1578044791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32074a0 0x559cc3389810 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb948007fd0 tx=0x7fb948012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82dc58da95695c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb93effd700 1 -- 172.21.15.19:0/1578044791 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948004bb0 con 0x559cc32074a0 2024-05-09T01:28:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb93effd700 1 -- 172.21.15.19:0/1578044791 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb948004d50 con 0x559cc32074a0 2024-05-09T01:28:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.376+0000 7fb93effd700 1 -- 172.21.15.19:0/1578044791 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948024790 con 0x559cc32074a0 2024-05-09T01:28:26.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 -- 172.21.15.19:0/1578044791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32074a0 msgr2=0x559cc3389810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 --2- 172.21.15.19:0/1578044791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32074a0 0x559cc3389810 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb948007fd0 tx=0x7fb948012000 comp rx=0 tx=0).stop 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 -- 172.21.15.19:0/1578044791 shutdown_connections 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 --2- 172.21.15.19:0/1578044791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc33846a0 0x559cc33a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 --2- 172.21.15.19:0/1578044791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32074a0 0x559cc3389810 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 --2- 172.21.15.19:0/1578044791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc32f1160 0x559cc33a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 -- 172.21.15.19:0/1578044791 >> 172.21.15.19:0/1578044791 conn(0x559cc32676a0 msgr2=0x559cc3389100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 -- 172.21.15.19:0/1578044791 shutdown_connections 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 -- 172.21.15.19:0/1578044791 wait complete. 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 -- start start 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.377+0000 7fb96e366b80 1 Processor -- start 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc3400060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc34005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb96e366b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 0x559cc3399c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb96e366b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb94400e890 con 0x559cc32f1160 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb96e366b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb94400e1c0 con 0x559cc32074a0 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb96e366b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb94400b3c0 con 0x559cc33846a0 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb96e366b80 10 client.0 _ll_register_callbacks cb 0x559cc31186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb95ceeb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc34005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb95ceeb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc34005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51504/0 (socket says 172.21.15.19:51504) 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb95d6ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc3400060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb95ceeb700 1 -- 172.21.15.19:0/875462598 learned_addr learned my addr 172.21.15.19:0/875462598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb95deed700 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 0x559cc3399c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb95ceeb700 1 -- 172.21.15.19:0/875462598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 msgr2=0x559cc3399c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb95ceeb700 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 0x559cc3399c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb95ceeb700 1 -- 172.21.15.19:0/875462598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 msgr2=0x559cc3400060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.378+0000 7fb95ceeb700 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc3400060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb95ceeb700 1 -- 172.21.15.19:0/875462598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cc339a1f0 con 0x559cc32f1160 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb95ceeb700 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc34005a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb94c026610 tx=0x7fb94c026cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f17677d16b994bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb93cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb93cff9700 1 -- 172.21.15.19:0/875462598 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c02b070 con 0x559cc32f1160 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb93cff9700 1 -- 172.21.15.19:0/875462598 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb94c0043e0 con 0x559cc32f1160 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb96e366b80 1 -- 172.21.15.19:0/875462598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb948004d50 con 0x559cc32f1160 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb93cff9700 1 -- 172.21.15.19:0/875462598 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb94c010e90 con 0x559cc32f1160 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb96e366b80 10 client.15912 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb96e366b80 1 -- 172.21.15.19:0/875462598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559cc34087f0 con 0x559cc32f1160 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.379+0000 7fb93cff9700 1 -- 172.21.15.19:0/875462598 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb948004d50 con 0x559cc32f1160 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb93cff9700 1 -- 172.21.15.19:0/875462598 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb94c0053f0 con 0x559cc32f1160 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb96e366b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb96e366b80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb96e366b80 20 client.15912 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb96e366b80 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559cc3408ea0 0x559cc340b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb96e366b80 4 client.15912 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb96e366b80 1 -- 172.21.15.19:0/875462598 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cc32aea70 con 0x559cc3408ea0 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559cc3408ea0 0x559cc340b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb95d6ec700 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559cc3408ea0 0x559cc340b2c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17ce43caee50d3c0 server_cookie=de8f1e879201e018 in_seq=0 out_seq=0 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.380+0000 7fb93cff9700 10 client.15912 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb93cff9700 1 -- 172.21.15.19:0/875462598 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cc32aea70 con 0x559cc3408ea0 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb93cff9700 10 client.15912 handle_command_reply: tid=0 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 client.15912 shutdown 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 -- 172.21.15.19:0/875462598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 msgr2=0x559cc34005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc34005a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb94c026610 tx=0x7fb94c026cf0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 -- 172.21.15.19:0/875462598 shutdown_connections 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559cc3408ea0 0x559cc340b2c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cc33846a0 0x559cc3399c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cc32074a0 0x559cc3400060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb93cff9700 0 client.15912 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 --2- 172.21.15.19:0/875462598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cc32f1160 0x559cc34005a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 -- 172.21.15.19:0/875462598 >> 172.21.15.19:0/875462598 conn(0x559cc32676a0 msgr2=0x559cc3387e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:26.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 -- 172.21.15.19:0/875462598 shutdown_connections 2024-05-09T01:28:26.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 1 -- 172.21.15.19:0/875462598 wait complete. 2024-05-09T01:28:26.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.381+0000 7fb96e366b80 20 client.15912 trim_cache size 0 max 16384 2024-05-09T01:28:26.383 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 -- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9400b2880 msgr2=0x7fb9400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9400b2880 0x7fb9400b4d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb9480060b0 tx=0x7fb94800c040 comp rx=0 tx=0).stop 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 -- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 msgr2=0x7fb95819f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 0x7fb95819f5a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb950009990 tx=0x7fb9500047b0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 -- 172.21.15.19:0/2227591048 shutdown_connections 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9400b2880 0x7fb9400b4d90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb958109850 0x7fb95819f5a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9581045d0 0x7fb95819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 --2- 172.21.15.19:0/2227591048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb958108f00 0x7fb95819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 -- 172.21.15.19:0/2227591048 >> 172.21.15.19:0/2227591048 conn(0x7fb9580ffe80 msgr2=0x7fb958108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 -- 172.21.15.19:0/2227591048 shutdown_connections 2024-05-09T01:28:26.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:26.382+0000 7fb95f950700 1 -- 172.21.15.19:0/2227591048 wait complete. 2024-05-09T01:28:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:26 smithi059 ceph-mon[34351]: pgmap v465: 97 pgs: 97 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-05-09T01:28:26.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:26 smithi019 ceph-mon[30158]: pgmap v465: 97 pgs: 97 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-05-09T01:28:26.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:26 smithi196 ceph-mon[35041]: pgmap v465: 97 pgs: 97 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-05-09T01:28:28.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:28:28.185 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:28:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:28.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:28 smithi019 ceph-mon[30158]: pgmap v466: 97 pgs: 97 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:28:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:28 smithi059 ceph-mon[34351]: pgmap v466: 97 pgs: 97 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:28:28.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:28 smithi196 ceph-mon[35041]: pgmap v466: 97 pgs: 97 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:28:30.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.214+0000 7fab2c51e700 1 -- 172.21.15.19:0/1926094206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 msgr2=0x7fab24103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:30.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.214+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1926094206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 0x7fab24103960 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fab200099c0 tx=0x7fab20009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.214+0000 7fab2c51e700 1 -- 172.21.15.19:0/1926094206 shutdown_connections 2024-05-09T01:28:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.214+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1926094206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab24108780 0x7fab241100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.214+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1926094206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 0x7fab24103960 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.214+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1926094206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab24107e30 0x7fab24108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.214+0000 7fab2c51e700 1 -- 172.21.15.19:0/1926094206 >> 172.21.15.19:0/1926094206 conn(0x7fab24078e00 msgr2=0x7fab24079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.214+0000 7fab2c51e700 1 -- 172.21.15.19:0/1926094206 shutdown_connections 2024-05-09T01:28:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.214+0000 7fab2c51e700 1 -- 172.21.15.19:0/1926094206 wait complete. 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2c51e700 1 Processor -- start 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2c51e700 1 -- start start 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2c51e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 0x7fab2419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2c51e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab24107e30 0x7fab2419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2c51e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab24108780 0x7fab2419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2c51e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab24114500 con 0x7fab24103550 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2c51e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab24114380 con 0x7fab24107e30 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2c51e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab24114680 con 0x7fab24108780 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2a2ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 0x7fab2419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2a2ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 0x7fab2419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59902/0 (socket says 172.21.15.19:59902) 2024-05-09T01:28:30.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2a2ba700 1 -- 172.21.15.19:0/1221885485 learned_addr learned my addr 172.21.15.19:0/1221885485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab29ab9700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab24107e30 0x7fab2419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.215+0000 7fab2aabb700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab24108780 0x7fab2419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab2a2ba700 1 -- 172.21.15.19:0/1221885485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab24108780 msgr2=0x7fab2419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab24108780 0x7fab2419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab2a2ba700 1 -- 172.21.15.19:0/1221885485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab24107e30 msgr2=0x7fab2419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab24107e30 0x7fab2419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab2a2ba700 1 -- 172.21.15.19:0/1221885485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2419fc10 con 0x7fab24103550 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab2aabb700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab24108780 0x7fab2419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 0x7fab2419adf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fab200098e0 tx=0x7fab200057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab1b7fe700 1 -- 172.21.15.19:0/1221885485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20027070 con 0x7fab24103550 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab1b7fe700 1 -- 172.21.15.19:0/1221885485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab20003f20 con 0x7fab24103550 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab2c51e700 1 -- 172.21.15.19:0/1221885485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2419fea0 con 0x7fab24103550 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab1b7fe700 1 -- 172.21.15.19:0/1221885485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20022da0 con 0x7fab24103550 2024-05-09T01:28:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.216+0000 7fab2c51e700 1 -- 172.21.15.19:0/1221885485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab241a03d0 con 0x7fab24103550 2024-05-09T01:28:30.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.219+0000 7fab1b7fe700 1 -- 172.21.15.19:0/1221885485 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fab2002a070 con 0x7fab24103550 2024-05-09T01:28:30.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.220+0000 7fab1b7fe700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab100b2900 0x7fab100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.220+0000 7fab29ab9700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab100b2900 0x7fab100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.220+0000 7fab1b7fe700 1 -- 172.21.15.19:0/1221885485 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fab200e32e0 con 0x7fab24103550 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.220+0000 7fab3af34b80 1 Processor -- start 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab29ab9700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab100b2900 0x7fab100b4e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fab24079250 tx=0x7fab1400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab3af34b80 1 -- start start 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1664cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a164cb520 0x560a1664d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16651260 0x560a16651670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab3af34b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a1656b120 con 0x560a16651260 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab3af34b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560a16510a20 con 0x560a16545d80 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab3af34b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560a165670e0 con 0x560a164cb520 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab2aabb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16651260 0x560a16651670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab2aabb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16651260 0x560a16651670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59910/0 (socket says 172.21.15.19:59910) 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab2a2ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1664cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab2aabb700 1 -- 172.21.15.19:0/1353028205 learned_addr learned my addr 172.21.15.19:0/1353028205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.221+0000 7fab29ab9700 1 --2- 172.21.15.19:0/1353028205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a164cb520 0x560a1664d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab2a2ba700 1 -- 172.21.15.19:0/1353028205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a164cb520 msgr2=0x560a1664d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/1353028205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a164cb520 0x560a1664d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab2a2ba700 1 -- 172.21.15.19:0/1353028205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16651260 msgr2=0x560a16651670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/1353028205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16651260 0x560a16651670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab2a2ba700 1 -- 172.21.15.19:0/1353028205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a16501580 con 0x560a16545d80 2024-05-09T01:28:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab2aabb700 1 --2- 172.21.15.19:0/1353028205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16651260 0x560a16651670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:30.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/1353028205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1664cb50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c010 tx=0x7fab20022470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b74128af50c33a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab18ff9700 1 -- 172.21.15.19:0/1353028205 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab200aa480 con 0x560a16545d80 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab18ff9700 1 -- 172.21.15.19:0/1353028205 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab200ab070 con 0x560a16545d80 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.222+0000 7fab18ff9700 1 -- 172.21.15.19:0/1353028205 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab200aabd0 con 0x560a16545d80 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.223+0000 7fab3af34b80 1 -- 172.21.15.19:0/1353028205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 msgr2=0x560a1664cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.223+0000 7fab3af34b80 1 --2- 172.21.15.19:0/1353028205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1664cb50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c010 tx=0x7fab20022470 comp rx=0 tx=0).stop 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.223+0000 7fab3af34b80 1 -- 172.21.15.19:0/1353028205 shutdown_connections 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.223+0000 7fab3af34b80 1 --2- 172.21.15.19:0/1353028205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16651260 0x560a16651670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.223+0000 7fab3af34b80 1 --2- 172.21.15.19:0/1353028205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1664cb50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.223+0000 7fab3af34b80 1 --2- 172.21.15.19:0/1353028205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a164cb520 0x560a1664d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.223+0000 7fab3af34b80 1 -- 172.21.15.19:0/1353028205 >> 172.21.15.19:0/1353028205 conn(0x560a164ceff0 msgr2=0x560a166432a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.223+0000 7fab3af34b80 1 -- 172.21.15.19:0/1353028205 shutdown_connections 2024-05-09T01:28:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.223+0000 7fab3af34b80 1 -- 172.21.15.19:0/1353028205 wait complete. 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab3af34b80 1 -- start start 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab3af34b80 1 Processor -- start 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a1665bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1665c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 0x560a166563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab3af34b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a16651dd0 con 0x560a164cb520 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab3af34b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560a16505ba0 con 0x560a16545d80 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab3af34b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560a16651f50 con 0x560a16651260 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab2a2ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a1665bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab3af34b80 10 client.0 _ll_register_callbacks cb 0x560a163dc740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab2a2ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a1665bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59918/0 (socket says 172.21.15.19:59918) 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab2a2ba700 1 -- 172.21.15.19:0/2190476373 learned_addr learned my addr 172.21.15.19:0/2190476373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab2aabb700 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 0x560a166563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.224+0000 7fab29ab9700 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1665c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab2a2ba700 1 -- 172.21.15.19:0/2190476373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 msgr2=0x560a166563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 0x560a166563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab2a2ba700 1 -- 172.21.15.19:0/2190476373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 msgr2=0x560a1665c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1665c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab2a2ba700 1 -- 172.21.15.19:0/2190476373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a163e6510 con 0x560a164cb520 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab29ab9700 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1665c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a1665bfe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fab200aa670 tx=0x7fab200a6200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5322af50caf4a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab0dffb700 1 -- 172.21.15.19:0/2190476373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab200c4070 con 0x560a164cb520 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab0dffb700 1 -- 172.21.15.19:0/2190476373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fab200a6f60 con 0x560a164cb520 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab3af34b80 1 -- 172.21.15.19:0/2190476373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab200ab070 con 0x560a164cb520 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab0dffb700 1 -- 172.21.15.19:0/2190476373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab200aa480 con 0x560a164cb520 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab3af34b80 10 client.15924 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.225+0000 7fab3af34b80 1 -- 172.21.15.19:0/2190476373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a16651dd0 con 0x560a164cb520 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.226+0000 7fab0dffb700 1 -- 172.21.15.19:0/2190476373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab200ab070 con 0x560a164cb520 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.226+0000 7fab0dffb700 1 -- 172.21.15.19:0/2190476373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fab200b3340 con 0x560a164cb520 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.226+0000 7fab3af34b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.226+0000 7fab3af34b80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.226+0000 7fab3af34b80 20 client.15924 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.226+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560a16666090 0x560a166684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.226+0000 7fab3af34b80 4 client.15924 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.226+0000 7fab3af34b80 1 -- 172.21.15.19:0/2190476373 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a16578b00 con 0x560a16666090 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.226+0000 7fab29ab9700 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560a16666090 0x560a166684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.227+0000 7fab29ab9700 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560a16666090 0x560a166684e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92bad09882c0b7a2 server_cookie=eecd55901a2732b6 in_seq=0 out_seq=0 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.227+0000 7fab0dffb700 10 client.15924 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.227+0000 7fab0dffb700 1 -- 172.21.15.19:0/2190476373 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a16578b00 con 0x560a16666090 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.227+0000 7fab0dffb700 10 client.15924 handle_command_reply: tid=0 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.227+0000 7fab3af34b80 1 client.15924 shutdown 2024-05-09T01:28:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 -- 172.21.15.19:0/2190476373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 msgr2=0x560a1665bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a1665bfe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fab200aa670 tx=0x7fab200a6200 comp rx=0 tx=0).stop 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 -- 172.21.15.19:0/2190476373 shutdown_connections 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560a16666090 0x560a166684e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 0x560a166563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a1665bfe0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab0dffb700 0 client.15924 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2190476373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16545d80 0x560a1665c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 -- 172.21.15.19:0/2190476373 >> 172.21.15.19:0/2190476373 conn(0x560a164ceff0 msgr2=0x560a1663d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 -- 172.21.15.19:0/2190476373 shutdown_connections 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 1 -- 172.21.15.19:0/2190476373 wait complete. 2024-05-09T01:28:30.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.228+0000 7fab3af34b80 20 client.15924 trim_cache size 0 max 16384 2024-05-09T01:28:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab3af34b80 1 Processor -- start 2024-05-09T01:28:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab3af34b80 1 -- start start 2024-05-09T01:28:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a16643310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16545d80 0x560a16650b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16651260 0x560a1666dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab3af34b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab241ac920 con 0x560a164cb520 2024-05-09T01:28:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab3af34b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab241ae460 con 0x560a16651260 2024-05-09T01:28:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab3af34b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab241acff0 con 0x560a16545d80 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab2a2ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a16643310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab2a2ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a16643310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59922/0 (socket says 172.21.15.19:59922) 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab2a2ba700 1 -- 172.21.15.19:0/2297940945 learned_addr learned my addr 172.21.15.19:0/2297940945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab29ab9700 1 --2- 172.21.15.19:0/2297940945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16545d80 0x560a16650b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab2aabb700 1 --2- 172.21.15.19:0/2297940945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16651260 0x560a1666dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab2a2ba700 1 -- 172.21.15.19:0/2297940945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16545d80 msgr2=0x560a16650b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/2297940945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16545d80 0x560a16650b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab2a2ba700 1 -- 172.21.15.19:0/2297940945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16651260 msgr2=0x560a1666dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/2297940945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16651260 0x560a1666dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.249+0000 7fab2a2ba700 1 -- 172.21.15.19:0/2297940945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1663fa50 con 0x560a164cb520 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.250+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/2297940945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a16643310 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c010 tx=0x7fab20022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513d77a1effdd4c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.250+0000 7fab0ffff700 1 -- 172.21.15.19:0/2297940945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab200c4070 con 0x560a164cb520 2024-05-09T01:28:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.250+0000 7fab0ffff700 1 -- 172.21.15.19:0/2297940945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab200ab070 con 0x560a164cb520 2024-05-09T01:28:30.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.250+0000 7fab0ffff700 1 -- 172.21.15.19:0/2297940945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab200a9d70 con 0x560a164cb520 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 -- 172.21.15.19:0/2297940945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 msgr2=0x560a16643310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2297940945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a16643310 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c010 tx=0x7fab20022470 comp rx=0 tx=0).stop 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 -- 172.21.15.19:0/2297940945 shutdown_connections 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2297940945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a16651260 0x560a1666dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2297940945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a164cb520 0x560a16643310 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 --2- 172.21.15.19:0/2297940945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16545d80 0x560a16650b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 -- 172.21.15.19:0/2297940945 >> 172.21.15.19:0/2297940945 conn(0x560a164ceff0 msgr2=0x560a166503f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 -- 172.21.15.19:0/2297940945 shutdown_connections 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 -- 172.21.15.19:0/2297940945 wait complete. 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 -- start start 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.251+0000 7fab3af34b80 1 Processor -- start 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a164cb520 0x560a1665ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16545d80 0x560a1665f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab3af34b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 0x560a1665f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab3af34b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab241add90 con 0x560a16545d80 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab3af34b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab241ad6c0 con 0x560a164cb520 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab3af34b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab241aa8c0 con 0x560a16651260 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab3af34b80 10 client.0 _ll_register_callbacks cb 0x560a163dc740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab29ab9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16545d80 0x560a1665f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab29ab9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16545d80 0x560a1665f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59928/0 (socket says 172.21.15.19:59928) 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab29ab9700 1 -- 172.21.15.19:0/4005203382 learned_addr learned my addr 172.21.15.19:0/4005203382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a164cb520 0x560a1665ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab2aabb700 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 0x560a1665f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab29ab9700 1 -- 172.21.15.19:0/4005203382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 msgr2=0x560a1665f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab29ab9700 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 0x560a1665f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab29ab9700 1 -- 172.21.15.19:0/4005203382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a164cb520 msgr2=0x560a1665ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab29ab9700 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a164cb520 0x560a1665ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab29ab9700 1 -- 172.21.15.19:0/4005203382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a166cf890 con 0x560a16545d80 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab29ab9700 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16545d80 0x560a1665f130 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fab1400aa30 tx=0x7fab140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6e730cb2ceaf9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.252+0000 7fab0dffb700 1 -- 172.21.15.19:0/4005203382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14016070 con 0x560a16545d80 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab0dffb700 1 -- 172.21.15.19:0/4005203382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fab14029850 con 0x560a16545d80 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab3af34b80 1 -- 172.21.15.19:0/4005203382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab200ab070 con 0x560a16545d80 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab0dffb700 1 -- 172.21.15.19:0/4005203382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab14037070 con 0x560a16545d80 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab3af34b80 10 client.15930 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab3af34b80 1 -- 172.21.15.19:0/4005203382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a166d0b10 con 0x560a16545d80 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab0dffb700 1 -- 172.21.15.19:0/4005203382 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab200ab070 con 0x560a16545d80 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab0dffb700 1 -- 172.21.15.19:0/4005203382 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fab14031410 con 0x560a16545d80 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab3af34b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab3af34b80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab3af34b80 20 client.15930 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:30.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab3af34b80 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560a166d1230 0x560a166d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab3af34b80 4 client.15930 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab3af34b80 1 -- 172.21.15.19:0/4005203382 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560a16572ba0 con 0x560a166d1230 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.253+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560a166d1230 0x560a166d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.254+0000 7fab0dffb700 10 client.15930 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.254+0000 7fab2a2ba700 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560a166d1230 0x560a166d3650 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590c236773a1945b server_cookie=53dd29f16a6610ee in_seq=0 out_seq=0 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab0dffb700 1 -- 172.21.15.19:0/4005203382 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560a16572ba0 con 0x560a166d1230 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab0dffb700 10 client.15930 handle_command_reply: tid=0 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 client.15930 shutdown 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 -- 172.21.15.19:0/4005203382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16545d80 msgr2=0x560a1665f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16545d80 0x560a1665f130 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fab1400aa30 tx=0x7fab140047c0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 -- 172.21.15.19:0/4005203382 shutdown_connections 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560a166d1230 0x560a166d3650 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a16651260 0x560a1665f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a164cb520 0x560a1665ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 --2- 172.21.15.19:0/4005203382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a16545d80 0x560a1665f130 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 -- 172.21.15.19:0/4005203382 >> 172.21.15.19:0/4005203382 conn(0x560a164ceff0 msgr2=0x560a1663d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab0dffb700 0 client.15930 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 -- 172.21.15.19:0/4005203382 shutdown_connections 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.255+0000 7fab3af34b80 1 -- 172.21.15.19:0/4005203382 wait complete. 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab3af34b80 20 client.15930 trim_cache size 0 max 16384 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "82ea64a1-c2fa-42cf-a625-d1afe424df97", 2024-05-09T01:28:30.257 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:28:30.258 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:28:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 -- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab100b2900 msgr2=0x7fab100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab100b2900 0x7fab100b4e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fab24079250 tx=0x7fab1400a380 comp rx=0 tx=0).stop 2024-05-09T01:28:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 -- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 msgr2=0x7fab2419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 0x7fab2419adf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fab200098e0 tx=0x7fab200057c0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 -- 172.21.15.19:0/1221885485 shutdown_connections 2024-05-09T01:28:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab24108780 0x7fab2419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fab100b2900 0x7fab100b4e10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab24103550 0x7fab2419adf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 --2- 172.21.15.19:0/1221885485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab24107e30 0x7fab2419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 -- 172.21.15.19:0/1221885485 >> 172.21.15.19:0/1221885485 conn(0x7fab24078e00 msgr2=0x7fab24101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 -- 172.21.15.19:0/1221885485 shutdown_connections 2024-05-09T01:28:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:30.256+0000 7fab2c51e700 1 -- 172.21.15.19:0/1221885485 wait complete. 2024-05-09T01:28:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:30 smithi059 ceph-mon[34351]: pgmap v467: 97 pgs: 97 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:30.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:30 smithi019 ceph-mon[30158]: pgmap v467: 97 pgs: 97 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:30 smithi196 ceph-mon[35041]: pgmap v467: 97 pgs: 97 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:31.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:28:31.370 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:31.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:31 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:28:31.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:31 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:31.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:31 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:28:31.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:31 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:31.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:31 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:28:31.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:31 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:31.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:31 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:28:31.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:31 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:31.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:31 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:28:31.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:31 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:31.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:31 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:28:31.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:31 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:32.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:32 smithi196 ceph-mon[35041]: pgmap v468: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:28:32.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:32 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:28:32.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:32 smithi059 ceph-mon[34351]: pgmap v468: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:28:32.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:32 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:28:32.877 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:32 smithi019 ceph-mon[30158]: pgmap v468: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:28:32.878 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:32 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:28:33.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 -- 172.21.15.19:0/1204397333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 msgr2=0x7fef20108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 --2- 172.21.15.19:0/1204397333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 0x7fef20108200 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fef080099c0 tx=0x7fef08009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 -- 172.21.15.19:0/1204397333 shutdown_connections 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 --2- 172.21.15.19:0/1204397333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef20108740 0x7fef201100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 --2- 172.21.15.19:0/1204397333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef201034c0 0x7fef201038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 --2- 172.21.15.19:0/1204397333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 0x7fef20108200 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 -- 172.21.15.19:0/1204397333 >> 172.21.15.19:0/1204397333 conn(0x7fef200fedf0 msgr2=0x7fef20101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 -- 172.21.15.19:0/1204397333 shutdown_connections 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 -- 172.21.15.19:0/1204397333 wait complete. 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 Processor -- start 2024-05-09T01:28:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.110+0000 7fef26ea1700 1 -- start start 2024-05-09T01:28:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef26ea1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef201034c0 0x7fef2019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef26ea1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 0x7fef2019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef26ea1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef20108740 0x7fef2019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef26ea1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef20113350 con 0x7fef20107df0 2024-05-09T01:28:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef26ea1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef201131d0 con 0x7fef201034c0 2024-05-09T01:28:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef26ea1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef201134d0 con 0x7fef20108740 2024-05-09T01:28:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 0x7fef2019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 0x7fef2019c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59954/0 (socket says 172.21.15.19:59954) 2024-05-09T01:28:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef1ffff700 1 -- 172.21.15.19:0/636193111 learned_addr learned my addr 172.21.15.19:0/636193111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef2543e700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef20108740 0x7fef2019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef24c3d700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef201034c0 0x7fef2019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef1ffff700 1 -- 172.21.15.19:0/636193111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef20108740 msgr2=0x7fef2019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef1ffff700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef20108740 0x7fef2019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef1ffff700 1 -- 172.21.15.19:0/636193111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef201034c0 msgr2=0x7fef2019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef1ffff700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef201034c0 0x7fef2019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef1ffff700 1 -- 172.21.15.19:0/636193111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef201a7040 con 0x7fef20107df0 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.111+0000 7fef1ffff700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 0x7fef2019c230 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fef08006010 tx=0x7fef080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.112+0000 7fef1dffb700 1 -- 172.21.15.19:0/636193111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08027070 con 0x7fef20107df0 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.112+0000 7fef26ea1700 1 -- 172.21.15.19:0/636193111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef201a72d0 con 0x7fef20107df0 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.112+0000 7fef1dffb700 1 -- 172.21.15.19:0/636193111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef08003fa0 con 0x7fef20107df0 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.112+0000 7fef1dffb700 1 -- 172.21.15.19:0/636193111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08022cd0 con 0x7fef20107df0 2024-05-09T01:28:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.112+0000 7fef26ea1700 1 -- 172.21.15.19:0/636193111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef201a7780 con 0x7fef20107df0 2024-05-09T01:28:33.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.115+0000 7fef1dffb700 1 -- 172.21.15.19:0/636193111 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fef0802a070 con 0x7fef20107df0 2024-05-09T01:28:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef1dffb700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fef0c0b2880 0x7fef0c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef24c3d700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fef0c0b2880 0x7fef0c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef358b7b80 1 Processor -- start 2024-05-09T01:28:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef1dffb700 1 -- 172.21.15.19:0/636193111 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fef080e3100 con 0x7fef20107df0 2024-05-09T01:28:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef358b7b80 1 -- start start 2024-05-09T01:28:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef24c3d700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fef0c0b2880 0x7fef0c0b4d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fef10009d10 tx=0x7fef10009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e4b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 0x560996e4ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996e456a0 0x560996e45ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef358b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560996d67ff0 con 0x560996db2160 2024-05-09T01:28:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef358b7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560996d067f0 con 0x560996e456a0 2024-05-09T01:28:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.116+0000 7fef358b7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560996d63fb0 con 0x560996cc84a0 2024-05-09T01:28:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef24c3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e4b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef24c3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e4b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59966/0 (socket says 172.21.15.19:59966) 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef24c3d700 1 -- 172.21.15.19:0/2249572334 learned_addr learned my addr 172.21.15.19:0/2249572334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef2543e700 1 --2- 172.21.15.19:0/2249572334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996e456a0 0x560996e45ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef1ffff700 1 --2- 172.21.15.19:0/2249572334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 0x560996e4ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef24c3d700 1 -- 172.21.15.19:0/2249572334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 msgr2=0x560996e4ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef24c3d700 1 --2- 172.21.15.19:0/2249572334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 0x560996e4ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef24c3d700 1 -- 172.21.15.19:0/2249572334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996e456a0 msgr2=0x560996e45ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef24c3d700 1 --2- 172.21.15.19:0/2249572334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996e456a0 0x560996e45ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef24c3d700 1 -- 172.21.15.19:0/2249572334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560996cfe370 con 0x560996db2160 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef24c3d700 1 --2- 172.21.15.19:0/2249572334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e4b4f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fef100164a0 tx=0x7fef10012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a205beafc662c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef16ffd700 1 -- 172.21.15.19:0/2249572334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10026070 con 0x560996db2160 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef16ffd700 1 -- 172.21.15.19:0/2249572334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef10016e50 con 0x560996db2160 2024-05-09T01:28:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.117+0000 7fef16ffd700 1 -- 172.21.15.19:0/2249572334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10014940 con 0x560996db2160 2024-05-09T01:28:33.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.118+0000 7fef358b7b80 1 -- 172.21.15.19:0/2249572334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 msgr2=0x560996e4b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.118+0000 7fef358b7b80 1 --2- 172.21.15.19:0/2249572334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e4b4f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fef100164a0 tx=0x7fef10012000 comp rx=0 tx=0).stop 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.118+0000 7fef358b7b80 1 -- 172.21.15.19:0/2249572334 shutdown_connections 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.118+0000 7fef358b7b80 1 --2- 172.21.15.19:0/2249572334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996e456a0 0x560996e45ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.118+0000 7fef358b7b80 1 --2- 172.21.15.19:0/2249572334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e4b4f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.118+0000 7fef358b7b80 1 --2- 172.21.15.19:0/2249572334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 0x560996e4ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.118+0000 7fef358b7b80 1 -- 172.21.15.19:0/2249572334 >> 172.21.15.19:0/2249572334 conn(0x560996d286a0 msgr2=0x560996e40400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.118+0000 7fef358b7b80 1 -- 172.21.15.19:0/2249572334 shutdown_connections 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.118+0000 7fef358b7b80 1 -- 172.21.15.19:0/2249572334 wait complete. 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef358b7b80 1 -- start start 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef358b7b80 1 Processor -- start 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996db2160 0x560996e4f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e4fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef358b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560996be3ba0 con 0x560996e456a0 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef358b7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560996d22150 con 0x560996cc84a0 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef358b7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560996e521d0 con 0x560996db2160 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef358b7b80 10 client.0 _ll_register_callbacks cb 0x560996bd96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.119+0000 7fef2543e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e4fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef24c3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef24c3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:34060/0 (socket says 172.21.15.19:34060) 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef24c3d700 1 -- 172.21.15.19:0/1978786700 learned_addr learned my addr 172.21.15.19:0/1978786700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef2543e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e4fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59982/0 (socket says 172.21.15.19:59982) 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef1ffff700 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996db2160 0x560996e4f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef2543e700 1 -- 172.21.15.19:0/1978786700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996db2160 msgr2=0x560996e4f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef2543e700 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996db2160 0x560996e4f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef2543e700 1 -- 172.21.15.19:0/1978786700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 msgr2=0x560996e4f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef2543e700 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef2543e700 1 -- 172.21.15.19:0/1978786700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560996be3490 con 0x560996e456a0 2024-05-09T01:28:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef2543e700 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e4fc50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fef20076ad0 tx=0x7fef18011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8526fbd5704bd7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7feefffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7feefffff700 1 -- 172.21.15.19:0/1978786700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18011de0 con 0x560996e456a0 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7feefffff700 1 -- 172.21.15.19:0/1978786700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fef18004830 con 0x560996e456a0 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef358b7b80 1 -- 172.21.15.19:0/1978786700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef10016e50 con 0x560996e456a0 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7feefffff700 1 -- 172.21.15.19:0/1978786700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef18004a50 con 0x560996e456a0 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef358b7b80 10 client.15942 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.120+0000 7fef358b7b80 1 -- 172.21.15.19:0/1978786700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560996e5cc00 con 0x560996e456a0 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.121+0000 7feefffff700 1 -- 172.21.15.19:0/1978786700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef10016e50 con 0x560996e456a0 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.121+0000 7feefffff700 1 -- 172.21.15.19:0/1978786700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fef180265e0 con 0x560996e456a0 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.121+0000 7fef358b7b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.121+0000 7fef358b7b80 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.121+0000 7fef358b7b80 20 client.15942 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.121+0000 7fef358b7b80 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560996e5f240 0x560996e61680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.121+0000 7fef358b7b80 4 client.15942 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.121+0000 7fef358b7b80 1 -- 172.21.15.19:0/1978786700 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560996d759d0 con 0x560996e5f240 2024-05-09T01:28:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.121+0000 7fef24c3d700 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560996e5f240 0x560996e61680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.122+0000 7fef24c3d700 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560996e5f240 0x560996e61680 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1366650b014e2bbb server_cookie=bb16e04a4965c222 in_seq=0 out_seq=0 2024-05-09T01:28:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.122+0000 7feefffff700 10 client.15942 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7feefffff700 1 -- 172.21.15.19:0/1978786700 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560996d759d0 con 0x560996e5f240 2024-05-09T01:28:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7feefffff700 10 client.15942 handle_command_reply: tid=0 2024-05-09T01:28:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 client.15942 shutdown 2024-05-09T01:28:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 -- 172.21.15.19:0/1978786700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 msgr2=0x560996e4fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e4fc50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fef20076ad0 tx=0x7fef18011720 comp rx=0 tx=0).stop 2024-05-09T01:28:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 -- 172.21.15.19:0/1978786700 shutdown_connections 2024-05-09T01:28:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560996e5f240 0x560996e61680 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e4fc50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 --2- 172.21.15.19:0/1978786700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996db2160 0x560996e4f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 -- 172.21.15.19:0/1978786700 >> 172.21.15.19:0/1978786700 conn(0x560996d286a0 msgr2=0x560996e401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7feefffff700 0 client.15942 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 -- 172.21.15.19:0/1978786700 shutdown_connections 2024-05-09T01:28:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 1 -- 172.21.15.19:0/1978786700 wait complete. 2024-05-09T01:28:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.123+0000 7fef358b7b80 20 client.15942 trim_cache size 0 max 16384 2024-05-09T01:28:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.143+0000 7fef358b7b80 1 Processor -- start 2024-05-09T01:28:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.143+0000 7fef358b7b80 1 -- start start 2024-05-09T01:28:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e49510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996e456a0 0x560996e5afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef358b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef201acaf0 con 0x560996db2160 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef358b7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef201ae630 con 0x560996cc84a0 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef358b7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef201ad1c0 con 0x560996e456a0 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e49510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e49510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59996/0 (socket says 172.21.15.19:59996) 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef1ffff700 1 -- 172.21.15.19:0/924002187 learned_addr learned my addr 172.21.15.19:0/924002187 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef2543e700 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996e456a0 0x560996e5afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef24c3d700 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef1ffff700 1 -- 172.21.15.19:0/924002187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996e456a0 msgr2=0x560996e5afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef1ffff700 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996e456a0 0x560996e5afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef1ffff700 1 -- 172.21.15.19:0/924002187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 msgr2=0x560996e4e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef1ffff700 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef1ffff700 1 -- 172.21.15.19:0/924002187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560996e39f60 con 0x560996db2160 2024-05-09T01:28:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef24c3d700 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.144+0000 7fef1ffff700 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e49510 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fef080aa6d0 tx=0x7fef080b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3448155c7fc53b2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.145+0000 7fef167fc700 1 -- 172.21.15.19:0/924002187 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef080095c0 con 0x560996db2160 2024-05-09T01:28:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.145+0000 7fef167fc700 1 -- 172.21.15.19:0/924002187 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef080aa2a0 con 0x560996db2160 2024-05-09T01:28:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.145+0000 7fef167fc700 1 -- 172.21.15.19:0/924002187 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef080aab70 con 0x560996db2160 2024-05-09T01:28:33.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.145+0000 7fef358b7b80 1 -- 172.21.15.19:0/924002187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 msgr2=0x560996e49510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e49510 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fef080aa6d0 tx=0x7fef080b1040 comp rx=0 tx=0).stop 2024-05-09T01:28:33.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 -- 172.21.15.19:0/924002187 shutdown_connections 2024-05-09T01:28:33.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996e456a0 0x560996e5afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996cc84a0 0x560996e4e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 --2- 172.21.15.19:0/924002187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996db2160 0x560996e49510 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 -- 172.21.15.19:0/924002187 >> 172.21.15.19:0/924002187 conn(0x560996d286a0 msgr2=0x560996e48e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 -- 172.21.15.19:0/924002187 shutdown_connections 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 -- 172.21.15.19:0/924002187 wait complete. 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 -- start start 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.146+0000 7fef358b7b80 1 Processor -- start 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 0x560996e575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996db2160 0x560996e540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef358b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e54610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef358b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef201adf60 con 0x560996e456a0 2024-05-09T01:28:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef358b7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef201ad890 con 0x560996db2160 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef358b7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef201aaab0 con 0x560996cc84a0 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef358b7b80 10 client.0 _ll_register_callbacks cb 0x560996bd96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef2543e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e54610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996db2160 0x560996e540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef24c3d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 0x560996e575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef2543e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e54610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60008/0 (socket says 172.21.15.19:60008) 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef2543e700 1 -- 172.21.15.19:0/717665756 learned_addr learned my addr 172.21.15.19:0/717665756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef24c3d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 0x560996e575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36062/0 (socket says 172.21.15.19:36062) 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef2543e700 1 -- 172.21.15.19:0/717665756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 msgr2=0x560996e575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef2543e700 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 0x560996e575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef2543e700 1 -- 172.21.15.19:0/717665756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996db2160 msgr2=0x560996e540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef2543e700 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996db2160 0x560996e540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef2543e700 1 -- 172.21.15.19:0/717665756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560996e54b50 con 0x560996e456a0 2024-05-09T01:28:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7fef2543e700 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e54610 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fef180059d0 tx=0x7fef18004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1682f43a04d49f82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.147+0000 7feefffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7feefffff700 1 -- 172.21.15.19:0/717665756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18035060 con 0x560996e456a0 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7feefffff700 1 -- 172.21.15.19:0/717665756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fef18026630 con 0x560996e456a0 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7fef358b7b80 1 -- 172.21.15.19:0/717665756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef080aa2a0 con 0x560996e456a0 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7feefffff700 1 -- 172.21.15.19:0/717665756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef180048b0 con 0x560996e456a0 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7fef358b7b80 10 client.15948 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7fef358b7b80 1 -- 172.21.15.19:0/717665756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560996ed2db0 con 0x560996e456a0 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7feefffff700 1 -- 172.21.15.19:0/717665756 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef080aa2a0 con 0x560996e456a0 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7feefffff700 1 -- 172.21.15.19:0/717665756 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fef18030430 con 0x560996e456a0 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7fef358b7b80 10 client.15948 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7fef358b7b80 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7fef358b7b80 20 client.15948 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7fef358b7b80 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560996ed3490 0x560996ed58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7fef358b7b80 4 client.15948 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.148+0000 7fef358b7b80 1 -- 172.21.15.19:0/717665756 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560996d6fa70 con 0x560996ed3490 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.149+0000 7fef24c3d700 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560996ed3490 0x560996ed58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.149+0000 7fef24c3d700 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560996ed3490 0x560996ed58b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69aab7a4e78bdf1 server_cookie=4a94001ccad7f0d5 in_seq=0 out_seq=0 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.149+0000 7feefffff700 10 client.15948 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.149+0000 7feefffff700 1 -- 172.21.15.19:0/717665756 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560996d6fa70 con 0x560996ed3490 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.149+0000 7feefffff700 10 client.15948 handle_command_reply: tid=0 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.149+0000 7fef358b7b80 1 client.15948 shutdown 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 -- 172.21.15.19:0/717665756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 msgr2=0x560996e54610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e54610 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fef180059d0 tx=0x7fef18004690 comp rx=0 tx=0).stop 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 -- 172.21.15.19:0/717665756 shutdown_connections 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560996ed3490 0x560996ed58b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560996e456a0 0x560996e54610 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560996cc84a0 0x560996e575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 --2- 172.21.15.19:0/717665756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560996db2160 0x560996e540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 -- 172.21.15.19:0/717665756 >> 172.21.15.19:0/717665756 conn(0x560996d286a0 msgr2=0x560996e4e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7feefffff700 0 client.15948 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 -- 172.21.15.19:0/717665756 shutdown_connections 2024-05-09T01:28:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 1 -- 172.21.15.19:0/717665756 wait complete. 2024-05-09T01:28:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.150+0000 7fef358b7b80 20 client.15948 trim_cache size 0 max 16384 2024-05-09T01:28:33.153 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:28:33.153 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:28:33.153 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:28:33.153 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 -- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fef0c0b2880 msgr2=0x7fef0c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fef0c0b2880 0x7fef0c0b4d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fef10009d10 tx=0x7fef10009040 comp rx=0 tx=0).stop 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 -- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 msgr2=0x7fef2019c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 0x7fef2019c230 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fef08006010 tx=0x7fef080047b0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 -- 172.21.15.19:0/636193111 shutdown_connections 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef20108740 0x7fef2019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef201034c0 0x7fef2019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fef0c0b2880 0x7fef0c0b4d90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 --2- 172.21.15.19:0/636193111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef20107df0 0x7fef2019c230 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 -- 172.21.15.19:0/636193111 >> 172.21.15.19:0/636193111 conn(0x7fef200fedf0 msgr2=0x7fef20101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 -- 172.21.15.19:0/636193111 shutdown_connections 2024-05-09T01:28:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:33.151+0000 7fef26ea1700 1 -- 172.21.15.19:0/636193111 wait complete. 2024-05-09T01:28:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 -- 172.21.15.19:0/2672514916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102b40 msgr2=0x7f0cbc103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 --2- 172.21.15.19:0/2672514916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102b40 0x7f0cbc103010 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009900 tx=0x7f0cb002a410 comp rx=0 tx=0).stop 2024-05-09T01:28:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 -- 172.21.15.19:0/2672514916 shutdown_connections 2024-05-09T01:28:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 --2- 172.21.15.19:0/2672514916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cbc1036a0 0x7f0cbc103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 --2- 172.21.15.19:0/2672514916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102b40 0x7f0cbc103010 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 --2- 172.21.15.19:0/2672514916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cbc102160 0x7f0cbc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 -- 172.21.15.19:0/2672514916 >> 172.21.15.19:0/2672514916 conn(0x7f0cbc0fdd30 msgr2=0x7f0cbc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 -- 172.21.15.19:0/2672514916 shutdown_connections 2024-05-09T01:28:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 -- 172.21.15.19:0/2672514916 wait complete. 2024-05-09T01:28:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.455+0000 7f0cc1134640 1 Processor -- start 2024-05-09T01:28:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cc1134640 1 -- start start 2024-05-09T01:28:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cc1134640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102160 0x7f0cbc19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cc1134640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cbc102b40 0x7f0cbc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cc1134640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cbc1036a0 0x7f0cbc1a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cc1134640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc111940 con 0x7f0cbc102160 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cc1134640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1117c0 con 0x7f0cbc1036a0 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cc1134640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc111ac0 con 0x7f0cbc102b40 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cbad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102160 0x7f0cbc19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cbad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102160 0x7f0cbc19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60020/0 (socket says 172.21.15.19:60020) 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cbad9d640 1 -- 172.21.15.19:0/1281484359 learned_addr learned my addr 172.21.15.19:0/1281484359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cba59c640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cbc102b40 0x7f0cbc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cbc1036a0 0x7f0cbc1a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cbad9d640 1 -- 172.21.15.19:0/1281484359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cbc102b40 msgr2=0x7f0cbc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cbad9d640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cbc102b40 0x7f0cbc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cbad9d640 1 -- 172.21.15.19:0/1281484359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cbc1036a0 msgr2=0x7f0cbc1a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cbad9d640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cbc1036a0 0x7f0cbc1a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.456+0000 7f0cbad9d640 1 -- 172.21.15.19:0/1281484359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc1a40c0 con 0x7f0cbc102160 2024-05-09T01:28:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.457+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cbc1036a0 0x7f0cbc1a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.457+0000 7f0cbad9d640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102160 0x7f0cbc19c580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d950 tx=0x7f0cac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.457+0000 7f0ca3fff640 1 -- 172.21.15.19:0/1281484359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac017070 con 0x7f0cbc102160 2024-05-09T01:28:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.457+0000 7f0cc1134640 1 -- 172.21.15.19:0/1281484359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cbc1a43b0 con 0x7f0cbc102160 2024-05-09T01:28:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.457+0000 7f0ca3fff640 1 -- 172.21.15.19:0/1281484359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cac009c30 con 0x7f0cbc102160 2024-05-09T01:28:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.457+0000 7f0ca3fff640 1 -- 172.21.15.19:0/1281484359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00f620 con 0x7f0cbc102160 2024-05-09T01:28:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.457+0000 7f0cc1134640 1 -- 172.21.15.19:0/1281484359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cbc1a4900 con 0x7f0cbc102160 2024-05-09T01:28:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cc7fe5740 1 Processor -- start 2024-05-09T01:28:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cc7fe5740 1 -- start start 2024-05-09T01:28:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0ca3fff640 1 -- 172.21.15.19:0/1281484359 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0cac01a070 con 0x7f0cbc102160 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54c3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b4c54277c0 con 0x55b4c511f1b0 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b4c53c30e0 con 0x55b4c50f62f0 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b4c5423780 con 0x55b4c50fe4d0 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54c3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60030/0 (socket says 172.21.15.19:60030) 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cbb59e640 1 -- 172.21.15.19:0/3775249667 learned_addr learned my addr 172.21.15.19:0/3775249667 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cbad9d640 1 --2- 172.21.15.19:0/3775249667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.460+0000 7f0cba59c640 1 --2- 172.21.15.19:0/3775249667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0cbb59e640 1 -- 172.21.15.19:0/3775249667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 msgr2=0x55b4c54ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/3775249667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0cbb59e640 1 -- 172.21.15.19:0/3775249667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 msgr2=0x55b4c54c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/3775249667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0cbb59e640 1 -- 172.21.15.19:0/3775249667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c53ec7c0 con 0x55b4c511f1b0 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/3775249667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54c3500 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009fb0 tx=0x7f0cb4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b5280a71de6a8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0ca3fff640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ca40b27e0 0x7f0ca40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0ca17fa640 1 -- 172.21.15.19:0/3775249667 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4016070 con 0x55b4c511f1b0 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0cbad9d640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ca40b27e0 0x7f0ca40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0ca17fa640 1 -- 172.21.15.19:0/3775249667 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb4012bc0 con 0x55b4c511f1b0 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0ca17fa640 1 -- 172.21.15.19:0/3775249667 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4011bc0 con 0x55b4c511f1b0 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.461+0000 7f0ca3fff640 1 -- 172.21.15.19:0/1281484359 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0cac0e1cd0 con 0x7f0cbc102160 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/3775249667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 msgr2=0x55b4c54c3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/3775249667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54c3500 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009fb0 tx=0x7f0cb4010800 comp rx=0 tx=0).stop 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cbad9d640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ca40b27e0 0x7f0ca40b4cf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55b4c54c48e0 tx=0x7f0cac0e57b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/3775249667 shutdown_connections 2024-05-09T01:28:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/3775249667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/3775249667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/3775249667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54c3500 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/3775249667 >> 172.21.15.19:0/3775249667 conn(0x55b4c5113300 msgr2=0x55b4c54bee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/3775249667 shutdown_connections 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/3775249667 wait complete. 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 -- start start 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.462+0000 7f0cc7fe5740 1 Processor -- start 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54d46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b4c54cb3d0 con 0x55b4c511f1b0 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b4c53ae9c0 con 0x55b4c50f62f0 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b4c54cb550 con 0x55b4c50fe4d0 2024-05-09T01:28:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cc7fe5740 10 client.0 _ll_register_callbacks cb 0x55b4c53db020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54d46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54d46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60038/0 (socket says 172.21.15.19:60038) 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 -- 172.21.15.19:0/505936289 learned_addr learned my addr 172.21.15.19:0/505936289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cba59c640 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 -- 172.21.15.19:0/505936289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 msgr2=0x55b4c54cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 -- 172.21.15.19:0/505936289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 msgr2=0x55b4c54cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 -- 172.21.15.19:0/505936289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c53d51f0 con 0x55b4c511f1b0 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54d46d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4011010 tx=0x7f0cb40098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bd2eb9b99c186c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0c9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.463+0000 7f0c9eff5640 1 -- 172.21.15.19:0/505936289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4016040 con 0x55b4c511f1b0 2024-05-09T01:28:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0c9eff5640 1 -- 172.21.15.19:0/505936289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0cb4004c50 con 0x55b4c511f1b0 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/505936289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb4012bc0 con 0x55b4c511f1b0 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0c9eff5640 1 -- 172.21.15.19:0/505936289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cb4031070 con 0x55b4c511f1b0 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cc7fe5740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/505936289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b4c518a790 con 0x55b4c511f1b0 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0c9eff5640 1 -- 172.21.15.19:0/505936289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb4012bc0 con 0x55b4c511f1b0 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0c9eff5640 1 -- 172.21.15.19:0/505936289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0cb400f040 con 0x55b4c511f1b0 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cc7fe5740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cc7fe5740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cc7fe5740 20 client.15960 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b4c50ddd80 0x55b4c54da9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cc7fe5740 4 client.15960 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/505936289 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4c54353d0 con 0x55b4c50ddd80 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.464+0000 7f0cba59c640 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b4c50ddd80 0x55b4c54da9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.465+0000 7f0c9eff5640 10 client.15960 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.465+0000 7f0cba59c640 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b4c50ddd80 0x55b4c54da9c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33729d5f05a9d288 server_cookie=25b01c3e035c8de1 in_seq=0 out_seq=0 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0c9eff5640 1 -- 172.21.15.19:0/505936289 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b4c54353d0 con 0x55b4c50ddd80 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0c9eff5640 10 client.15960 handle_command_reply: tid=0 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 client.15960 shutdown 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/505936289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 msgr2=0x55b4c54d46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54d46d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4011010 tx=0x7f0cb40098a0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/505936289 shutdown_connections 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b4c50ddd80 0x55b4c54da9c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0c9eff5640 0 client.15960 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c511f1b0 0x55b4c54d46d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/505936289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/505936289 >> 172.21.15.19:0/505936289 conn(0x55b4c5113300 msgr2=0x55b4c54c1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/505936289 shutdown_connections 2024-05-09T01:28:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/505936289 wait complete. 2024-05-09T01:28:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.466+0000 7f0cc7fe5740 20 client.15960 trim_cache size 0 max 16384 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.481+0000 7f0cc7fe5740 1 Processor -- start 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.481+0000 7f0cc7fe5740 1 -- start start 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.481+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50ddd80 0x55b4c54c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.481+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.481+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.481+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1ad6d0 con 0x55b4c50ddd80 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.481+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1af210 con 0x55b4c50f62f0 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.481+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1adda0 con 0x55b4c50fe4d0 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.481+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50ddd80 0x55b4c54c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50ddd80 0x55b4c54c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60054/0 (socket says 172.21.15.19:60054) 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cbb59e640 1 -- 172.21.15.19:0/4147994342 learned_addr learned my addr 172.21.15.19:0/4147994342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cba59c640 1 --2- 172.21.15.19:0/4147994342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cbb59e640 1 -- 172.21.15.19:0/4147994342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 msgr2=0x55b4c54cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/4147994342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cbb59e640 1 -- 172.21.15.19:0/4147994342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 msgr2=0x55b4c54c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/4147994342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cbb59e640 1 -- 172.21.15.19:0/4147994342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c52015f0 con 0x55b4c50ddd80 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cba59c640 1 --2- 172.21.15.19:0/4147994342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/4147994342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50ddd80 0x55b4c54c7620 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4013eb0 tx=0x7f0cb4009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a839d67eb97d75d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0ca0ff9640 1 -- 172.21.15.19:0/4147994342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb403c070 con 0x55b4c50ddd80 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0ca0ff9640 1 -- 172.21.15.19:0/4147994342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb4011450 con 0x55b4c50ddd80 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.482+0000 7f0ca0ff9640 1 -- 172.21.15.19:0/4147994342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb401f070 con 0x55b4c50ddd80 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/4147994342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50ddd80 msgr2=0x55b4c54c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/4147994342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50ddd80 0x55b4c54c7620 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4013eb0 tx=0x7f0cb4009e80 comp rx=0 tx=0).stop 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/4147994342 shutdown_connections 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/4147994342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50fe4d0 0x55b4c54cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/4147994342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50f62f0 0x55b4c54c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/4147994342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50ddd80 0x55b4c54c7620 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/4147994342 >> 172.21.15.19:0/4147994342 conn(0x55b4c5113300 msgr2=0x55b4c54c34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/4147994342 shutdown_connections 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/4147994342 wait complete. 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 -- start start 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.483+0000 7f0cc7fe5740 1 Processor -- start 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50ddd80 0x55b4c54d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50f62f0 0x55b4c54d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cc7fe5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50fe4d0 0x55b4c5540d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1aeb40 con 0x55b4c50fe4d0 2024-05-09T01:28:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1ae470 con 0x55b4c50ddd80 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cc7fe5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1ab6b0 con 0x55b4c50f62f0 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cc7fe5740 10 client.0 _ll_register_callbacks cb 0x7f0cac0e5680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50fe4d0 0x55b4c5540d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50fe4d0 0x55b4c5540d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60066/0 (socket says 172.21.15.19:60066) 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 -- 172.21.15.19:0/488903770 learned_addr learned my addr 172.21.15.19:0/488903770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cba59c640 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50f62f0 0x55b4c54d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50ddd80 0x55b4c54d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 -- 172.21.15.19:0/488903770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50f62f0 msgr2=0x55b4c54d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50f62f0 0x55b4c54d0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 -- 172.21.15.19:0/488903770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50ddd80 msgr2=0x55b4c54d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50ddd80 0x55b4c54d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 -- 172.21.15.19:0/488903770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c53ebed0 con 0x55b4c50fe4d0 2024-05-09T01:28:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cba59c640 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50f62f0 0x55b4c54d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cbb59e640 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50fe4d0 0x55b4c5540d60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0cb403a040 tx=0x7f0cb4005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbf601d1b8249d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0c9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0c9eff5640 1 -- 172.21.15.19:0/488903770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb401f070 con 0x55b4c50fe4d0 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0c9eff5640 1 -- 172.21.15.19:0/488903770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0cb402d8e0 con 0x55b4c50fe4d0 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.484+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/488903770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb4011450 con 0x55b4c50fe4d0 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0c9eff5640 1 -- 172.21.15.19:0/488903770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cb403c030 con 0x55b4c50fe4d0 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0cc7fe5740 10 client.15966 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/488903770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b4c5541f90 con 0x55b4c50fe4d0 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0c9eff5640 1 -- 172.21.15.19:0/488903770 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb4011450 con 0x55b4c50fe4d0 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0c9eff5640 1 -- 172.21.15.19:0/488903770 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0cb4038540 con 0x55b4c50fe4d0 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0cc7fe5740 10 client.15966 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0cc7fe5740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0cc7fe5740 20 client.15966 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b4c511f1b0 0x55b4c5544830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0cc7fe5740 4 client.15966 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:28:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/488903770 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4c542f240 con 0x55b4c511f1b0 2024-05-09T01:28:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.485+0000 7f0cba59c640 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b4c511f1b0 0x55b4c5544830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.486+0000 7f0c9eff5640 10 client.15966 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.486+0000 7f0cba59c640 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b4c511f1b0 0x55b4c5544830 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97c69f6e8336590 server_cookie=f2c56f3bb501a681 in_seq=0 out_seq=0 2024-05-09T01:28:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.500+0000 7f0c9eff5640 1 -- 172.21.15.19:0/488903770 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b4c542f240 con 0x55b4c511f1b0 2024-05-09T01:28:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.500+0000 7f0c9eff5640 10 client.15966 handle_command_reply: tid=0 2024-05-09T01:28:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.500+0000 7f0cc7fe5740 1 client.15966 shutdown 2024-05-09T01:28:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/488903770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50fe4d0 msgr2=0x55b4c5540d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50fe4d0 0x55b4c5540d60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0cb403a040 tx=0x7f0cb4005d10 comp rx=0 tx=0).stop 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/488903770 shutdown_connections 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b4c511f1b0 0x55b4c5544830 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b4c50fe4d0 0x55b4c5540d60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0c9eff5640 0 client.15966 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4c50f62f0 0x55b4c54d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 --2- 172.21.15.19:0/488903770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4c50ddd80 0x55b4c54d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/488903770 >> 172.21.15.19:0/488903770 conn(0x55b4c5113300 msgr2=0x55b4c54c3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/488903770 shutdown_connections 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 1 -- 172.21.15.19:0/488903770 wait complete. 2024-05-09T01:28:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc7fe5740 20 client.15966 trim_cache size 0 max 16384 2024-05-09T01:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:28:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 -- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ca40b27e0 msgr2=0x7f0ca40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ca40b27e0 0x7f0ca40b4cf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55b4c54c48e0 tx=0x7f0cac0e57b0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 -- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102160 msgr2=0x7f0cbc19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102160 0x7f0cbc19c580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d950 tx=0x7f0cac00de10 comp rx=0 tx=0).stop 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 -- 172.21.15.19:0/1281484359 shutdown_connections 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ca40b27e0 0x7f0ca40b4cf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cbc1036a0 0x7f0cbc1a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cbc102b40 0x7f0cbc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 --2- 172.21.15.19:0/1281484359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cbc102160 0x7f0cbc19c580 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.501+0000 7f0cc1134640 1 -- 172.21.15.19:0/1281484359 >> 172.21.15.19:0/1281484359 conn(0x7f0cbc0fdd30 msgr2=0x7f0cbc10a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.502+0000 7f0cc1134640 1 -- 172.21.15.19:0/1281484359 shutdown_connections 2024-05-09T01:28:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:33.502+0000 7f0cc1134640 1 -- 172.21.15.19:0/1281484359 wait complete. 2024-05-09T01:28:33.912 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:28:33.912 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:28:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:34.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:34 smithi019 ceph-mon[30158]: pgmap v469: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:28:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:34 smithi059 ceph-mon[34351]: pgmap v469: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:28:34.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:34 smithi196 ceph-mon[35041]: pgmap v469: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.956+0000 7fbcb9590700 1 -- 172.21.15.19:0/2426545918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb4109850 msgr2=0x7fbcb41111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.956+0000 7fbcb9590700 1 --2- 172.21.15.19:0/2426545918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb4109850 0x7fbcb41111b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0099c0 tx=0x7fbcac009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.956+0000 7fbcb9590700 1 -- 172.21.15.19:0/2426545918 shutdown_connections 2024-05-09T01:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.956+0000 7fbcb9590700 1 --2- 172.21.15.19:0/2426545918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb4109850 0x7fbcb41111b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.956+0000 7fbcb9590700 1 --2- 172.21.15.19:0/2426545918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb41045d0 0x7fbcb41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.956+0000 7fbcb9590700 1 --2- 172.21.15.19:0/2426545918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4108f00 0x7fbcb4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.956+0000 7fbcb9590700 1 -- 172.21.15.19:0/2426545918 >> 172.21.15.19:0/2426545918 conn(0x7fbcb40ffea0 msgr2=0x7fbcb41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.956+0000 7fbcb9590700 1 -- 172.21.15.19:0/2426545918 shutdown_connections 2024-05-09T01:28:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.956+0000 7fbcb9590700 1 -- 172.21.15.19:0/2426545918 wait complete. 2024-05-09T01:28:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.957+0000 7fbcb9590700 1 Processor -- start 2024-05-09T01:28:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb9590700 1 -- start start 2024-05-09T01:28:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb9590700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb41045d0 0x7fbcb419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb9590700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb4108f00 0x7fbcb419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb9590700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4109850 0x7fbcb419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb9590700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4114480 con 0x7fbcb4108f00 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb9590700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4114300 con 0x7fbcb4109850 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb9590700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4114600 con 0x7fbcb41045d0 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb4108f00 0x7fbcb419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb2ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb41045d0 0x7fbcb419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4109850 0x7fbcb419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb2ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb41045d0 0x7fbcb419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36142/0 (socket says 172.21.15.19:36142) 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.958+0000 7fbcb2ffd700 1 -- 172.21.15.19:0/3560100003 learned_addr learned my addr 172.21.15.19:0/3560100003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbcb2ffd700 1 -- 172.21.15.19:0/3560100003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4109850 msgr2=0x7fbcb419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbcb2ffd700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4109850 0x7fbcb419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbcb2ffd700 1 -- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb4108f00 msgr2=0x7fbcb419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbcb2ffd700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb4108f00 0x7fbcb419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbcb2ffd700 1 -- 172.21.15.19:0/3560100003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb419fc10 con 0x7fbcb41045d0 2024-05-09T01:28:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbcb27fc700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb4108f00 0x7fbcb419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:35.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbcb2ffd700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb41045d0 0x7fbcb419adf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbca4007e20 tx=0x7fbca400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:35.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbc9bfff700 1 -- 172.21.15.19:0/3560100003 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4016020 con 0x7fbcb41045d0 2024-05-09T01:28:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb419ff00 con 0x7fbcb41045d0 2024-05-09T01:28:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbc9bfff700 1 -- 172.21.15.19:0/3560100003 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca400ac90 con 0x7fbcb41045d0 2024-05-09T01:28:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbc9bfff700 1 -- 172.21.15.19:0/3560100003 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca40115e0 con 0x7fbcb41045d0 2024-05-09T01:28:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.959+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb41a78d0 con 0x7fbcb41045d0 2024-05-09T01:28:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.960+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb40512c0 con 0x7fbcb41045d0 2024-05-09T01:28:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.962+0000 7fbc9bfff700 1 -- 172.21.15.19:0/3560100003 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbca4019020 con 0x7fbcb41045d0 2024-05-09T01:28:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.963+0000 7fbc9bfff700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc9c0b2880 0x7fbc9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.963+0000 7fbcb27fc700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc9c0b2880 0x7fbc9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.963+0000 7fbc9bfff700 1 -- 172.21.15.19:0/3560100003 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbca40e22f0 con 0x7fbcb41045d0 2024-05-09T01:28:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.963+0000 7fbcb27fc700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc9c0b2880 0x7fbc9c0b4d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4103880 tx=0x7fbca800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:35.967+0000 7fbc9bfff700 1 -- 172.21.15.19:0/3560100003 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbca4020070 con 0x7fbcb41045d0 2024-05-09T01:28:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.228+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcb419c150 con 0x7fbcb41045d0 2024-05-09T01:28:36.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.229+0000 7fbc9bfff700 1 -- 172.21.15.19:0/3560100003 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fbca40a9ff0 con 0x7fbcb41045d0 2024-05-09T01:28:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:28:36.227 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:28:36.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc9c0b2880 msgr2=0x7fbc9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:36.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc9c0b2880 0x7fbc9c0b4d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4103880 tx=0x7fbca800a300 comp rx=0 tx=0).stop 2024-05-09T01:28:36.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb41045d0 msgr2=0x7fbcb419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:36.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb41045d0 0x7fbcb419adf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbca4007e20 tx=0x7fbca400a340 comp rx=0 tx=0).stop 2024-05-09T01:28:36.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 shutdown_connections 2024-05-09T01:28:36.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4109850 0x7fbcb419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:36.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb41045d0 0x7fbcb419adf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:36.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc9c0b2880 0x7fbc9c0b4d90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:36.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 --2- 172.21.15.19:0/3560100003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb4108f00 0x7fbcb419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:36.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 >> 172.21.15.19:0/3560100003 conn(0x7fbcb40ffea0 msgr2=0x7fbcb41086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:36.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.231+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 shutdown_connections 2024-05-09T01:28:36.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:36.232+0000 7fbcb9590700 1 -- 172.21.15.19:0/3560100003 wait complete. 2024-05-09T01:28:36.231 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:28:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:36 smithi059 ceph-mon[34351]: pgmap v470: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:28:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:36 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3560100003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:28:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:36 smithi019 ceph-mon[30158]: pgmap v470: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:28:36.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:36 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3560100003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:28:36.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:28:36.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:36 smithi196 ceph-mon[35041]: pgmap v470: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:28:36.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:36 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3560100003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:28:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:38.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:38 smithi019 ceph-mon[30158]: pgmap v471: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:28:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:38 smithi059 ceph-mon[34351]: pgmap v471: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:28:38.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:38 smithi196 ceph-mon[35041]: pgmap v471: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:28:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.090+0000 7fa45c9c2700 1 -- 172.21.15.19:0/22394118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458108740 msgr2=0x7fa4581100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.090+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/22394118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458108740 0x7fa4581100a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa45000b0d0 tx=0x7fa45000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.090+0000 7fa45c9c2700 1 -- 172.21.15.19:0/22394118 shutdown_connections 2024-05-09T01:28:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.090+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/22394118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458108740 0x7fa4581100a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.090+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/22394118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4581034c0 0x7fa4581038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.090+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/22394118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa458107df0 0x7fa458108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.091+0000 7fa45c9c2700 1 -- 172.21.15.19:0/22394118 >> 172.21.15.19:0/22394118 conn(0x7fa4580fedc0 msgr2=0x7fa4581011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.091+0000 7fa45c9c2700 1 -- 172.21.15.19:0/22394118 shutdown_connections 2024-05-09T01:28:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.091+0000 7fa45c9c2700 1 -- 172.21.15.19:0/22394118 wait complete. 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.091+0000 7fa45c9c2700 1 Processor -- start 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.091+0000 7fa45c9c2700 1 -- start start 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa45c9c2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4581034c0 0x7fa458199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa45c9c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458107df0 0x7fa45819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa45c9c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa458108740 0x7fa45819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa45c9c2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa458113370 con 0x7fa458107df0 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa45c9c2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581131f0 con 0x7fa458108740 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa45c9c2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581134f0 con 0x7fa4581034c0 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa455d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458107df0 0x7fa45819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa455d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458107df0 0x7fa45819a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60080/0 (socket says 172.21.15.19:60080) 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa455d9b700 1 -- 172.21.15.19:0/3687886437 learned_addr learned my addr 172.21.15.19:0/3687886437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa45659c700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4581034c0 0x7fa458199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa456d9d700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa458108740 0x7fa45819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa455d9b700 1 -- 172.21.15.19:0/3687886437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4581034c0 msgr2=0x7fa458199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa455d9b700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4581034c0 0x7fa458199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa455d9b700 1 -- 172.21.15.19:0/3687886437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa458108740 msgr2=0x7fa45819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa455d9b700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa458108740 0x7fa45819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa455d9b700 1 -- 172.21.15.19:0/3687886437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45819eb40 con 0x7fa458107df0 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.092+0000 7fa456d9d700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa458108740 0x7fa45819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.093+0000 7fa455d9b700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458107df0 0x7fa45819a2c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c9e0 tx=0x7fa44c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.093+0000 7fa4477fe700 1 -- 172.21.15.19:0/3687886437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c007aa0 con 0x7fa458107df0 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.093+0000 7fa4477fe700 1 -- 172.21.15.19:0/3687886437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa44c007c40 con 0x7fa458107df0 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.093+0000 7fa4477fe700 1 -- 172.21.15.19:0/3687886437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c018660 con 0x7fa458107df0 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.093+0000 7fa45c9c2700 1 -- 172.21.15.19:0/3687886437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45819ee30 con 0x7fa458107df0 2024-05-09T01:28:39.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.093+0000 7fa45c9c2700 1 -- 172.21.15.19:0/3687886437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45819f370 con 0x7fa458107df0 2024-05-09T01:28:39.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.096+0000 7fa4477fe700 1 -- 172.21.15.19:0/3687886437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa44c018800 con 0x7fa458107df0 2024-05-09T01:28:39.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.097+0000 7fa46b3d8b80 1 Processor -- start 2024-05-09T01:28:39.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.097+0000 7fa4477fe700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4400b2880 0x7fa4400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.097+0000 7fa456d9d700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4400b2880 0x7fa4400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.097+0000 7fa46b3d8b80 1 -- start start 2024-05-09T01:28:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.097+0000 7fa4477fe700 1 -- 172.21.15.19:0/3687886437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa44c0e21d0 con 0x7fa458107df0 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca878cd00 0x555ca8895240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca87124a0 0x555ca8895780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca888f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ca87b20a0 con 0x555ca878cd00 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555ca87579a0 con 0x555ca888f400 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ca87ae060 con 0x555ca87124a0 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa456d9d700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4400b2880 0x7fa4400b4d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa45000d010 tx=0x7fa45000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa45659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca878cd00 0x555ca8895240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa45659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca878cd00 0x555ca8895240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60084/0 (socket says 172.21.15.19:60084) 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa45659c700 1 -- 172.21.15.19:0/910595642 learned_addr learned my addr 172.21.15.19:0/910595642 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa455d9b700 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca87124a0 0x555ca8895780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa456d9d700 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca888f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa45659c700 1 -- 172.21.15.19:0/910595642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca87124a0 msgr2=0x555ca8895780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa45659c700 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca87124a0 0x555ca8895780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa45659c700 1 -- 172.21.15.19:0/910595642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 msgr2=0x555ca888f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa45659c700 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca888f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa45659c700 1 -- 172.21.15.19:0/910595642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca8748500 con 0x555ca878cd00 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa456d9d700 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca888f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.098+0000 7fa455d9b700 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca87124a0 0x555ca8895780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.099+0000 7fa45659c700 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca878cd00 0x555ca8895240 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa44800bd70 tx=0x7fa4480106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8519ba38a64e95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.099+0000 7fa444ff9700 1 -- 172.21.15.19:0/910595642 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44801c480 con 0x555ca878cd00 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.099+0000 7fa444ff9700 1 -- 172.21.15.19:0/910595642 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa44801cad0 con 0x555ca878cd00 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.099+0000 7fa444ff9700 1 -- 172.21.15.19:0/910595642 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4480119e0 con 0x555ca878cd00 2024-05-09T01:28:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/910595642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca878cd00 msgr2=0x555ca8895240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca878cd00 0x555ca8895240 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa44800bd70 tx=0x7fa4480106b0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/910595642 shutdown_connections 2024-05-09T01:28:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca888f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca878cd00 0x555ca8895240 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/910595642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca87124a0 0x555ca8895780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/910595642 >> 172.21.15.19:0/910595642 conn(0x555ca8715f70 msgr2=0x555ca888a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/910595642 shutdown_connections 2024-05-09T01:28:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/910595642 wait complete. 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 -- start start 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.100+0000 7fa46b3d8b80 1 Processor -- start 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca87124a0 0x555ca8898f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca88994d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca8899a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ca8890160 con 0x555ca87124a0 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555ca874cb20 con 0x555ca888f400 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ca889e230 con 0x555ca878cd00 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa46b3d8b80 10 client.0 _ll_register_callbacks cb 0x555ca86236c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:39.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa45659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca87124a0 0x555ca8898f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa45659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca87124a0 0x555ca8898f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60100/0 (socket says 172.21.15.19:60100) 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa45659c700 1 -- 172.21.15.19:0/1899081264 learned_addr learned my addr 172.21.15.19:0/1899081264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa456d9d700 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca8899a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa455d9b700 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca88994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa45659c700 1 -- 172.21.15.19:0/1899081264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 msgr2=0x555ca88994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa45659c700 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca88994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa45659c700 1 -- 172.21.15.19:0/1899081264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 msgr2=0x555ca8899a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa45659c700 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca8899a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa45659c700 1 -- 172.21.15.19:0/1899081264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca862d490 con 0x555ca87124a0 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.101+0000 7fa456d9d700 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca8899a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa45659c700 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca87124a0 0x555ca8898f90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa448011620 tx=0x7fa448010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4990f15c7249a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa43dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa43dffb700 1 -- 172.21.15.19:0/1899081264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448004030 con 0x555ca87124a0 2024-05-09T01:28:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa43dffb700 1 -- 172.21.15.19:0/1899081264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa4480041d0 con 0x555ca87124a0 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/1899081264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa44801cad0 con 0x555ca87124a0 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa43dffb700 1 -- 172.21.15.19:0/1899081264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4480043f0 con 0x555ca87124a0 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa46b3d8b80 10 client.15984 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/1899081264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ca88a8a50 con 0x555ca87124a0 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa43dffb700 1 -- 172.21.15.19:0/1899081264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa44801cad0 con 0x555ca87124a0 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa43dffb700 1 -- 172.21.15.19:0/1899081264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa448004af0 con 0x555ca87124a0 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa46b3d8b80 10 client.15984 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa46b3d8b80 10 client.15984 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa46b3d8b80 20 client.15984 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:39.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.102+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x555ca88aafd0 0x555ca88ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.103+0000 7fa46b3d8b80 4 client.15984 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.103+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/1899081264 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ca87bfa80 con 0x555ca88aafd0 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.103+0000 7fa455d9b700 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x555ca88aafd0 0x555ca88ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.103+0000 7fa455d9b700 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x555ca88aafd0 0x555ca88ad3f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d1faf43c1d8e598 server_cookie=b43140b84a5de4db in_seq=0 out_seq=0 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.103+0000 7fa43dffb700 10 client.15984 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa43dffb700 1 -- 172.21.15.19:0/1899081264 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ca87bfa80 con 0x555ca88aafd0 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa43dffb700 10 client.15984 handle_command_reply: tid=0 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa46b3d8b80 1 client.15984 shutdown 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/1899081264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca87124a0 msgr2=0x555ca8898f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca87124a0 0x555ca8898f90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa448011620 tx=0x7fa448010f70 comp rx=0 tx=0).stop 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/1899081264 shutdown_connections 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x555ca88aafd0 0x555ca88ad3f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca888f400 0x555ca8899a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca87124a0 0x555ca8898f90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/1899081264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca88994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/1899081264 >> 172.21.15.19:0/1899081264 conn(0x555ca8715f70 msgr2=0x555ca8883ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.104+0000 7fa43dffb700 0 client.15984 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:28:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.105+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/1899081264 shutdown_connections 2024-05-09T01:28:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.105+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/1899081264 wait complete. 2024-05-09T01:28:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.105+0000 7fa46b3d8b80 20 client.15984 trim_cache size 0 max 16384 2024-05-09T01:28:39.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa46b3d8b80 1 Processor -- start 2024-05-09T01:28:39.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa46b3d8b80 1 -- start start 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 0x555ca88982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca8894760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca8899850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581aca50 con 0x555ca888f400 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581ae590 con 0x555ca87124a0 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581ad120 con 0x555ca878cd00 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa456d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca8899850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa456d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca8899850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60102/0 (socket says 172.21.15.19:60102) 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa456d9d700 1 -- 172.21.15.19:0/2542081825 learned_addr learned my addr 172.21.15.19:0/2542081825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.120+0000 7fa45659c700 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 0x555ca88982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa455d9b700 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca8894760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa456d9d700 1 -- 172.21.15.19:0/2542081825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 msgr2=0x555ca8894760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa456d9d700 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca8894760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa456d9d700 1 -- 172.21.15.19:0/2542081825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 msgr2=0x555ca88982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa456d9d700 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 0x555ca88982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa456d9d700 1 -- 172.21.15.19:0/2542081825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca88869d0 con 0x555ca888f400 2024-05-09T01:28:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa455d9b700 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca8894760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa456d9d700 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca8899850 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa45000b0d0 tx=0x7fa45001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e6b1ca3faa5f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa43ffff700 1 -- 172.21.15.19:0/2542081825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45002d030 con 0x555ca888f400 2024-05-09T01:28:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa43ffff700 1 -- 172.21.15.19:0/2542081825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa450023040 con 0x555ca888f400 2024-05-09T01:28:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.121+0000 7fa43ffff700 1 -- 172.21.15.19:0/2542081825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450005820 con 0x555ca888f400 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/2542081825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 msgr2=0x555ca8899850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca8899850 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa45000b0d0 tx=0x7fa45001c000 comp rx=0 tx=0).stop 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/2542081825 shutdown_connections 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca8899850 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 0x555ca88982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/2542081825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca8894760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/2542081825 >> 172.21.15.19:0/2542081825 conn(0x555ca8715f70 msgr2=0x555ca8883ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/2542081825 shutdown_connections 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/2542081825 wait complete. 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.122+0000 7fa46b3d8b80 1 -- start start 2024-05-09T01:28:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa46b3d8b80 1 Processor -- start 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 0x555ca88b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca88a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa46b3d8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca88a3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581adec0 con 0x555ca888f400 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581ad7f0 con 0x555ca87124a0 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa46b3d8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4581aa9f0 con 0x555ca878cd00 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa46b3d8b80 10 client.0 _ll_register_callbacks cb 0x555ca86236c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa456d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca88a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa456d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca88a3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60112/0 (socket says 172.21.15.19:60112) 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa456d9d700 1 -- 172.21.15.19:0/195814520 learned_addr learned my addr 172.21.15.19:0/195814520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa45659c700 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 0x555ca88b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa455d9b700 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca88a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa456d9d700 1 -- 172.21.15.19:0/195814520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 msgr2=0x555ca88a32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa456d9d700 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca88a32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa456d9d700 1 -- 172.21.15.19:0/195814520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 msgr2=0x555ca88b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa456d9d700 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 0x555ca88b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa456d9d700 1 -- 172.21.15.19:0/195814520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca88b3440 con 0x555ca888f400 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.123+0000 7fa455d9b700 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca88a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa456d9d700 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca88a3820 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa450009a90 tx=0x7fa450007aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd790da3b70c168 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa43dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa43dffb700 1 -- 172.21.15.19:0/195814520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45002d040 con 0x555ca888f400 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa43dffb700 1 -- 172.21.15.19:0/195814520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa450005360 con 0x555ca888f400 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/195814520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa450023040 con 0x555ca888f400 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa43dffb700 1 -- 172.21.15.19:0/195814520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa450005560 con 0x555ca888f400 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa46b3d8b80 10 client.15990 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/195814520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ca89146e0 con 0x555ca888f400 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa43dffb700 1 -- 172.21.15.19:0/195814520 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa450023040 con 0x555ca888f400 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa43dffb700 1 -- 172.21.15.19:0/195814520 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa450028040 con 0x555ca888f400 2024-05-09T01:28:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa46b3d8b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.124+0000 7fa46b3d8b80 10 client.15990 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.125+0000 7fa46b3d8b80 20 client.15990 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.125+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x555ca8914bd0 0x555ca8916ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.125+0000 7fa46b3d8b80 4 client.15990 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.125+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/195814520 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ca87b9b20 con 0x555ca8914bd0 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.125+0000 7fa45659c700 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x555ca8914bd0 0x555ca8916ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.125+0000 7fa45659c700 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x555ca8914bd0 0x555ca8916ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fc8c88d4eabb9d7 server_cookie=14258814877c29be in_seq=0 out_seq=0 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.125+0000 7fa43dffb700 10 client.15990 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa43dffb700 1 -- 172.21.15.19:0/195814520 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ca87b9b20 con 0x555ca8914bd0 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa43dffb700 10 client.15990 handle_command_reply: tid=0 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 client.15990 shutdown 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/195814520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 msgr2=0x555ca88a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca88a3820 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa450009a90 tx=0x7fa450007aa0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/195814520 shutdown_connections 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x555ca8914bd0 0x555ca8916ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa43dffb700 0 client.15990 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ca888f400 0x555ca88a3820 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ca87124a0 0x555ca88b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 --2- 172.21.15.19:0/195814520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ca878cd00 0x555ca88a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/195814520 >> 172.21.15.19:0/195814520 conn(0x555ca8715f70 msgr2=0x555ca8898010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/195814520 shutdown_connections 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 1 -- 172.21.15.19:0/195814520 wait complete. 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.126+0000 7fa46b3d8b80 20 client.15990 trim_cache size 0 max 16384 2024-05-09T01:28:39.128 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:39.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 -- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4400b2880 msgr2=0x7fa4400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:39.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4400b2880 0x7fa4400b4d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa45000d010 tx=0x7fa45000b040 comp rx=0 tx=0).stop 2024-05-09T01:28:39.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 -- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458107df0 msgr2=0x7fa45819a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:39.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458107df0 0x7fa45819a2c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c9e0 tx=0x7fa44c00ccf0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 -- 172.21.15.19:0/3687886437 shutdown_connections 2024-05-09T01:28:39.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa458108740 0x7fa45819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4581034c0 0x7fa458199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4400b2880 0x7fa4400b4d90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 --2- 172.21.15.19:0/3687886437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa458107df0 0x7fa45819a2c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 -- 172.21.15.19:0/3687886437 >> 172.21.15.19:0/3687886437 conn(0x7fa4580fedc0 msgr2=0x7fa458107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 -- 172.21.15.19:0/3687886437 shutdown_connections 2024-05-09T01:28:39.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:39.127+0000 7fa45c9c2700 1 -- 172.21.15.19:0/3687886437 wait complete. 2024-05-09T01:28:39.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:28:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:40.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:40 smithi019 ceph-mon[30158]: pgmap v472: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:28:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:40 smithi059 ceph-mon[34351]: pgmap v472: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:28:40.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:40 smithi196 ceph-mon[35041]: pgmap v472: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:28:41.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.988+0000 7f4896594700 1 -- 172.21.15.19:0/1802825380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 msgr2=0x7f4890104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:41.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.988+0000 7f4896594700 1 --2- 172.21.15.19:0/1802825380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 0x7f4890104a00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f48800099c0 tx=0x7f4880009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.989+0000 7f4896594700 1 -- 172.21.15.19:0/1802825380 shutdown_connections 2024-05-09T01:28:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.989+0000 7f4896594700 1 --2- 172.21.15.19:0/1802825380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4890109870 0x7f48901111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.989+0000 7f4896594700 1 --2- 172.21.15.19:0/1802825380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 0x7f4890104a00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.989+0000 7f4896594700 1 --2- 172.21.15.19:0/1802825380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4890108f20 0x7f4890109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.989+0000 7f4896594700 1 -- 172.21.15.19:0/1802825380 >> 172.21.15.19:0/1802825380 conn(0x7f4890100040 msgr2=0x7f4890102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.989+0000 7f4896594700 1 -- 172.21.15.19:0/1802825380 shutdown_connections 2024-05-09T01:28:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.989+0000 7f4896594700 1 -- 172.21.15.19:0/1802825380 wait complete. 2024-05-09T01:28:41.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.989+0000 7f4896594700 1 Processor -- start 2024-05-09T01:28:41.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.989+0000 7f4896594700 1 -- start start 2024-05-09T01:28:41.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f4896594700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 0x7f4890075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f4896594700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4890108f20 0x7f4890076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f4896594700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4890109870 0x7f4890076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f4896594700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4890114560 con 0x7f48901045f0 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f4896594700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48901143e0 con 0x7f4890108f20 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f4896594700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48901146e0 con 0x7f4890109870 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 0x7f4890075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 0x7f4890075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37040/0 (socket says 172.21.15.19:37040) 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488ffff700 1 -- 172.21.15.19:0/3302419041 learned_addr learned my addr 172.21.15.19:0/3302419041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f4894b31700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4890109870 0x7f4890076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488f7fe700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4890108f20 0x7f4890076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488ffff700 1 -- 172.21.15.19:0/3302419041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4890109870 msgr2=0x7f4890076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488ffff700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4890109870 0x7f4890076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488ffff700 1 -- 172.21.15.19:0/3302419041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4890108f20 msgr2=0x7f4890076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:41.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488ffff700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4890108f20 0x7f4890076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488ffff700 1 -- 172.21.15.19:0/3302419041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48901a92b0 con 0x7f48901045f0 2024-05-09T01:28:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f4894b31700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4890109870 0x7f4890076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.990+0000 7f488ffff700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 0x7f4890075ac0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4880009990 tx=0x7f48800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.991+0000 7f488d7fa700 1 -- 172.21.15.19:0/3302419041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880027070 con 0x7f48901045f0 2024-05-09T01:28:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.991+0000 7f488d7fa700 1 -- 172.21.15.19:0/3302419041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4880004190 con 0x7f48901045f0 2024-05-09T01:28:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.991+0000 7f4896594700 1 -- 172.21.15.19:0/3302419041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48901a9540 con 0x7f48901045f0 2024-05-09T01:28:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.991+0000 7f488d7fa700 1 -- 172.21.15.19:0/3302419041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488000f460 con 0x7f48901045f0 2024-05-09T01:28:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.991+0000 7f4896594700 1 -- 172.21.15.19:0/3302419041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48901a9990 con 0x7f48901045f0 2024-05-09T01:28:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.993+0000 7f488d7fa700 1 -- 172.21.15.19:0/3302419041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f488002a070 con 0x7f48901045f0 2024-05-09T01:28:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.994+0000 7f488d7fa700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f48780b2880 0x7f48780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.994+0000 7f488d7fa700 1 -- 172.21.15.19:0/3302419041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f48800e3360 con 0x7f48901045f0 2024-05-09T01:28:41.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.994+0000 7f488f7fe700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f48780b2880 0x7f48780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f488f7fe700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f48780b2880 0x7f48780b4d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4884006fd0 tx=0x7f4884008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f48a4faab80 1 Processor -- start 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f48a4faab80 1 -- start start 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0794240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd0794780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd078e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f48a4faab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ccd06b10a0 con 0x55ccd068bd00 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f48a4faab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ccd06569a0 con 0x55ccd06114a0 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f48a4faab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ccd06ad060 con 0x55ccd078e400 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f488ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0794240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f488ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0794240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37050/0 (socket says 172.21.15.19:37050) 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f488ffff700 1 -- 172.21.15.19:0/4096995048 learned_addr learned my addr 172.21.15.19:0/4096995048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.995+0000 7f488f7fe700 1 --2- 172.21.15.19:0/4096995048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd0794780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f4894b31700 1 --2- 172.21.15.19:0/4096995048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd078e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f488ffff700 1 -- 172.21.15.19:0/4096995048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 msgr2=0x55ccd078e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f488ffff700 1 --2- 172.21.15.19:0/4096995048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd078e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f488ffff700 1 -- 172.21.15.19:0/4096995048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 msgr2=0x55ccd0794780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f488ffff700 1 --2- 172.21.15.19:0/4096995048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd0794780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f488ffff700 1 -- 172.21.15.19:0/4096995048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccd0647500 con 0x55ccd068bd00 2024-05-09T01:28:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f488ffff700 1 --2- 172.21.15.19:0/4096995048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0794240 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f48800aa920 tx=0x7f4880009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb653a3d581411df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f48767fc700 1 -- 172.21.15.19:0/4096995048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48800b30b0 con 0x55ccd068bd00 2024-05-09T01:28:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f48767fc700 1 -- 172.21.15.19:0/4096995048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48800aa4a0 con 0x55ccd068bd00 2024-05-09T01:28:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.996+0000 7f48767fc700 1 -- 172.21.15.19:0/4096995048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48800b10b0 con 0x55ccd068bd00 2024-05-09T01:28:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 -- 172.21.15.19:0/4096995048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 msgr2=0x55ccd0794240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 --2- 172.21.15.19:0/4096995048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0794240 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f48800aa920 tx=0x7f4880009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 -- 172.21.15.19:0/4096995048 shutdown_connections 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 --2- 172.21.15.19:0/4096995048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd078e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 --2- 172.21.15.19:0/4096995048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0794240 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 --2- 172.21.15.19:0/4096995048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd0794780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 -- 172.21.15.19:0/4096995048 >> 172.21.15.19:0/4096995048 conn(0x55ccd0614f70 msgr2=0x55ccd0789220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 -- 172.21.15.19:0/4096995048 shutdown_connections 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 -- 172.21.15.19:0/4096995048 wait complete. 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.997+0000 7f48a4faab80 1 -- start start 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f48a4faab80 1 Processor -- start 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd079dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd079e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0797ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f48a4faab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ccd06b10a0 con 0x55ccd068bd00 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f48a4faab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ccd064bb20 con 0x55ccd06114a0 2024-05-09T01:28:41.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f48a4faab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ccd078efb0 con 0x55ccd078e400 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f48a4faab80 10 client.0 _ll_register_callbacks cb 0x55ccd05226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd079e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f488ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd079dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd079e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37054/0 (socket says 172.21.15.19:37054) 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f488f7fe700 1 -- 172.21.15.19:0/2117973729 learned_addr learned my addr 172.21.15.19:0/2117973729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.998+0000 7f4894b31700 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0797ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f488f7fe700 1 -- 172.21.15.19:0/2117973729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 msgr2=0x55ccd0797ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f488f7fe700 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0797ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f488f7fe700 1 -- 172.21.15.19:0/2117973729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 msgr2=0x55ccd079dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f488f7fe700 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd079dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f488f7fe700 1 -- 172.21.15.19:0/2117973729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccd052c490 con 0x55ccd068bd00 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f4894b31700 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0797ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f488f7fe700 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd079e010 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f488400dde0 tx=0x7f4884004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956530647705cf56 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f486f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:42.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f486f7fe700 1 -- 172.21.15.19:0/2117973729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884024020 con 0x55ccd068bd00 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f48a4faab80 1 -- 172.21.15.19:0/2117973729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48800aa4a0 con 0x55ccd068bd00 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f486f7fe700 1 -- 172.21.15.19:0/2117973729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f488400f040 con 0x55ccd068bd00 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f486f7fe700 1 -- 172.21.15.19:0/2117973729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f488402f050 con 0x55ccd068bd00 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f48a4faab80 10 client.16002 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f48a4faab80 1 -- 172.21.15.19:0/2117973729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccd07a5750 con 0x55ccd068bd00 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:41.999+0000 7f486f7fe700 1 -- 172.21.15.19:0/2117973729 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48800aa4a0 con 0x55ccd068bd00 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.000+0000 7f486f7fe700 1 -- 172.21.15.19:0/2117973729 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4884013070 con 0x55ccd068bd00 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.000+0000 7f48a4faab80 10 client.16002 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.000+0000 7f48a4faab80 10 client.16002 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.000+0000 7f48a4faab80 20 client.16002 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.000+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ccd07a7de0 0x55ccd07aa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.000+0000 7f48a4faab80 4 client.16002 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.000+0000 7f48a4faab80 1 -- 172.21.15.19:0/2117973729 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccd06bea80 con 0x55ccd07a7de0 2024-05-09T01:28:42.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.000+0000 7f488ffff700 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ccd07a7de0 0x55ccd07aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.001+0000 7f486f7fe700 10 client.16002 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:28:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.001+0000 7f488ffff700 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ccd07a7de0 0x55ccd07aa230 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7fcf522a08fb4b22 server_cookie=9dcd4c812b80cf58 in_seq=0 out_seq=0 2024-05-09T01:28:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.001+0000 7f486f7fe700 1 -- 172.21.15.19:0/2117973729 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ccd06bea80 con 0x55ccd07a7de0 2024-05-09T01:28:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.001+0000 7f486f7fe700 10 client.16002 handle_command_reply: tid=0 2024-05-09T01:28:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.001+0000 7f48a4faab80 1 client.16002 shutdown 2024-05-09T01:28:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 -- 172.21.15.19:0/2117973729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 msgr2=0x55ccd079e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd079e010 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f488400dde0 tx=0x7f4884004860 comp rx=0 tx=0).stop 2024-05-09T01:28:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 -- 172.21.15.19:0/2117973729 shutdown_connections 2024-05-09T01:28:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ccd07a7de0 0x55ccd07aa230 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0797ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd079dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2117973729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd079e010 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 -- 172.21.15.19:0/2117973729 >> 172.21.15.19:0/2117973729 conn(0x55ccd0614f70 msgr2=0x55ccd0782ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f486f7fe700 0 client.16002 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:28:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 -- 172.21.15.19:0/2117973729 shutdown_connections 2024-05-09T01:28:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 1 -- 172.21.15.19:0/2117973729 wait complete. 2024-05-09T01:28:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.002+0000 7f48a4faab80 20 client.16002 trim_cache size 0 max 16384 2024-05-09T01:28:42.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.017+0000 7f48a4faab80 1 Processor -- start 2024-05-09T01:28:42.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.017+0000 7f48a4faab80 1 -- start start 2024-05-09T01:28:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.017+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd0789290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.017+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0796ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.017+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0795250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.017+0000 7f48a4faab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f48901b0e40 con 0x55ccd068bd00 2024-05-09T01:28:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.017+0000 7f48a4faab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48901b2980 con 0x55ccd06114a0 2024-05-09T01:28:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.017+0000 7f48a4faab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48901b1510 con 0x55ccd078e400 2024-05-09T01:28:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0796ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0796ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37058/0 (socket says 172.21.15.19:37058) 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488f7fe700 1 -- 172.21.15.19:0/1115342446 learned_addr learned my addr 172.21.15.19:0/1115342446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488ffff700 1 --2- 172.21.15.19:0/1115342446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd0789290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488f7fe700 1 -- 172.21.15.19:0/1115342446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 msgr2=0x55ccd0795250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488f7fe700 1 --2- 172.21.15.19:0/1115342446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0795250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488f7fe700 1 -- 172.21.15.19:0/1115342446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 msgr2=0x55ccd0789290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488f7fe700 1 --2- 172.21.15.19:0/1115342446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd0789290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488f7fe700 1 -- 172.21.15.19:0/1115342446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccd079e360 con 0x55ccd068bd00 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488ffff700 1 --2- 172.21.15.19:0/1115342446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd0789290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f488f7fe700 1 --2- 172.21.15.19:0/1115342446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0796ed0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4884022000 tx=0x7f48840125a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56c83e898e175b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f4875ffb700 1 -- 172.21.15.19:0/1115342446 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884038070 con 0x55ccd068bd00 2024-05-09T01:28:42.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f4875ffb700 1 -- 172.21.15.19:0/1115342446 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4884012c80 con 0x55ccd068bd00 2024-05-09T01:28:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.018+0000 7f4875ffb700 1 -- 172.21.15.19:0/1115342446 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884024040 con 0x55ccd068bd00 2024-05-09T01:28:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.019+0000 7f48a4faab80 1 -- 172.21.15.19:0/1115342446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 msgr2=0x55ccd0796ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.019+0000 7f48a4faab80 1 --2- 172.21.15.19:0/1115342446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0796ed0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4884022000 tx=0x7f48840125a0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.019+0000 7f48a4faab80 1 -- 172.21.15.19:0/1115342446 shutdown_connections 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.019+0000 7f48a4faab80 1 --2- 172.21.15.19:0/1115342446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0795250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.019+0000 7f48a4faab80 1 --2- 172.21.15.19:0/1115342446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd06114a0 0x55ccd0789290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.019+0000 7f48a4faab80 1 --2- 172.21.15.19:0/1115342446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd068bd00 0x55ccd0796ed0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.019+0000 7f48a4faab80 1 -- 172.21.15.19:0/1115342446 >> 172.21.15.19:0/1115342446 conn(0x55ccd0614f70 msgr2=0x55ccd07967c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 -- 172.21.15.19:0/1115342446 shutdown_connections 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 -- 172.21.15.19:0/1115342446 wait complete. 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 -- start start 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 Processor -- start 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd06114a0 0x55ccd07a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd068bd00 0x55ccd07a1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0810580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f48901b22b0 con 0x55ccd06114a0 2024-05-09T01:28:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48901b1be0 con 0x55ccd068bd00 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.020+0000 7f48a4faab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48901aede0 con 0x55ccd078e400 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f48a4faab80 10 client.0 _ll_register_callbacks cb 0x55ccd05226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd06114a0 0x55ccd07a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd06114a0 0x55ccd07a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37074/0 (socket says 172.21.15.19:37074) 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488ffff700 1 -- 172.21.15.19:0/2717018231 learned_addr learned my addr 172.21.15.19:0/2717018231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488f7fe700 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd068bd00 0x55ccd07a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f4894b31700 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0810580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488ffff700 1 -- 172.21.15.19:0/2717018231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 msgr2=0x55ccd0810580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488ffff700 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0810580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488ffff700 1 -- 172.21.15.19:0/2717018231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd068bd00 msgr2=0x55ccd07a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488ffff700 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd068bd00 0x55ccd07a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488ffff700 1 -- 172.21.15.19:0/2717018231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccd0810ac0 con 0x55ccd06114a0 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f4894b31700 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0810580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f488ffff700 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd06114a0 0x55ccd07a12c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f48800aa690 tx=0x7f48800b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afcb74b9eda43e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f486f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f486f7fe700 1 -- 172.21.15.19:0/2717018231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48800aa4a0 con 0x55ccd06114a0 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f486f7fe700 1 -- 172.21.15.19:0/2717018231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4880071170 con 0x55ccd06114a0 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.021+0000 7f48a4faab80 1 -- 172.21.15.19:0/2717018231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4884012c80 con 0x55ccd06114a0 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f486f7fe700 1 -- 172.21.15.19:0/2717018231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f48800ab090 con 0x55ccd06114a0 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f48a4faab80 10 client.16008 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f48a4faab80 1 -- 172.21.15.19:0/2717018231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccd0811d00 con 0x55ccd06114a0 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f486f7fe700 1 -- 172.21.15.19:0/2717018231 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4884012c80 con 0x55ccd06114a0 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f486f7fe700 1 -- 172.21.15.19:0/2717018231 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f48800aabf0 con 0x55ccd06114a0 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f48a4faab80 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f48a4faab80 10 client.16008 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f48a4faab80 20 client.16008 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ccd0812220 0x55ccd0814640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f48a4faab80 4 client.16008 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f48a4faab80 1 -- 172.21.15.19:0/2717018231 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccd0798310 con 0x55ccd0812220 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.022+0000 7f488f7fe700 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ccd0812220 0x55ccd0814640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.023+0000 7f486f7fe700 10 client.16008 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.023+0000 7f488f7fe700 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ccd0812220 0x55ccd0814640 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61ea9fecdda9ec8b server_cookie=fcbea2b49c617cb6 in_seq=0 out_seq=0 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f486f7fe700 1 -- 172.21.15.19:0/2717018231 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccd0798310 con 0x55ccd0812220 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f486f7fe700 10 client.16008 handle_command_reply: tid=0 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 client.16008 shutdown 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 -- 172.21.15.19:0/2717018231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd06114a0 msgr2=0x55ccd07a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd06114a0 0x55ccd07a12c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f48800aa690 tx=0x7f48800b1040 comp rx=0 tx=0).stop 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 -- 172.21.15.19:0/2717018231 shutdown_connections 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ccd0812220 0x55ccd0814640 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccd078e400 0x55ccd0810580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ccd06114a0 0x55ccd07a12c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 --2- 172.21.15.19:0/2717018231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccd068bd00 0x55ccd07a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f486f7fe700 0 client.16008 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:28:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 -- 172.21.15.19:0/2717018231 >> 172.21.15.19:0/2717018231 conn(0x55ccd0614f70 msgr2=0x55ccd07937b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 -- 172.21.15.19:0/2717018231 shutdown_connections 2024-05-09T01:28:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 1 -- 172.21.15.19:0/2717018231 wait complete. 2024-05-09T01:28:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.024+0000 7f48a4faab80 20 client.16008 trim_cache size 0 max 16384 2024-05-09T01:28:42.026 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 -- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f48780b2880 msgr2=0x7f48780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f48780b2880 0x7f48780b4d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4884006fd0 tx=0x7f4884008040 comp rx=0 tx=0).stop 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 -- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 msgr2=0x7f4890075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 0x7f4890075ac0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4880009990 tx=0x7f48800047b0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 -- 172.21.15.19:0/3302419041 shutdown_connections 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4890109870 0x7f4890076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f48780b2880 0x7f48780b4d90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48901045f0 0x7f4890075ac0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 --2- 172.21.15.19:0/3302419041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4890108f20 0x7f4890076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 -- 172.21.15.19:0/3302419041 >> 172.21.15.19:0/3302419041 conn(0x7f4890100040 msgr2=0x7f48901086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 -- 172.21.15.19:0/3302419041 shutdown_connections 2024-05-09T01:28:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:42.025+0000 7f4896594700 1 -- 172.21.15.19:0/3302419041 wait complete. 2024-05-09T01:28:42.633 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:42 smithi019 ceph-mon[30158]: pgmap v473: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:28:42.781 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:28:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:42 smithi059 ceph-mon[34351]: pgmap v473: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:28:42.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:42 smithi196 ceph-mon[35041]: pgmap v473: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:28:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:43.284 INFO:tasks.workunit.client.0.smithi019.stdout:1/600: open fd = 3 2024-05-09T01:28:43.284 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.284 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.284 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.284 INFO:tasks.workunit.client.0.smithi019.stdout:2/600: open fd = 3 2024-05-09T01:28:43.284 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.284 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.284 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.285 INFO:tasks.workunit.client.0.smithi019.stdout:3/600: open fd = 3 2024-05-09T01:28:43.285 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.285 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.285 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.285 INFO:tasks.workunit.client.0.smithi019.stdout:4/600: open fd = 3 2024-05-09T01:28:43.285 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.285 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.286 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.286 INFO:tasks.workunit.client.0.smithi019.stdout:5/600: open fd = 3 2024-05-09T01:28:43.286 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.286 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.286 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.286 INFO:tasks.workunit.client.0.smithi019.stdout:6/600: open fd = 3 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:7/600: open fd = 3 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:8/600: open fd = 3 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:9/600: open fd = 3 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.287 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:10/600: open fd = 3 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:11/600: open fd = 3 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:12/600: open fd = 3 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:13/600: open fd = 3 2024-05-09T01:28:43.288 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:14/600: open fd = 3 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:15/600: open fd = 3 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:16/600: open fd = 3 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.289 INFO:tasks.workunit.client.0.smithi019.stdout:17/600: open fd = 3 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:18/600: open fd = 3 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:19/600: open fd = 3 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:20/600: open fd = 3 2024-05-09T01:28:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:21/600: open fd = 3 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:22/600: open fd = 3 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:23/600: open fd = 3 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.291 INFO:tasks.workunit.client.0.smithi019.stdout:24/600: open fd = 3 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:25/600: open fd = 3 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:26/600: open fd = 3 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:27/600: open fd = 3 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:28/600: open fd = 3 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:29/600: open fd = 3 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:30/600: open fd = 3 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:31/600: open fd = 3 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.293 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:32/600: open fd = 3 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:33/600: open fd = 3 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:34/600: open fd = 3 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:35/600: open fd = 3 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:36/600: open fd = 3 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:37/600: open fd = 3 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:38/600: open fd = 3 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:39/600: open fd = 3 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:40/600: open fd = 3 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:41/600: open fd = 3 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:42/600: open fd = 3 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:43/600: open fd = 3 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:44/600: open fd = 3 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:45/600: open fd = 3 2024-05-09T01:28:43.297 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:46/600: open fd = 3 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:47/600: open fd = 3 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:48/600: open fd = 3 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.298 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:49/600: open fd = 3 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:50/600: open fd = 3 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:51/600: open fd = 3 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:52/600: open fd = 3 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:53/600: open fd = 3 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:54/600: open fd = 3 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:55/600: open fd = 3 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:56/600: open fd = 3 2024-05-09T01:28:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:57/600: open fd = 3 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:58/600: open fd = 3 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:59/600: open fd = 3 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:60/600: open fd = 3 2024-05-09T01:28:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:61/600: open fd = 3 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:62/600: open fd = 3 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:63/600: open fd = 3 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:28:43.302 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:28:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.773+0000 7ff49d028700 1 -- 172.21.15.19:0/1012684941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 msgr2=0x7ff498104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.773+0000 7ff49d028700 1 --2- 172.21.15.19:0/1012684941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 0x7ff498104a60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff4880099c0 tx=0x7ff488009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.773+0000 7ff49d028700 1 -- 172.21.15.19:0/1012684941 shutdown_connections 2024-05-09T01:28:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.773+0000 7ff49d028700 1 --2- 172.21.15.19:0/1012684941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff498109880 0x7ff4981111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.773+0000 7ff49d028700 1 --2- 172.21.15.19:0/1012684941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 0x7ff498104a60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.773+0000 7ff49d028700 1 --2- 172.21.15.19:0/1012684941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff498108f30 0x7ff498109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.773+0000 7ff49d028700 1 -- 172.21.15.19:0/1012684941 >> 172.21.15.19:0/1012684941 conn(0x7ff498079f00 msgr2=0x7ff49807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.773+0000 7ff49d028700 1 -- 172.21.15.19:0/1012684941 shutdown_connections 2024-05-09T01:28:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.773+0000 7ff49d028700 1 -- 172.21.15.19:0/1012684941 wait complete. 2024-05-09T01:28:44.771 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:44 smithi019 ceph-mon[30158]: pgmap v474: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:28:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.774+0000 7ff49d028700 1 Processor -- start 2024-05-09T01:28:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.774+0000 7ff49d028700 1 -- start start 2024-05-09T01:28:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.774+0000 7ff49d028700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 0x7ff498075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.774+0000 7ff49d028700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff498108f30 0x7ff498076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.774+0000 7ff49d028700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff498109880 0x7ff498076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff49d028700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4981144b0 con 0x7ff498104650 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff49d028700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff498114330 con 0x7ff498109880 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff49d028700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff498114630 con 0x7ff498108f30 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 0x7ff498075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 0x7ff498075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37090/0 (socket says 172.21.15.19:37090) 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff496d9d700 1 -- 172.21.15.19:0/2895530817 learned_addr learned my addr 172.21.15.19:0/2895530817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff48e59c700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff498108f30 0x7ff498076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff49759e700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff498109880 0x7ff498076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff496d9d700 1 -- 172.21.15.19:0/2895530817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff498108f30 msgr2=0x7ff498076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff496d9d700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff498108f30 0x7ff498076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff496d9d700 1 -- 172.21.15.19:0/2895530817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff498109880 msgr2=0x7ff498076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff496d9d700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff498109880 0x7ff498076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff496d9d700 1 -- 172.21.15.19:0/2895530817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4981a91c0 con 0x7ff498104650 2024-05-09T01:28:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff496d9d700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 0x7ff498075ac0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b580 tx=0x7ff4880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.775+0000 7ff494d99700 1 -- 172.21.15.19:0/2895530817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488027070 con 0x7ff498104650 2024-05-09T01:28:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.776+0000 7ff494d99700 1 -- 172.21.15.19:0/2895530817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff488003f20 con 0x7ff498104650 2024-05-09T01:28:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.776+0000 7ff49d028700 1 -- 172.21.15.19:0/2895530817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4981a9450 con 0x7ff498104650 2024-05-09T01:28:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.776+0000 7ff494d99700 1 -- 172.21.15.19:0/2895530817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488022d10 con 0x7ff498104650 2024-05-09T01:28:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.776+0000 7ff49d028700 1 -- 172.21.15.19:0/2895530817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4981a98a0 con 0x7ff498104650 2024-05-09T01:28:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.778+0000 7ff494d99700 1 -- 172.21.15.19:0/2895530817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff488022470 con 0x7ff498104650 2024-05-09T01:28:44.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.779+0000 7ff494d99700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4840b2880 0x7ff4840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.779+0000 7ff494d99700 1 -- 172.21.15.19:0/2895530817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff4880e2e60 con 0x7ff498104650 2024-05-09T01:28:44.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.779+0000 7ff48e59c700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4840b2880 0x7ff4840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.780+0000 7ff48e59c700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4840b2880 0x7ff4840b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff49807a4c0 tx=0x7ff48000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.780+0000 7ff4aba3eb80 1 Processor -- start 2024-05-09T01:28:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.780+0000 7ff4aba3eb80 1 -- start start 2024-05-09T01:28:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80248a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc80242ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc80164ff0 con 0x55bc801af160 2024-05-09T01:28:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bc801037f0 con 0x55bc800c54a0 2024-05-09T01:28:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc80160fb0 con 0x55bc802426a0 2024-05-09T01:28:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37092/0 (socket says 172.21.15.19:37092) 2024-05-09T01:28:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff496d9d700 1 -- 172.21.15.19:0/3347020543 learned_addr learned my addr 172.21.15.19:0/3347020543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff49759e700 1 --2- 172.21.15.19:0/3347020543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc80242ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff48e59c700 1 --2- 172.21.15.19:0/3347020543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80248a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff496d9d700 1 -- 172.21.15.19:0/3347020543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 msgr2=0x55bc80242ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff496d9d700 1 --2- 172.21.15.19:0/3347020543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc80242ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff496d9d700 1 -- 172.21.15.19:0/3347020543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 msgr2=0x55bc80248a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff496d9d700 1 --2- 172.21.15.19:0/3347020543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80248a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.781+0000 7ff496d9d700 1 -- 172.21.15.19:0/3347020543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc800fb370 con 0x55bc801af160 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.782+0000 7ff496d9d700 1 --2- 172.21.15.19:0/3347020543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802484f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff4880aa1f0 tx=0x7ff488003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e0cb31fcf8bb40 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.782+0000 7ff48d59a700 1 -- 172.21.15.19:0/3347020543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4880aabf0 con 0x55bc801af160 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.782+0000 7ff48d59a700 1 -- 172.21.15.19:0/3347020543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4880aa4f0 con 0x55bc801af160 2024-05-09T01:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.782+0000 7ff48d59a700 1 -- 172.21.15.19:0/3347020543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4880b3090 con 0x55bc801af160 2024-05-09T01:28:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3347020543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 msgr2=0x55bc802484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3347020543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802484f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff4880aa1f0 tx=0x7ff488003780 comp rx=0 tx=0).stop 2024-05-09T01:28:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3347020543 shutdown_connections 2024-05-09T01:28:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3347020543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc80242ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3347020543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802484f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3347020543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80248a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3347020543 >> 172.21.15.19:0/3347020543 conn(0x55bc801256a0 msgr2=0x55bc8023d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3347020543 shutdown_connections 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3347020543 wait complete. 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 -- start start 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.783+0000 7ff4aba3eb80 1 Processor -- start 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80251c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc8024c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc7ffe0ba0 con 0x55bc801af160 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bc8011f150 con 0x55bc800c54a0 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc80243360 con 0x55bc802426a0 2024-05-09T01:28:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff4aba3eb80 10 client.0 _ll_register_callbacks cb 0x55bc7ffd66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff48e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff48e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37100/0 (socket says 172.21.15.19:37100) 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff48e59c700 1 -- 172.21.15.19:0/3119929166 learned_addr learned my addr 172.21.15.19:0/3119929166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff496d9d700 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80251c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff49759e700 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc8024c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff48e59c700 1 -- 172.21.15.19:0/3119929166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 msgr2=0x55bc8024c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff48e59c700 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc8024c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff48e59c700 1 -- 172.21.15.19:0/3119929166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 msgr2=0x55bc80251c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff48e59c700 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80251c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff48e59c700 1 -- 172.21.15.19:0/3119929166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc7ffe0490 con 0x55bc801af160 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff48e59c700 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802521c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff4800116c0 tx=0x7ff4800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfc5b4063a7c4e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.784+0000 7ff47e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff47e7fc700 1 -- 172.21.15.19:0/3119929166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480004a30 con 0x55bc801af160 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3119929166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4880aa4f0 con 0x55bc801af160 2024-05-09T01:28:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff47e7fc700 1 -- 172.21.15.19:0/3119929166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff480004bd0 con 0x55bc801af160 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff47e7fc700 1 -- 172.21.15.19:0/3119929166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff48002d070 con 0x55bc801af160 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff4aba3eb80 10 client.16020 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3119929166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc80259950 con 0x55bc801af160 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff47e7fc700 1 -- 172.21.15.19:0/3119929166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4880aa4f0 con 0x55bc801af160 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff47e7fc700 1 -- 172.21.15.19:0/3119929166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff48001f6d0 con 0x55bc801af160 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff4aba3eb80 10 client.16020 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff4aba3eb80 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff4aba3eb80 20 client.16020 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55bc8025bee0 0x55bc8025e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff4aba3eb80 4 client.16020 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.785+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3119929166 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc801729d0 con 0x55bc8025bee0 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.786+0000 7ff496d9d700 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55bc8025bee0 0x55bc8025e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.786+0000 7ff47e7fc700 10 client.16020 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:28:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.786+0000 7ff496d9d700 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55bc8025bee0 0x55bc8025e300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f389eeb67b0a3033 server_cookie=29039b76b399411f in_seq=0 out_seq=0 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.787+0000 7ff47e7fc700 1 -- 172.21.15.19:0/3119929166 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc801729d0 con 0x55bc8025bee0 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.787+0000 7ff47e7fc700 10 client.16020 handle_command_reply: tid=0 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.787+0000 7ff4aba3eb80 1 client.16020 shutdown 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.787+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3119929166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 msgr2=0x55bc802521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.787+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802521c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff4800116c0 tx=0x7ff4800047c0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.787+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3119929166 shutdown_connections 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.787+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc8024c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.787+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55bc8025bee0 0x55bc8025e300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.788+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80251c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.788+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3119929166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc801af160 0x55bc802521c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.788+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3119929166 >> 172.21.15.19:0/3119929166 conn(0x55bc801256a0 msgr2=0x55bc8023d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.788+0000 7ff47e7fc700 0 client.16020 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:28:44.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.788+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3119929166 shutdown_connections 2024-05-09T01:28:44.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.788+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3119929166 wait complete. 2024-05-09T01:28:44.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.788+0000 7ff4aba3eb80 20 client.16020 trim_cache size 0 max 16384 2024-05-09T01:28:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff4aba3eb80 1 Processor -- start 2024-05-09T01:28:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff4aba3eb80 1 -- start start 2024-05-09T01:28:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80244e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc801af160 0x55bc80263b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc802426a0 0x55bc80264040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4981b0e30 con 0x55bc802426a0 2024-05-09T01:28:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4981b2970 con 0x55bc800c54a0 2024-05-09T01:28:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4981b1500 con 0x55bc801af160 2024-05-09T01:28:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff49759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc802426a0 0x55bc80264040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff49759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc802426a0 0x55bc80264040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37110/0 (socket says 172.21.15.19:37110) 2024-05-09T01:28:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff49759e700 1 -- 172.21.15.19:0/3080125801 learned_addr learned my addr 172.21.15.19:0/3080125801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.803+0000 7ff496d9d700 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80244e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff48e59c700 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc801af160 0x55bc80263b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff49759e700 1 -- 172.21.15.19:0/3080125801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc801af160 msgr2=0x55bc80263b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff49759e700 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc801af160 0x55bc80263b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff49759e700 1 -- 172.21.15.19:0/3080125801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 msgr2=0x55bc80244e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:44.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff49759e700 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80244e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff49759e700 1 -- 172.21.15.19:0/3080125801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc80236f60 con 0x55bc802426a0 2024-05-09T01:28:44.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff48e59c700 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc801af160 0x55bc80263b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:44.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff49759e700 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc802426a0 0x55bc80264040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff490009910 tx=0x7ff490009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae7d35108294b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:44.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff48cd99700 1 -- 172.21.15.19:0/3080125801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49001a070 con 0x55bc802426a0 2024-05-09T01:28:44.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff48cd99700 1 -- 172.21.15.19:0/3080125801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff490007500 con 0x55bc802426a0 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.804+0000 7ff48cd99700 1 -- 172.21.15.19:0/3080125801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490016690 con 0x55bc802426a0 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.805+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3080125801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc802426a0 msgr2=0x55bc80264040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.805+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc802426a0 0x55bc80264040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff490009910 tx=0x7ff490009940 comp rx=0 tx=0).stop 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.805+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3080125801 shutdown_connections 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.805+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc802426a0 0x55bc80264040 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.805+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc800c54a0 0x55bc80244e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.805+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/3080125801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc801af160 0x55bc80263b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.805+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3080125801 >> 172.21.15.19:0/3080125801 conn(0x55bc801256a0 msgr2=0x55bc8025def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.805+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3080125801 shutdown_connections 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.805+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/3080125801 wait complete. 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff4aba3eb80 1 -- start start 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff4aba3eb80 1 Processor -- start 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc800c54a0 0x55bc80252f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc801af160 0x55bc802534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff4aba3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc802c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4981b22a0 con 0x55bc800c54a0 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4981b1bd0 con 0x55bc801af160 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff4aba3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4981aedd0 con 0x55bc802426a0 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff4aba3eb80 10 client.0 _ll_register_callbacks cb 0x55bc7ffd66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc800c54a0 0x55bc80252f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc800c54a0 0x55bc80252f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37122/0 (socket says 172.21.15.19:37122) 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff496d9d700 1 -- 172.21.15.19:0/4148422936 learned_addr learned my addr 172.21.15.19:0/4148422936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff48e59c700 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc801af160 0x55bc802534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.806+0000 7ff49759e700 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc802c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff496d9d700 1 -- 172.21.15.19:0/4148422936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 msgr2=0x55bc802c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff496d9d700 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc802c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff496d9d700 1 -- 172.21.15.19:0/4148422936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc801af160 msgr2=0x55bc802534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff496d9d700 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc801af160 0x55bc802534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff496d9d700 1 -- 172.21.15.19:0/4148422936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc80253ce0 con 0x55bc800c54a0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff49759e700 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc802c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff496d9d700 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc800c54a0 0x55bc80252f70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff4880a7760 tx=0x7ff4880aa140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80ebafd3df5965c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff47e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff47e7fc700 1 -- 172.21.15.19:0/4148422936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4880aabf0 con 0x55bc800c54a0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff47e7fc700 1 -- 172.21.15.19:0/4148422936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff4880a6ae0 con 0x55bc800c54a0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/4148422936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff490007500 con 0x55bc800c54a0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff47e7fc700 1 -- 172.21.15.19:0/4148422936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4880aa4f0 con 0x55bc800c54a0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff4aba3eb80 10 client.16026 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.807+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/4148422936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc802c5ba0 con 0x55bc800c54a0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.808+0000 7ff47e7fc700 1 -- 172.21.15.19:0/4148422936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff490007500 con 0x55bc800c54a0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.808+0000 7ff47e7fc700 1 -- 172.21.15.19:0/4148422936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff4880b1c70 con 0x55bc800c54a0 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.808+0000 7ff4aba3eb80 10 client.16026 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.808+0000 7ff4aba3eb80 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:28:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.808+0000 7ff4aba3eb80 20 client.16026 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.808+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55bc802c60e0 0x55bc802c8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.808+0000 7ff4aba3eb80 4 client.16026 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.808+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/4148422936 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc8016ca70 con 0x55bc802c60e0 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.808+0000 7ff48e59c700 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55bc802c60e0 0x55bc802c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff48e59c700 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55bc802c60e0 0x55bc802c8500 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7563ab726f8ce741 server_cookie=81ed216be6a7fc63 in_seq=0 out_seq=0 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff47e7fc700 10 client.16026 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff47e7fc700 1 -- 172.21.15.19:0/4148422936 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc8016ca70 con 0x55bc802c60e0 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff47e7fc700 10 client.16026 handle_command_reply: tid=0 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 client.16026 shutdown 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/4148422936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc800c54a0 msgr2=0x55bc80252f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc800c54a0 0x55bc80252f70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff4880a7760 tx=0x7ff4880aa140 comp rx=0 tx=0).stop 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/4148422936 shutdown_connections 2024-05-09T01:28:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc802426a0 0x55bc802c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc800c54a0 0x55bc80252f70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55bc802c60e0 0x55bc802c8500 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 --2- 172.21.15.19:0/4148422936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc801af160 0x55bc802534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/4148422936 >> 172.21.15.19:0/4148422936 conn(0x55bc801256a0 msgr2=0x55bc80245e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:44.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff47e7fc700 0 client.16026 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:28:44.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/4148422936 shutdown_connections 2024-05-09T01:28:44.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 1 -- 172.21.15.19:0/4148422936 wait complete. 2024-05-09T01:28:44.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.809+0000 7ff4aba3eb80 20 client.16026 trim_cache size 0 max 16384 2024-05-09T01:28:44.812 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 -- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4840b2880 msgr2=0x7ff4840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4840b2880 0x7ff4840b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff49807a4c0 tx=0x7ff48000b410 comp rx=0 tx=0).stop 2024-05-09T01:28:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 -- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 msgr2=0x7ff498075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 0x7ff498075ac0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b580 tx=0x7ff4880047b0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 -- 172.21.15.19:0/2895530817 shutdown_connections 2024-05-09T01:28:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff498109880 0x7ff498076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4840b2880 0x7ff4840b4d90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498104650 0x7ff498075ac0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 --2- 172.21.15.19:0/2895530817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff498108f30 0x7ff498076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 -- 172.21.15.19:0/2895530817 >> 172.21.15.19:0/2895530817 conn(0x7ff498079f00 msgr2=0x7ff4981020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 -- 172.21.15.19:0/2895530817 shutdown_connections 2024-05-09T01:28:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:44.810+0000 7ff49d028700 1 -- 172.21.15.19:0/2895530817 wait complete. 2024-05-09T01:28:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:44 smithi059 ceph-mon[34351]: pgmap v474: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:28:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:44 smithi196 ceph-mon[35041]: pgmap v474: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:28:45.587 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:28:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.615+0000 7f54999a0640 1 -- 172.21.15.19:0/928347375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 msgr2=0x7f549410acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.615+0000 7f54999a0640 1 --2- 172.21.15.19:0/928347375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 0x7f549410acf0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f547c009900 tx=0x7f547c02a410 comp rx=0 tx=0).stop 2024-05-09T01:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.616+0000 7f54999a0640 1 -- 172.21.15.19:0/928347375 shutdown_connections 2024-05-09T01:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.616+0000 7f54999a0640 1 --2- 172.21.15.19:0/928347375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 0x7f549410acf0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.616+0000 7f54999a0640 1 --2- 172.21.15.19:0/928347375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5494103bd0 0x7f5494103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.616+0000 7f54999a0640 1 --2- 172.21.15.19:0/928347375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5494102990 0x7f5494103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.616+0000 7f54999a0640 1 -- 172.21.15.19:0/928347375 >> 172.21.15.19:0/928347375 conn(0x7f54940fdb60 msgr2=0x7f5494100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.616+0000 7f54999a0640 1 -- 172.21.15.19:0/928347375 shutdown_connections 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.616+0000 7f54999a0640 1 -- 172.21.15.19:0/928347375 wait complete. 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.616+0000 7f54999a0640 1 Processor -- start 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.616+0000 7f54999a0640 1 -- start start 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f54999a0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5494102990 0x7f549406bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f54999a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5494103bd0 0x7f549419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f54999a0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 0x7f54941a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f54999a0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f549410e4a0 con 0x7f54941088d0 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f54999a0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f549410e320 con 0x7f5494102990 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f54999a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f549410e620 con 0x7f5494103bd0 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549919f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 0x7f54941a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549919f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 0x7f54941a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37140/0 (socket says 172.21.15.19:37140) 2024-05-09T01:28:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549919f640 1 -- 172.21.15.19:0/3522266205 learned_addr learned my addr 172.21.15.19:0/3522266205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f5493fff640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5494103bd0 0x7f549419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549899e640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5494102990 0x7f549406bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549919f640 1 -- 172.21.15.19:0/3522266205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5494103bd0 msgr2=0x7f549419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549919f640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5494103bd0 0x7f549419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549919f640 1 -- 172.21.15.19:0/3522266205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5494102990 msgr2=0x7f549406bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549919f640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5494102990 0x7f549406bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549919f640 1 -- 172.21.15.19:0/3522266205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54941a1c00 con 0x7f54941088d0 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.617+0000 7f549919f640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 0x7f54941a13e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f547c009230 tx=0x7f547c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.618+0000 7f5491ffb640 1 -- 172.21.15.19:0/3522266205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c03f030 con 0x7f54941088d0 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.618+0000 7f54999a0640 1 -- 172.21.15.19:0/3522266205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54941a1e90 con 0x7f54941088d0 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.618+0000 7f5491ffb640 1 -- 172.21.15.19:0/3522266205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f547c003aa0 con 0x7f54941088d0 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.618+0000 7f5491ffb640 1 -- 172.21.15.19:0/3522266205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c035070 con 0x7f54941088d0 2024-05-09T01:28:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.618+0000 7f54999a0640 1 -- 172.21.15.19:0/3522266205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54941a22a0 con 0x7f54941088d0 2024-05-09T01:28:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.620+0000 7f54a1b8b740 1 Processor -- start 2024-05-09T01:28:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f5491ffb640 1 -- 172.21.15.19:0/3522266205 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f547c042030 con 0x7f54941088d0 2024-05-09T01:28:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f54a1b8b740 1 -- start start 2024-05-09T01:28:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9331b0 0x56419acd76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a90a2f0 0x56419acd7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 0x56419acdeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56419ac3b7c0 con 0x56419a90a2f0 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56419abd70e0 con 0x56419a9124d0 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56419ac37780 con 0x56419a9331b0 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f549899e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a90a2f0 0x56419acd7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f549899e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a90a2f0 0x56419acd7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37154/0 (socket says 172.21.15.19:37154) 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f549899e640 1 -- 172.21.15.19:0/3782697332 learned_addr learned my addr 172.21.15.19:0/3782697332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f5493fff640 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9331b0 0x56419acd76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f5493fff640 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 0x56419acdeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f549899e640 1 -- 172.21.15.19:0/3782697332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9331b0 msgr2=0x56419acd76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f549899e640 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9331b0 0x56419acd76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f549899e640 1 -- 172.21.15.19:0/3782697332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 msgr2=0x56419acdeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f549899e640 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 0x56419acdeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f549899e640 1 -- 172.21.15.19:0/3782697332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419ac007c0 con 0x56419a90a2f0 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f5493fff640 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 0x56419acdeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.621+0000 7f5491ffb640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f54800b27e0 0x7f54800b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.622+0000 7f5493fff640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f54800b27e0 0x7f54800b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.622+0000 7f549899e640 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a90a2f0 0x56419acd7c40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f548c00b750 tx=0x7f548c00bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d283d79351703390 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.622+0000 7f5491ffb640 1 -- 172.21.15.19:0/3522266205 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f547c102ad0 con 0x7f54941088d0 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.622+0000 7f548affd640 1 -- 172.21.15.19:0/3782697332 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c00be60 con 0x56419a90a2f0 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.622+0000 7f548affd640 1 -- 172.21.15.19:0/3782697332 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548c015070 con 0x56419a90a2f0 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.622+0000 7f548affd640 1 -- 172.21.15.19:0/3782697332 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0106a0 con 0x56419a90a2f0 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 -- 172.21.15.19:0/3782697332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a90a2f0 msgr2=0x56419acd7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f5493fff640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f54800b27e0 0x7f54800b4cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x56419acd8890 tx=0x7f5484009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a90a2f0 0x56419acd7c40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f548c00b750 tx=0x7f548c00bc10 comp rx=0 tx=0).stop 2024-05-09T01:28:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 -- 172.21.15.19:0/3782697332 shutdown_connections 2024-05-09T01:28:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 0x56419acdeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a90a2f0 0x56419acd7c40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/3782697332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9331b0 0x56419acd76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 -- 172.21.15.19:0/3782697332 >> 172.21.15.19:0/3782697332 conn(0x56419a927300 msgr2=0x56419acd3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 -- 172.21.15.19:0/3782697332 shutdown_connections 2024-05-09T01:28:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 -- 172.21.15.19:0/3782697332 wait complete. 2024-05-09T01:28:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 -- start start 2024-05-09T01:28:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.623+0000 7f54a1b8b740 1 Processor -- start 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a90a2f0 0x56419ace13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 0x56419ace1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a9331b0 0x56419ace8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56419acdf590 con 0x56419a9331b0 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56419abc29c0 con 0x56419a9124d0 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56419acdf710 con 0x56419a90a2f0 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f54a1b8b740 10 client.0 _ll_register_callbacks cb 0x56419abef020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f549919f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a9331b0 0x56419ace8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f549919f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a9331b0 0x56419ace8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37170/0 (socket says 172.21.15.19:37170) 2024-05-09T01:28:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f549919f640 1 -- 172.21.15.19:0/1623711001 learned_addr learned my addr 172.21.15.19:0/1623711001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f549899e640 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a90a2f0 0x56419ace13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f5493fff640 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 0x56419ace1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f549919f640 1 -- 172.21.15.19:0/1623711001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a90a2f0 msgr2=0x56419ace13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f549919f640 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a90a2f0 0x56419ace13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f549919f640 1 -- 172.21.15.19:0/1623711001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 msgr2=0x56419ace1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f549919f640 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 0x56419ace1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.624+0000 7f549919f640 1 -- 172.21.15.19:0/1623711001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419abe91f0 con 0x56419a9331b0 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f549919f640 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a9331b0 0x56419ace8890 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f547c0c9e80 tx=0x7f547c0c9b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155504862be479ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f5473fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f5473fff640 1 -- 172.21.15.19:0/1623711001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c049040 con 0x56419a9331b0 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f5473fff640 1 -- 172.21.15.19:0/1623711001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f547c0ce160 con 0x56419a9331b0 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1623711001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f548c015070 con 0x56419a9331b0 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f5473fff640 1 -- 172.21.15.19:0/1623711001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f547c0ce480 con 0x56419a9331b0 2024-05-09T01:28:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f54a1b8b740 10 client.16038 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1623711001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56419a99e790 con 0x56419a9331b0 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f5473fff640 1 -- 172.21.15.19:0/1623711001 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548c015070 con 0x56419a9331b0 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.625+0000 7f5473fff640 1 -- 172.21.15.19:0/1623711001 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f547c0d2aa0 con 0x56419a9331b0 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.626+0000 7f54a1b8b740 10 client.16038 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.626+0000 7f54a1b8b740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.626+0000 7f54a1b8b740 20 client.16038 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.626+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56419a8f1d80 0x56419aceee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.626+0000 7f54a1b8b740 4 client.16038 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.626+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1623711001 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56419ac493d0 con 0x56419a8f1d80 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.626+0000 7f549899e640 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56419a8f1d80 0x56419aceee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.626+0000 7f5473fff640 10 client.16038 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.626+0000 7f549899e640 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56419a8f1d80 0x56419aceee20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b0acb4d976576c server_cookie=1dab007f14afc371 in_seq=0 out_seq=0 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.627+0000 7f5473fff640 1 -- 172.21.15.19:0/1623711001 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56419ac493d0 con 0x56419a8f1d80 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.627+0000 7f5473fff640 10 client.16038 handle_command_reply: tid=0 2024-05-09T01:28:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.627+0000 7f54a1b8b740 1 client.16038 shutdown 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.627+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1623711001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a9331b0 msgr2=0x56419ace8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.627+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a9331b0 0x56419ace8890 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f547c0c9e80 tx=0x7f547c0c9b90 comp rx=0 tx=0).stop 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.627+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1623711001 shutdown_connections 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.628+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56419a8f1d80 0x56419aceee20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.628+0000 7f5473fff640 0 client.16038 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.628+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a9331b0 0x56419ace8890 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.628+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a9124d0 0x56419ace1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.628+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1623711001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a90a2f0 0x56419ace13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.628+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1623711001 >> 172.21.15.19:0/1623711001 conn(0x56419a927300 msgr2=0x56419acd1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.628+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1623711001 shutdown_connections 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.628+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1623711001 wait complete. 2024-05-09T01:28:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.628+0000 7f54a1b8b740 20 client.16038 trim_cache size 0 max 16384 2024-05-09T01:28:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f54a1b8b740 1 Processor -- start 2024-05-09T01:28:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f54a1b8b740 1 -- start start 2024-05-09T01:28:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419acd3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 0x56419acdfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 0x56419ace0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54941ab1c0 con 0x56419a8f1d80 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f54941acd00 con 0x56419a90a2f0 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54941ab890 con 0x56419a9124d0 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f549899e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419acd3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f549899e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419acd3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37186/0 (socket says 172.21.15.19:37186) 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.643+0000 7f549899e640 1 -- 172.21.15.19:0/1611829030 learned_addr learned my addr 172.21.15.19:0/1611829030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f5493fff640 1 --2- 172.21.15.19:0/1611829030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 0x56419acdfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f549919f640 1 --2- 172.21.15.19:0/1611829030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 0x56419ace0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f549899e640 1 -- 172.21.15.19:0/1611829030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 msgr2=0x56419ace0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f549899e640 1 --2- 172.21.15.19:0/1611829030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 0x56419ace0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f549899e640 1 -- 172.21.15.19:0/1611829030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 msgr2=0x56419acdfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f549899e640 1 --2- 172.21.15.19:0/1611829030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 0x56419acdfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f549899e640 1 -- 172.21.15.19:0/1611829030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419aa155f0 con 0x56419a8f1d80 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f549899e640 1 --2- 172.21.15.19:0/1611829030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419acd3120 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f548c0039c0 tx=0x7f548c003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd875435a974581 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f548a7fc640 1 -- 172.21.15.19:0/1611829030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c005240 con 0x56419a8f1d80 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f548a7fc640 1 -- 172.21.15.19:0/1611829030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548c015040 con 0x56419a8f1d80 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.644+0000 7f548a7fc640 1 -- 172.21.15.19:0/1611829030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c02a100 con 0x56419a8f1d80 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1611829030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 msgr2=0x56419acd3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1611829030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419acd3120 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f548c0039c0 tx=0x7f548c003e80 comp rx=0 tx=0).stop 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1611829030 shutdown_connections 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1611829030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 0x56419ace0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1611829030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 0x56419acdfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1611829030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419acd3120 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1611829030 >> 172.21.15.19:0/1611829030 conn(0x56419a927300 msgr2=0x56419acd2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1611829030 shutdown_connections 2024-05-09T01:28:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1611829030 wait complete. 2024-05-09T01:28:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 -- start start 2024-05-09T01:28:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.645+0000 7f54a1b8b740 1 Processor -- start 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419ace5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 0x56419ace57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f54a1b8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 0x56419ace66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54941ac630 con 0x56419a8f1d80 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f54941abf60 con 0x56419a90a2f0 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f54a1b8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54941a91a0 con 0x56419a9124d0 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f54a1b8b740 10 client.0 _ll_register_callbacks cb 0x7f547c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549899e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419ace5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549899e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419ace5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37192/0 (socket says 172.21.15.19:37192) 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549899e640 1 -- 172.21.15.19:0/1078740888 learned_addr learned my addr 172.21.15.19:0/1078740888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549919f640 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 0x56419ace66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f5493fff640 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 0x56419ace57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549899e640 1 -- 172.21.15.19:0/1078740888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 msgr2=0x56419ace66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549899e640 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 0x56419ace66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549899e640 1 -- 172.21.15.19:0/1078740888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 msgr2=0x56419ace57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549899e640 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 0x56419ace57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549899e640 1 -- 172.21.15.19:0/1078740888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419abffed0 con 0x56419a8f1d80 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f5493fff640 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 0x56419ace57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f549899e640 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419ace5290 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f548c0043f0 tx=0x7f548c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed09a5ce74a65e78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.646+0000 7f5473fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f5473fff640 1 -- 172.21.15.19:0/1078740888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c039060 con 0x56419a8f1d80 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f5473fff640 1 -- 172.21.15.19:0/1078740888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f548c004830 con 0x56419a8f1d80 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1078740888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f548c015040 con 0x56419a8f1d80 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f5473fff640 1 -- 172.21.15.19:0/1078740888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f548c005040 con 0x56419a8f1d80 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f54a1b8b740 10 client.16044 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1078740888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56419ace7d30 con 0x56419a8f1d80 2024-05-09T01:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f5473fff640 1 -- 172.21.15.19:0/1078740888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548c015040 con 0x56419a8f1d80 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f5473fff640 1 -- 172.21.15.19:0/1078740888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f548c02a530 con 0x56419a8f1d80 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f54a1b8b740 10 client.16044 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f54a1b8b740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f54a1b8b740 20 client.16044 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56419a9331b0 0x56419ad592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f54a1b8b740 4 client.16044 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1078740888 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56419ac43240 con 0x56419a9331b0 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.647+0000 7f5493fff640 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56419a9331b0 0x56419ad592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.648+0000 7f5493fff640 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56419a9331b0 0x56419ad592d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcdfd2892172072d server_cookie=bc343f65238e1215 in_seq=0 out_seq=0 2024-05-09T01:28:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.648+0000 7f5473fff640 10 client.16044 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f5473fff640 1 -- 172.21.15.19:0/1078740888 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56419ac43240 con 0x56419a9331b0 2024-05-09T01:28:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f5473fff640 10 client.16044 handle_command_reply: tid=0 2024-05-09T01:28:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 client.16044 shutdown 2024-05-09T01:28:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1078740888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 msgr2=0x56419ace5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419ace5290 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f548c0043f0 tx=0x7f548c003a70 comp rx=0 tx=0).stop 2024-05-09T01:28:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1078740888 shutdown_connections 2024-05-09T01:28:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56419a9331b0 0x56419ad592d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56419a9124d0 0x56419ace66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56419a90a2f0 0x56419ace57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 --2- 172.21.15.19:0/1078740888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56419a8f1d80 0x56419ace5290 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f5473fff640 0 client.16044 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1078740888 >> 172.21.15.19:0/1078740888 conn(0x56419a927300 msgr2=0x56419acd5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1078740888 shutdown_connections 2024-05-09T01:28:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 1 -- 172.21.15.19:0/1078740888 wait complete. 2024-05-09T01:28:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.664+0000 7f54a1b8b740 20 client.16044 trim_cache size 0 max 16384 2024-05-09T01:28:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:28:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:28:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:28:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 -- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f54800b27e0 msgr2=0x7f54800b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f54800b27e0 0x7f54800b4cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x56419acd8890 tx=0x7f5484009680 comp rx=0 tx=0).stop 2024-05-09T01:28:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 -- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 msgr2=0x7f54941a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 0x7f54941a13e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f547c009230 tx=0x7f547c031a50 comp rx=0 tx=0).stop 2024-05-09T01:28:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 -- 172.21.15.19:0/3522266205 shutdown_connections 2024-05-09T01:28:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f54800b27e0 0x7f54800b4cf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54941088d0 0x7f54941a13e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5494103bd0 0x7f549419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 --2- 172.21.15.19:0/3522266205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5494102990 0x7f549406bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 -- 172.21.15.19:0/3522266205 >> 172.21.15.19:0/3522266205 conn(0x7f54940fdb60 msgr2=0x7f54941048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 -- 172.21.15.19:0/3522266205 shutdown_connections 2024-05-09T01:28:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:46.665+0000 7f54999a0640 1 -- 172.21.15.19:0/3522266205 wait complete. 2024-05-09T01:28:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:46 smithi059 ceph-mon[34351]: pgmap v475: 97 pgs: 97 active+clean; 1.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:46.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:46 smithi019 ceph-mon[30158]: pgmap v475: 97 pgs: 97 active+clean; 1.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:46.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:46 smithi196 ceph-mon[35041]: pgmap v475: 97 pgs: 97 active+clean; 1.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.594+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1809557266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e041034c0 msgr2=0x7f3e041038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.594+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1809557266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e041034c0 0x7f3e041038d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b0d0 tx=0x7f3df400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.595+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1809557266 shutdown_connections 2024-05-09T01:28:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.595+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1809557266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e04108740 0x7f3e041100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.595+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1809557266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e041034c0 0x7f3e041038d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.595+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1809557266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e04107df0 0x7f3e04108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.595+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1809557266 >> 172.21.15.19:0/1809557266 conn(0x7f3e040fedc0 msgr2=0x7f3e041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.595+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1809557266 shutdown_connections 2024-05-09T01:28:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.595+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1809557266 wait complete. 2024-05-09T01:28:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0c28a700 1 Processor -- start 2024-05-09T01:28:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0c28a700 1 -- start start 2024-05-09T01:28:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0c28a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e041034c0 0x7f3e0419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0c28a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e04107df0 0x7f3e0419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0c28a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e04108740 0x7f3e0419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0c28a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04113370 con 0x7f3e04108740 2024-05-09T01:28:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0c28a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041131f0 con 0x7f3e04107df0 2024-05-09T01:28:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0c28a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041134f0 con 0x7f3e041034c0 2024-05-09T01:28:47.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0a827700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e04108740 0x7f3e0419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0a827700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e04108740 0x7f3e0419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37224/0 (socket says 172.21.15.19:37224) 2024-05-09T01:28:47.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0a827700 1 -- 172.21.15.19:0/1491717097 learned_addr learned my addr 172.21.15.19:0/1491717097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:47.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0a026700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e041034c0 0x7f3e0419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e09825700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e04107df0 0x7f3e0419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0a827700 1 -- 172.21.15.19:0/1491717097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e041034c0 msgr2=0x7f3e0419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0a827700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e041034c0 0x7f3e0419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0a827700 1 -- 172.21.15.19:0/1491717097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e04107df0 msgr2=0x7f3e0419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0a827700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e04107df0 0x7f3e0419c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.596+0000 7f3e0a827700 1 -- 172.21.15.19:0/1491717097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e041a6f30 con 0x7f3e04108740 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.597+0000 7f3e0a827700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e04108740 0x7f3e0419c710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00c900 tx=0x7f3dfc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.597+0000 7f3e037fe700 1 -- 172.21.15.19:0/1491717097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0078d0 con 0x7f3e04108740 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.597+0000 7f3e037fe700 1 -- 172.21.15.19:0/1491717097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dfc00f450 con 0x7f3e04108740 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.597+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1491717097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e041a7220 con 0x7f3e04108740 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.597+0000 7f3e037fe700 1 -- 172.21.15.19:0/1491717097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc018620 con 0x7f3e04108740 2024-05-09T01:28:47.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.597+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1491717097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a7710 con 0x7f3e04108740 2024-05-09T01:28:47.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.600+0000 7f3e037fe700 1 -- 172.21.15.19:0/1491717097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3dfc007a70 con 0x7f3e04108740 2024-05-09T01:28:47.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.601+0000 7f3e037fe700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dec0b2880 0x7f3dec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.601+0000 7f3e0a026700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dec0b2880 0x7f3dec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.601+0000 7f3e037fe700 1 -- 172.21.15.19:0/1491717097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3dfc0e2050 con 0x7f3e04108740 2024-05-09T01:28:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.601+0000 7f3e0a026700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dec0b2880 0x7f3dec0b4d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009900 tx=0x7f3df400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.601+0000 7f3e1aca0b80 1 Processor -- start 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e1aca0b80 1 -- start start 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 0x55f8317c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8316404a0 0x55f8317c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 0x55f8317bdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f8316dfff0 con 0x55f8316404a0 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f83167e7f0 con 0x55f8317bd6a0 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f8316dbfb0 con 0x55f83172a160 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e09825700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8316404a0 0x55f8317c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e09825700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8316404a0 0x55f8317c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37234/0 (socket says 172.21.15.19:37234) 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e09825700 1 -- 172.21.15.19:0/2238117563 learned_addr learned my addr 172.21.15.19:0/2238117563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e0a827700 1 --2- 172.21.15.19:0/2238117563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 0x55f8317bdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e0a026700 1 --2- 172.21.15.19:0/2238117563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 0x55f8317c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e09825700 1 -- 172.21.15.19:0/2238117563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 msgr2=0x55f8317c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e09825700 1 --2- 172.21.15.19:0/2238117563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 0x55f8317c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e09825700 1 -- 172.21.15.19:0/2238117563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 msgr2=0x55f8317bdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.602+0000 7f3e09825700 1 --2- 172.21.15.19:0/2238117563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 0x55f8317bdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.603+0000 7f3e09825700 1 -- 172.21.15.19:0/2238117563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f831676370 con 0x55f8316404a0 2024-05-09T01:28:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.603+0000 7f3e09825700 1 --2- 172.21.15.19:0/2238117563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8316404a0 0x55f8317c3a30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3df800bcf0 tx=0x7f3df80103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b19dada90ac856 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.603+0000 7f3e00d47700 1 -- 172.21.15.19:0/2238117563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8010d10 con 0x55f8316404a0 2024-05-09T01:28:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.603+0000 7f3e00d47700 1 -- 172.21.15.19:0/2238117563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df80044e0 con 0x55f8316404a0 2024-05-09T01:28:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.603+0000 7f3e00d47700 1 -- 172.21.15.19:0/2238117563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8005280 con 0x55f8316404a0 2024-05-09T01:28:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2238117563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8316404a0 msgr2=0x55f8317c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2238117563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8316404a0 0x55f8317c3a30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3df800bcf0 tx=0x7f3df80103d0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2238117563 shutdown_connections 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2238117563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 0x55f8317bdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2238117563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 0x55f8317c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2238117563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8316404a0 0x55f8317c3a30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2238117563 >> 172.21.15.19:0/2238117563 conn(0x55f8316a06a0 msgr2=0x55f8317b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2238117563 shutdown_connections 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2238117563 wait complete. 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.604+0000 7f3e1aca0b80 1 -- start start 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e1aca0b80 1 Processor -- start 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8316404a0 0x55f8317c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 0x55f8317c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f83155bba0 con 0x55f83172a160 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f83169a150 con 0x55f8317bd6a0 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f8317ca1d0 con 0x55f8316404a0 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e1aca0b80 10 client.0 _ll_register_callbacks cb 0x55f8315516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e09825700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e09825700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37246/0 (socket says 172.21.15.19:37246) 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e09825700 1 -- 172.21.15.19:0/4223880753 learned_addr learned my addr 172.21.15.19:0/4223880753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e0a026700 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8316404a0 0x55f8317c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e0a827700 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 0x55f8317c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e09825700 1 -- 172.21.15.19:0/4223880753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8316404a0 msgr2=0x55f8317c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e09825700 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8316404a0 0x55f8317c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e09825700 1 -- 172.21.15.19:0/4223880753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 msgr2=0x55f8317c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e09825700 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 0x55f8317c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.605+0000 7f3e09825700 1 -- 172.21.15.19:0/4223880753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83155b490 con 0x55f83172a160 2024-05-09T01:28:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.606+0000 7f3e09825700 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c76b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000c00 tx=0x7f3df8004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a06808659a656ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.606+0000 7f3de9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.606+0000 7f3de9ffb700 1 -- 172.21.15.19:0/4223880753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8010e20 con 0x55f83172a160 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.606+0000 7f3de9ffb700 1 -- 172.21.15.19:0/4223880753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3df8004dc0 con 0x55f83172a160 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.606+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/4223880753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3df80044e0 con 0x55f83172a160 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.606+0000 7f3de9ffb700 1 -- 172.21.15.19:0/4223880753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3df8004fc0 con 0x55f83172a160 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.606+0000 7f3e1aca0b80 10 client.16056 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.606+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/4223880753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8317d4b40 con 0x55f83172a160 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.606+0000 7f3de9ffb700 1 -- 172.21.15.19:0/4223880753 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df80044e0 con 0x55f83172a160 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3de9ffb700 1 -- 172.21.15.19:0/4223880753 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3df8011a50 con 0x55f83172a160 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3e1aca0b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3e1aca0b80 10 client.16056 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3e1aca0b80 20 client.16056 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f8317d7010 0x55f8317d9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3e1aca0b80 4 client.16056 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/4223880753 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8316ed9d0 con 0x55f8317d7010 2024-05-09T01:28:47.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3e0a026700 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f8317d7010 0x55f8317d9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3e0a026700 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f8317d7010 0x55f8317d9430 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2a301338cbe77d0 server_cookie=760445a04cdf0332 in_seq=0 out_seq=0 2024-05-09T01:28:47.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.607+0000 7f3de9ffb700 10 client.16056 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3de9ffb700 1 -- 172.21.15.19:0/4223880753 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f8316ed9d0 con 0x55f8317d7010 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3de9ffb700 10 client.16056 handle_command_reply: tid=0 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 client.16056 shutdown 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/4223880753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 msgr2=0x55f8317c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c76b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000c00 tx=0x7f3df8004a30 comp rx=0 tx=0).stop 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/4223880753 shutdown_connections 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f8317d7010 0x55f8317d9430 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8317bd6a0 0x55f8317c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8316404a0 0x55f8317c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/4223880753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c76b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3de9ffb700 0 client.16056 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:28:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/4223880753 >> 172.21.15.19:0/4223880753 conn(0x55f8316a06a0 msgr2=0x55f8317b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/4223880753 shutdown_connections 2024-05-09T01:28:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.608+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/4223880753 wait complete. 2024-05-09T01:28:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.609+0000 7f3e1aca0b80 20 client.16056 trim_cache size 0 max 16384 2024-05-09T01:28:47.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e1aca0b80 1 Processor -- start 2024-05-09T01:28:47.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e1aca0b80 1 -- start start 2024-05-09T01:28:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8317bd6a0 0x55f8317d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041aec70 con 0x55f83172a160 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041b07b0 con 0x55f8316404a0 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041af340 con 0x55f8317bd6a0 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e09825700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e09825700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37256/0 (socket says 172.21.15.19:37256) 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e09825700 1 -- 172.21.15.19:0/465361208 learned_addr learned my addr 172.21.15.19:0/465361208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e0a026700 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.624+0000 7f3e0a827700 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8317bd6a0 0x55f8317d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3e09825700 1 -- 172.21.15.19:0/465361208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8317bd6a0 msgr2=0x55f8317d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3e09825700 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8317bd6a0 0x55f8317d2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3e09825700 1 -- 172.21.15.19:0/465361208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 msgr2=0x55f8317c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3e09825700 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3e09825700 1 -- 172.21.15.19:0/465361208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8317c9a10 con 0x55f83172a160 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3e0a026700 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3e09825700 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c28b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3df8011ef0 tx=0x7f3df800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab64305a4fb6c502 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3debfff700 1 -- 172.21.15.19:0/465361208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8004c80 con 0x55f83172a160 2024-05-09T01:28:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3debfff700 1 -- 172.21.15.19:0/465361208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df8035020 con 0x55f83172a160 2024-05-09T01:28:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.625+0000 7f3debfff700 1 -- 172.21.15.19:0/465361208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8005160 con 0x55f83172a160 2024-05-09T01:28:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.626+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/465361208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 msgr2=0x55f8317c28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.626+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c28b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3df8011ef0 tx=0x7f3df800b410 comp rx=0 tx=0).stop 2024-05-09T01:28:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.626+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/465361208 shutdown_connections 2024-05-09T01:28:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.626+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8317bd6a0 0x55f8317d2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.626+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.626+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/465361208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f83172a160 0x55f8317c28b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.626+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/465361208 >> 172.21.15.19:0/465361208 conn(0x55f8316a06a0 msgr2=0x55f8317c21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.626+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/465361208 shutdown_connections 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.626+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/465361208 wait complete. 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e1aca0b80 1 -- start start 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e1aca0b80 1 Processor -- start 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 0x55f8317cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e1aca0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8317bd6a0 0x55f8318395f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041b00e0 con 0x55f8317bd6a0 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041afa10 con 0x55f8316404a0 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e1aca0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041acbf0 con 0x55f83172a160 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e1aca0b80 10 client.0 _ll_register_callbacks cb 0x55f8315516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e0a827700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8317bd6a0 0x55f8318395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e0a026700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e0a827700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8317bd6a0 0x55f8318395f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37258/0 (socket says 172.21.15.19:37258) 2024-05-09T01:28:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e0a827700 1 -- 172.21.15.19:0/2568930175 learned_addr learned my addr 172.21.15.19:0/2568930175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.627+0000 7f3e09825700 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 0x55f8317cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e0a827700 1 -- 172.21.15.19:0/2568930175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 msgr2=0x55f8317cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e0a827700 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 0x55f8317cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e0a827700 1 -- 172.21.15.19:0/2568930175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 msgr2=0x55f8317cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e0a827700 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e0a827700 1 -- 172.21.15.19:0/2568930175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f831839be0 con 0x55f8317bd6a0 2024-05-09T01:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e0a026700 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e0a827700 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8317bd6a0 0x55f8318395f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0a68a0 tx=0x7f3dfc0a9190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ea76038e2d4617 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3de9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3de9ffb700 1 -- 172.21.15.19:0/2568930175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0c2020 con 0x55f8317bd6a0 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3de9ffb700 1 -- 172.21.15.19:0/2568930175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3dfc007d60 con 0x55f8317bd6a0 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2568930175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3df8035020 con 0x55f8317bd6a0 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3de9ffb700 1 -- 172.21.15.19:0/2568930175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3dfc0c8050 con 0x55f8317bd6a0 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e1aca0b80 10 client.16062 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.628+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2568930175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f831840f00 con 0x55f8317bd6a0 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.629+0000 7f3de9ffb700 1 -- 172.21.15.19:0/2568930175 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df8035020 con 0x55f8317bd6a0 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.629+0000 7f3de9ffb700 1 -- 172.21.15.19:0/2568930175 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3dfc0a9890 con 0x55f8317bd6a0 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.629+0000 7f3e1aca0b80 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.629+0000 7f3e1aca0b80 10 client.16062 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.629+0000 7f3e1aca0b80 20 client.16062 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.629+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f831841660 0x55f831843a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.629+0000 7f3e1aca0b80 4 client.16062 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.629+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2568930175 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8316e7a70 con 0x55f831841660 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.629+0000 7f3e0a026700 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f831841660 0x55f831843a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e0a026700 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f831841660 0x55f831843a80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ecc6f968433acd7 server_cookie=291a18de3c4577a1 in_seq=0 out_seq=0 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3de9ffb700 10 client.16062 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3de9ffb700 1 -- 172.21.15.19:0/2568930175 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8316e7a70 con 0x55f831841660 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3de9ffb700 10 client.16062 handle_command_reply: tid=0 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 client.16062 shutdown 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2568930175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8317bd6a0 msgr2=0x55f8318395f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8317bd6a0 0x55f8318395f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0a68a0 tx=0x7f3dfc0a9190 comp rx=0 tx=0).stop 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2568930175 shutdown_connections 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f831841660 0x55f831843a80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3de9ffb700 0 client.16062 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:28:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8317bd6a0 0x55f8318395f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f8316404a0 0x55f8317cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 --2- 172.21.15.19:0/2568930175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f83172a160 0x55f8317cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2568930175 >> 172.21.15.19:0/2568930175 conn(0x55f8316a06a0 msgr2=0x55f8317c0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2568930175 shutdown_connections 2024-05-09T01:28:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 1 -- 172.21.15.19:0/2568930175 wait complete. 2024-05-09T01:28:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.630+0000 7f3e1aca0b80 20 client.16062 trim_cache size 0 max 16384 2024-05-09T01:28:47.632 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dec0b2880 msgr2=0x7f3dec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dec0b2880 0x7f3dec0b4d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009900 tx=0x7f3df400b040 comp rx=0 tx=0).stop 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e04108740 msgr2=0x7f3e0419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e04108740 0x7f3e0419c710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00c900 tx=0x7f3dfc00cc10 comp rx=0 tx=0).stop 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1491717097 shutdown_connections 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3dec0b2880 0x7f3dec0b4d90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e04108740 0x7f3e0419c710 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e041034c0 0x7f3e0419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 --2- 172.21.15.19:0/1491717097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e04107df0 0x7f3e0419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1491717097 >> 172.21.15.19:0/1491717097 conn(0x7f3e040fedc0 msgr2=0x7f3e041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1491717097 shutdown_connections 2024-05-09T01:28:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:47.631+0000 7f3e0c28a700 1 -- 172.21.15.19:0/1491717097 wait complete. 2024-05-09T01:28:48.378 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:28:48.635 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:48.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:48 smithi019 ceph-mon[30158]: pgmap v476: 97 pgs: 97 active+clean; 1.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-05-09T01:28:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:48 smithi059 ceph-mon[34351]: pgmap v476: 97 pgs: 97 active+clean; 1.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-05-09T01:28:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:48 smithi196 ceph-mon[35041]: pgmap v476: 97 pgs: 97 active+clean; 1.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-05-09T01:28:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.389+0000 7f047b8ad700 1 -- 172.21.15.19:0/2250785641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0474109870 msgr2=0x7f04741111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.389+0000 7f047b8ad700 1 --2- 172.21.15.19:0/2250785641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0474109870 0x7f04741111d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f046c00b0d0 tx=0x7f046c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.389+0000 7f047b8ad700 1 -- 172.21.15.19:0/2250785641 shutdown_connections 2024-05-09T01:28:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.389+0000 7f047b8ad700 1 --2- 172.21.15.19:0/2250785641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0474109870 0x7f04741111d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.389+0000 7f047b8ad700 1 --2- 172.21.15.19:0/2250785641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04741045f0 0x7f0474104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.389+0000 7f047b8ad700 1 --2- 172.21.15.19:0/2250785641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0474108f20 0x7f0474109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.389+0000 7f047b8ad700 1 -- 172.21.15.19:0/2250785641 >> 172.21.15.19:0/2250785641 conn(0x7f04740ffec0 msgr2=0x7f0474102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.389+0000 7f047b8ad700 1 -- 172.21.15.19:0/2250785641 shutdown_connections 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.389+0000 7f047b8ad700 1 -- 172.21.15.19:0/2250785641 wait complete. 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.390+0000 7f047b8ad700 1 Processor -- start 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.390+0000 7f047b8ad700 1 -- start start 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.390+0000 7f047b8ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04741045f0 0x7f047419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.390+0000 7f047b8ad700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0474108f20 0x7f047419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.390+0000 7f047b8ad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0474109870 0x7f047419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.390+0000 7f047b8ad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04741144a0 con 0x7f0474109870 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.390+0000 7f047b8ad700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0474114320 con 0x7f04741045f0 2024-05-09T01:28:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f047b8ad700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0474114620 con 0x7f0474108f20 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479649700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04741045f0 0x7f047419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479649700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04741045f0 0x7f047419aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:32848/0 (socket says 172.21.15.19:32848) 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479649700 1 -- 172.21.15.19:0/1988843581 learned_addr learned my addr 172.21.15.19:0/1988843581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479e4a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0474109870 0x7f047419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0478e48700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0474108f20 0x7f047419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479649700 1 -- 172.21.15.19:0/1988843581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0474108f20 msgr2=0x7f047419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479649700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0474108f20 0x7f047419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479649700 1 -- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0474109870 msgr2=0x7f047419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479649700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0474109870 0x7f047419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479649700 1 -- 172.21.15.19:0/1988843581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047419fda0 con 0x7f04741045f0 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479e4a700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0474109870 0x7f047419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.391+0000 7f0479649700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04741045f0 0x7f047419aed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f047000d950 tx=0x7f047000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.392+0000 7f046a7fc700 1 -- 172.21.15.19:0/1988843581 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470016070 con 0x7f04741045f0 2024-05-09T01:28:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.392+0000 7f047b8ad700 1 -- 172.21.15.19:0/1988843581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04741a0090 con 0x7f04741045f0 2024-05-09T01:28:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.392+0000 7f046a7fc700 1 -- 172.21.15.19:0/1988843581 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047000fd40 con 0x7f04741045f0 2024-05-09T01:28:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.392+0000 7f046a7fc700 1 -- 172.21.15.19:0/1988843581 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047000e3b0 con 0x7f04741045f0 2024-05-09T01:28:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.392+0000 7f047b8ad700 1 -- 172.21.15.19:0/1988843581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04741a79b0 con 0x7f04741045f0 2024-05-09T01:28:50.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.395+0000 7f046a7fc700 1 -- 172.21.15.19:0/1988843581 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0470019070 con 0x7f04741045f0 2024-05-09T01:28:50.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f046a7fc700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04600b6c90 0x7f04600b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f0478e48700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04600b6c90 0x7f04600b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f046a7fc700 1 -- 172.21.15.19:0/1988843581 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f04700e20c0 con 0x7f04741045f0 2024-05-09T01:28:50.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f048a2c3b80 1 Processor -- start 2024-05-09T01:28:50.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f048a2c3b80 1 -- start start 2024-05-09T01:28:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 0x56181280d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f0478e48700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04600b6c90 0x7f04600b91a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0474103460 tx=0x7f0464009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x56181280d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812807810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56181272a0a0 con 0x56181268a4a0 2024-05-09T01:28:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5618126cf9a0 con 0x561812704d00 2024-05-09T01:28:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561812726060 con 0x561812807400 2024-05-09T01:28:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f0478e48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x56181280d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.396+0000 7f0478e48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x56181280d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58602/0 (socket says 172.21.15.19:58602) 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0478e48700 1 -- 172.21.15.19:0/3422429385 learned_addr learned my addr 172.21.15.19:0/3422429385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0479649700 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 0x56181280d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0479e4a700 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812807810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0478e48700 1 -- 172.21.15.19:0/3422429385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 msgr2=0x561812807810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0478e48700 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812807810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0478e48700 1 -- 172.21.15.19:0/3422429385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 msgr2=0x56181280d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0478e48700 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 0x56181280d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0478e48700 1 -- 172.21.15.19:0/3422429385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618126c0500 con 0x56181268a4a0 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0479e4a700 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812807810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f0478e48700 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x56181280d780 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0464011d10 tx=0x7f0464014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6713c90f6b9cbef5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f045f7fe700 1 -- 172.21.15.19:0/3422429385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464011d40 con 0x56181268a4a0 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f045f7fe700 1 -- 172.21.15.19:0/3422429385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0464004020 con 0x56181268a4a0 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.397+0000 7f045f7fe700 1 -- 172.21.15.19:0/3422429385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464003b60 con 0x56181268a4a0 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.398+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3422429385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 msgr2=0x56181280d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.398+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x56181280d780 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0464011d10 tx=0x7f0464014000 comp rx=0 tx=0).stop 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.398+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3422429385 shutdown_connections 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.398+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812807810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.398+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 0x56181280d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.398+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3422429385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x56181280d780 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.398+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3422429385 >> 172.21.15.19:0/3422429385 conn(0x56181268df70 msgr2=0x561812802220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.398+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3422429385 shutdown_connections 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.398+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3422429385 wait complete. 2024-05-09T01:28:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.399+0000 7f048a2c3b80 1 -- start start 2024-05-09T01:28:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.399+0000 7f048a2c3b80 1 Processor -- start 2024-05-09T01:28:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.399+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56181268a4a0 0x561812810fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.399+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812704d00 0x5618128114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.399+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812811a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.399+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561812808160 con 0x561812704d00 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.399+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5618126c4b20 con 0x56181268a4a0 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.399+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561812814020 con 0x561812807400 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.399+0000 7f048a2c3b80 10 client.0 _ll_register_callbacks cb 0x56181259b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0478e48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812704d00 0x5618128114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0478e48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812704d00 0x5618128114f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58608/0 (socket says 172.21.15.19:58608) 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0478e48700 1 -- 172.21.15.19:0/3932076658 learned_addr learned my addr 172.21.15.19:0/3932076658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0479649700 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56181268a4a0 0x561812810fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0479e4a700 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812811a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0478e48700 1 -- 172.21.15.19:0/3932076658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 msgr2=0x561812811a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0478e48700 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812811a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0478e48700 1 -- 172.21.15.19:0/3932076658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56181268a4a0 msgr2=0x561812810fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0478e48700 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56181268a4a0 0x561812810fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0478e48700 1 -- 172.21.15.19:0/3932076658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618125a5490 con 0x561812704d00 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0479e4a700 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812811a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f0478e48700 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812704d00 0x5618128114f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f04640036c0 tx=0x7f0464005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c6de62f94aa349 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f045cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f045cff9700 1 -- 172.21.15.19:0/3932076658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464003a60 con 0x561812704d00 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f045cff9700 1 -- 172.21.15.19:0/3932076658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0464003c00 con 0x561812704d00 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3932076658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0464004020 con 0x561812704d00 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f045cff9700 1 -- 172.21.15.19:0/3932076658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0464003e00 con 0x561812704d00 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f048a2c3b80 10 client.16074 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.400+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3932076658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56181281e9d0 con 0x561812704d00 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.401+0000 7f045cff9700 1 -- 172.21.15.19:0/3932076658 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0464004020 con 0x561812704d00 2024-05-09T01:28:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.401+0000 7f045cff9700 1 -- 172.21.15.19:0/3932076658 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f046402e070 con 0x561812704d00 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.401+0000 7f048a2c3b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.401+0000 7f048a2c3b80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.401+0000 7f048a2c3b80 20 client.16074 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.401+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561812820ea0 0x5618128232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.401+0000 7f048a2c3b80 4 client.16074 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.401+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3932076658 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561812737a80 con 0x561812820ea0 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.401+0000 7f0479649700 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561812820ea0 0x5618128232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.402+0000 7f045cff9700 10 client.16074 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.402+0000 7f0479649700 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561812820ea0 0x5618128232c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3a81e6d76419a4 server_cookie=f952f98e87b3e092 in_seq=0 out_seq=0 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.402+0000 7f045cff9700 1 -- 172.21.15.19:0/3932076658 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561812737a80 con 0x561812820ea0 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.402+0000 7f045cff9700 10 client.16074 handle_command_reply: tid=0 2024-05-09T01:28:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.402+0000 7f048a2c3b80 1 client.16074 shutdown 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3932076658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812704d00 msgr2=0x5618128114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812704d00 0x5618128114f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f04640036c0 tx=0x7f0464005550 comp rx=0 tx=0).stop 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3932076658 shutdown_connections 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561812820ea0 0x5618128232c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812807400 0x561812811a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56181268a4a0 0x561812810fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/3932076658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812704d00 0x5618128114f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3932076658 >> 172.21.15.19:0/3932076658 conn(0x56181268df70 msgr2=0x5618127fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f045cff9700 0 client.16074 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3932076658 shutdown_connections 2024-05-09T01:28:50.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 1 -- 172.21.15.19:0/3932076658 wait complete. 2024-05-09T01:28:50.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.403+0000 7f048a2c3b80 20 client.16074 trim_cache size 0 max 16384 2024-05-09T01:28:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f048a2c3b80 1 Processor -- start 2024-05-09T01:28:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f048a2c3b80 1 -- start start 2024-05-09T01:28:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x561812811870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812704d00 0x56181281d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812807400 0x5618127fc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04741acb20 con 0x56181268a4a0 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04741ae660 con 0x561812807400 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04741ad1f0 con 0x561812704d00 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f0479649700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x561812811870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f0479649700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x561812811870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58614/0 (socket says 172.21.15.19:58614) 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f0479649700 1 -- 172.21.15.19:0/2139743289 learned_addr learned my addr 172.21.15.19:0/2139743289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f0478e48700 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812704d00 0x56181281d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f0479e4a700 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812807400 0x5618127fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f0479649700 1 -- 172.21.15.19:0/2139743289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812704d00 msgr2=0x56181281d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f0479649700 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812704d00 0x56181281d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.418+0000 7f0479649700 1 -- 172.21.15.19:0/2139743289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812807400 msgr2=0x5618127fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.419+0000 7f0479649700 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812807400 0x5618127fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.419+0000 7f0479649700 1 -- 172.21.15.19:0/2139743289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561812802f40 con 0x56181268a4a0 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.419+0000 7f0479e4a700 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812807400 0x5618127fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.419+0000 7f0479649700 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x561812811870 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f04700dff10 tx=0x7f04700dfa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd42a4f9cfc8feb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.419+0000 7f045effd700 1 -- 172.21.15.19:0/2139743289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470020040 con 0x56181268a4a0 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.419+0000 7f045effd700 1 -- 172.21.15.19:0/2139743289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04700b2520 con 0x56181268a4a0 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.419+0000 7f045effd700 1 -- 172.21.15.19:0/2139743289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04700b2090 con 0x56181268a4a0 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2139743289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 msgr2=0x561812811870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x561812811870 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f04700dff10 tx=0x7f04700dfa70 comp rx=0 tx=0).stop 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2139743289 shutdown_connections 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812807400 0x5618127fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56181268a4a0 0x561812811870 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2139743289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561812704d00 0x56181281d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2139743289 >> 172.21.15.19:0/2139743289 conn(0x56181268df70 msgr2=0x5618127fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2139743289 shutdown_connections 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2139743289 wait complete. 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 -- start start 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.420+0000 7f048a2c3b80 1 Processor -- start 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56181268a4a0 0x561812810ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 0x561812810ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f048a2c3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812807400 0x5618128893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04741adf90 con 0x561812807400 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04741ad8c0 con 0x561812704d00 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f048a2c3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04741aaac0 con 0x56181268a4a0 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f048a2c3b80 10 client.0 _ll_register_callbacks cb 0x56181259b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479e4a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812807400 0x5618128893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479e4a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812807400 0x5618128893e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58618/0 (socket says 172.21.15.19:58618) 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479e4a700 1 -- 172.21.15.19:0/2121384320 learned_addr learned my addr 172.21.15.19:0/2121384320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479649700 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56181268a4a0 0x561812810ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0478e48700 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 0x561812810ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479e4a700 1 -- 172.21.15.19:0/2121384320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56181268a4a0 msgr2=0x561812810ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479e4a700 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56181268a4a0 0x561812810ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479e4a700 1 -- 172.21.15.19:0/2121384320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 msgr2=0x561812810ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479e4a700 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 0x561812810ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479e4a700 1 -- 172.21.15.19:0/2121384320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561812889920 con 0x561812807400 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f0479e4a700 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812807400 0x5618128893e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f046c003a30 tx=0x7f046c003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d36b43fd00e304d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.421+0000 7f045cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f045cff9700 1 -- 172.21.15.19:0/2121384320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c0075c0 con 0x561812807400 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f045cff9700 1 -- 172.21.15.19:0/2121384320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f046c007760 con 0x561812807400 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2121384320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04700b2520 con 0x561812807400 2024-05-09T01:28:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f045cff9700 1 -- 172.21.15.19:0/2121384320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f046c007960 con 0x561812807400 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f048a2c3b80 10 client.16080 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2121384320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56181288aa20 con 0x561812807400 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f045cff9700 1 -- 172.21.15.19:0/2121384320 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04700b2520 con 0x561812807400 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f045cff9700 1 -- 172.21.15.19:0/2121384320 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f046c011070 con 0x561812807400 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f048a2c3b80 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f048a2c3b80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f048a2c3b80 20 client.16080 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56181288b100 0x56181288d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f048a2c3b80 4 client.16080 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.422+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2121384320 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561812731b20 con 0x56181288b100 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f0479649700 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56181288b100 0x56181288d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f0479649700 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56181288b100 0x56181288d520 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d238338d895f07 server_cookie=3c34fef205c84902 in_seq=0 out_seq=0 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f045cff9700 10 client.16080 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f045cff9700 1 -- 172.21.15.19:0/2121384320 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561812731b20 con 0x56181288b100 2024-05-09T01:28:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f045cff9700 10 client.16080 handle_command_reply: tid=0 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f048a2c3b80 1 client.16080 shutdown 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2121384320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812807400 msgr2=0x5618128893e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812807400 0x5618128893e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f046c003a30 tx=0x7f046c003a60 comp rx=0 tx=0).stop 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2121384320 shutdown_connections 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56181288b100 0x56181288d520 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561812807400 0x5618128893e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56181268a4a0 0x561812810ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f048a2c3b80 1 --2- 172.21.15.19:0/2121384320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561812704d00 0x561812810ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2121384320 >> 172.21.15.19:0/2121384320 conn(0x56181268df70 msgr2=0x5618128063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.423+0000 7f045cff9700 0 client.16080 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2121384320 shutdown_connections 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f048a2c3b80 1 -- 172.21.15.19:0/2121384320 wait complete. 2024-05-09T01:28:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f048a2c3b80 20 client.16080 trim_cache size 0 max 16384 2024-05-09T01:28:50.432 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:28:50.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 -- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04600b6c90 msgr2=0x7f04600b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:50.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04600b6c90 0x7f04600b91a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0474103460 tx=0x7f0464009260 comp rx=0 tx=0).stop 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 -- 172.21.15.19:0/1988843581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04741045f0 msgr2=0x7f047419aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04741045f0 0x7f047419aed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f047000d950 tx=0x7f047000dc60 comp rx=0 tx=0).stop 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 -- 172.21.15.19:0/1988843581 shutdown_connections 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f04600b6c90 0x7f04600b91a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0474109870 0x7f047419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04741045f0 0x7f047419aed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 --2- 172.21.15.19:0/1988843581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0474108f20 0x7f047419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 -- 172.21.15.19:0/1988843581 >> 172.21.15.19:0/1988843581 conn(0x7f04740ffec0 msgr2=0x7f0474108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 -- 172.21.15.19:0/1988843581 shutdown_connections 2024-05-09T01:28:50.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:50.424+0000 7f047b8ad700 1 -- 172.21.15.19:0/1988843581 wait complete. 2024-05-09T01:28:50.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:50 smithi019 ceph-mon[30158]: pgmap v477: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:28:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:50 smithi059 ceph-mon[34351]: pgmap v477: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:28:50.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:50 smithi196 ceph-mon[35041]: pgmap v477: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:28:51.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:28:51.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:28:51.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:28:51.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:51 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:28:51.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:28:51.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:28:51.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:28:51.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:51 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:28:51.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:28:51.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:28:51.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:28:51.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:51 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:28:52.191 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:28:52.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:28:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:52.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:52 smithi059 ceph-mon[34351]: pgmap v478: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:52.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:52 smithi019 ceph-mon[30158]: pgmap v478: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:52.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:52 smithi196 ceph-mon[35041]: pgmap v478: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:28:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.240+0000 7ff136383700 1 -- 172.21.15.19:0/1851449869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 msgr2=0x7ff1301049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.240+0000 7ff136383700 1 --2- 172.21.15.19:0/1851449869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 0x7ff1301049e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff1200099c0 tx=0x7ff120009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.240+0000 7ff136383700 1 -- 172.21.15.19:0/1851449869 shutdown_connections 2024-05-09T01:28:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.240+0000 7ff136383700 1 --2- 172.21.15.19:0/1851449869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff130109850 0x7ff1301111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.240+0000 7ff136383700 1 --2- 172.21.15.19:0/1851449869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 0x7ff1301049e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.240+0000 7ff136383700 1 --2- 172.21.15.19:0/1851449869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff130108f00 0x7ff130109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.240+0000 7ff136383700 1 -- 172.21.15.19:0/1851449869 >> 172.21.15.19:0/1851449869 conn(0x7ff1300ffea0 msgr2=0x7ff1301022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.240+0000 7ff136383700 1 -- 172.21.15.19:0/1851449869 shutdown_connections 2024-05-09T01:28:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.241+0000 7ff136383700 1 -- 172.21.15.19:0/1851449869 wait complete. 2024-05-09T01:28:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.241+0000 7ff136383700 1 Processor -- start 2024-05-09T01:28:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.241+0000 7ff136383700 1 -- start start 2024-05-09T01:28:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff136383700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 0x7ff13019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff136383700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff130108f00 0x7ff13019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff136383700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff130109850 0x7ff13019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff136383700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff130114480 con 0x7ff1301045d0 2024-05-09T01:28:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff136383700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff130114300 con 0x7ff130109850 2024-05-09T01:28:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff136383700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff130114600 con 0x7ff130108f00 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 0x7ff13019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 0x7ff13019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58636/0 (socket says 172.21.15.19:58636) 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12ffff700 1 -- 172.21.15.19:0/987020496 learned_addr learned my addr 172.21.15.19:0/987020496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff127fff700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff130108f00 0x7ff13019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff134920700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff130109850 0x7ff13019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12ffff700 1 -- 172.21.15.19:0/987020496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff130108f00 msgr2=0x7ff13019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12ffff700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff130108f00 0x7ff13019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12ffff700 1 -- 172.21.15.19:0/987020496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff130109850 msgr2=0x7ff13019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12ffff700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff130109850 0x7ff13019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12ffff700 1 -- 172.21.15.19:0/987020496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13019fbf0 con 0x7ff1301045d0 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff134920700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff130109850 0x7ff13019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12ffff700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 0x7ff13019add0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff1200098e0 tx=0x7ff120005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12dffb700 1 -- 172.21.15.19:0/987020496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120027070 con 0x7ff1301045d0 2024-05-09T01:28:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12dffb700 1 -- 172.21.15.19:0/987020496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff120003fa0 con 0x7ff1301045d0 2024-05-09T01:28:54.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff136383700 1 -- 172.21.15.19:0/987020496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13019fe80 con 0x7ff1301045d0 2024-05-09T01:28:54.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.242+0000 7ff12dffb700 1 -- 172.21.15.19:0/987020496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120022d40 con 0x7ff1301045d0 2024-05-09T01:28:54.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.243+0000 7ff136383700 1 -- 172.21.15.19:0/987020496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1301a03b0 con 0x7ff1301045d0 2024-05-09T01:28:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.245+0000 7ff12dffb700 1 -- 172.21.15.19:0/987020496 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff12002a070 con 0x7ff1301045d0 2024-05-09T01:28:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.246+0000 7ff12dffb700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff11c0b2900 0x7ff11c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.246+0000 7ff127fff700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff11c0b2900 0x7ff11c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.246+0000 7ff12dffb700 1 -- 172.21.15.19:0/987020496 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff1200e2fd0 con 0x7ff1301045d0 2024-05-09T01:28:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff127fff700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff11c0b2900 0x7ff11c0b4e10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff118005950 tx=0x7ff1180058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff144d99b80 1 Processor -- start 2024-05-09T01:28:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff144d99b80 1 -- start start 2024-05-09T01:28:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 0x55e4a99bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 0x55e4a99c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff144d99b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e4a98de120 con 0x55e4a983e520 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff144d99b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9883a20 con 0x55e4a99c4230 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff144d99b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4a98da0e0 con 0x55e4a98b8d80 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff127fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff127fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99c00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58650/0 (socket says 172.21.15.19:58650) 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff127fff700 1 -- 172.21.15.19:0/271988211 learned_addr learned my addr 172.21.15.19:0/271988211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff134920700 1 --2- 172.21.15.19:0/271988211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 0x55e4a99c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.247+0000 7ff12ffff700 1 --2- 172.21.15.19:0/271988211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 0x55e4a99bfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.248+0000 7ff127fff700 1 -- 172.21.15.19:0/271988211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 msgr2=0x55e4a99bfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.248+0000 7ff127fff700 1 --2- 172.21.15.19:0/271988211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 0x55e4a99bfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.248+0000 7ff127fff700 1 -- 172.21.15.19:0/271988211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 msgr2=0x55e4a99c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.248+0000 7ff127fff700 1 --2- 172.21.15.19:0/271988211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 0x55e4a99c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.248+0000 7ff127fff700 1 -- 172.21.15.19:0/271988211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a9874580 con 0x55e4a983e520 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.248+0000 7ff127fff700 1 --2- 172.21.15.19:0/271988211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99c00d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff11800fd10 tx=0x7ff118012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93173fe76aa21ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.248+0000 7ff1267fc700 1 -- 172.21.15.19:0/271988211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118022070 con 0x55e4a983e520 2024-05-09T01:28:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.248+0000 7ff1267fc700 1 -- 172.21.15.19:0/271988211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11801e450 con 0x55e4a983e520 2024-05-09T01:28:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.248+0000 7ff1267fc700 1 -- 172.21.15.19:0/271988211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11801d610 con 0x55e4a983e520 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 -- 172.21.15.19:0/271988211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 msgr2=0x55e4a99c00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 --2- 172.21.15.19:0/271988211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99c00d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff11800fd10 tx=0x7ff118012000 comp rx=0 tx=0).stop 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 -- 172.21.15.19:0/271988211 shutdown_connections 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 --2- 172.21.15.19:0/271988211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 0x55e4a99c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 --2- 172.21.15.19:0/271988211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 0x55e4a99bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 --2- 172.21.15.19:0/271988211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99c00d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 -- 172.21.15.19:0/271988211 >> 172.21.15.19:0/271988211 conn(0x55e4a9841ff0 msgr2=0x55e4a99b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 -- 172.21.15.19:0/271988211 shutdown_connections 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 -- 172.21.15.19:0/271988211 wait complete. 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.249+0000 7ff144d99b80 1 -- start start 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff144d99b80 1 Processor -- start 2024-05-09T01:28:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 0x55e4a99c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 0x55e4a99c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff144d99b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e4a99c4f50 con 0x55e4a98b8d80 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff144d99b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9878ba0 con 0x55e4a983e520 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff144d99b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4a99cc4e0 con 0x55e4a99c4230 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff144d99b80 10 client.0 _ll_register_callbacks cb 0x55e4a974f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff127fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff127fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58658/0 (socket says 172.21.15.19:58658) 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff127fff700 1 -- 172.21.15.19:0/2684037722 learned_addr learned my addr 172.21.15.19:0/2684037722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff12ffff700 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 0x55e4a99c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.250+0000 7ff134920700 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 0x55e4a99c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff127fff700 1 -- 172.21.15.19:0/2684037722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 msgr2=0x55e4a99c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff127fff700 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 0x55e4a99c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff127fff700 1 -- 172.21.15.19:0/2684037722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 msgr2=0x55e4a99c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff127fff700 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 0x55e4a99c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff127fff700 1 -- 172.21.15.19:0/2684037722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a9759510 con 0x55e4a98b8d80 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff134920700 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 0x55e4a99c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff127fff700 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99c99f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff11801e880 tx=0x7ff11801e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5190354e5ef7ee74 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff10f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff10f7fe700 1 -- 172.21.15.19:0/2684037722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118036060 con 0x55e4a98b8d80 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff144d99b80 1 -- 172.21.15.19:0/2684037722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff11801e450 con 0x55e4a98b8d80 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff10f7fe700 1 -- 172.21.15.19:0/2684037722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff118011ae0 con 0x55e4a98b8d80 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff10f7fe700 1 -- 172.21.15.19:0/2684037722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff118022050 con 0x55e4a98b8d80 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff144d99b80 10 client.16092 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff144d99b80 1 -- 172.21.15.19:0/2684037722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4a99d6e40 con 0x55e4a98b8d80 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.251+0000 7ff10f7fe700 1 -- 172.21.15.19:0/2684037722 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11801e450 con 0x55e4a98b8d80 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.252+0000 7ff10f7fe700 1 -- 172.21.15.19:0/2684037722 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff118027450 con 0x55e4a98b8d80 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.252+0000 7ff144d99b80 10 client.16092 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.252+0000 7ff144d99b80 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.252+0000 7ff144d99b80 20 client.16092 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.252+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e4a99d9370 0x55e4a99db790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.252+0000 7ff144d99b80 4 client.16092 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.252+0000 7ff144d99b80 1 -- 172.21.15.19:0/2684037722 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a98ebb00 con 0x55e4a99d9370 2024-05-09T01:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.252+0000 7ff12ffff700 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e4a99d9370 0x55e4a99db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.253+0000 7ff12ffff700 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e4a99d9370 0x55e4a99db790 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c8206ef674e140 server_cookie=316a8441ca78b711 in_seq=0 out_seq=0 2024-05-09T01:28:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.253+0000 7ff10f7fe700 10 client.16092 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.253+0000 7ff10f7fe700 1 -- 172.21.15.19:0/2684037722 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4a98ebb00 con 0x55e4a99d9370 2024-05-09T01:28:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.253+0000 7ff10f7fe700 10 client.16092 handle_command_reply: tid=0 2024-05-09T01:28:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.253+0000 7ff144d99b80 1 client.16092 shutdown 2024-05-09T01:28:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 -- 172.21.15.19:0/2684037722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 msgr2=0x55e4a99c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99c99f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff11801e880 tx=0x7ff11801e960 comp rx=0 tx=0).stop 2024-05-09T01:28:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 -- 172.21.15.19:0/2684037722 shutdown_connections 2024-05-09T01:28:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e4a99d9370 0x55e4a99db790 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 0x55e4a99c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 0x55e4a99c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff10f7fe700 0 client.16092 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2684037722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99c99f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 -- 172.21.15.19:0/2684037722 >> 172.21.15.19:0/2684037722 conn(0x55e4a9841ff0 msgr2=0x55e4a99ba470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 -- 172.21.15.19:0/2684037722 shutdown_connections 2024-05-09T01:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 1 -- 172.21.15.19:0/2684037722 wait complete. 2024-05-09T01:28:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.254+0000 7ff144d99b80 20 client.16092 trim_cache size 0 max 16384 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.275+0000 7ff144d99b80 1 Processor -- start 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.275+0000 7ff144d99b80 1 -- start start 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.275+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 0x55e4a99baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.275+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.275+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 0x55e4a99c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.275+0000 7ff144d99b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301ac930 con 0x55e4a98b8d80 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.275+0000 7ff144d99b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301ae470 con 0x55e4a983e520 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.275+0000 7ff144d99b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301ad000 con 0x55e4a99c4230 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.275+0000 7ff127fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff127fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99d5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58664/0 (socket says 172.21.15.19:58664) 2024-05-09T01:28:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff127fff700 1 -- 172.21.15.19:0/2537898522 learned_addr learned my addr 172.21.15.19:0/2537898522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff12ffff700 1 --2- 172.21.15.19:0/2537898522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 0x55e4a99baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff134920700 1 --2- 172.21.15.19:0/2537898522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 0x55e4a99c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff127fff700 1 -- 172.21.15.19:0/2537898522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 msgr2=0x55e4a99c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff127fff700 1 --2- 172.21.15.19:0/2537898522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 0x55e4a99c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff127fff700 1 -- 172.21.15.19:0/2537898522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 msgr2=0x55e4a99baea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff127fff700 1 --2- 172.21.15.19:0/2537898522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 0x55e4a99baea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff127fff700 1 -- 172.21.15.19:0/2537898522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a99b7f00 con 0x55e4a98b8d80 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff127fff700 1 --2- 172.21.15.19:0/2537898522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99d5630 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff11800ba40 tx=0x7ff118027f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cf1db7ea146945 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff125ffb700 1 -- 172.21.15.19:0/2537898522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118040070 con 0x55e4a98b8d80 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff125ffb700 1 -- 172.21.15.19:0/2537898522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11801eb40 con 0x55e4a98b8d80 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.276+0000 7ff125ffb700 1 -- 172.21.15.19:0/2537898522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118015070 con 0x55e4a98b8d80 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 -- 172.21.15.19:0/2537898522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 msgr2=0x55e4a99d5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2537898522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99d5630 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff11800ba40 tx=0x7ff118027f70 comp rx=0 tx=0).stop 2024-05-09T01:28:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 -- 172.21.15.19:0/2537898522 shutdown_connections 2024-05-09T01:28:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2537898522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a99c4230 0x55e4a99c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2537898522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a983e520 0x55e4a99baea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 --2- 172.21.15.19:0/2537898522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a98b8d80 0x55e4a99d5630 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 -- 172.21.15.19:0/2537898522 >> 172.21.15.19:0/2537898522 conn(0x55e4a9841ff0 msgr2=0x55e4a99b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 -- 172.21.15.19:0/2537898522 shutdown_connections 2024-05-09T01:28:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 -- 172.21.15.19:0/2537898522 wait complete. 2024-05-09T01:28:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.277+0000 7ff144d99b80 1 -- start start 2024-05-09T01:28:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff144d99b80 1 Processor -- start 2024-05-09T01:28:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99de7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 0x55e4a99ded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff144d99b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 0x55e4a99db790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff144d99b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301adda0 con 0x55e4a983e520 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff144d99b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301ad6d0 con 0x55e4a99c4230 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff144d99b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1301aa8f0 con 0x55e4a98b8d80 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff144d99b80 10 client.0 _ll_register_callbacks cb 0x55e4a974f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff12ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99de7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff12ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99de7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58674/0 (socket says 172.21.15.19:58674) 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff12ffff700 1 -- 172.21.15.19:0/1250790352 learned_addr learned my addr 172.21.15.19:0/1250790352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff127fff700 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 0x55e4a99ded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.278+0000 7ff134920700 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 0x55e4a99db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff12ffff700 1 -- 172.21.15.19:0/1250790352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 msgr2=0x55e4a99ded00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff12ffff700 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 0x55e4a99ded00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff12ffff700 1 -- 172.21.15.19:0/1250790352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 msgr2=0x55e4a99db790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff12ffff700 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 0x55e4a99db790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff12ffff700 1 -- 172.21.15.19:0/1250790352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a99dbcd0 con 0x55e4a983e520 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff12ffff700 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99de7c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff1200e0ab0 tx=0x7ff1200b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c816c88d7f39bf22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff10f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff10f7fe700 1 -- 172.21.15.19:0/1250790352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200b3430 con 0x55e4a983e520 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff10f7fe700 1 -- 172.21.15.19:0/1250790352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff120004150 con 0x55e4a983e520 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff144d99b80 1 -- 172.21.15.19:0/1250790352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff11801eb40 con 0x55e4a983e520 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff10f7fe700 1 -- 172.21.15.19:0/1250790352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff1200aa8c0 con 0x55e4a983e520 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff144d99b80 10 client.16098 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff144d99b80 1 -- 172.21.15.19:0/1250790352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4a9a440c0 con 0x55e4a983e520 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.279+0000 7ff10f7fe700 1 -- 172.21.15.19:0/1250790352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11801eb40 con 0x55e4a983e520 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff10f7fe700 1 -- 172.21.15.19:0/1250790352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff1200b2fa0 con 0x55e4a983e520 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff144d99b80 10 client.16098 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff144d99b80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff144d99b80 20 client.16098 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff144d99b80 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e4a9a447d0 0x55e4a9a46bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff144d99b80 4 client.16098 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff144d99b80 1 -- 172.21.15.19:0/1250790352 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e4a98e5ba0 con 0x55e4a9a447d0 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff127fff700 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e4a9a447d0 0x55e4a9a46bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff127fff700 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e4a9a447d0 0x55e4a9a46bf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfbc1756ec53ba22 server_cookie=c562ecfb2de04c83 in_seq=0 out_seq=0 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.280+0000 7ff10f7fe700 10 client.16098 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff10f7fe700 1 -- 172.21.15.19:0/1250790352 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e4a98e5ba0 con 0x55e4a9a447d0 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff10f7fe700 10 client.16098 handle_command_reply: tid=0 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 client.16098 shutdown 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 -- 172.21.15.19:0/1250790352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 msgr2=0x55e4a99de7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99de7c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff1200e0ab0 tx=0x7ff1200b1040 comp rx=0 tx=0).stop 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 -- 172.21.15.19:0/1250790352 shutdown_connections 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e4a9a447d0 0x55e4a9a46bf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4a99c4230 0x55e4a99db790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff10f7fe700 0 client.16098 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e4a983e520 0x55e4a99de7c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 --2- 172.21.15.19:0/1250790352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4a98b8d80 0x55e4a99ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 -- 172.21.15.19:0/1250790352 >> 172.21.15.19:0/1250790352 conn(0x55e4a9841ff0 msgr2=0x55e4a99c7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 -- 172.21.15.19:0/1250790352 shutdown_connections 2024-05-09T01:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 1 -- 172.21.15.19:0/1250790352 wait complete. 2024-05-09T01:28:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.282+0000 7ff144d99b80 20 client.16098 trim_cache size 0 max 16384 2024-05-09T01:28:54.284 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:28:54.284 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:28:54.284 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "1cf51d31-cb14-46fa-aeb3-1e76f078ad1e", 2024-05-09T01:28:54.285 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:28:54.285 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 -- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff11c0b2900 msgr2=0x7ff11c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff11c0b2900 0x7ff11c0b4e10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff118005950 tx=0x7ff1180058e0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 -- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 msgr2=0x7ff13019add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 0x7ff13019add0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff1200098e0 tx=0x7ff120005e80 comp rx=0 tx=0).stop 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 -- 172.21.15.19:0/987020496 shutdown_connections 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff130109850 0x7ff13019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff11c0b2900 0x7ff11c0b4e10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1301045d0 0x7ff13019add0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 --2- 172.21.15.19:0/987020496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff130108f00 0x7ff13019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 -- 172.21.15.19:0/987020496 >> 172.21.15.19:0/987020496 conn(0x7ff1300ffea0 msgr2=0x7ff13010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 -- 172.21.15.19:0/987020496 shutdown_connections 2024-05-09T01:28:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:54.283+0000 7ff136383700 1 -- 172.21.15.19:0/987020496 wait complete. 2024-05-09T01:28:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:54 smithi059 ceph-mon[34351]: pgmap v479: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:28:54.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:54 smithi019 ceph-mon[30158]: pgmap v479: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:28:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:54 smithi196 ceph-mon[35041]: pgmap v479: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:28:55.075 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:28:55.331 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:55.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:55 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:28:55.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:55 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:55.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:55 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:28:55.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:55 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:55 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:28:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:55 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:55 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:28:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:55 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:55.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:55 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:28:55.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:55 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:55.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:55 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:28:55.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:55 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:28:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:56 smithi059 ceph-mon[34351]: pgmap v480: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:56 smithi019 ceph-mon[30158]: pgmap v480: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:56 smithi196 ceph-mon[35041]: pgmap v480: 97 pgs: 97 active+clean; 1.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:28:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.176+0000 7f55d1239700 1 -- 172.21.15.19:0/1303613461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc108740 msgr2=0x7f55cc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.176+0000 7f55d1239700 1 --2- 172.21.15.19:0/1303613461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc108740 0x7f55cc1100a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f55c400b0d0 tx=0x7f55c400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.176+0000 7f55d1239700 1 -- 172.21.15.19:0/1303613461 shutdown_connections 2024-05-09T01:28:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.176+0000 7f55d1239700 1 --2- 172.21.15.19:0/1303613461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc108740 0x7f55cc1100a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.176+0000 7f55d1239700 1 --2- 172.21.15.19:0/1303613461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55cc1034c0 0x7f55cc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.176+0000 7f55d1239700 1 --2- 172.21.15.19:0/1303613461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55cc107df0 0x7f55cc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.176+0000 7f55d1239700 1 -- 172.21.15.19:0/1303613461 >> 172.21.15.19:0/1303613461 conn(0x7f55cc0fedc0 msgr2=0x7f55cc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.176+0000 7f55d1239700 1 -- 172.21.15.19:0/1303613461 shutdown_connections 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.176+0000 7f55d1239700 1 -- 172.21.15.19:0/1303613461 wait complete. 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.177+0000 7f55d1239700 1 Processor -- start 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.177+0000 7f55d1239700 1 -- start start 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.177+0000 7f55d1239700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc1034c0 0x7f55cc19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.177+0000 7f55d1239700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55cc107df0 0x7f55cc19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.177+0000 7f55d1239700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55cc108740 0x7f55cc19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.177+0000 7f55d1239700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc113460 con 0x7f55cc1034c0 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.177+0000 7f55d1239700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1132e0 con 0x7f55cc107df0 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.177+0000 7f55d1239700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1135e0 con 0x7f55cc108740 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc1034c0 0x7f55cc19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55cc107df0 0x7f55cc19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc1034c0 0x7f55cc19bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58698/0 (socket says 172.21.15.19:58698) 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cad9d700 1 -- 172.21.15.19:0/3089500491 learned_addr learned my addr 172.21.15.19:0/3089500491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cb59e700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55cc108740 0x7f55cc19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cad9d700 1 -- 172.21.15.19:0/3089500491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55cc108740 msgr2=0x7f55cc19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cad9d700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55cc108740 0x7f55cc19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cad9d700 1 -- 172.21.15.19:0/3089500491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55cc107df0 msgr2=0x7f55cc19c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cad9d700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55cc107df0 0x7f55cc19c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cad9d700 1 -- 172.21.15.19:0/3089500491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc1a70d0 con 0x7f55cc1034c0 2024-05-09T01:28:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55ca59c700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55cc107df0 0x7f55cc19c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55cad9d700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc1034c0 0x7f55cc19bd60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00b530 tx=0x7f55bc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.178+0000 7f55b3fff700 1 -- 172.21.15.19:0/3089500491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc015020 con 0x7f55cc1034c0 2024-05-09T01:28:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.179+0000 7f55d1239700 1 -- 172.21.15.19:0/3089500491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc1a73c0 con 0x7f55cc1034c0 2024-05-09T01:28:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.179+0000 7f55b3fff700 1 -- 172.21.15.19:0/3089500491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55bc011ce0 con 0x7f55cc1034c0 2024-05-09T01:28:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.179+0000 7f55d1239700 1 -- 172.21.15.19:0/3089500491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc1a7950 con 0x7f55cc1034c0 2024-05-09T01:28:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.179+0000 7f55b3fff700 1 -- 172.21.15.19:0/3089500491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc010dd0 con 0x7f55cc1034c0 2024-05-09T01:28:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.181+0000 7f55b3fff700 1 -- 172.21.15.19:0/3089500491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f55bc018020 con 0x7f55cc1034c0 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.182+0000 7f55b3fff700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55b40b2900 0x7f55b40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.182+0000 7f55dfc4fb80 1 Processor -- start 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.182+0000 7f55dfc4fb80 1 -- start start 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.182+0000 7f55ca59c700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55b40b2900 0x7f55b40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.182+0000 7f55b3fff700 1 -- 172.21.15.19:0/3089500491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f55bc0e18f0 con 0x7f55cc1034c0 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b907fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b9067a4a0 0x563b907fd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b907f7400 0x563b907f7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563b9071a0a0 con 0x563b9067a4a0 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563b906bf9a0 con 0x563b907f7400 2024-05-09T01:28:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563b90716060 con 0x563b906f4d00 2024-05-09T01:28:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b9067a4a0 0x563b907fd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b9067a4a0 0x563b907fd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58710/0 (socket says 172.21.15.19:58710) 2024-05-09T01:28:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55ca59c700 1 -- 172.21.15.19:0/2392699881 learned_addr learned my addr 172.21.15.19:0/2392699881 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55cad9d700 1 --2- 172.21.15.19:0/2392699881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b907fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55ca59c700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55b40b2900 0x7f55b40b4e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f55cc1024b0 tx=0x7f55c0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55cb59e700 1 --2- 172.21.15.19:0/2392699881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b907f7400 0x563b907f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55ca59c700 1 -- 172.21.15.19:0/2392699881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 msgr2=0x563b907fd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55ca59c700 1 --2- 172.21.15.19:0/2392699881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b907fd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55ca59c700 1 -- 172.21.15.19:0/2392699881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b907f7400 msgr2=0x563b907f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55ca59c700 1 --2- 172.21.15.19:0/2392699881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b907f7400 0x563b907f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.183+0000 7f55ca59c700 1 -- 172.21.15.19:0/2392699881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b906b0500 con 0x563b9067a4a0 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.184+0000 7f55ca59c700 1 --2- 172.21.15.19:0/2392699881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b9067a4a0 0x563b907fd780 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f55c000edb0 tx=0x7f55c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a75d94474403eb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.184+0000 7f55b17fa700 1 -- 172.21.15.19:0/2392699881 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c001b040 con 0x563b9067a4a0 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.184+0000 7f55b17fa700 1 -- 172.21.15.19:0/2392699881 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55c0024070 con 0x563b9067a4a0 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.184+0000 7f55b17fa700 1 -- 172.21.15.19:0/2392699881 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0013a80 con 0x563b9067a4a0 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2392699881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b9067a4a0 msgr2=0x563b907fd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2392699881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b9067a4a0 0x563b907fd780 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f55c000edb0 tx=0x7f55c0011000 comp rx=0 tx=0).stop 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2392699881 shutdown_connections 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2392699881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b907f7400 0x563b907f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2392699881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b907fd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2392699881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b9067a4a0 0x563b907fd780 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2392699881 >> 172.21.15.19:0/2392699881 conn(0x563b9067df70 msgr2=0x563b907f2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2392699881 shutdown_connections 2024-05-09T01:28:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2392699881 wait complete. 2024-05-09T01:28:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 -- start start 2024-05-09T01:28:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.185+0000 7f55dfc4fb80 1 Processor -- start 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b90800f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b906f4d00 0x563b908014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b907f7400 0x563b90801a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563b907f8160 con 0x563b906f4d00 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563b906b4b20 con 0x563b9067a4a0 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563b90804020 con 0x563b907f7400 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55dfc4fb80 10 client.0 _ll_register_callbacks cb 0x563b9058b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b906f4d00 0x563b908014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55cad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b90800f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b906f4d00 0x563b908014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58716/0 (socket says 172.21.15.19:58716) 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55ca59c700 1 -- 172.21.15.19:0/498660090 learned_addr learned my addr 172.21.15.19:0/498660090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.186+0000 7f55cb59e700 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b907f7400 0x563b90801a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55ca59c700 1 -- 172.21.15.19:0/498660090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b907f7400 msgr2=0x563b90801a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55ca59c700 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b907f7400 0x563b90801a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55ca59c700 1 -- 172.21.15.19:0/498660090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 msgr2=0x563b90800f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55ca59c700 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b90800f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55ca59c700 1 -- 172.21.15.19:0/498660090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b90595490 con 0x563b906f4d00 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55cb59e700 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b907f7400 0x563b90801a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55cad9d700 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b90800f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55ca59c700 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b906f4d00 0x563b908014d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f55c00048a0 tx=0x7f55c0015df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88d67b894f72582 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55aeff5700 1 -- 172.21.15.19:0/498660090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c002d070 con 0x563b906f4d00 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55aeff5700 1 -- 172.21.15.19:0/498660090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f55c0013900 con 0x563b906f4d00 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/498660090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c0024070 con 0x563b906f4d00 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55aeff5700 1 -- 172.21.15.19:0/498660090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55c0013b00 con 0x563b906f4d00 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55dfc4fb80 10 client.16110 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.187+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/498660090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b9080e9a0 con 0x563b906f4d00 2024-05-09T01:28:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.188+0000 7f55aeff5700 1 -- 172.21.15.19:0/498660090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55c0024070 con 0x563b906f4d00 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.188+0000 7f55aeff5700 1 -- 172.21.15.19:0/498660090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f55c001e030 con 0x563b906f4d00 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.188+0000 7f55dfc4fb80 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.188+0000 7f55dfc4fb80 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.188+0000 7f55dfc4fb80 20 client.16110 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.188+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563b90810f80 0x563b908133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.188+0000 7f55dfc4fb80 4 client.16110 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.188+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/498660090 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b90727a80 con 0x563b90810f80 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.188+0000 7f55cad9d700 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563b90810f80 0x563b908133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.189+0000 7f55cad9d700 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563b90810f80 0x563b908133a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f042f2773570c3fb server_cookie=f9d092bbf16470ba in_seq=0 out_seq=0 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.189+0000 7f55aeff5700 10 client.16110 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.189+0000 7f55aeff5700 1 -- 172.21.15.19:0/498660090 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563b90727a80 con 0x563b90810f80 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.189+0000 7f55aeff5700 10 client.16110 handle_command_reply: tid=0 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 client.16110 shutdown 2024-05-09T01:28:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/498660090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b906f4d00 msgr2=0x563b908014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b906f4d00 0x563b908014d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f55c00048a0 tx=0x7f55c0015df0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/498660090 shutdown_connections 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563b90810f80 0x563b908133a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b907f7400 0x563b90801a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b90800f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55aeff5700 0 client.16110 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/498660090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b906f4d00 0x563b908014d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/498660090 >> 172.21.15.19:0/498660090 conn(0x563b9067df70 msgr2=0x563b907ebff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/498660090 shutdown_connections 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/498660090 wait complete. 2024-05-09T01:28:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.190+0000 7f55dfc4fb80 20 client.16110 trim_cache size 0 max 16384 2024-05-09T01:28:57.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.210+0000 7f55dfc4fb80 1 Processor -- start 2024-05-09T01:28:57.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.210+0000 7f55dfc4fb80 1 -- start start 2024-05-09T01:28:57.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b907fb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b9080d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90801850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1acc80 con 0x563b907f7400 2024-05-09T01:28:57.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ae7c0 con 0x563b9067a4a0 2024-05-09T01:28:57.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ad350 con 0x563b906f4d00 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90801850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90801850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58732/0 (socket says 172.21.15.19:58732) 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cb59e700 1 -- 172.21.15.19:0/889489976 learned_addr learned my addr 172.21.15.19:0/889489976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cad9d700 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b907fb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55ca59c700 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b9080d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cb59e700 1 -- 172.21.15.19:0/889489976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 msgr2=0x563b9080d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cb59e700 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b9080d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cb59e700 1 -- 172.21.15.19:0/889489976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 msgr2=0x563b907fb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cb59e700 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b907fb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cb59e700 1 -- 172.21.15.19:0/889489976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b907fdaf0 con 0x563b907f7400 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cad9d700 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b907fb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.211+0000 7f55cb59e700 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90801850 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f55c400b820 tx=0x7f55c4003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6735ab204c4b9ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.212+0000 7f55b0ff9700 1 -- 172.21.15.19:0/889489976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c402a070 con 0x563b907f7400 2024-05-09T01:28:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.212+0000 7f55b0ff9700 1 -- 172.21.15.19:0/889489976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55c4003c50 con 0x563b907f7400 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.212+0000 7f55b0ff9700 1 -- 172.21.15.19:0/889489976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40078d0 con 0x563b907f7400 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/889489976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 msgr2=0x563b90801850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90801850 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f55c400b820 tx=0x7f55c4003ab0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/889489976 shutdown_connections 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90801850 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b907fb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/889489976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b9080d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/889489976 >> 172.21.15.19:0/889489976 conn(0x563b9067df70 msgr2=0x563b907ec660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/889489976 shutdown_connections 2024-05-09T01:28:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/889489976 wait complete. 2024-05-09T01:28:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 -- start start 2024-05-09T01:28:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.213+0000 7f55dfc4fb80 1 Processor -- start 2024-05-09T01:28:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b90816770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b90813af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55dfc4fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90814030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ae0f0 con 0x563b907f7400 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1ada20 con 0x563b9067a4a0 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55dfc4fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1aac20 con 0x563b906f4d00 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55dfc4fb80 10 client.0 _ll_register_callbacks cb 0x563b9058b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90814030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90814030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58740/0 (socket says 172.21.15.19:58740) 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cb59e700 1 -- 172.21.15.19:0/2944415775 learned_addr learned my addr 172.21.15.19:0/2944415775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cad9d700 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b90816770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55ca59c700 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b90813af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cb59e700 1 -- 172.21.15.19:0/2944415775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 msgr2=0x563b90813af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cb59e700 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b90813af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cb59e700 1 -- 172.21.15.19:0/2944415775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 msgr2=0x563b90816770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cb59e700 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b90816770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cb59e700 1 -- 172.21.15.19:0/2944415775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b908145f0 con 0x563b907f7400 2024-05-09T01:28:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55cb59e700 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90814030 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005bd0 tx=0x7f55c4005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189d6cc32c29d5ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.214+0000 7f55aeff5700 1 -- 172.21.15.19:0/2944415775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4033060 con 0x563b907f7400 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55aeff5700 1 -- 172.21.15.19:0/2944415775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f55c4022040 con 0x563b907f7400 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2944415775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c4003c50 con 0x563b907f7400 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55aeff5700 1 -- 172.21.15.19:0/2944415775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55c402a030 con 0x563b907f7400 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55dfc4fb80 10 client.16116 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2944415775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b9087b180 con 0x563b907f7400 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55aeff5700 1 -- 172.21.15.19:0/2944415775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55c4003c50 con 0x563b907f7400 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55aeff5700 1 -- 172.21.15.19:0/2944415775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f55c4011070 con 0x563b907f7400 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55dfc4fb80 10 client.16116 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55dfc4fb80 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55dfc4fb80 20 client.16116 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563b9087b6d0 0x563b9087daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55dfc4fb80 4 client.16116 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.215+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2944415775 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b90721b20 con 0x563b9087b6d0 2024-05-09T01:28:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55cad9d700 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563b9087b6d0 0x563b9087daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55aeff5700 10 client.16116 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55cad9d700 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563b9087b6d0 0x563b9087daf0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6c5d782b2913bc server_cookie=597bcf9eb909c359 in_seq=0 out_seq=0 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55aeff5700 1 -- 172.21.15.19:0/2944415775 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b90721b20 con 0x563b9087b6d0 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55aeff5700 10 client.16116 handle_command_reply: tid=0 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55dfc4fb80 1 client.16116 shutdown 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2944415775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 msgr2=0x563b90814030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90814030 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005bd0 tx=0x7f55c4005750 comp rx=0 tx=0).stop 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2944415775 shutdown_connections 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563b9087b6d0 0x563b9087daf0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b907f7400 0x563b90814030 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b9067a4a0 0x563b90816770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55aeff5700 0 client.16116 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55dfc4fb80 1 --2- 172.21.15.19:0/2944415775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b906f4d00 0x563b90813af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.216+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2944415775 >> 172.21.15.19:0/2944415775 conn(0x563b9067df70 msgr2=0x563b907f63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:57.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2944415775 shutdown_connections 2024-05-09T01:28:57.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55dfc4fb80 1 -- 172.21.15.19:0/2944415775 wait complete. 2024-05-09T01:28:57.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55dfc4fb80 20 client.16116 trim_cache size 0 max 16384 2024-05-09T01:28:57.219 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:28:57.219 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:28:57.219 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:28:57.219 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:28:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 -- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55b40b2900 msgr2=0x7f55b40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55b40b2900 0x7f55b40b4e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f55cc1024b0 tx=0x7f55c0009190 comp rx=0 tx=0).stop 2024-05-09T01:28:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 -- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc1034c0 msgr2=0x7f55cc19bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc1034c0 0x7f55cc19bd60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00b530 tx=0x7f55bc00b840 comp rx=0 tx=0).stop 2024-05-09T01:28:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 -- 172.21.15.19:0/3089500491 shutdown_connections 2024-05-09T01:28:57.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55cc108740 0x7f55cc19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f55b40b2900 0x7f55b40b4e10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55cc1034c0 0x7f55cc19bd60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 --2- 172.21.15.19:0/3089500491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55cc107df0 0x7f55cc19c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 -- 172.21.15.19:0/3089500491 >> 172.21.15.19:0/3089500491 conn(0x7f55cc0fedc0 msgr2=0x7f55cc1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 -- 172.21.15.19:0/3089500491 shutdown_connections 2024-05-09T01:28:57.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:57.217+0000 7f55d1239700 1 -- 172.21.15.19:0/3089500491 wait complete. 2024-05-09T01:28:57.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:57 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:28:57.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:57 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:28:57.992 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:28:57.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:28:58.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:57 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:28:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:28:58.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:28:58 smithi019 ceph-mon[30158]: pgmap v481: 97 pgs: 97 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:28:58.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:28:58 smithi196 ceph-mon[35041]: pgmap v481: 97 pgs: 97 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:28:59.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:28:58 smithi059 ceph-mon[34351]: pgmap v481: 97 pgs: 97 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:28:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.776+0000 7ff62d984640 1 -- 172.21.15.19:0/875428681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628103db0 msgr2=0x7ff6281041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.776+0000 7ff62d984640 1 --2- 172.21.15.19:0/875428681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628103db0 0x7ff6281041c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff610009900 tx=0x7ff61002a410 comp rx=0 tx=0).stop 2024-05-09T01:28:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.776+0000 7ff62d984640 1 -- 172.21.15.19:0/875428681 shutdown_connections 2024-05-09T01:28:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.776+0000 7ff62d984640 1 --2- 172.21.15.19:0/875428681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff628108ab0 0x7ff62810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.776+0000 7ff62d984640 1 --2- 172.21.15.19:0/875428681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628103db0 0x7ff6281041c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.776+0000 7ff62d984640 1 --2- 172.21.15.19:0/875428681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff628102b70 0x7ff628103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.776+0000 7ff62d984640 1 -- 172.21.15.19:0/875428681 >> 172.21.15.19:0/875428681 conn(0x7ff6280fdd60 msgr2=0x7ff628100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.776+0000 7ff62d984640 1 -- 172.21.15.19:0/875428681 shutdown_connections 2024-05-09T01:28:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.776+0000 7ff62d984640 1 -- 172.21.15.19:0/875428681 wait complete. 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff62d984640 1 Processor -- start 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff62d984640 1 -- start start 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff62d984640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628102b70 0x7ff62806be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff62d984640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff628103db0 0x7ff62819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff62d984640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff628108ab0 0x7ff6281a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff62d984640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810e690 con 0x7ff628102b70 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff62d984640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810e510 con 0x7ff628108ab0 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff62d984640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810e810 con 0x7ff628103db0 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff626ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628102b70 0x7ff62806be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff626ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628102b70 0x7ff62806be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49906/0 (socket says 172.21.15.19:49906) 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.777+0000 7ff626ffd640 1 -- 172.21.15.19:0/117182425 learned_addr learned my addr 172.21.15.19:0/117182425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff6277fe640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff628108ab0 0x7ff6281a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff626ffd640 1 -- 172.21.15.19:0/117182425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff628103db0 msgr2=0x7ff62819f100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T01:28:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff626ffd640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff628103db0 0x7ff62819f100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff626ffd640 1 -- 172.21.15.19:0/117182425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff628108ab0 msgr2=0x7ff6281a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff626ffd640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff628108ab0 0x7ff6281a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff626ffd640 1 -- 172.21.15.19:0/117182425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6281a1e90 con 0x7ff628102b70 2024-05-09T01:28:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff626ffd640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628102b70 0x7ff62806be30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff618005d00 tx=0x7ff61800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff62c982640 1 -- 172.21.15.19:0/117182425 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618017070 con 0x7ff628102b70 2024-05-09T01:28:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff62c982640 1 -- 172.21.15.19:0/117182425 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff618009c30 con 0x7ff628102b70 2024-05-09T01:28:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff62d984640 1 -- 172.21.15.19:0/117182425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6281a2180 con 0x7ff628102b70 2024-05-09T01:28:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff62c982640 1 -- 172.21.15.19:0/117182425 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61800f620 con 0x7ff628102b70 2024-05-09T01:28:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.778+0000 7ff62d984640 1 -- 172.21.15.19:0/117182425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6281a2680 con 0x7ff628102b70 2024-05-09T01:28:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff63483b740 1 Processor -- start 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff63483b740 1 -- start start 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff63483b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 0x55d96be6b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff63483b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 0x55d96be6ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff62c982640 1 -- 172.21.15.19:0/117182425 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff61801a070 con 0x7ff628102b70 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff63483b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be72980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff63483b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d96bdcf7c0 con 0x55d96baa64d0 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff63483b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d96bd6b0e0 con 0x55d96bac71b0 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff63483b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d96bdcb780 con 0x55d96ba9e2f0 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff61ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be72980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff6277fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 0x55d96be6b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff61ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be72980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49912/0 (socket says 172.21.15.19:49912) 2024-05-09T01:28:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.781+0000 7ff61ffff640 1 -- 172.21.15.19:0/2433072502 learned_addr learned my addr 172.21.15.19:0/2433072502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.782+0000 7ff6277fe640 1 -- 172.21.15.19:0/2433072502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 msgr2=0x55d96be6ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.782+0000 7ff6277fe640 1 --2- 172.21.15.19:0/2433072502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 0x55d96be6ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.782+0000 7ff6277fe640 1 -- 172.21.15.19:0/2433072502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 msgr2=0x55d96be72980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.782+0000 7ff6277fe640 1 --2- 172.21.15.19:0/2433072502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be72980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.782+0000 7ff6277fe640 1 -- 172.21.15.19:0/2433072502 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96bd947c0 con 0x55d96bac71b0 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.782+0000 7ff61ffff640 1 --2- 172.21.15.19:0/2433072502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be72980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.782+0000 7ff62c982640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6140b27e0 0x7ff6140b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.782+0000 7ff626ffd640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6140b27e0 0x7ff6140b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.782+0000 7ff62c982640 1 -- 172.21.15.19:0/117182425 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff6180e1d60 con 0x7ff628102b70 2024-05-09T01:28:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.783+0000 7ff626ffd640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6140b27e0 0x7ff6140b4cf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff6180a53e0 tx=0x7ff618091040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.789+0000 7ff6277fe640 1 --2- 172.21.15.19:0/2433072502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 0x55d96be6b500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff620009ed0 tx=0x7ff6200107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be5e7a165fcb9e77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.789+0000 7ff61dffb640 1 -- 172.21.15.19:0/2433072502 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620003b70 con 0x55d96bac71b0 2024-05-09T01:28:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.789+0000 7ff61dffb640 1 -- 172.21.15.19:0/2433072502 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff620015070 con 0x55d96bac71b0 2024-05-09T01:28:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.789+0000 7ff61dffb640 1 -- 172.21.15.19:0/2433072502 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6200117b0 con 0x55d96bac71b0 2024-05-09T01:28:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 -- 172.21.15.19:0/2433072502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 msgr2=0x55d96be6b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 --2- 172.21.15.19:0/2433072502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 0x55d96be6b500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff620009ed0 tx=0x7ff6200107a0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 -- 172.21.15.19:0/2433072502 shutdown_connections 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 --2- 172.21.15.19:0/2433072502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be72980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 --2- 172.21.15.19:0/2433072502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 0x55d96be6ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 --2- 172.21.15.19:0/2433072502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 0x55d96be6b500 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 -- 172.21.15.19:0/2433072502 >> 172.21.15.19:0/2433072502 conn(0x55d96babb300 msgr2=0x55d96be66e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 -- 172.21.15.19:0/2433072502 shutdown_connections 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 -- 172.21.15.19:0/2433072502 wait complete. 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.790+0000 7ff63483b740 1 -- start start 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff63483b740 1 Processor -- start 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff63483b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 0x55d96be752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff63483b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be757e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff63483b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 0x55d96be7c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff63483b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d96be733d0 con 0x55d96baa64d0 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff63483b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d96bd569c0 con 0x55d96bac71b0 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff63483b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d96be73550 con 0x55d96ba9e2f0 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff63483b740 10 client.0 _ll_register_callbacks cb 0x55d96bd83020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff61ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff61ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be757e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49914/0 (socket says 172.21.15.19:49914) 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff61ffff640 1 -- 172.21.15.19:0/2618021194 learned_addr learned my addr 172.21.15.19:0/2618021194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff6277fe640 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 0x55d96be7c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff6277fe640 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 0x55d96be752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff61ffff640 1 -- 172.21.15.19:0/2618021194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 msgr2=0x55d96be752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff61ffff640 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 0x55d96be752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff61ffff640 1 -- 172.21.15.19:0/2618021194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 msgr2=0x55d96be7c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff61ffff640 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 0x55d96be7c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff61ffff640 1 -- 172.21.15.19:0/2618021194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96bd7d1f0 con 0x55d96baa64d0 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.791+0000 7ff6277fe640 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 0x55d96be752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff61ffff640 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be757e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff610032d50 tx=0x7ff610032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f5d44664b02a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff606ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff606ffd640 1 -- 172.21.15.19:0/2618021194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610040070 con 0x55d96baa64d0 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff606ffd640 1 -- 172.21.15.19:0/2618021194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff610004040 con 0x55d96baa64d0 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff63483b740 1 -- 172.21.15.19:0/2618021194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff620015070 con 0x55d96baa64d0 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff606ffd640 1 -- 172.21.15.19:0/2618021194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff61004a070 con 0x55d96baa64d0 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff63483b740 10 client.16128 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff63483b740 1 -- 172.21.15.19:0/2618021194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d96bb32790 con 0x55d96baa64d0 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff606ffd640 1 -- 172.21.15.19:0/2618021194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff620015070 con 0x55d96baa64d0 2024-05-09T01:28:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff606ffd640 1 -- 172.21.15.19:0/2618021194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff6100046c0 con 0x55d96baa64d0 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff63483b740 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff63483b740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff63483b740 20 client.16128 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff63483b740 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d96ba85d80 0x55d96be82c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff63483b740 4 client.16128 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.792+0000 7ff63483b740 1 -- 172.21.15.19:0/2618021194 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96bddd3d0 con 0x55d96ba85d80 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.793+0000 7ff6277fe640 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d96ba85d80 0x55d96be82c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.793+0000 7ff606ffd640 10 client.16128 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.793+0000 7ff6277fe640 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d96ba85d80 0x55d96be82c30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a17d2393b768d8a server_cookie=e3c90eb83aaa24f9 in_seq=0 out_seq=0 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff606ffd640 1 -- 172.21.15.19:0/2618021194 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d96bddd3d0 con 0x55d96ba85d80 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff606ffd640 10 client.16128 handle_command_reply: tid=0 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 client.16128 shutdown 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 -- 172.21.15.19:0/2618021194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 msgr2=0x55d96be757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be757e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff610032d50 tx=0x7ff610032d80 comp rx=0 tx=0).stop 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 -- 172.21.15.19:0/2618021194 shutdown_connections 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d96ba85d80 0x55d96be82c30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96bac71b0 0x55d96be7c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96baa64d0 0x55d96be757e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff606ffd640 0 client.16128 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 --2- 172.21.15.19:0/2618021194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96ba9e2f0 0x55d96be752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 -- 172.21.15.19:0/2618021194 >> 172.21.15.19:0/2618021194 conn(0x55d96babb300 msgr2=0x55d96be69da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 -- 172.21.15.19:0/2618021194 shutdown_connections 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 1 -- 172.21.15.19:0/2618021194 wait complete. 2024-05-09T01:28:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.794+0000 7ff63483b740 20 client.16128 trim_cache size 0 max 16384 2024-05-09T01:28:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff63483b740 1 Processor -- start 2024-05-09T01:28:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff63483b740 1 -- start start 2024-05-09T01:28:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff63483b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96babb300 0x55d96be6a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff63483b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be6a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff63483b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 0x55d96be88480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff63483b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281ab4a0 con 0x55d96ba9e2f0 2024-05-09T01:28:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff63483b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281acfe0 con 0x55d96baa64d0 2024-05-09T01:28:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff63483b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281abb70 con 0x55d96babb300 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff61ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be6a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff61ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be6a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49922/0 (socket says 172.21.15.19:49922) 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff61ffff640 1 -- 172.21.15.19:0/536317662 learned_addr learned my addr 172.21.15.19:0/536317662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff61ffff640 1 -- 172.21.15.19:0/536317662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96babb300 msgr2=0x55d96be6a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff61ffff640 1 --2- 172.21.15.19:0/536317662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96babb300 0x55d96be6a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff6277fe640 1 --2- 172.21.15.19:0/536317662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 0x55d96be88480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff61ffff640 1 -- 172.21.15.19:0/536317662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 msgr2=0x55d96be88480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff61ffff640 1 --2- 172.21.15.19:0/536317662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 0x55d96be88480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.810+0000 7ff61ffff640 1 -- 172.21.15.19:0/536317662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96bba95f0 con 0x55d96ba9e2f0 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff61ffff640 1 --2- 172.21.15.19:0/536317662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be6a8e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff610009850 tx=0x7ff61002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9650650816f2b20a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff61d7fa640 1 -- 172.21.15.19:0/536317662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610040070 con 0x55d96ba9e2f0 2024-05-09T01:28:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff61d7fa640 1 -- 172.21.15.19:0/536317662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff610031c60 con 0x55d96ba9e2f0 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff61d7fa640 1 -- 172.21.15.19:0/536317662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610035040 con 0x55d96ba9e2f0 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff63483b740 1 -- 172.21.15.19:0/536317662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 msgr2=0x55d96be6a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff63483b740 1 --2- 172.21.15.19:0/536317662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be6a8e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff610009850 tx=0x7ff61002a410 comp rx=0 tx=0).stop 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff63483b740 1 -- 172.21.15.19:0/536317662 shutdown_connections 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff63483b740 1 --2- 172.21.15.19:0/536317662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 0x55d96be88480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff63483b740 1 --2- 172.21.15.19:0/536317662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be6a8e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff63483b740 1 --2- 172.21.15.19:0/536317662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96babb300 0x55d96be6a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.811+0000 7ff63483b740 1 -- 172.21.15.19:0/536317662 >> 172.21.15.19:0/536317662 conn(0x55d96be6ff10 msgr2=0x55d96be69c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 -- 172.21.15.19:0/536317662 shutdown_connections 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 -- 172.21.15.19:0/536317662 wait complete. 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 -- start start 2024-05-09T01:28:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 Processor -- start 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be7a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 0x55d96be7a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96babb300 0x55d96be7b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281ac910 con 0x55d96ba9e2f0 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281ac240 con 0x55d96baa64d0 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.812+0000 7ff63483b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281a9480 con 0x55d96babb300 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff63483b740 10 client.0 _ll_register_callbacks cb 0x7ff610009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff6277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be7a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff6277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be7a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49926/0 (socket says 172.21.15.19:49926) 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff6277fe640 1 -- 172.21.15.19:0/511839225 learned_addr learned my addr 172.21.15.19:0/511839225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff61ffff640 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 0x55d96be7a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff6277fe640 1 -- 172.21.15.19:0/511839225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96babb300 msgr2=0x55d96be7b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff6277fe640 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96babb300 0x55d96be7b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff6277fe640 1 -- 172.21.15.19:0/511839225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 msgr2=0x55d96be7a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff6277fe640 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 0x55d96be7a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff6277fe640 1 -- 172.21.15.19:0/511839225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96bd93ed0 con 0x55d96ba9e2f0 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff61ffff640 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 0x55d96be7a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff6277fe640 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be7a3e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff620005fd0 tx=0x7ff620009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59f516bc6bb38d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff606ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff606ffd640 1 -- 172.21.15.19:0/511839225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620005790 con 0x55d96ba9e2f0 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff606ffd640 1 -- 172.21.15.19:0/511839225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff62002ac70 con 0x55d96ba9e2f0 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff63483b740 1 -- 172.21.15.19:0/511839225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff610031c60 con 0x55d96ba9e2f0 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff606ffd640 1 -- 172.21.15.19:0/511839225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff620003c30 con 0x55d96ba9e2f0 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff63483b740 10 client.16134 fetch_fsmap learned FSMap version 28 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.813+0000 7ff63483b740 1 -- 172.21.15.19:0/511839225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d96be7ced0 con 0x55d96ba9e2f0 2024-05-09T01:28:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.814+0000 7ff606ffd640 1 -- 172.21.15.19:0/511839225 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff610031c60 con 0x55d96ba9e2f0 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.814+0000 7ff606ffd640 1 -- 172.21.15.19:0/511839225 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff620028070 con 0x55d96ba9e2f0 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.814+0000 7ff63483b740 10 client.16134 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.814+0000 7ff63483b740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.814+0000 7ff63483b740 20 client.16134 populate_metadata read hostname 'smithi019' 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.814+0000 7ff63483b740 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d96bac71b0 0x55d96beed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.814+0000 7ff63483b740 4 client.16134 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.814+0000 7ff63483b740 1 -- 172.21.15.19:0/511839225 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d96be6b890 con 0x55d96bac71b0 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.814+0000 7ff61ffff640 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d96bac71b0 0x55d96beed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.815+0000 7ff61ffff640 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d96bac71b0 0x55d96beed2c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12d6e4df6968f7a server_cookie=30e8f8badaa04c86 in_seq=0 out_seq=0 2024-05-09T01:28:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.815+0000 7ff606ffd640 10 client.16134 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff606ffd640 1 -- 172.21.15.19:0/511839225 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d96be6b890 con 0x55d96bac71b0 2024-05-09T01:28:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff606ffd640 10 client.16134 handle_command_reply: tid=0 2024-05-09T01:28:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 client.16134 shutdown 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 -- 172.21.15.19:0/511839225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 msgr2=0x55d96be7a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be7a3e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff620005fd0 tx=0x7ff620009970 comp rx=0 tx=0).stop 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 -- 172.21.15.19:0/511839225 shutdown_connections 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d96bac71b0 0x55d96beed2c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96babb300 0x55d96be7b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96baa64d0 0x55d96be7a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff606ffd640 0 client.16134 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 --2- 172.21.15.19:0/511839225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96ba9e2f0 0x55d96be7a3e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 -- 172.21.15.19:0/511839225 >> 172.21.15.19:0/511839225 conn(0x55d96be6ff10 msgr2=0x55d96be6a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 -- 172.21.15.19:0/511839225 shutdown_connections 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 1 -- 172.21.15.19:0/511839225 wait complete. 2024-05-09T01:28:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff63483b740 20 client.16134 trim_cache size 0 max 16384 2024-05-09T01:28:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:28:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:28:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 -- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6140b27e0 msgr2=0x7ff6140b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6140b27e0 0x7ff6140b4cf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff6180a53e0 tx=0x7ff618091040 comp rx=0 tx=0).stop 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 -- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628102b70 msgr2=0x7ff62806be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628102b70 0x7ff62806be30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff618005d00 tx=0x7ff61800dde0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 -- 172.21.15.19:0/117182425 shutdown_connections 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6140b27e0 0x7ff6140b4cf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff628108ab0 0x7ff6281a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff628103db0 0x7ff62819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 --2- 172.21.15.19:0/117182425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff628102b70 0x7ff62806be30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 -- 172.21.15.19:0/117182425 >> 172.21.15.19:0/117182425 conn(0x7ff6280fdd60 msgr2=0x7ff628104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 -- 172.21.15.19:0/117182425 shutdown_connections 2024-05-09T01:28:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:28:59.832+0000 7ff62d984640 1 -- 172.21.15.19:0/117182425 wait complete. 2024-05-09T01:28:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.955+0000 7f7cdcb36700 1 -- 172.21.15.19:0/2420371658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8108760 msgr2=0x7f7cd81100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.955+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/2420371658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8108760 0x7f7cd81100c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00099c0 tx=0x7f7cd0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.955+0000 7f7cdcb36700 1 -- 172.21.15.19:0/2420371658 shutdown_connections 2024-05-09T01:28:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.955+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/2420371658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8108760 0x7f7cd81100c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.955+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/2420371658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd81034e0 0x7f7cd81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.956+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/2420371658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd8107e10 0x7f7cd8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.956+0000 7f7cdcb36700 1 -- 172.21.15.19:0/2420371658 >> 172.21.15.19:0/2420371658 conn(0x7f7cd80fef30 msgr2=0x7f7cd8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:28:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.956+0000 7f7cdcb36700 1 -- 172.21.15.19:0/2420371658 shutdown_connections 2024-05-09T01:28:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.956+0000 7f7cdcb36700 1 -- 172.21.15.19:0/2420371658 wait complete. 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.956+0000 7f7cdcb36700 1 Processor -- start 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.956+0000 7f7cdcb36700 1 -- start start 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cdcb36700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd81034e0 0x7f7cd80ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cdcb36700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8107e10 0x7f7cd80ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cdcb36700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd8108760 0x7f7cd80ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cdcb36700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd8113390 con 0x7f7cd8107e10 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cdcb36700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd8113210 con 0x7f7cd81034e0 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cdcb36700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd8113510 con 0x7f7cd8108760 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8107e10 0x7f7cd80ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8107e10 0x7f7cd80ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49950/0 (socket says 172.21.15.19:49950) 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd5d9b700 1 -- 172.21.15.19:0/1896082112 learned_addr learned my addr 172.21.15.19:0/1896082112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd659c700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd81034e0 0x7f7cd80ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd6d9d700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd8108760 0x7f7cd80ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd5d9b700 1 -- 172.21.15.19:0/1896082112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd8108760 msgr2=0x7f7cd80ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd5d9b700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd8108760 0x7f7cd80ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd5d9b700 1 -- 172.21.15.19:0/1896082112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd81034e0 msgr2=0x7f7cd80ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd5d9b700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd81034e0 0x7f7cd80ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd5d9b700 1 -- 172.21.15.19:0/1896082112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd81a9130 con 0x7f7cd8107e10 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.957+0000 7f7cd5d9b700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8107e10 0x7f7cd80ff950 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00d950 tx=0x7f7ccc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.958+0000 7f7cc77fe700 1 -- 172.21.15.19:0/1896082112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc016070 con 0x7f7cd8107e10 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.958+0000 7f7cc77fe700 1 -- 172.21.15.19:0/1896082112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ccc004830 con 0x7f7cd8107e10 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.958+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd81a9420 con 0x7f7cd8107e10 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.958+0000 7f7cc77fe700 1 -- 172.21.15.19:0/1896082112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc005440 con 0x7f7cd8107e10 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.958+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd81a99b0 con 0x7f7cd8107e10 2024-05-09T01:28:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.959+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd80512c0 con 0x7f7cd8107e10 2024-05-09T01:28:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.960+0000 7f7cc77fe700 1 -- 172.21.15.19:0/1896082112 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7ccc0055e0 con 0x7f7cd8107e10 2024-05-09T01:28:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.961+0000 7f7cc77fe700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7cc00b2900 0x7f7cc00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:28:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.961+0000 7f7cd659c700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7cc00b2900 0x7f7cc00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:28:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.961+0000 7f7cc77fe700 1 -- 172.21.15.19:0/1896082112 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7ccc0e2460 con 0x7f7cd8107e10 2024-05-09T01:28:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.962+0000 7f7cd659c700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7cc00b2900 0x7f7cc00b4e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009550 tx=0x7f7cc80091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:28:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:28:59.965+0000 7f7cc77fe700 1 -- 172.21.15.19:0/1896082112 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ccc01f020 con 0x7f7cd8107e10 2024-05-09T01:29:00.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.225+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cd802e5f0 con 0x7f7cd8107e10 2024-05-09T01:29:00.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.226+0000 7f7cc77fe700 1 -- 172.21.15.19:0/1896082112 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f7ccc0aa100 con 0x7f7cd8107e10 2024-05-09T01:29:00.223 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:29:00.223 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:29:00.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7cc00b2900 msgr2=0x7f7cc00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:00.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7cc00b2900 0x7f7cc00b4e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009550 tx=0x7f7cc80091f0 comp rx=0 tx=0).stop 2024-05-09T01:29:00.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8107e10 msgr2=0x7f7cd80ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:00.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8107e10 0x7f7cd80ff950 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00d950 tx=0x7f7ccc00dc60 comp rx=0 tx=0).stop 2024-05-09T01:29:00.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 shutdown_connections 2024-05-09T01:29:00.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd8108760 0x7f7cd80ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:00.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd81034e0 0x7f7cd80ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:00.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7cc00b2900 0x7f7cc00b4e10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:00.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 --2- 172.21.15.19:0/1896082112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cd8107e10 0x7f7cd80ff950 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:00.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 >> 172.21.15.19:0/1896082112 conn(0x7f7cd80fef30 msgr2=0x7f7cd806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:00.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 shutdown_connections 2024-05-09T01:29:00.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:00.228+0000 7f7cdcb36700 1 -- 172.21.15.19:0/1896082112 wait complete. 2024-05-09T01:29:00.226 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:29:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:00 smithi019 ceph-mon[30158]: pgmap v482: 97 pgs: 97 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:29:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:00 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1896082112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:29:00.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:00 smithi196 ceph-mon[35041]: pgmap v482: 97 pgs: 97 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:29:00.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:00 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1896082112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:29:00.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:29:01.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:00 smithi059 ceph-mon[34351]: pgmap v482: 97 pgs: 97 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:29:01.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:00 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1896082112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:29:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:02.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:02 smithi019 ceph-mon[30158]: pgmap v483: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:29:02.898 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:02 smithi059 ceph-mon[34351]: pgmap v483: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:29:02.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:02 smithi196 ceph-mon[35041]: pgmap v483: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:29:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.990+0000 7fcb34098700 1 -- 172.21.15.19:0/1040057471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 msgr2=0x7fcb2c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.990+0000 7fcb34098700 1 --2- 172.21.15.19:0/1040057471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 0x7fcb2c1100c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b0d0 tx=0x7fcb2400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:29:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.990+0000 7fcb34098700 1 -- 172.21.15.19:0/1040057471 shutdown_connections 2024-05-09T01:29:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.990+0000 7fcb34098700 1 --2- 172.21.15.19:0/1040057471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 0x7fcb2c1100c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.990+0000 7fcb34098700 1 --2- 172.21.15.19:0/1040057471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb2c1034e0 0x7fcb2c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.990+0000 7fcb34098700 1 --2- 172.21.15.19:0/1040057471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb2c107e10 0x7fcb2c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.990+0000 7fcb34098700 1 -- 172.21.15.19:0/1040057471 >> 172.21.15.19:0/1040057471 conn(0x7fcb2c0fef30 msgr2=0x7fcb2c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.990+0000 7fcb34098700 1 -- 172.21.15.19:0/1040057471 shutdown_connections 2024-05-09T01:29:02.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.990+0000 7fcb34098700 1 -- 172.21.15.19:0/1040057471 wait complete. 2024-05-09T01:29:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.991+0000 7fcb34098700 1 Processor -- start 2024-05-09T01:29:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.991+0000 7fcb34098700 1 -- start start 2024-05-09T01:29:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.991+0000 7fcb34098700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb2c1034e0 0x7fcb2c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.991+0000 7fcb34098700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb2c107e10 0x7fcb2c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.991+0000 7fcb34098700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 0x7fcb2c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.991+0000 7fcb34098700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1144e0 con 0x7fcb2c108760 2024-05-09T01:29:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.991+0000 7fcb34098700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c114360 con 0x7fcb2c1034e0 2024-05-09T01:29:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.991+0000 7fcb34098700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c114660 con 0x7fcb2c107e10 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb32635700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 0x7fcb2c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb32635700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 0x7fcb2c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49966/0 (socket says 172.21.15.19:49966) 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb32635700 1 -- 172.21.15.19:0/3647028140 learned_addr learned my addr 172.21.15.19:0/3647028140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb31633700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb2c107e10 0x7fcb2c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb31e34700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb2c1034e0 0x7fcb2c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb32635700 1 -- 172.21.15.19:0/3647028140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb2c107e10 msgr2=0x7fcb2c19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb32635700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb2c107e10 0x7fcb2c19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb32635700 1 -- 172.21.15.19:0/3647028140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb2c1034e0 msgr2=0x7fcb2c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb32635700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb2c1034e0 0x7fcb2c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb32635700 1 -- 172.21.15.19:0/3647028140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c19fd20 con 0x7fcb2c108760 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb32635700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 0x7fcb2c19f680 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcb24007a00 tx=0x7fcb24007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb1effd700 1 -- 172.21.15.19:0/3647028140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2401f070 con 0x7fcb2c108760 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb1effd700 1 -- 172.21.15.19:0/3647028140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb24004c50 con 0x7fcb2c108760 2024-05-09T01:29:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb34098700 1 -- 172.21.15.19:0/3647028140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c19ffb0 con 0x7fcb2c108760 2024-05-09T01:29:02.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.992+0000 7fcb1effd700 1 -- 172.21.15.19:0/3647028140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2401b690 con 0x7fcb2c108760 2024-05-09T01:29:02.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.993+0000 7fcb34098700 1 -- 172.21.15.19:0/3647028140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c1a7930 con 0x7fcb2c108760 2024-05-09T01:29:02.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.995+0000 7fcb1effd700 1 -- 172.21.15.19:0/3647028140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcb24022070 con 0x7fcb2c108760 2024-05-09T01:29:02.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.996+0000 7fcb1effd700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb180b2900 0x7fcb180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:02.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.996+0000 7fcb31e34700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb180b2900 0x7fcb180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:02.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.996+0000 7fcb1effd700 1 -- 172.21.15.19:0/3647028140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcb2400f020 con 0x7fcb2c108760 2024-05-09T01:29:02.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb42aaeb80 1 Processor -- start 2024-05-09T01:29:02.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb31e34700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb180b2900 0x7fcb180b4e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009c60 tx=0x7fcb20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:02.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb42aaeb80 1 -- start start 2024-05-09T01:29:02.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x5574831294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:02.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x557483129a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:02.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574831236a0 0x557483123ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:02.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557483045ff0 con 0x557482fa64a0 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557482fe47f0 con 0x557483090160 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557483041fb0 con 0x5574831236a0 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb31633700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x557483129a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb31633700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x557483129a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49968/0 (socket says 172.21.15.19:49968) 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb31633700 1 -- 172.21.15.19:0/2540792362 learned_addr learned my addr 172.21.15.19:0/2540792362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb31e34700 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x5574831294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.997+0000 7fcb32635700 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574831236a0 0x557483123ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb31633700 1 -- 172.21.15.19:0/2540792362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574831236a0 msgr2=0x557483123ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb31633700 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574831236a0 0x557483123ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb31633700 1 -- 172.21.15.19:0/2540792362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 msgr2=0x5574831294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb31633700 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x5574831294f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb31633700 1 -- 172.21.15.19:0/2540792362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557482fdc370 con 0x557482fa64a0 2024-05-09T01:29:02.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb32635700 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574831236a0 0x557483123ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:02.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb31633700 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x557483129a30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009cb0 tx=0x7fcb28011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dab7ce554d630 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:02.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb17fff700 1 -- 172.21.15.19:0/2540792362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28011d20 con 0x557482fa64a0 2024-05-09T01:29:02.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb17fff700 1 -- 172.21.15.19:0/2540792362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb28004500 con 0x557482fa64a0 2024-05-09T01:29:02.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.998+0000 7fcb17fff700 1 -- 172.21.15.19:0/2540792362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28005260 con 0x557482fa64a0 2024-05-09T01:29:02.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.999+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/2540792362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 msgr2=0x557483129a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.999+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x557483129a30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009cb0 tx=0x7fcb28011440 comp rx=0 tx=0).stop 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.999+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/2540792362 shutdown_connections 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.999+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574831236a0 0x557483123ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.999+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x5574831294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.999+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/2540792362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x557483129a30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.999+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/2540792362 >> 172.21.15.19:0/2540792362 conn(0x5574830066a0 msgr2=0x55748311e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.999+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/2540792362 shutdown_connections 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:02.999+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/2540792362 wait complete. 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb42aaeb80 1 -- start start 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb42aaeb80 1 Processor -- start 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x55748312d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557483090160 0x55748312d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574831236a0 0x55748312dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557482ec1ba0 con 0x557482fa64a0 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557483000150 con 0x5574831236a0 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5574831301d0 con 0x557483090160 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb42aaeb80 10 client.0 _ll_register_callbacks cb 0x557482eb76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb31e34700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x55748312d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb31e34700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x55748312d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49982/0 (socket says 172.21.15.19:49982) 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb31e34700 1 -- 172.21.15.19:0/315381205 learned_addr learned my addr 172.21.15.19:0/315381205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb32635700 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574831236a0 0x55748312dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.000+0000 7fcb31633700 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557483090160 0x55748312d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb31e34700 1 -- 172.21.15.19:0/315381205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557483090160 msgr2=0x55748312d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb31e34700 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557483090160 0x55748312d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb31e34700 1 -- 172.21.15.19:0/315381205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574831236a0 msgr2=0x55748312dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb31e34700 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574831236a0 0x55748312dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb31e34700 1 -- 172.21.15.19:0/315381205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557482ec1490 con 0x557482fa64a0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb31e34700 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x55748312d140 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcb20005910 tx=0x7fcb200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513c89f9761be88c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb157fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb157fa700 1 -- 172.21.15.19:0/315381205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2001d720 con 0x557482fa64a0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb157fa700 1 -- 172.21.15.19:0/315381205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcb2001d8c0 con 0x557482fa64a0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/315381205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb28004500 con 0x557482fa64a0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb157fa700 1 -- 172.21.15.19:0/315381205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcb2001dac0 con 0x557482fa64a0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb42aaeb80 10 client.16152 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.001+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/315381205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55748313ac40 con 0x557482fa64a0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.002+0000 7fcb157fa700 1 -- 172.21.15.19:0/315381205 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb28004500 con 0x557482fa64a0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.002+0000 7fcb157fa700 1 -- 172.21.15.19:0/315381205 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcb20027420 con 0x557482fa64a0 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.002+0000 7fcb42aaeb80 10 client.16152 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.002+0000 7fcb42aaeb80 10 client.16152 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:29:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.002+0000 7fcb42aaeb80 20 client.16152 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:03.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.002+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55748313d220 0x55748313f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.002+0000 7fcb42aaeb80 4 client.16152 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:03.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.002+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/315381205 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574830539d0 con 0x55748313d220 2024-05-09T01:29:03.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.002+0000 7fcb31633700 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55748313d220 0x55748313f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.003+0000 7fcb31633700 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55748313d220 0x55748313f6c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3496eef1efe7a132 server_cookie=17a84b99f558d4a in_seq=0 out_seq=0 2024-05-09T01:29:03.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.003+0000 7fcb157fa700 10 client.16152 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:29:03.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.003+0000 7fcb157fa700 1 -- 172.21.15.19:0/315381205 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574830539d0 con 0x55748313d220 2024-05-09T01:29:03.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.003+0000 7fcb157fa700 10 client.16152 handle_command_reply: tid=0 2024-05-09T01:29:03.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.003+0000 7fcb42aaeb80 1 client.16152 shutdown 2024-05-09T01:29:03.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/315381205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 msgr2=0x55748312d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:03.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x55748312d140 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcb20005910 tx=0x7fcb200047c0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/315381205 shutdown_connections 2024-05-09T01:29:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55748313d220 0x55748313f6c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574831236a0 0x55748312dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557482fa64a0 0x55748312d140 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb157fa700 0 client.16152 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:29:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/315381205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557483090160 0x55748312d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/315381205 >> 172.21.15.19:0/315381205 conn(0x5574830066a0 msgr2=0x55748311e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/315381205 shutdown_connections 2024-05-09T01:29:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/315381205 wait complete. 2024-05-09T01:29:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.004+0000 7fcb42aaeb80 20 client.16152 trim_cache size 0 max 16384 2024-05-09T01:29:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb42aaeb80 1 Processor -- start 2024-05-09T01:29:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb42aaeb80 1 -- start start 2024-05-09T01:29:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 0x55748312c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x55748312ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1aec30 con 0x5574831236a0 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1b0770 con 0x557483090160 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1af300 con 0x557482fa64a0 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb32635700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb32635700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831390a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49994/0 (socket says 172.21.15.19:49994) 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb32635700 1 -- 172.21.15.19:0/1689648237 learned_addr learned my addr 172.21.15.19:0/1689648237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb31633700 1 --2- 172.21.15.19:0/1689648237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x55748312ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.019+0000 7fcb31e34700 1 --2- 172.21.15.19:0/1689648237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 0x55748312c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.020+0000 7fcb32635700 1 -- 172.21.15.19:0/1689648237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 msgr2=0x55748312c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.020+0000 7fcb32635700 1 --2- 172.21.15.19:0/1689648237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 0x55748312c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.020+0000 7fcb32635700 1 -- 172.21.15.19:0/1689648237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 msgr2=0x55748312ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.020+0000 7fcb32635700 1 --2- 172.21.15.19:0/1689648237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x55748312ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.020+0000 7fcb32635700 1 -- 172.21.15.19:0/1689648237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748311fd00 con 0x5574831236a0 2024-05-09T01:29:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.020+0000 7fcb32635700 1 --2- 172.21.15.19:0/1689648237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831390a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcb240a68c0 tx=0x7fcb240a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade73f909a6210d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.020+0000 7fcb177fe700 1 -- 172.21.15.19:0/1689648237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24004e00 con 0x5574831236a0 2024-05-09T01:29:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.020+0000 7fcb177fe700 1 -- 172.21.15.19:0/1689648237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb240aa350 con 0x5574831236a0 2024-05-09T01:29:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.020+0000 7fcb177fe700 1 -- 172.21.15.19:0/1689648237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240b10b0 con 0x5574831236a0 2024-05-09T01:29:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/1689648237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 msgr2=0x5574831390a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/1689648237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831390a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcb240a68c0 tx=0x7fcb240a6040 comp rx=0 tx=0).stop 2024-05-09T01:29:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/1689648237 shutdown_connections 2024-05-09T01:29:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/1689648237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831390a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/1689648237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 0x55748312c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/1689648237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x55748312ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/1689648237 >> 172.21.15.19:0/1689648237 conn(0x5574830066a0 msgr2=0x557483127470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/1689648237 shutdown_connections 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/1689648237 wait complete. 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 -- start start 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.021+0000 7fcb42aaeb80 1 Processor -- start 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 0x557483144a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x557483144f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb42aaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1b00a0 con 0x5574831236a0 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1af9d0 con 0x557483090160 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb42aaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1acbd0 con 0x557482fa64a0 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb42aaeb80 10 client.0 _ll_register_callbacks cb 0x557482eb76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb32635700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb32635700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831a6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50010/0 (socket says 172.21.15.19:50010) 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb32635700 1 -- 172.21.15.19:0/3777530134 learned_addr learned my addr 172.21.15.19:0/3777530134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb31e34700 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 0x557483144a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb31633700 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x557483144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb32635700 1 -- 172.21.15.19:0/3777530134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 msgr2=0x557483144a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb32635700 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 0x557483144a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb32635700 1 -- 172.21.15.19:0/3777530134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 msgr2=0x557483144f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb32635700 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x557483144f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.022+0000 7fcb32635700 1 -- 172.21.15.19:0/3777530134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574831457d0 con 0x5574831236a0 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.023+0000 7fcb32635700 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831a6a30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009fd0 tx=0x7fcb2400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5d06fdd979b4c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.023+0000 7fcb157fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.023+0000 7fcb157fa700 1 -- 172.21.15.19:0/3777530134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240c3030 con 0x5574831236a0 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.023+0000 7fcb157fa700 1 -- 172.21.15.19:0/3777530134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcb240a6e30 con 0x5574831236a0 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.023+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/3777530134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb240aa350 con 0x5574831236a0 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.023+0000 7fcb157fa700 1 -- 172.21.15.19:0/3777530134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcb240aa840 con 0x5574831236a0 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.023+0000 7fcb42aaeb80 10 client.16158 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.023+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/3777530134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574831a7f80 con 0x5574831236a0 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.024+0000 7fcb157fa700 1 -- 172.21.15.19:0/3777530134 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb240aa350 con 0x5574831236a0 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.024+0000 7fcb157fa700 1 -- 172.21.15.19:0/3777530134 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcb24011070 con 0x5574831236a0 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.024+0000 7fcb42aaeb80 10 client.16158 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.024+0000 7fcb42aaeb80 10 client.16158 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.024+0000 7fcb42aaeb80 20 client.16158 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.024+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5574831a8570 0x5574831aa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.024+0000 7fcb42aaeb80 4 client.16158 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.024+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/3777530134 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55748304da70 con 0x5574831a8570 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.024+0000 7fcb31e34700 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5574831a8570 0x5574831aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb31e34700 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5574831a8570 0x5574831aa990 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e16ccf03d786aaa server_cookie=e8b2f791586e83bd in_seq=0 out_seq=0 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb157fa700 10 client.16158 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb157fa700 1 -- 172.21.15.19:0/3777530134 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55748304da70 con 0x5574831a8570 2024-05-09T01:29:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb157fa700 10 client.16158 handle_command_reply: tid=0 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 client.16158 shutdown 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/3777530134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 msgr2=0x5574831a6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831a6a30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009fd0 tx=0x7fcb2400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/3777530134 shutdown_connections 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x5574831a8570 0x5574831aa990 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574831236a0 0x5574831a6a30 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557482fa64a0 0x557483144a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb157fa700 0 client.16158 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 --2- 172.21.15.19:0/3777530134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557483090160 0x557483144f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/3777530134 >> 172.21.15.19:0/3777530134 conn(0x5574830066a0 msgr2=0x557483127470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/3777530134 shutdown_connections 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.025+0000 7fcb42aaeb80 1 -- 172.21.15.19:0/3777530134 wait complete. 2024-05-09T01:29:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb42aaeb80 20 client.16158 trim_cache size 0 max 16384 2024-05-09T01:29:03.028 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:29:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 -- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb180b2900 msgr2=0x7fcb180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb180b2900 0x7fcb180b4e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009c60 tx=0x7fcb20009260 comp rx=0 tx=0).stop 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 -- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 msgr2=0x7fcb2c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 0x7fcb2c19f680 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcb24007a00 tx=0x7fcb24007a30 comp rx=0 tx=0).stop 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 -- 172.21.15.19:0/3647028140 shutdown_connections 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcb180b2900 0x7fcb180b4e10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb2c108760 0x7fcb2c19f680 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb2c1034e0 0x7fcb2c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 --2- 172.21.15.19:0/3647028140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb2c107e10 0x7fcb2c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 -- 172.21.15.19:0/3647028140 >> 172.21.15.19:0/3647028140 conn(0x7fcb2c0fef30 msgr2=0x7fcb2c107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 -- 172.21.15.19:0/3647028140 shutdown_connections 2024-05-09T01:29:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:03.026+0000 7fcb34098700 1 -- 172.21.15.19:0/3647028140 wait complete. 2024-05-09T01:29:03.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:29:03.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:03 smithi196 ceph-mon[35041]: pgmap v484: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:29:04.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:03 smithi059 ceph-mon[34351]: pgmap v484: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:29:04.133 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:04.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:03 smithi019 ceph-mon[30158]: pgmap v484: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.877+0000 7fa491197700 1 -- 172.21.15.19:0/1992505078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 msgr2=0x7fa48c108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.877+0000 7fa491197700 1 --2- 172.21.15.19:0/1992505078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 0x7fa48c108230 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa4740099c0 tx=0x7fa474009cd0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.877+0000 7fa491197700 1 -- 172.21.15.19:0/1992505078 shutdown_connections 2024-05-09T01:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.877+0000 7fa491197700 1 --2- 172.21.15.19:0/1992505078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa48c108770 0x7fa48c1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.877+0000 7fa491197700 1 --2- 172.21.15.19:0/1992505078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c103540 0x7fa48c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.877+0000 7fa491197700 1 --2- 172.21.15.19:0/1992505078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 0x7fa48c108230 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.877+0000 7fa491197700 1 -- 172.21.15.19:0/1992505078 >> 172.21.15.19:0/1992505078 conn(0x7fa48c078e50 msgr2=0x7fa48c079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.877+0000 7fa491197700 1 -- 172.21.15.19:0/1992505078 shutdown_connections 2024-05-09T01:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.877+0000 7fa491197700 1 -- 172.21.15.19:0/1992505078 wait complete. 2024-05-09T01:29:05.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.878+0000 7fa491197700 1 Processor -- start 2024-05-09T01:29:05.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.878+0000 7fa491197700 1 -- start start 2024-05-09T01:29:05.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.878+0000 7fa491197700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c103540 0x7fa48c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.878+0000 7fa491197700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 0x7fa48c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa491197700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa48c108770 0x7fa48c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa491197700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c113380 con 0x7fa48c107e20 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa491197700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c113200 con 0x7fa48c108770 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa491197700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c113500 con 0x7fa48c103540 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 0x7fa48c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 0x7fa48c19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50024/0 (socket says 172.21.15.19:50024) 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48a59c700 1 -- 172.21.15.19:0/1844592469 learned_addr learned my addr 172.21.15.19:0/1844592469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48b59e700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa48c108770 0x7fa48c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c103540 0x7fa48c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48a59c700 1 -- 172.21.15.19:0/1844592469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c103540 msgr2=0x7fa48c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48a59c700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c103540 0x7fa48c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48a59c700 1 -- 172.21.15.19:0/1844592469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa48c108770 msgr2=0x7fa48c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48a59c700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa48c108770 0x7fa48c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48a59c700 1 -- 172.21.15.19:0/1844592469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c19fbe0 con 0x7fa48c107e20 2024-05-09T01:29:05.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa48a59c700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 0x7fa48c19b360 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098e0 tx=0x7fa4740057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:05.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.879+0000 7fa483fff700 1 -- 172.21.15.19:0/1844592469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474027070 con 0x7fa48c107e20 2024-05-09T01:29:05.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.880+0000 7fa491197700 1 -- 172.21.15.19:0/1844592469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c19fe70 con 0x7fa48c107e20 2024-05-09T01:29:05.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.880+0000 7fa483fff700 1 -- 172.21.15.19:0/1844592469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa474004020 con 0x7fa48c107e20 2024-05-09T01:29:05.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.880+0000 7fa483fff700 1 -- 172.21.15.19:0/1844592469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474022e30 con 0x7fa48c107e20 2024-05-09T01:29:05.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.880+0000 7fa491197700 1 -- 172.21.15.19:0/1844592469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c1a03a0 con 0x7fa48c107e20 2024-05-09T01:29:05.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.882+0000 7fa483fff700 1 -- 172.21.15.19:0/1844592469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa47402a070 con 0x7fa48c107e20 2024-05-09T01:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.883+0000 7fa483fff700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4780b2880 0x7fa4780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.883+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4780b2880 0x7fa4780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa483fff700 1 -- 172.21.15.19:0/1844592469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa4740e2f60 con 0x7fa48c107e20 2024-05-09T01:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa49fbadb80 1 Processor -- start 2024-05-09T01:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa49fbadb80 1 -- start start 2024-05-09T01:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4780b2880 0x7fa4780b4d90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e270 tx=0x7fa484009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 0x565413681d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 0x5654136822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x565413686810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56541359fff0 con 0x565413686400 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56541353e7f0 con 0x5654135004a0 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56541359bfb0 con 0x5654135ea160 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.884+0000 7fa48b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x565413686810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x565413686810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50030/0 (socket says 172.21.15.19:50030) 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48b59e700 1 -- 172.21.15.19:0/1906043536 learned_addr learned my addr 172.21.15.19:0/1906043536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48a59c700 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 0x5654136822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 0x565413681d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48b59e700 1 -- 172.21.15.19:0/1906043536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 msgr2=0x565413681d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48b59e700 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 0x565413681d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48b59e700 1 -- 172.21.15.19:0/1906043536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 msgr2=0x5654136822f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48b59e700 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 0x5654136822f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48b59e700 1 -- 172.21.15.19:0/1906043536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565413536370 con 0x565413686400 2024-05-09T01:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48a59c700 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 0x5654136822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa48b59e700 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x565413686810 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00ac30 tx=0x7fa47c00af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdea1e6e537c6e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa4817fa700 1 -- 172.21.15.19:0/1906043536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c00fe90 con 0x565413686400 2024-05-09T01:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa4817fa700 1 -- 172.21.15.19:0/1906043536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c004ce0 con 0x565413686400 2024-05-09T01:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.885+0000 7fa4817fa700 1 -- 172.21.15.19:0/1906043536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0186a0 con 0x565413686400 2024-05-09T01:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.886+0000 7fa49fbadb80 1 -- 172.21.15.19:0/1906043536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 msgr2=0x565413686810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.886+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x565413686810 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00ac30 tx=0x7fa47c00af40 comp rx=0 tx=0).stop 2024-05-09T01:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.886+0000 7fa49fbadb80 1 -- 172.21.15.19:0/1906043536 shutdown_connections 2024-05-09T01:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.886+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x565413686810 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.886+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 0x565413681d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.886+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/1906043536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 0x5654136822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.886+0000 7fa49fbadb80 1 -- 172.21.15.19:0/1906043536 >> 172.21.15.19:0/1906043536 conn(0x5654135606a0 msgr2=0x565413678400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.886+0000 7fa49fbadb80 1 -- 172.21.15.19:0/1906043536 shutdown_connections 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.886+0000 7fa49fbadb80 1 -- 172.21.15.19:0/1906043536 wait complete. 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.887+0000 7fa49fbadb80 1 -- start start 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.887+0000 7fa49fbadb80 1 Processor -- start 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.887+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 0x56541368b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.887+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 0x56541368bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.887+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541368c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.887+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56541341bba0 con 0x565413686400 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.887+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56541355a150 con 0x5654135004a0 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.887+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654136871b0 con 0x5654135ea160 2024-05-09T01:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.887+0000 7fa49fbadb80 10 client.0 _ll_register_callbacks cb 0x5654134116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541368c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 0x56541368bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541368c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50032/0 (socket says 172.21.15.19:50032) 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48b59e700 1 -- 172.21.15.19:0/432938935 learned_addr learned my addr 172.21.15.19:0/432938935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 0x56541368b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48b59e700 1 -- 172.21.15.19:0/432938935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 msgr2=0x56541368bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48b59e700 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 0x56541368bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48b59e700 1 -- 172.21.15.19:0/432938935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 msgr2=0x56541368b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48b59e700 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 0x56541368b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48b59e700 1 -- 172.21.15.19:0/432938935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541341b490 con 0x565413686400 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa48b59e700 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541368c010 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00a560 tx=0x7fa47c0050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f1e6bc370033a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa4727fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa4727fc700 1 -- 172.21.15.19:0/432938935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c026020 con 0x565413686400 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa4727fc700 1 -- 172.21.15.19:0/432938935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa47c004040 con 0x565413686400 2024-05-09T01:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa49fbadb80 1 -- 172.21.15.19:0/432938935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47c004ce0 con 0x565413686400 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa4727fc700 1 -- 172.21.15.19:0/432938935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa47c031050 con 0x565413686400 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa49fbadb80 10 client.16170 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.888+0000 7fa49fbadb80 1 -- 172.21.15.19:0/432938935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565413698ea0 con 0x565413686400 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.889+0000 7fa4727fc700 1 -- 172.21.15.19:0/432938935 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c004ce0 con 0x565413686400 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.889+0000 7fa4727fc700 1 -- 172.21.15.19:0/432938935 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa47c003ca0 con 0x565413686400 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.889+0000 7fa49fbadb80 10 client.16170 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.889+0000 7fa49fbadb80 10 client.16170 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.889+0000 7fa49fbadb80 20 client.16170 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.889+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56541369b370 0x56541369d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.889+0000 7fa49fbadb80 4 client.16170 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.889+0000 7fa49fbadb80 1 -- 172.21.15.19:0/432938935 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654135ad9d0 con 0x56541369b370 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.889+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56541369b370 0x56541369d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.890+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56541369b370 0x56541369d790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=110e124a216d7a89 server_cookie=d53865f67e7e07f4 in_seq=0 out_seq=0 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.890+0000 7fa4727fc700 10 client.16170 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa4727fc700 1 -- 172.21.15.19:0/432938935 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654135ad9d0 con 0x56541369b370 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa4727fc700 10 client.16170 handle_command_reply: tid=0 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 client.16170 shutdown 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 -- 172.21.15.19:0/432938935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 msgr2=0x56541368c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541368c010 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00a560 tx=0x7fa47c0050c0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 -- 172.21.15.19:0/432938935 shutdown_connections 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56541369b370 0x56541369d790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541368c010 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa4727fc700 0 client.16170 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135004a0 0x56541368b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/432938935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135ea160 0x56541368bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 -- 172.21.15.19:0/432938935 >> 172.21.15.19:0/432938935 conn(0x5654135606a0 msgr2=0x5654136781c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 -- 172.21.15.19:0/432938935 shutdown_connections 2024-05-09T01:29:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 1 -- 172.21.15.19:0/432938935 wait complete. 2024-05-09T01:29:05.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.891+0000 7fa49fbadb80 20 client.16170 trim_cache size 0 max 16384 2024-05-09T01:29:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.906+0000 7fa49fbadb80 1 Processor -- start 2024-05-09T01:29:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.906+0000 7fa49fbadb80 1 -- start start 2024-05-09T01:29:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654135004a0 0x565413697790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 0x565413685c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565413686400 0x565413684230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1ac940 con 0x5654135004a0 2024-05-09T01:29:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1ae480 con 0x5654135ea160 2024-05-09T01:29:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1ad010 con 0x565413686400 2024-05-09T01:29:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654135004a0 0x565413697790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654135004a0 0x565413697790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50046/0 (socket says 172.21.15.19:50046) 2024-05-09T01:29:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48ad9d700 1 -- 172.21.15.19:0/2013947993 learned_addr learned my addr 172.21.15.19:0/2013947993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48a59c700 1 --2- 172.21.15.19:0/2013947993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 0x565413685c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48b59e700 1 --2- 172.21.15.19:0/2013947993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565413686400 0x565413684230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48ad9d700 1 -- 172.21.15.19:0/2013947993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565413686400 msgr2=0x565413684230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/2013947993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565413686400 0x565413684230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48ad9d700 1 -- 172.21.15.19:0/2013947993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 msgr2=0x565413685c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/2013947993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 0x565413685c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48ad9d700 1 -- 172.21.15.19:0/2013947993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565413689b00 con 0x5654135004a0 2024-05-09T01:29:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/2013947993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654135004a0 0x565413697790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e9c0 tx=0x7fa484014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260cb902825a6c8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.907+0000 7fa480ff9700 1 -- 172.21.15.19:0/2013947993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484028150 con 0x5654135004a0 2024-05-09T01:29:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.908+0000 7fa480ff9700 1 -- 172.21.15.19:0/2013947993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4840287a0 con 0x5654135004a0 2024-05-09T01:29:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.908+0000 7fa480ff9700 1 -- 172.21.15.19:0/2013947993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484024a40 con 0x5654135004a0 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.908+0000 7fa49fbadb80 1 -- 172.21.15.19:0/2013947993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654135004a0 msgr2=0x565413697790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.908+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/2013947993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654135004a0 0x565413697790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e9c0 tx=0x7fa484014000 comp rx=0 tx=0).stop 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.908+0000 7fa49fbadb80 1 -- 172.21.15.19:0/2013947993 shutdown_connections 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.908+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/2013947993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565413686400 0x565413684230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.908+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/2013947993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654135004a0 0x565413697790 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.908+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/2013947993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 0x565413685c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 -- 172.21.15.19:0/2013947993 >> 172.21.15.19:0/2013947993 conn(0x5654135606a0 msgr2=0x565413685570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 -- 172.21.15.19:0/2013947993 shutdown_connections 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 -- 172.21.15.19:0/2013947993 wait complete. 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 -- start start 2024-05-09T01:29:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 Processor -- start 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135004a0 0x5654136fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 0x5654136fd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541369db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1addb0 con 0x565413686400 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1ad6e0 con 0x5654135ea160 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1aa8e0 con 0x5654135004a0 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.909+0000 7fa49fbadb80 10 client.0 _ll_register_callbacks cb 0x5654134116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 0x5654136fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541369db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 0x5654136fd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56184/0 (socket says 172.21.15.19:56184) 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541369db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50048/0 (socket says 172.21.15.19:50048) 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48a59c700 1 -- 172.21.15.19:0/3486643460 learned_addr learned my addr 172.21.15.19:0/3486643460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135004a0 0x5654136fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48b59e700 1 -- 172.21.15.19:0/3486643460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135004a0 msgr2=0x5654136fcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48b59e700 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135004a0 0x5654136fcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48b59e700 1 -- 172.21.15.19:0/3486643460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 msgr2=0x5654136fd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48b59e700 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 0x5654136fd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48b59e700 1 -- 172.21.15.19:0/3486643460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541369e290 con 0x565413686400 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa48b59e700 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541369db90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00ab50 tx=0x7fa47c005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e383b22c298419 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa4727fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa4727fc700 1 -- 172.21.15.19:0/3486643460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c039020 con 0x565413686400 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa4727fc700 1 -- 172.21.15.19:0/3486643460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa47c004960 con 0x565413686400 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa49fbadb80 1 -- 172.21.15.19:0/3486643460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4840287a0 con 0x565413686400 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa4727fc700 1 -- 172.21.15.19:0/3486643460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa47c044050 con 0x565413686400 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa49fbadb80 10 client.16176 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.910+0000 7fa49fbadb80 1 -- 172.21.15.19:0/3486643460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654137052b0 con 0x565413686400 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.911+0000 7fa4727fc700 1 -- 172.21.15.19:0/3486643460 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4840287a0 con 0x565413686400 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.911+0000 7fa4727fc700 1 -- 172.21.15.19:0/3486643460 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa47c018a10 con 0x565413686400 2024-05-09T01:29:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.911+0000 7fa49fbadb80 10 client.16176 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.911+0000 7fa49fbadb80 10 client.16176 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.911+0000 7fa49fbadb80 20 client.16176 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.911+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56541369b370 0x565413707900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.911+0000 7fa49fbadb80 4 client.16176 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.911+0000 7fa49fbadb80 1 -- 172.21.15.19:0/3486643460 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654135a7a70 con 0x56541369b370 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.911+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56541369b370 0x565413707900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa48ad9d700 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56541369b370 0x565413707900 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4fe757d3fad237d7 server_cookie=79382f38ca699610 in_seq=0 out_seq=0 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa4727fc700 10 client.16176 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa4727fc700 1 -- 172.21.15.19:0/3486643460 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654135a7a70 con 0x56541369b370 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa4727fc700 10 client.16176 handle_command_reply: tid=0 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 client.16176 shutdown 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 -- 172.21.15.19:0/3486643460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 msgr2=0x56541369db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541369db90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00ab50 tx=0x7fa47c005510 comp rx=0 tx=0).stop 2024-05-09T01:29:05.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 -- 172.21.15.19:0/3486643460 shutdown_connections 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56541369b370 0x565413707900 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565413686400 0x56541369db90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654135004a0 0x5654136fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 --2- 172.21.15.19:0/3486643460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654135ea160 0x5654136fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 -- 172.21.15.19:0/3486643460 >> 172.21.15.19:0/3486643460 conn(0x5654135606a0 msgr2=0x56541367c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa4727fc700 0 client.16176 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 -- 172.21.15.19:0/3486643460 shutdown_connections 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 1 -- 172.21.15.19:0/3486643460 wait complete. 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.912+0000 7fa49fbadb80 20 client.16176 trim_cache size 0 max 16384 2024-05-09T01:29:05.914 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:29:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 -- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4780b2880 msgr2=0x7fa4780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4780b2880 0x7fa4780b4d90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e270 tx=0x7fa484009040 comp rx=0 tx=0).stop 2024-05-09T01:29:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 -- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 msgr2=0x7fa48c19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 0x7fa48c19b360 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098e0 tx=0x7fa4740057c0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 -- 172.21.15.19:0/1844592469 shutdown_connections 2024-05-09T01:29:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa48c108770 0x7fa48c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c103540 0x7fa48c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa4780b2880 0x7fa4780b4d90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 --2- 172.21.15.19:0/1844592469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa48c107e20 0x7fa48c19b360 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 -- 172.21.15.19:0/1844592469 >> 172.21.15.19:0/1844592469 conn(0x7fa48c078e50 msgr2=0x7fa48c075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 -- 172.21.15.19:0/1844592469 shutdown_connections 2024-05-09T01:29:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:05.913+0000 7fa491197700 1 -- 172.21.15.19:0/1844592469 wait complete. 2024-05-09T01:29:06.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:06 smithi059 ceph-mon[34351]: pgmap v485: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:29:06.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:06 smithi019 ceph-mon[30158]: pgmap v485: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:29:06.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:29:06.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:06 smithi196 ceph-mon[35041]: pgmap v485: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:29:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:08.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:08 smithi019 ceph-mon[30158]: pgmap v486: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:29:08.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:08 smithi059 ceph-mon[34351]: pgmap v486: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:29:08.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.693+0000 7f264d5e4700 1 -- 172.21.15.19:0/2391438889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648108740 msgr2=0x7f26481100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.693+0000 7f264d5e4700 1 --2- 172.21.15.19:0/2391438889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648108740 0x7f26481100a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f264000b0d0 tx=0x7f264000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.693+0000 7f264d5e4700 1 -- 172.21.15.19:0/2391438889 shutdown_connections 2024-05-09T01:29:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.693+0000 7f264d5e4700 1 --2- 172.21.15.19:0/2391438889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648108740 0x7f26481100a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.693+0000 7f264d5e4700 1 --2- 172.21.15.19:0/2391438889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26481034c0 0x7f26481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.693+0000 7f264d5e4700 1 --2- 172.21.15.19:0/2391438889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2648107df0 0x7f2648108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.693+0000 7f264d5e4700 1 -- 172.21.15.19:0/2391438889 >> 172.21.15.19:0/2391438889 conn(0x7f26480fedf0 msgr2=0x7f2648101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.693+0000 7f264d5e4700 1 -- 172.21.15.19:0/2391438889 shutdown_connections 2024-05-09T01:29:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.693+0000 7f264d5e4700 1 -- 172.21.15.19:0/2391438889 wait complete. 2024-05-09T01:29:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f264d5e4700 1 Processor -- start 2024-05-09T01:29:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f264d5e4700 1 -- start start 2024-05-09T01:29:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f264d5e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26481034c0 0x7f26480ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f264d5e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648107df0 0x7f26480ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f264d5e4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2648108740 0x7f26480ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f264d5e4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2648113370 con 0x7f2648107df0 2024-05-09T01:29:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f264d5e4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26481131f0 con 0x7f26481034c0 2024-05-09T01:29:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f264d5e4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26481134f0 con 0x7f2648108740 2024-05-09T01:29:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f263ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648107df0 0x7f26480ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f263ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648107df0 0x7f26480ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50072/0 (socket says 172.21.15.19:50072) 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.694+0000 7f263ffff700 1 -- 172.21.15.19:0/4125141624 learned_addr learned my addr 172.21.15.19:0/4125141624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f2646ffd700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26481034c0 0x7f26480ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f26477fe700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2648108740 0x7f26480ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f263ffff700 1 -- 172.21.15.19:0/4125141624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2648108740 msgr2=0x7f26480ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f263ffff700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2648108740 0x7f26480ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f263ffff700 1 -- 172.21.15.19:0/4125141624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26481034c0 msgr2=0x7f26480ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f263ffff700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26481034c0 0x7f26480ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f263ffff700 1 -- 172.21.15.19:0/4125141624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26481a9110 con 0x7f2648107df0 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f263ffff700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648107df0 0x7f26480ff810 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f263000e970 tx=0x7f263000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f2644ff9700 1 -- 172.21.15.19:0/4125141624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263000cbd0 con 0x7f2648107df0 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f2644ff9700 1 -- 172.21.15.19:0/4125141624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f263000cd70 con 0x7f2648107df0 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f264d5e4700 1 -- 172.21.15.19:0/4125141624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26481a9400 con 0x7f2648107df0 2024-05-09T01:29:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f2644ff9700 1 -- 172.21.15.19:0/4125141624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630010430 con 0x7f2648107df0 2024-05-09T01:29:08.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.695+0000 7f264d5e4700 1 -- 172.21.15.19:0/4125141624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26481a9940 con 0x7f2648107df0 2024-05-09T01:29:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:08 smithi196 ceph-mon[35041]: pgmap v486: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:29:08.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.698+0000 7f2644ff9700 1 -- 172.21.15.19:0/4125141624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f26300105d0 con 0x7f2648107df0 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.699+0000 7f2644ff9700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26340b2880 0x7f26340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.699+0000 7f265bffab80 1 Processor -- start 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.699+0000 7f2646ffd700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26340b2880 0x7f26340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.699+0000 7f2644ff9700 1 -- 172.21.15.19:0/4125141624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f26300e2510 con 0x7f2648107df0 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.699+0000 7f265bffab80 1 -- start start 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26340b2880 0x7f26340b4d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2638009bf0 tx=0x7f2638009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f265bffab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a59240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f265bffab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a398d64a0 0x555a39a59780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f265bffab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 0x555a39a53810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f265bffab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a399760a0 con 0x555a39950d00 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f265bffab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555a3991b9a0 con 0x555a398d64a0 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f265bffab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555a39972060 con 0x555a39a53400 2024-05-09T01:29:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a59240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a59240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50084/0 (socket says 172.21.15.19:50084) 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 -- 172.21.15.19:0/3114574588 learned_addr learned my addr 172.21.15.19:0/3114574588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f263ffff700 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a398d64a0 0x555a39a59780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f26477fe700 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 0x555a39a53810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 -- 172.21.15.19:0/3114574588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 msgr2=0x555a39a53810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 0x555a39a53810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 -- 172.21.15.19:0/3114574588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a398d64a0 msgr2=0x555a39a59780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a398d64a0 0x555a39a59780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 -- 172.21.15.19:0/3114574588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3990c500 con 0x555a39950d00 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f263ffff700 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a398d64a0 0x555a39a59780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.700+0000 7f2646ffd700 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a59240 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2638011e40 tx=0x7f2638014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d65ec870781d09b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.701+0000 7f263d7fa700 1 -- 172.21.15.19:0/3114574588 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638012dc0 con 0x555a39950d00 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.701+0000 7f263d7fa700 1 -- 172.21.15.19:0/3114574588 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2638011e70 con 0x555a39950d00 2024-05-09T01:29:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.701+0000 7f263d7fa700 1 -- 172.21.15.19:0/3114574588 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263801f420 con 0x555a39950d00 2024-05-09T01:29:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 -- 172.21.15.19:0/3114574588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 msgr2=0x555a39a59240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a59240 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2638011e40 tx=0x7f2638014000 comp rx=0 tx=0).stop 2024-05-09T01:29:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 -- 172.21.15.19:0/3114574588 shutdown_connections 2024-05-09T01:29:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 0x555a39a53810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a59240 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 --2- 172.21.15.19:0/3114574588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a398d64a0 0x555a39a59780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 -- 172.21.15.19:0/3114574588 >> 172.21.15.19:0/3114574588 conn(0x555a398d9f70 msgr2=0x555a39a4e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 -- 172.21.15.19:0/3114574588 shutdown_connections 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 -- 172.21.15.19:0/3114574588 wait complete. 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 -- start start 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.702+0000 7f265bffab80 1 Processor -- start 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f265bffab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a398d64a0 0x555a39a5d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f265bffab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a5d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f265bffab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 0x555a39a5dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f265bffab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a39a54160 con 0x555a39950d00 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f265bffab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555a39910b20 con 0x555a39a53400 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f265bffab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555a39a60020 con 0x555a398d64a0 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f263ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a5d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f265bffab80 10 client.0 _ll_register_callbacks cb 0x555a397e76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f263ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a5d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50100/0 (socket says 172.21.15.19:50100) 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f263ffff700 1 -- 172.21.15.19:0/45870891 learned_addr learned my addr 172.21.15.19:0/45870891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f2646ffd700 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a398d64a0 0x555a39a5d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f26477fe700 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 0x555a39a5dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f263ffff700 1 -- 172.21.15.19:0/45870891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a398d64a0 msgr2=0x555a39a5d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f263ffff700 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a398d64a0 0x555a39a5d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f263ffff700 1 -- 172.21.15.19:0/45870891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 msgr2=0x555a39a5dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f263ffff700 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 0x555a39a5dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f263ffff700 1 -- 172.21.15.19:0/45870891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a397f1490 con 0x555a39950d00 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f263ffff700 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a5d580 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f26300dff90 tx=0x7f2630005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e885b2b5a11c499c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f262e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f262e7fc700 1 -- 172.21.15.19:0/45870891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300e2300 con 0x555a39950d00 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.703+0000 7f262e7fc700 1 -- 172.21.15.19:0/45870891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f26300a60a0 con 0x555a39950d00 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f265bffab80 1 -- 172.21.15.19:0/45870891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2638011e70 con 0x555a39950d00 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f262e7fc700 1 -- 172.21.15.19:0/45870891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26300c4070 con 0x555a39950d00 2024-05-09T01:29:08.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f265bffab80 10 client.16188 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f265bffab80 1 -- 172.21.15.19:0/45870891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a39a6aa70 con 0x555a39950d00 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f262e7fc700 1 -- 172.21.15.19:0/45870891 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2638011e70 con 0x555a39950d00 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f262e7fc700 1 -- 172.21.15.19:0/45870891 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f26300a9d10 con 0x555a39950d00 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f265bffab80 10 client.16188 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f265bffab80 10 client.16188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f265bffab80 20 client.16188 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f265bffab80 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555a39a6d0b0 0x555a39a6f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f265bffab80 4 client.16188 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f265bffab80 1 -- 172.21.15.19:0/45870891 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a39983a80 con 0x555a39a6d0b0 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.704+0000 7f2646ffd700 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555a39a6d0b0 0x555a39a6f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.705+0000 7f2646ffd700 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555a39a6d0b0 0x555a39a6f4d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c82b6771927e591f server_cookie=28d24ba9698248ab in_seq=0 out_seq=0 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.705+0000 7f262e7fc700 10 client.16188 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f262e7fc700 1 -- 172.21.15.19:0/45870891 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a39983a80 con 0x555a39a6d0b0 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f262e7fc700 10 client.16188 handle_command_reply: tid=0 2024-05-09T01:29:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 client.16188 shutdown 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 -- 172.21.15.19:0/45870891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 msgr2=0x555a39a5d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a5d580 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f26300dff90 tx=0x7f2630005720 comp rx=0 tx=0).stop 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 -- 172.21.15.19:0/45870891 shutdown_connections 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555a39a6d0b0 0x555a39a6f4d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 0x555a39a5dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a398d64a0 0x555a39a5d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 --2- 172.21.15.19:0/45870891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a39950d00 0x555a39a5d580 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 -- 172.21.15.19:0/45870891 >> 172.21.15.19:0/45870891 conn(0x555a398d9f70 msgr2=0x555a39a47ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f262e7fc700 0 client.16188 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 -- 172.21.15.19:0/45870891 shutdown_connections 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 1 -- 172.21.15.19:0/45870891 wait complete. 2024-05-09T01:29:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.706+0000 7f265bffab80 20 client.16188 trim_cache size 0 max 16384 2024-05-09T01:29:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.722+0000 7f265bffab80 1 Processor -- start 2024-05-09T01:29:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.722+0000 7f265bffab80 1 -- start start 2024-05-09T01:29:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.722+0000 7f265bffab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.722+0000 7f265bffab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39950d00 0x555a39a5acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.722+0000 7f265bffab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 0x555a39a5d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.722+0000 7f265bffab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b0fa0 con 0x555a398d64a0 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.722+0000 7f265bffab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b2ae0 con 0x555a39a53400 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.722+0000 7f265bffab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b1670 con 0x555a39950d00 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.722+0000 7f2646ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f2646ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50114/0 (socket says 172.21.15.19:50114) 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f2646ffd700 1 -- 172.21.15.19:0/3854633813 learned_addr learned my addr 172.21.15.19:0/3854633813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f26477fe700 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 0x555a39a5d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f263ffff700 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39950d00 0x555a39a5acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f2646ffd700 1 -- 172.21.15.19:0/3854633813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39950d00 msgr2=0x555a39a5acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f2646ffd700 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39950d00 0x555a39a5acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f2646ffd700 1 -- 172.21.15.19:0/3854633813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 msgr2=0x555a39a5d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f2646ffd700 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 0x555a39a5d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f2646ffd700 1 -- 172.21.15.19:0/3854633813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a39a5c360 con 0x555a398d64a0 2024-05-09T01:29:08.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f263ffff700 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39950d00 0x555a39a5acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:29:08.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f2646ffd700 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a588b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2638012a10 tx=0x7f2638011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69ef45c8100b057 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:08.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f263cff9700 1 -- 172.21.15.19:0/3854633813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263803a020 con 0x555a398d64a0 2024-05-09T01:29:08.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f263cff9700 1 -- 172.21.15.19:0/3854633813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2638011e70 con 0x555a398d64a0 2024-05-09T01:29:08.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.723+0000 7f263cff9700 1 -- 172.21.15.19:0/3854633813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638035e60 con 0x555a398d64a0 2024-05-09T01:29:08.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.724+0000 7f265bffab80 1 -- 172.21.15.19:0/3854633813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 msgr2=0x555a39a588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:08.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.724+0000 7f265bffab80 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a588b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2638012a10 tx=0x7f2638011a00 comp rx=0 tx=0).stop 2024-05-09T01:29:08.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.724+0000 7f265bffab80 1 -- 172.21.15.19:0/3854633813 shutdown_connections 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.724+0000 7f265bffab80 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39a53400 0x555a39a5d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.724+0000 7f265bffab80 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a588b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.724+0000 7f265bffab80 1 --2- 172.21.15.19:0/3854633813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39950d00 0x555a39a5acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.724+0000 7f265bffab80 1 -- 172.21.15.19:0/3854633813 >> 172.21.15.19:0/3854633813 conn(0x555a398d9f70 msgr2=0x555a39a581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.724+0000 7f265bffab80 1 -- 172.21.15.19:0/3854633813 shutdown_connections 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.724+0000 7f265bffab80 1 -- 172.21.15.19:0/3854633813 wait complete. 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f265bffab80 1 -- start start 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f265bffab80 1 Processor -- start 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f265bffab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a75020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f265bffab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39950d00 0x555a39a651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f265bffab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 0x555a39a656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f265bffab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b2410 con 0x555a398d64a0 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f265bffab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b1d40 con 0x555a39950d00 2024-05-09T01:29:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f265bffab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26481aef40 con 0x555a39a53400 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f265bffab80 10 client.0 _ll_register_callbacks cb 0x555a397e76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f2646ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a75020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f2646ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a75020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50118/0 (socket says 172.21.15.19:50118) 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f2646ffd700 1 -- 172.21.15.19:0/4088971899 learned_addr learned my addr 172.21.15.19:0/4088971899 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f26477fe700 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 0x555a39a656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.725+0000 7f263ffff700 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39950d00 0x555a39a651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f2646ffd700 1 -- 172.21.15.19:0/4088971899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 msgr2=0x555a39a656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f2646ffd700 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 0x555a39a656f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f2646ffd700 1 -- 172.21.15.19:0/4088971899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39950d00 msgr2=0x555a39a651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f2646ffd700 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39950d00 0x555a39a651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f2646ffd700 1 -- 172.21.15.19:0/4088971899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a39a73c10 con 0x555a398d64a0 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f2646ffd700 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a75020 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f26380097d0 tx=0x7f26380354d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebd4dd0c18bd931 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f262e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f262e7fc700 1 -- 172.21.15.19:0/4088971899 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263803a050 con 0x555a398d64a0 2024-05-09T01:29:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f262e7fc700 1 -- 172.21.15.19:0/4088971899 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2638035b90 con 0x555a398d64a0 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f265bffab80 1 -- 172.21.15.19:0/4088971899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2638011e70 con 0x555a398d64a0 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f262e7fc700 1 -- 172.21.15.19:0/4088971899 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f263804e050 con 0x555a398d64a0 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f265bffab80 10 client.16194 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.726+0000 7f265bffab80 1 -- 172.21.15.19:0/4088971899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a39ad6890 con 0x555a398d64a0 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.727+0000 7f262e7fc700 1 -- 172.21.15.19:0/4088971899 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2638011e70 con 0x555a398d64a0 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.727+0000 7f262e7fc700 1 -- 172.21.15.19:0/4088971899 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2630004d70 con 0x555a398d64a0 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.727+0000 7f265bffab80 10 client.16194 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.727+0000 7f265bffab80 10 client.16194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.727+0000 7f265bffab80 20 client.16194 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.727+0000 7f265bffab80 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555a39ad6d80 0x555a39ad91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.727+0000 7f265bffab80 4 client.16194 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.727+0000 7f265bffab80 1 -- 172.21.15.19:0/4088971899 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a3997db20 con 0x555a39ad6d80 2024-05-09T01:29:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.727+0000 7f263ffff700 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555a39ad6d80 0x555a39ad91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f262e7fc700 10 client.16194 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f263ffff700 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555a39ad6d80 0x555a39ad91a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19f292e3a1497e2d server_cookie=ad27e0dd759e6cd in_seq=0 out_seq=0 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f262e7fc700 1 -- 172.21.15.19:0/4088971899 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a3997db20 con 0x555a39ad6d80 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f262e7fc700 10 client.16194 handle_command_reply: tid=0 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 client.16194 shutdown 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 -- 172.21.15.19:0/4088971899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 msgr2=0x555a39a75020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a75020 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f26380097d0 tx=0x7f26380354d0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 -- 172.21.15.19:0/4088971899 shutdown_connections 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a39a53400 0x555a39a656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a398d64a0 0x555a39a75020 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555a39ad6d80 0x555a39ad91a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 --2- 172.21.15.19:0/4088971899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a39950d00 0x555a39a651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f262e7fc700 0 client.16194 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 -- 172.21.15.19:0/4088971899 >> 172.21.15.19:0/4088971899 conn(0x555a398d9f70 msgr2=0x555a39a47ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 -- 172.21.15.19:0/4088971899 shutdown_connections 2024-05-09T01:29:08.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 1 -- 172.21.15.19:0/4088971899 wait complete. 2024-05-09T01:29:08.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.728+0000 7f265bffab80 20 client.16194 trim_cache size 0 max 16384 2024-05-09T01:29:08.729 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:29:08.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 -- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26340b2880 msgr2=0x7f26340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:08.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26340b2880 0x7f26340b4d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2638009bf0 tx=0x7f2638009260 comp rx=0 tx=0).stop 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 -- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648107df0 msgr2=0x7f26480ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648107df0 0x7f26480ff810 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f263000e970 tx=0x7f263000ed30 comp rx=0 tx=0).stop 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 -- 172.21.15.19:0/4125141624 shutdown_connections 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2648108740 0x7f26480ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26481034c0 0x7f26480ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f26340b2880 0x7f26340b4d90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 --2- 172.21.15.19:0/4125141624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2648107df0 0x7f26480ff810 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 -- 172.21.15.19:0/4125141624 >> 172.21.15.19:0/4125141624 conn(0x7f26480fedf0 msgr2=0x7f2648075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 -- 172.21.15.19:0/4125141624 shutdown_connections 2024-05-09T01:29:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:08.729+0000 7f264d5e4700 1 -- 172.21.15.19:0/4125141624 wait complete. 2024-05-09T01:29:09.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:29:09.819 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:10.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:10 smithi059 ceph-mon[34351]: pgmap v487: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:10.629 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:10 smithi019 ceph-mon[30158]: pgmap v487: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:10.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:10 smithi196 ceph-mon[35041]: pgmap v487: 97 pgs: 97 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.618+0000 7f0676c90700 1 -- 172.21.15.19:0/1762854782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 msgr2=0x7f0670108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.618+0000 7f0676c90700 1 --2- 172.21.15.19:0/1762854782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 0x7f0670108200 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f065c0099c0 tx=0x7f065c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.618+0000 7f0676c90700 1 -- 172.21.15.19:0/1762854782 shutdown_connections 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.618+0000 7f0676c90700 1 --2- 172.21.15.19:0/1762854782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0670108740 0x7f06701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.618+0000 7f0676c90700 1 --2- 172.21.15.19:0/1762854782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06701034c0 0x7f06701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.618+0000 7f0676c90700 1 --2- 172.21.15.19:0/1762854782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 0x7f0670108200 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.618+0000 7f0676c90700 1 -- 172.21.15.19:0/1762854782 >> 172.21.15.19:0/1762854782 conn(0x7f06700fedf0 msgr2=0x7f0670101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.618+0000 7f0676c90700 1 -- 172.21.15.19:0/1762854782 shutdown_connections 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.618+0000 7f0676c90700 1 -- 172.21.15.19:0/1762854782 wait complete. 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.619+0000 7f0676c90700 1 Processor -- start 2024-05-09T01:29:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.619+0000 7f0676c90700 1 -- start start 2024-05-09T01:29:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.619+0000 7f0676c90700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06701034c0 0x7f067019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.619+0000 7f0676c90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 0x7f067019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.619+0000 7f0676c90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0670108740 0x7f06701a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.619+0000 7f0676c90700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06701133a0 con 0x7f0670107df0 2024-05-09T01:29:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.619+0000 7f0676c90700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0670113220 con 0x7f0670108740 2024-05-09T01:29:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.619+0000 7f0676c90700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0670113520 con 0x7f06701034c0 2024-05-09T01:29:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 0x7f067019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 0x7f067019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47862/0 (socket says 172.21.15.19:47862) 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066ffff700 1 -- 172.21.15.19:0/5439857 learned_addr learned my addr 172.21.15.19:0/5439857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f0674a2c700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06701034c0 0x7f067019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f067522d700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0670108740 0x7f06701a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066ffff700 1 -- 172.21.15.19:0/5439857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06701034c0 msgr2=0x7f067019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066ffff700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06701034c0 0x7f067019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066ffff700 1 -- 172.21.15.19:0/5439857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0670108740 msgr2=0x7f06701a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066ffff700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0670108740 0x7f06701a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066ffff700 1 -- 172.21.15.19:0/5439857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06701a1e30 con 0x7f0670107df0 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066ffff700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 0x7f067019d5b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f065c009f50 tx=0x7f065c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066dffb700 1 -- 172.21.15.19:0/5439857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c026070 con 0x7f0670107df0 2024-05-09T01:29:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066dffb700 1 -- 172.21.15.19:0/5439857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f065c003f20 con 0x7f0670107df0 2024-05-09T01:29:11.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f0676c90700 1 -- 172.21.15.19:0/5439857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06701a20c0 con 0x7f0670107df0 2024-05-09T01:29:11.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.620+0000 7f066dffb700 1 -- 172.21.15.19:0/5439857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c00fd30 con 0x7f0670107df0 2024-05-09T01:29:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.621+0000 7f0676c90700 1 -- 172.21.15.19:0/5439857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06701a25f0 con 0x7f0670107df0 2024-05-09T01:29:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.623+0000 7f066dffb700 1 -- 172.21.15.19:0/5439857 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f065c029070 con 0x7f0670107df0 2024-05-09T01:29:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.624+0000 7f066dffb700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06600b2900 0x7f06600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.624+0000 7f0674a2c700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06600b2900 0x7f06600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.624+0000 7f066dffb700 1 -- 172.21.15.19:0/5439857 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f065c0e2b30 con 0x7f0670107df0 2024-05-09T01:29:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.624+0000 7f06856a6b80 1 Processor -- start 2024-05-09T01:29:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f06856a6b80 1 -- start start 2024-05-09T01:29:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fd3dd00 0x558c0fe46240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f0674a2c700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06600b2900 0x7f06600b4e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f066400ac50 tx=0x7f066400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 0x558c0fe46780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fe40400 0x558c0fe40810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f06856a6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c0fd630a0 con 0x558c0fd3dd00 2024-05-09T01:29:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f06856a6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558c0fd089a0 con 0x558c0fcc34a0 2024-05-09T01:29:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f06856a6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c0fd5f060 con 0x558c0fe40400 2024-05-09T01:29:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f0674a2c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fd3dd00 0x558c0fe46240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f0674a2c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fd3dd00 0x558c0fe46240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47868/0 (socket says 172.21.15.19:47868) 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f0674a2c700 1 -- 172.21.15.19:0/1622702882 learned_addr learned my addr 172.21.15.19:0/1622702882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f067522d700 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fe40400 0x558c0fe40810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f066ffff700 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 0x558c0fe46780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f0674a2c700 1 -- 172.21.15.19:0/1622702882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fe40400 msgr2=0x558c0fe40810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.625+0000 7f0674a2c700 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fe40400 0x558c0fe40810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.626+0000 7f0674a2c700 1 -- 172.21.15.19:0/1622702882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 msgr2=0x558c0fe46780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.626+0000 7f0674a2c700 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 0x558c0fe46780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.626+0000 7f0674a2c700 1 -- 172.21.15.19:0/1622702882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0fcf9500 con 0x558c0fd3dd00 2024-05-09T01:29:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.626+0000 7f067522d700 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fe40400 0x558c0fe40810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.626+0000 7f0674a2c700 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fd3dd00 0x558c0fe46240 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f066400ff90 tx=0x7f0664012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fa18dcdd07a68d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.626+0000 7f065affd700 1 -- 172.21.15.19:0/1622702882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664016070 con 0x558c0fd3dd00 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.626+0000 7f065affd700 1 -- 172.21.15.19:0/1622702882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0664014070 con 0x558c0fd3dd00 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.626+0000 7f065affd700 1 -- 172.21.15.19:0/1622702882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664010e00 con 0x558c0fd3dd00 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 -- 172.21.15.19:0/1622702882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fd3dd00 msgr2=0x558c0fe46240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fd3dd00 0x558c0fe46240 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f066400ff90 tx=0x7f0664012000 comp rx=0 tx=0).stop 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 -- 172.21.15.19:0/1622702882 shutdown_connections 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fe40400 0x558c0fe40810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fd3dd00 0x558c0fe46240 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 --2- 172.21.15.19:0/1622702882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 0x558c0fe46780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 -- 172.21.15.19:0/1622702882 >> 172.21.15.19:0/1622702882 conn(0x558c0fcc6f70 msgr2=0x558c0fe3b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 -- 172.21.15.19:0/1622702882 shutdown_connections 2024-05-09T01:29:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 -- 172.21.15.19:0/1622702882 wait complete. 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.627+0000 7f06856a6b80 1 -- start start 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f06856a6b80 1 Processor -- start 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 0x558c0fe4a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe4a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f06856a6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c0fe41160 con 0x558c0fe40400 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f06856a6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558c0fcfdb20 con 0x558c0fd3dd00 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f06856a6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c0fe4d020 con 0x558c0fcc34a0 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f06856a6b80 10 client.0 _ll_register_callbacks cb 0x558c0fbd46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f067522d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f066ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f0674a2c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 0x558c0fe4a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f066ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe4a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:48062/0 (socket says 172.21.15.19:48062) 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f067522d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47884/0 (socket says 172.21.15.19:47884) 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.628+0000 7f066ffff700 1 -- 172.21.15.19:0/3982582640 learned_addr learned my addr 172.21.15.19:0/3982582640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f067522d700 1 -- 172.21.15.19:0/3982582640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 msgr2=0x558c0fe4a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f067522d700 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 0x558c0fe4a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f067522d700 1 -- 172.21.15.19:0/3982582640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 msgr2=0x558c0fe4a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f067522d700 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe4a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f067522d700 1 -- 172.21.15.19:0/3982582640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0fbde490 con 0x558c0fe40400 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f066ffff700 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe4a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:29:11.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f067522d700 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4aac0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0668012da0 tx=0x7f0668010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a195537c22a044 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f064ffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f064ffff700 1 -- 172.21.15.19:0/3982582640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668019070 con 0x558c0fe40400 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f064ffff700 1 -- 172.21.15.19:0/3982582640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0668010a80 con 0x558c0fe40400 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f06856a6b80 1 -- 172.21.15.19:0/3982582640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0664014070 con 0x558c0fe40400 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f064ffff700 1 -- 172.21.15.19:0/3982582640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0668025050 con 0x558c0fe40400 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f06856a6b80 10 client.16206 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.629+0000 7f06856a6b80 1 -- 172.21.15.19:0/3982582640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c0fe57a70 con 0x558c0fe40400 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.630+0000 7f064ffff700 1 -- 172.21.15.19:0/3982582640 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0664014070 con 0x558c0fe40400 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.630+0000 7f064ffff700 1 -- 172.21.15.19:0/3982582640 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0668004420 con 0x558c0fe40400 2024-05-09T01:29:11.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.630+0000 7f06856a6b80 10 client.16206 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.630+0000 7f06856a6b80 10 client.16206 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.630+0000 7f06856a6b80 20 client.16206 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.630+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x558c0fe5a000 0x558c0fe5c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.630+0000 7f06856a6b80 4 client.16206 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.630+0000 7f06856a6b80 1 -- 172.21.15.19:0/3982582640 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c0fd70a80 con 0x558c0fe5a000 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.630+0000 7f0674a2c700 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x558c0fe5a000 0x558c0fe5c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.631+0000 7f0674a2c700 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x558c0fe5a000 0x558c0fe5c450 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e196af0f5140641 server_cookie=a64c7679b2bd2364 in_seq=0 out_seq=0 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.631+0000 7f064ffff700 10 client.16206 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f064ffff700 1 -- 172.21.15.19:0/3982582640 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c0fd70a80 con 0x558c0fe5a000 2024-05-09T01:29:11.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f064ffff700 10 client.16206 handle_command_reply: tid=0 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 client.16206 shutdown 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 -- 172.21.15.19:0/3982582640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 msgr2=0x558c0fe4aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4aac0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0668012da0 tx=0x7f0668010520 comp rx=0 tx=0).stop 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 -- 172.21.15.19:0/3982582640 shutdown_connections 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x558c0fe5a000 0x558c0fe5c450 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4aac0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 0x558c0fe4a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3982582640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe4a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 -- 172.21.15.19:0/3982582640 >> 172.21.15.19:0/3982582640 conn(0x558c0fcc6f70 msgr2=0x558c0fe34ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f064ffff700 0 client.16206 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:29:11.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 -- 172.21.15.19:0/3982582640 shutdown_connections 2024-05-09T01:29:11.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 1 -- 172.21.15.19:0/3982582640 wait complete. 2024-05-09T01:29:11.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.632+0000 7f06856a6b80 20 client.16206 trim_cache size 0 max 16384 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.647+0000 7f06856a6b80 1 Processor -- start 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.647+0000 7f06856a6b80 1 -- start start 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 0x558c0fe4a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe35700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe55e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f06856a6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06701b0d80 con 0x558c0fe40400 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f06856a6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06701b28c0 con 0x558c0fd3dd00 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f06856a6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06701b1450 con 0x558c0fcc34a0 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f067522d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe55e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f067522d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe55e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47892/0 (socket says 172.21.15.19:47892) 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f067522d700 1 -- 172.21.15.19:0/1714297887 learned_addr learned my addr 172.21.15.19:0/1714297887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f066ffff700 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe35700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f0674a2c700 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 0x558c0fe4a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f067522d700 1 -- 172.21.15.19:0/1714297887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 msgr2=0x558c0fe4a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:11.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f067522d700 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 0x558c0fe4a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f067522d700 1 -- 172.21.15.19:0/1714297887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 msgr2=0x558c0fe35700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:11.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f067522d700 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe35700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f067522d700 1 -- 172.21.15.19:0/1714297887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0fe3bd20 con 0x558c0fe40400 2024-05-09T01:29:11.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f0674a2c700 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 0x558c0fe4a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:11.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.648+0000 7f067522d700 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe55e60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0668012ac0 tx=0x7f0668015650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5800d703dfa97e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:11.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f065a7fc700 1 -- 172.21.15.19:0/1714297887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668019070 con 0x558c0fe40400 2024-05-09T01:29:11.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f065a7fc700 1 -- 172.21.15.19:0/1714297887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0668005d60 con 0x558c0fe40400 2024-05-09T01:29:11.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f065a7fc700 1 -- 172.21.15.19:0/1714297887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668015b30 con 0x558c0fe40400 2024-05-09T01:29:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f06856a6b80 1 -- 172.21.15.19:0/1714297887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 msgr2=0x558c0fe55e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f06856a6b80 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe55e60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0668012ac0 tx=0x7f0668015650 comp rx=0 tx=0).stop 2024-05-09T01:29:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f06856a6b80 1 -- 172.21.15.19:0/1714297887 shutdown_connections 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f06856a6b80 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe55e60 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f06856a6b80 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fcc34a0 0x558c0fe4a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f06856a6b80 1 --2- 172.21.15.19:0/1714297887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fd3dd00 0x558c0fe35700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f06856a6b80 1 -- 172.21.15.19:0/1714297887 >> 172.21.15.19:0/1714297887 conn(0x558c0fcc6f70 msgr2=0x558c0fe34ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f06856a6b80 1 -- 172.21.15.19:0/1714297887 shutdown_connections 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.649+0000 7f06856a6b80 1 -- 172.21.15.19:0/1714297887 wait complete. 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.650+0000 7f06856a6b80 1 -- start start 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.650+0000 7f06856a6b80 1 Processor -- start 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.650+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 0x558c0fe528a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.650+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fd3dd00 0x558c0fe4f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.650+0000 7f06856a6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.650+0000 7f06856a6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06701b21f0 con 0x558c0fe40400 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.650+0000 7f06856a6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06701b1b20 con 0x558c0fcc34a0 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.650+0000 7f06856a6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06701aed20 con 0x558c0fd3dd00 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f06856a6b80 10 client.0 _ll_register_callbacks cb 0x558c0fbd46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f067522d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f067522d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47908/0 (socket says 172.21.15.19:47908) 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f0674a2c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 0x558c0fe528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f067522d700 1 -- 172.21.15.19:0/3674919474 learned_addr learned my addr 172.21.15.19:0/3674919474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f066ffff700 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fd3dd00 0x558c0fe4f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f067522d700 1 -- 172.21.15.19:0/3674919474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fd3dd00 msgr2=0x558c0fe4f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f067522d700 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fd3dd00 0x558c0fe4f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f067522d700 1 -- 172.21.15.19:0/3674919474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 msgr2=0x558c0fe528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f067522d700 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 0x558c0fe528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f067522d700 1 -- 172.21.15.19:0/3674919474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0fe4fef0 con 0x558c0fe40400 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f066ffff700 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fd3dd00 0x558c0fe4f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f067522d700 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4f750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0668006010 tx=0x7f0668010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ffb36016b5b7f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f064ffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f064ffff700 1 -- 172.21.15.19:0/3674919474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668019040 con 0x558c0fe40400 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f064ffff700 1 -- 172.21.15.19:0/3674919474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0668015770 con 0x558c0fe40400 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.651+0000 7f06856a6b80 1 -- 172.21.15.19:0/3674919474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0668005d60 con 0x558c0fe40400 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f064ffff700 1 -- 172.21.15.19:0/3674919474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0668037070 con 0x558c0fe40400 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f06856a6b80 10 client.16212 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f06856a6b80 1 -- 172.21.15.19:0/3674919474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c0fec3830 con 0x558c0fe40400 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f064ffff700 1 -- 172.21.15.19:0/3674919474 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0668005d60 con 0x558c0fe40400 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f064ffff700 1 -- 172.21.15.19:0/3674919474 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f066801e9a0 con 0x558c0fe40400 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f06856a6b80 10 client.16212 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f06856a6b80 10 client.16212 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:29:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f06856a6b80 20 client.16212 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x558c0fec3e20 0x558c0fec6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f06856a6b80 4 client.16212 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f06856a6b80 1 -- 172.21.15.19:0/3674919474 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c0fd6ab20 con 0x558c0fec3e20 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.652+0000 7f0674a2c700 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x558c0fec3e20 0x558c0fec6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f0674a2c700 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x558c0fec3e20 0x558c0fec6240 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a84039557fea0cfd server_cookie=97cb3d1de1086beb in_seq=0 out_seq=0 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f064ffff700 10 client.16212 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f064ffff700 1 -- 172.21.15.19:0/3674919474 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c0fd6ab20 con 0x558c0fec3e20 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f064ffff700 10 client.16212 handle_command_reply: tid=0 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 client.16212 shutdown 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 -- 172.21.15.19:0/3674919474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 msgr2=0x558c0fe4f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4f750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0668006010 tx=0x7f0668010cb0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 -- 172.21.15.19:0/3674919474 shutdown_connections 2024-05-09T01:29:11.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x558c0fec3e20 0x558c0fec6240 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c0fe40400 0x558c0fe4f750 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c0fcc34a0 0x558c0fe528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 --2- 172.21.15.19:0/3674919474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c0fd3dd00 0x558c0fe4f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 -- 172.21.15.19:0/3674919474 >> 172.21.15.19:0/3674919474 conn(0x558c0fcc6f70 msgr2=0x558c0fe43cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:11.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f064ffff700 0 client.16212 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:29:11.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 -- 172.21.15.19:0/3674919474 shutdown_connections 2024-05-09T01:29:11.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.653+0000 7f06856a6b80 1 -- 172.21.15.19:0/3674919474 wait complete. 2024-05-09T01:29:11.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f06856a6b80 20 client.16212 trim_cache size 0 max 16384 2024-05-09T01:29:11.656 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:29:11.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 -- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06600b2900 msgr2=0x7f06600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:11.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06600b2900 0x7f06600b4e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f066400ac50 tx=0x7f066400a380 comp rx=0 tx=0).stop 2024-05-09T01:29:11.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 -- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 msgr2=0x7f067019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:11.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 0x7f067019d5b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f065c009f50 tx=0x7f065c0057c0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 -- 172.21.15.19:0/5439857 shutdown_connections 2024-05-09T01:29:11.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0670108740 0x7f06701a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06701034c0 0x7f067019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06600b2900 0x7f06600b4e10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 --2- 172.21.15.19:0/5439857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0670107df0 0x7f067019d5b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:11.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 -- 172.21.15.19:0/5439857 >> 172.21.15.19:0/5439857 conn(0x7f06700fedf0 msgr2=0x7f067010ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:11.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 -- 172.21.15.19:0/5439857 shutdown_connections 2024-05-09T01:29:11.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:11.654+0000 7f0676c90700 1 -- 172.21.15.19:0/5439857 wait complete. 2024-05-09T01:29:12.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:29:12.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:12 smithi059 ceph-mon[34351]: pgmap v488: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:29:12.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:12 smithi019 ceph-mon[30158]: pgmap v488: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:29:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:12 smithi196 ceph-mon[35041]: pgmap v488: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:29:12.735 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 -- 172.21.15.19:0/1318703268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f0104760 msgr2=0x7fe3f0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 --2- 172.21.15.19:0/1318703268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f0104760 0x7fe3f0106b80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009900 tx=0x7fe3ec02a410 comp rx=0 tx=0).stop 2024-05-09T01:29:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 -- 172.21.15.19:0/1318703268 shutdown_connections 2024-05-09T01:29:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 --2- 172.21.15.19:0/1318703268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3f0109a20 0x7fe3f010be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 --2- 172.21.15.19:0/1318703268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3f01070c0 0x7fe3f01094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 --2- 172.21.15.19:0/1318703268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f0104760 0x7fe3f0106b80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 -- 172.21.15.19:0/1318703268 >> 172.21.15.19:0/1318703268 conn(0x7fe3f00fdd60 msgr2=0x7fe3f0100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 -- 172.21.15.19:0/1318703268 shutdown_connections 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 -- 172.21.15.19:0/1318703268 wait complete. 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 Processor -- start 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.936+0000 7fe3f7396640 1 -- start start 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f7396640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3f0104760 0x7fe3f019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f7396640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01070c0 0x7fe3f019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f7396640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3f0109a20 0x7fe3f01a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f7396640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f010f770 con 0x7fe3f01070c0 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f7396640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f010f5f0 con 0x7fe3f0104760 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f7396640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f010f8f0 con 0x7fe3f0109a20 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f4931640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01070c0 0x7fe3f019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f4931640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01070c0 0x7fe3f019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47918/0 (socket says 172.21.15.19:47918) 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f4931640 1 -- 172.21.15.19:0/4024199814 learned_addr learned my addr 172.21.15.19:0/4024199814 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f5933640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3f0109a20 0x7fe3f01a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f5132640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3f0104760 0x7fe3f019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f4931640 1 -- 172.21.15.19:0/4024199814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3f0109a20 msgr2=0x7fe3f01a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f4931640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3f0109a20 0x7fe3f01a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f4931640 1 -- 172.21.15.19:0/4024199814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3f0104760 msgr2=0x7fe3f019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f4931640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3f0104760 0x7fe3f019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f4931640 1 -- 172.21.15.19:0/4024199814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f01a1e90 con 0x7fe3f01070c0 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3f4931640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01070c0 0x7fe3f019a890 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000d950 tx=0x7fe3e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.937+0000 7fe3e67fc640 1 -- 172.21.15.19:0/4024199814 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0017030 con 0x7fe3f01070c0 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.938+0000 7fe3e67fc640 1 -- 172.21.15.19:0/4024199814 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3e0004020 con 0x7fe3f01070c0 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.938+0000 7fe3f7396640 1 -- 172.21.15.19:0/4024199814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f01a2180 con 0x7fe3f01070c0 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.938+0000 7fe3e67fc640 1 -- 172.21.15.19:0/4024199814 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00047a0 con 0x7fe3f01070c0 2024-05-09T01:29:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.938+0000 7fe3f7396640 1 -- 172.21.15.19:0/4024199814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f01a26d0 con 0x7fe3f01070c0 2024-05-09T01:29:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.940+0000 7fe3fe24b740 1 Processor -- start 2024-05-09T01:29:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.940+0000 7fe3fe24b740 1 -- start start 2024-05-09T01:29:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.940+0000 7fe3e67fc640 1 -- 172.21.15.19:0/4024199814 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe3e0009a30 con 0x7fe3f01070c0 2024-05-09T01:29:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.940+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 0x560fa008a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcbd2f0 0x560fa008aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcc54d0 0x560fa0091980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560f9ffee7c0 con 0x560f9fcbd2f0 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f9ff8a0e0 con 0x560f9fce61b0 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f9ffea780 con 0x560f9fcc54d0 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5132640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcbd2f0 0x560fa008aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5132640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcbd2f0 0x560fa008aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47926/0 (socket says 172.21.15.19:47926) 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5132640 1 -- 172.21.15.19:0/1682667372 learned_addr learned my addr 172.21.15.19:0/1682667372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5933640 1 --2- 172.21.15.19:0/1682667372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 0x560fa008a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f4931640 1 --2- 172.21.15.19:0/1682667372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcc54d0 0x560fa0091980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5132640 1 -- 172.21.15.19:0/1682667372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcc54d0 msgr2=0x560fa0091980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5132640 1 --2- 172.21.15.19:0/1682667372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcc54d0 0x560fa0091980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5132640 1 -- 172.21.15.19:0/1682667372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 msgr2=0x560fa008a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5132640 1 --2- 172.21.15.19:0/1682667372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 0x560fa008a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5132640 1 -- 172.21.15.19:0/1682667372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9ffb37c0 con 0x560f9fcbd2f0 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5132640 1 --2- 172.21.15.19:0/1682667372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcbd2f0 0x560fa008aa80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0075920 tx=0x7fe3ec032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3790934a8b2a6f54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3e67fc640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe3dc0b27e0 0x7fe3dc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3f5933640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe3dc0b27e0 0x7fe3dc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.942+0000 7fe3e67fc640 1 -- 172.21.15.19:0/4024199814 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe3e00e1dd0 con 0x7fe3f01070c0 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.941+0000 7fe3db7fe640 1 -- 172.21.15.19:0/1682667372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec03f070 con 0x560f9fcbd2f0 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.942+0000 7fe3db7fe640 1 -- 172.21.15.19:0/1682667372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ec004040 con 0x560f9fcbd2f0 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.942+0000 7fe3db7fe640 1 -- 172.21.15.19:0/1682667372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec035070 con 0x560f9fcbd2f0 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.942+0000 7fe3fe24b740 1 -- 172.21.15.19:0/1682667372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcbd2f0 msgr2=0x560fa008aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.942+0000 7fe3f5933640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe3dc0b27e0 0x7fe3dc0b4cf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x560fa008b6d0 tx=0x7fe3e8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.942+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/1682667372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcbd2f0 0x560fa008aa80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0075920 tx=0x7fe3ec032b20 comp rx=0 tx=0).stop 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.942+0000 7fe3fe24b740 1 -- 172.21.15.19:0/1682667372 shutdown_connections 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.942+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/1682667372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcc54d0 0x560fa0091980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.942+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/1682667372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcbd2f0 0x560fa008aa80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.943+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/1682667372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 0x560fa008a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.943+0000 7fe3fe24b740 1 -- 172.21.15.19:0/1682667372 >> 172.21.15.19:0/1682667372 conn(0x560f9fcda300 msgr2=0x560fa0085e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.943+0000 7fe3fe24b740 1 -- 172.21.15.19:0/1682667372 shutdown_connections 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.943+0000 7fe3fe24b740 1 -- 172.21.15.19:0/1682667372 wait complete. 2024-05-09T01:29:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.943+0000 7fe3fe24b740 1 -- start start 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.943+0000 7fe3fe24b740 1 Processor -- start 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.943+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa00941a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 0x560fa00946e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa009b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560fa0092370 con 0x560f9fce61b0 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f9ff759c0 con 0x560f9fcc54d0 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560fa00924f0 con 0x560f9fcbd2f0 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3fe24b740 10 client.0 _ll_register_callbacks cb 0x560f9ffa2020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5933640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa009b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5933640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa009b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47938/0 (socket says 172.21.15.19:47938) 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5933640 1 -- 172.21.15.19:0/24748697 learned_addr learned my addr 172.21.15.19:0/24748697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5132640 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa00941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f4931640 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 0x560fa00946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5933640 1 -- 172.21.15.19:0/24748697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 msgr2=0x560fa00941a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5933640 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa00941a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5933640 1 -- 172.21.15.19:0/24748697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 msgr2=0x560fa00946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5933640 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 0x560fa00946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5933640 1 -- 172.21.15.19:0/24748697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9ff9c1f0 con 0x560f9fce61b0 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f4931640 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 0x560fa00946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5132640 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa00941a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3f5933640 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa009b640 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe3e801f5a0 tx=0x7fe3e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b404b94a743174e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/24748697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80117d0 con 0x560f9fce61b0 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/24748697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe3e8011970 con 0x560f9fce61b0 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.944+0000 7fe3fe24b740 1 -- 172.21.15.19:0/24748697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3ec004040 con 0x560f9fce61b0 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/24748697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3e8011b70 con 0x560f9fce61b0 2024-05-09T01:29:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3fe24b740 10 client.16224 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3fe24b740 1 -- 172.21.15.19:0/24748697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f9fd51790 con 0x560f9fce61b0 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/24748697 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ec004040 con 0x560f9fce61b0 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/24748697 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe3e8026070 con 0x560f9fce61b0 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3fe24b740 10 client.16224 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3fe24b740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3fe24b740 20 client.16224 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560f9fca4d80 0x560fa00a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3fe24b740 4 client.16224 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3fe24b740 1 -- 172.21.15.19:0/24748697 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f9fffc3d0 con 0x560f9fca4d80 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.945+0000 7fe3f5132640 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560f9fca4d80 0x560fa00a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.946+0000 7fe3d8ff9640 10 client.16224 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.946+0000 7fe3f5132640 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560f9fca4d80 0x560fa00a1bd0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1783f2d7519a0f2 server_cookie=2ff21107bab07725 in_seq=0 out_seq=0 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/24748697 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560f9fffc3d0 con 0x560f9fca4d80 2024-05-09T01:29:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3d8ff9640 10 client.16224 handle_command_reply: tid=0 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 client.16224 shutdown 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 -- 172.21.15.19:0/24748697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 msgr2=0x560fa009b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa009b640 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe3e801f5a0 tx=0x7fe3e8004ab0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 -- 172.21.15.19:0/24748697 shutdown_connections 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560f9fca4d80 0x560fa00a1bd0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa009b640 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 0x560fa00946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/24748697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa00941a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 -- 172.21.15.19:0/24748697 >> 172.21.15.19:0/24748697 conn(0x560f9fcda300 msgr2=0x560fa0084700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3d8ff9640 0 client.16224 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 -- 172.21.15.19:0/24748697 shutdown_connections 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 1 -- 172.21.15.19:0/24748697 wait complete. 2024-05-09T01:29:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.947+0000 7fe3fe24b740 20 client.16224 trim_cache size 0 max 16384 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.961+0000 7fe3fe24b740 1 Processor -- start 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.961+0000 7fe3fe24b740 1 -- start start 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa0085f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 0x560fa009b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa0088c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01ab490 con 0x560f9fce61b0 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01acfd0 con 0x560f9fcc54d0 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01abb60 con 0x560f9fcbd2f0 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5933640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa0088c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5933640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa0088c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47952/0 (socket says 172.21.15.19:47952) 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5933640 1 -- 172.21.15.19:0/891177661 learned_addr learned my addr 172.21.15.19:0/891177661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5132640 1 --2- 172.21.15.19:0/891177661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa0085f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f4931640 1 --2- 172.21.15.19:0/891177661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 0x560fa009b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5933640 1 -- 172.21.15.19:0/891177661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 msgr2=0x560fa0085f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5933640 1 --2- 172.21.15.19:0/891177661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa0085f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5933640 1 -- 172.21.15.19:0/891177661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 msgr2=0x560fa009b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5933640 1 --2- 172.21.15.19:0/891177661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 0x560fa009b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5933640 1 -- 172.21.15.19:0/891177661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9fdc85f0 con 0x560f9fce61b0 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3f5933640 1 --2- 172.21.15.19:0/891177661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa0088c90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8028f10 tx=0x7fe3e8011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a16964e5978c65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3daffd640 1 -- 172.21.15.19:0/891177661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80325e0 con 0x560f9fce61b0 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3daffd640 1 -- 172.21.15.19:0/891177661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3e8037070 con 0x560f9fce61b0 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.962+0000 7fe3daffd640 1 -- 172.21.15.19:0/891177661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8028b20 con 0x560f9fce61b0 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.963+0000 7fe3fe24b740 1 -- 172.21.15.19:0/891177661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 msgr2=0x560fa0088c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.963+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/891177661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa0088c90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8028f10 tx=0x7fe3e8011f20 comp rx=0 tx=0).stop 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.963+0000 7fe3fe24b740 1 -- 172.21.15.19:0/891177661 shutdown_connections 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.963+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/891177661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fce61b0 0x560fa0088c90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.963+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/891177661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fcc54d0 0x560fa009b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.963+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/891177661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa0085f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.963+0000 7fe3fe24b740 1 -- 172.21.15.19:0/891177661 >> 172.21.15.19:0/891177661 conn(0x560f9fcda300 msgr2=0x560fa0085850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.963+0000 7fe3fe24b740 1 -- 172.21.15.19:0/891177661 shutdown_connections 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.963+0000 7fe3fe24b740 1 -- 172.21.15.19:0/891177661 wait complete. 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3fe24b740 1 -- start start 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3fe24b740 1 Processor -- start 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa0105580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcc54d0 0x560fa0105ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3fe24b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 0x560fa010ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01ac900 con 0x560f9fcc54d0 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01ac230 con 0x560f9fce61b0 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3fe24b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01a9470 con 0x560f9fcbd2f0 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3fe24b740 10 client.0 _ll_register_callbacks cb 0x7fe3e8003f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3f4931640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcc54d0 0x560fa0105ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3f4931640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcc54d0 0x560fa0105ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47968/0 (socket says 172.21.15.19:47968) 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3f4931640 1 -- 172.21.15.19:0/727656079 learned_addr learned my addr 172.21.15.19:0/727656079 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3f5933640 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 0x560fa010ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3f5132640 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa0105580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3f4931640 1 -- 172.21.15.19:0/727656079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 msgr2=0x560fa0105580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.964+0000 7fe3f4931640 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa0105580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3f4931640 1 -- 172.21.15.19:0/727656079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 msgr2=0x560fa010ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3f4931640 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 0x560fa010ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3f4931640 1 -- 172.21.15.19:0/727656079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9ffb2ed0 con 0x560f9fcc54d0 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3f4931640 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcc54d0 0x560fa0105ac0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00a5450 tx=0x7fe3e0009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e96d0d6c4782d3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/727656079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00a93d0 con 0x560f9fcc54d0 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/727656079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe3e00a5660 con 0x560f9fcc54d0 2024-05-09T01:29:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3fe24b740 1 -- 172.21.15.19:0/727656079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3e8037070 con 0x560f9fcc54d0 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/727656079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3e00ad780 con 0x560f9fcc54d0 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3fe24b740 10 client.16230 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3fe24b740 1 -- 172.21.15.19:0/727656079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560fa010e110 con 0x560f9fcc54d0 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/727656079 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3e8037070 con 0x560f9fcc54d0 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/727656079 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe3e00a3af0 con 0x560f9fcc54d0 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3fe24b740 10 client.16230 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3fe24b740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3fe24b740 20 client.16230 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560f9fc7a1c0 0x560fa0110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3fe24b740 4 client.16230 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.965+0000 7fe3fe24b740 1 -- 172.21.15.19:0/727656079 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f9fff6240 con 0x560f9fc7a1c0 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.966+0000 7fe3f5132640 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560f9fc7a1c0 0x560fa0110ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.966+0000 7fe3f5132640 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560f9fc7a1c0 0x560fa0110ab0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c850f7ee5649d2c2 server_cookie=8efffcd654664b9 in_seq=0 out_seq=0 2024-05-09T01:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.966+0000 7fe3d8ff9640 10 client.16230 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.983+0000 7fe3d8ff9640 1 -- 172.21.15.19:0/727656079 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560f9fff6240 con 0x560f9fc7a1c0 2024-05-09T01:29:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.983+0000 7fe3d8ff9640 10 client.16230 handle_command_reply: tid=0 2024-05-09T01:29:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.983+0000 7fe3fe24b740 1 client.16230 shutdown 2024-05-09T01:29:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.983+0000 7fe3fe24b740 1 -- 172.21.15.19:0/727656079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcc54d0 msgr2=0x560fa0105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcc54d0 0x560fa0105ac0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00a5450 tx=0x7fe3e0009f50 comp rx=0 tx=0).stop 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 1 -- 172.21.15.19:0/727656079 shutdown_connections 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560f9fc7a1c0 0x560fa0110ab0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3d8ff9640 0 client.16230 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f9fce61b0 0x560fa010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f9fcc54d0 0x560fa0105ac0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 1 --2- 172.21.15.19:0/727656079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f9fcbd2f0 0x560fa0105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 1 -- 172.21.15.19:0/727656079 >> 172.21.15.19:0/727656079 conn(0x560f9fcda300 msgr2=0x560fa0088c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 1 -- 172.21.15.19:0/727656079 shutdown_connections 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 1 -- 172.21.15.19:0/727656079 wait complete. 2024-05-09T01:29:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3fe24b740 20 client.16230 trim_cache size 0 max 16384 2024-05-09T01:29:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:29:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:29:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 -- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe3dc0b27e0 msgr2=0x7fe3dc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe3dc0b27e0 0x7fe3dc0b4cf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x560fa008b6d0 tx=0x7fe3e8009500 comp rx=0 tx=0).stop 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 -- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01070c0 msgr2=0x7fe3f019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01070c0 0x7fe3f019a890 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000d950 tx=0x7fe3e000de10 comp rx=0 tx=0).stop 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 -- 172.21.15.19:0/4024199814 shutdown_connections 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe3dc0b27e0 0x7fe3dc0b4cf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3f0109a20 0x7fe3f01a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01070c0 0x7fe3f019a890 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 --2- 172.21.15.19:0/4024199814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3f0104760 0x7fe3f019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.984+0000 7fe3f7396640 1 -- 172.21.15.19:0/4024199814 >> 172.21.15.19:0/4024199814 conn(0x7fe3f00fdd60 msgr2=0x7fe3f0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.985+0000 7fe3f7396640 1 -- 172.21.15.19:0/4024199814 shutdown_connections 2024-05-09T01:29:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:12.985+0000 7fe3f7396640 1 -- 172.21.15.19:0/4024199814 wait complete. 2024-05-09T01:29:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.513+0000 7f80251ec700 1 -- 172.21.15.19:0/3400927320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020108740 msgr2=0x7f80201100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.513+0000 7f80251ec700 1 --2- 172.21.15.19:0/3400927320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020108740 0x7f80201100a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f801800b0d0 tx=0x7f801800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.513+0000 7f80251ec700 1 -- 172.21.15.19:0/3400927320 shutdown_connections 2024-05-09T01:29:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.513+0000 7f80251ec700 1 --2- 172.21.15.19:0/3400927320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020108740 0x7f80201100a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.513+0000 7f80251ec700 1 --2- 172.21.15.19:0/3400927320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80201034c0 0x7f80201038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.513+0000 7f80251ec700 1 --2- 172.21.15.19:0/3400927320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8020107df0 0x7f8020108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.513+0000 7f80251ec700 1 -- 172.21.15.19:0/3400927320 >> 172.21.15.19:0/3400927320 conn(0x7f80200fedf0 msgr2=0x7f8020101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.513+0000 7f80251ec700 1 -- 172.21.15.19:0/3400927320 shutdown_connections 2024-05-09T01:29:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.513+0000 7f80251ec700 1 -- 172.21.15.19:0/3400927320 wait complete. 2024-05-09T01:29:14.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:14 smithi019 ceph-mon[30158]: pgmap v489: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.514+0000 7f80251ec700 1 Processor -- start 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.514+0000 7f80251ec700 1 -- start start 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.514+0000 7f80251ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80201034c0 0x7f802019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.514+0000 7f80251ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020107df0 0x7f802019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.514+0000 7f80251ec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8020108740 0x7f80201a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.514+0000 7f80251ec700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80201133c0 con 0x7f8020107df0 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.514+0000 7f80251ec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8020113240 con 0x7f80201034c0 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.514+0000 7f80251ec700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8020113540 con 0x7f8020108740 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020107df0 0x7f802019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020107df0 0x7f802019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47994/0 (socket says 172.21.15.19:47994) 2024-05-09T01:29:14.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801659c700 1 -- 172.21.15.19:0/2937907066 learned_addr learned my addr 172.21.15.19:0/2937907066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801ed9d700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80201034c0 0x7f802019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801f59e700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8020108740 0x7f80201a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801659c700 1 -- 172.21.15.19:0/2937907066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8020108740 msgr2=0x7f80201a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801659c700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8020108740 0x7f80201a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801659c700 1 -- 172.21.15.19:0/2937907066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80201034c0 msgr2=0x7f802019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801659c700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80201034c0 0x7f802019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801659c700 1 -- 172.21.15.19:0/2937907066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80201a1df0 con 0x7f8020107df0 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801659c700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020107df0 0x7f802019d570 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f800800d980 tx=0x7f800800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.515+0000 7f801cd99700 1 -- 172.21.15.19:0/2937907066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008017070 con 0x7f8020107df0 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.516+0000 7f80251ec700 1 -- 172.21.15.19:0/2937907066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80201a20e0 con 0x7f8020107df0 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.516+0000 7f801cd99700 1 -- 172.21.15.19:0/2937907066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f800800fd40 con 0x7f8020107df0 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.516+0000 7f801cd99700 1 -- 172.21.15.19:0/2937907066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800800e3b0 con 0x7f8020107df0 2024-05-09T01:29:14.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.516+0000 7f80251ec700 1 -- 172.21.15.19:0/2937907066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80201a9ab0 con 0x7f8020107df0 2024-05-09T01:29:14.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.520+0000 7f801cd99700 1 -- 172.21.15.19:0/2937907066 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f800800e550 con 0x7f8020107df0 2024-05-09T01:29:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.520+0000 7f801cd99700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f800c0b2900 0x7f800c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.520+0000 7f8033c02b80 1 Processor -- start 2024-05-09T01:29:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f8033c02b80 1 -- start start 2024-05-09T01:29:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f801ed9d700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f800c0b2900 0x7f800c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f801cd99700 1 -- 172.21.15.19:0/2937907066 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f80080e1f80 con 0x7f8020107df0 2024-05-09T01:29:14.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f801ed9d700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f800c0b2900 0x7f800c0b4e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f801000ac50 tx=0x7f801000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:14.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f042160 0x56067f0db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 0x56067f0dba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 0x56067f0d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f8033c02b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56067eff7ff0 con 0x56067f042160 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f8033c02b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56067ef967f0 con 0x56067ef584a0 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f8033c02b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56067eff3fb0 con 0x56067f0d56a0 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f801ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f042160 0x56067f0db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f801ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f042160 0x56067f0db4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48004/0 (socket says 172.21.15.19:48004) 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.521+0000 7f801ed9d700 1 -- 172.21.15.19:0/3231216904 learned_addr learned my addr 172.21.15.19:0/3231216904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801659c700 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 0x56067f0dba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801f59e700 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 0x56067f0d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801ed9d700 1 -- 172.21.15.19:0/3231216904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 msgr2=0x56067f0d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801ed9d700 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 0x56067f0d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801ed9d700 1 -- 172.21.15.19:0/3231216904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 msgr2=0x56067f0dba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801ed9d700 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 0x56067f0dba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801ed9d700 1 -- 172.21.15.19:0/3231216904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56067ef8e370 con 0x56067f042160 2024-05-09T01:29:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801f59e700 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 0x56067f0d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801659c700 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 0x56067f0dba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801ed9d700 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f042160 0x56067f0db4f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f801000ff90 tx=0x7f8010012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8808d251d62ed723 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801559a700 1 -- 172.21.15.19:0/3231216904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010016070 con 0x56067f042160 2024-05-09T01:29:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801559a700 1 -- 172.21.15.19:0/3231216904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8010014070 con 0x56067f042160 2024-05-09T01:29:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.522+0000 7f801559a700 1 -- 172.21.15.19:0/3231216904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801001d400 con 0x56067f042160 2024-05-09T01:29:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.523+0000 7f8033c02b80 1 -- 172.21.15.19:0/3231216904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f042160 msgr2=0x56067f0db4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.523+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f042160 0x56067f0db4f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f801000ff90 tx=0x7f8010012000 comp rx=0 tx=0).stop 2024-05-09T01:29:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.523+0000 7f8033c02b80 1 -- 172.21.15.19:0/3231216904 shutdown_connections 2024-05-09T01:29:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.523+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 0x56067f0d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.523+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f042160 0x56067f0db4f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.523+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3231216904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 0x56067f0dba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.523+0000 7f8033c02b80 1 -- 172.21.15.19:0/3231216904 >> 172.21.15.19:0/3231216904 conn(0x56067efb86a0 msgr2=0x56067f0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.523+0000 7f8033c02b80 1 -- 172.21.15.19:0/3231216904 shutdown_connections 2024-05-09T01:29:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.523+0000 7f8033c02b80 1 -- 172.21.15.19:0/3231216904 wait complete. 2024-05-09T01:29:14.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f8033c02b80 1 -- start start 2024-05-09T01:29:14.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f8033c02b80 1 Processor -- start 2024-05-09T01:29:14.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 0x56067f0df780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f0d56a0 0x56067f0dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f8033c02b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56067ee73ba0 con 0x56067ef584a0 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f8033c02b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56067efb2150 con 0x56067f0d56a0 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f8033c02b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56067f0e21d0 con 0x56067f042160 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f8033c02b80 10 client.0 _ll_register_callbacks cb 0x56067ee696c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f801ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f801ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0df240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48016/0 (socket says 172.21.15.19:48016) 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.524+0000 7f801ed9d700 1 -- 172.21.15.19:0/3356823593 learned_addr learned my addr 172.21.15.19:0/3356823593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f801659c700 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 0x56067f0df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f801f59e700 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f0d56a0 0x56067f0dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f801ed9d700 1 -- 172.21.15.19:0/3356823593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 msgr2=0x56067f0df780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:14.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f801ed9d700 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 0x56067f0df780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f801ed9d700 1 -- 172.21.15.19:0/3356823593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f0d56a0 msgr2=0x56067f0dfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f801ed9d700 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f0d56a0 0x56067f0dfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f801ed9d700 1 -- 172.21.15.19:0/3356823593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56067ee73490 con 0x56067ef584a0 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f801f59e700 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f0d56a0 0x56067f0dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f801ed9d700 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0df240 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f801000fa30 tx=0x7f8010010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c7eb5711383219 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f80067fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f80067fc700 1 -- 172.21.15.19:0/3356823593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010010880 con 0x56067ef584a0 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f80067fc700 1 -- 172.21.15.19:0/3356823593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8010003910 con 0x56067ef584a0 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f8033c02b80 1 -- 172.21.15.19:0/3356823593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8010014070 con 0x56067ef584a0 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f80067fc700 1 -- 172.21.15.19:0/3356823593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8010003b10 con 0x56067ef584a0 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f8033c02b80 10 client.16242 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.525+0000 7f8033c02b80 1 -- 172.21.15.19:0/3356823593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56067f0ecc10 con 0x56067ef584a0 2024-05-09T01:29:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.526+0000 7f80067fc700 1 -- 172.21.15.19:0/3356823593 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8010014070 con 0x56067ef584a0 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.526+0000 7f80067fc700 1 -- 172.21.15.19:0/3356823593 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f80100049b0 con 0x56067ef584a0 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.526+0000 7f8033c02b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.526+0000 7f8033c02b80 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.526+0000 7f8033c02b80 20 client.16242 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.526+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56067f0ef190 0x56067f0f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.526+0000 7f8033c02b80 4 client.16242 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.526+0000 7f8033c02b80 1 -- 172.21.15.19:0/3356823593 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56067f0059d0 con 0x56067f0ef190 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.526+0000 7f801659c700 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56067f0ef190 0x56067f0f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.527+0000 7f801659c700 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56067f0ef190 0x56067f0f15b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a00389ce1262bd server_cookie=365110a82ef5c539 in_seq=0 out_seq=0 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.527+0000 7f80067fc700 10 client.16242 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f80067fc700 1 -- 172.21.15.19:0/3356823593 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56067f0059d0 con 0x56067f0ef190 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f80067fc700 10 client.16242 handle_command_reply: tid=0 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 client.16242 shutdown 2024-05-09T01:29:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 -- 172.21.15.19:0/3356823593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 msgr2=0x56067f0df240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0df240 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f801000fa30 tx=0x7f8010010eb0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 -- 172.21.15.19:0/3356823593 shutdown_connections 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56067f0ef190 0x56067f0f15b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f0d56a0 0x56067f0dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0df240 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 --2- 172.21.15.19:0/3356823593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 0x56067f0df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 -- 172.21.15.19:0/3356823593 >> 172.21.15.19:0/3356823593 conn(0x56067efb86a0 msgr2=0x56067f0d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f80067fc700 0 client.16242 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 -- 172.21.15.19:0/3356823593 shutdown_connections 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 1 -- 172.21.15.19:0/3356823593 wait complete. 2024-05-09T01:29:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.528+0000 7f8033c02b80 20 client.16242 trim_cache size 0 max 16384 2024-05-09T01:29:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f8033c02b80 1 Processor -- start 2024-05-09T01:29:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f8033c02b80 1 -- start start 2024-05-09T01:29:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 0x56067f0de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 0x56067f0df200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f0d56a0 0x56067f0df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f8033c02b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80201b0f20 con 0x56067f0d56a0 2024-05-09T01:29:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f8033c02b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80201b2a60 con 0x56067ef584a0 2024-05-09T01:29:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f8033c02b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80201b15f0 con 0x56067f042160 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f801f59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f0d56a0 0x56067f0df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f801f59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f0d56a0 0x56067f0df740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48018/0 (socket says 172.21.15.19:48018) 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f801f59e700 1 -- 172.21.15.19:0/443614832 learned_addr learned my addr 172.21.15.19:0/443614832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f801659c700 1 --2- 172.21.15.19:0/443614832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 0x56067f0df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.544+0000 7f801ed9d700 1 --2- 172.21.15.19:0/443614832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 0x56067f0de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.545+0000 7f801f59e700 1 -- 172.21.15.19:0/443614832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 msgr2=0x56067f0df200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.545+0000 7f801f59e700 1 --2- 172.21.15.19:0/443614832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 0x56067f0df200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.545+0000 7f801f59e700 1 -- 172.21.15.19:0/443614832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 msgr2=0x56067f0de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.545+0000 7f801f59e700 1 --2- 172.21.15.19:0/443614832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 0x56067f0de580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.545+0000 7f801f59e700 1 -- 172.21.15.19:0/443614832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56067f0e1ae0 con 0x56067f0d56a0 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.545+0000 7f801f59e700 1 --2- 172.21.15.19:0/443614832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f0d56a0 0x56067f0df740 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f801800d010 tx=0x7f801800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c986f2b451d4f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.545+0000 7f8014d99700 1 -- 172.21.15.19:0/443614832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018009410 con 0x56067f0d56a0 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.545+0000 7f8014d99700 1 -- 172.21.15.19:0/443614832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80180075c0 con 0x56067f0d56a0 2024-05-09T01:29:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.545+0000 7f8014d99700 1 -- 172.21.15.19:0/443614832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018005310 con 0x56067f0d56a0 2024-05-09T01:29:14.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 -- 172.21.15.19:0/443614832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f0d56a0 msgr2=0x56067f0df740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:14.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 --2- 172.21.15.19:0/443614832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f0d56a0 0x56067f0df740 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f801800d010 tx=0x7f801800b040 comp rx=0 tx=0).stop 2024-05-09T01:29:14.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 -- 172.21.15.19:0/443614832 shutdown_connections 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 --2- 172.21.15.19:0/443614832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067f0d56a0 0x56067f0df740 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 --2- 172.21.15.19:0/443614832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067ef584a0 0x56067f0de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 --2- 172.21.15.19:0/443614832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f042160 0x56067f0df200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 -- 172.21.15.19:0/443614832 >> 172.21.15.19:0/443614832 conn(0x56067efb86a0 msgr2=0x56067f0da330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 -- 172.21.15.19:0/443614832 shutdown_connections 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 -- 172.21.15.19:0/443614832 wait complete. 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.546+0000 7f8033c02b80 1 -- start start 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f8033c02b80 1 Processor -- start 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0f1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f042160 0x56067f0f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f8033c02b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 0x56067f1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f8033c02b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80201b2390 con 0x56067ef584a0 2024-05-09T01:29:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f8033c02b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80201b1cc0 con 0x56067f042160 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f8033c02b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80201aeec0 con 0x56067f0d56a0 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f8033c02b80 10 client.0 _ll_register_callbacks cb 0x56067ee696c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0f1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48020/0 (socket says 172.21.15.19:48020) 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801ed9d700 1 -- 172.21.15.19:0/613812952 learned_addr learned my addr 172.21.15.19:0/613812952 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801659c700 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f042160 0x56067f0f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801f59e700 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 0x56067f1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801ed9d700 1 -- 172.21.15.19:0/613812952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 msgr2=0x56067f1576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801ed9d700 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 0x56067f1576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801ed9d700 1 -- 172.21.15.19:0/613812952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f042160 msgr2=0x56067f0f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801ed9d700 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f042160 0x56067f0f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.547+0000 7f801ed9d700 1 -- 172.21.15.19:0/613812952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56067f157c40 con 0x56067ef584a0 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f801ed9d700 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0f1560 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8010004dd0 tx=0x7f8010010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f47af6a1995a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f80067fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f80067fc700 1 -- 172.21.15.19:0/613812952 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801003f030 con 0x56067ef584a0 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f80067fc700 1 -- 172.21.15.19:0/613812952 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f801002c070 con 0x56067ef584a0 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f8033c02b80 1 -- 172.21.15.19:0/613812952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80180075c0 con 0x56067ef584a0 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f80067fc700 1 -- 172.21.15.19:0/613812952 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8010016040 con 0x56067ef584a0 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f8033c02b80 10 client.16248 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f8033c02b80 1 -- 172.21.15.19:0/613812952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56067f158e80 con 0x56067ef584a0 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f80067fc700 1 -- 172.21.15.19:0/613812952 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80180075c0 con 0x56067ef584a0 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f80067fc700 1 -- 172.21.15.19:0/613812952 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f801003bc80 con 0x56067ef584a0 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f8033c02b80 10 client.16248 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f8033c02b80 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.548+0000 7f8033c02b80 20 client.16248 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.549+0000 7f8033c02b80 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56067f159500 0x56067f15b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.549+0000 7f8033c02b80 4 client.16248 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.549+0000 7f8033c02b80 1 -- 172.21.15.19:0/613812952 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56067efffa70 con 0x56067f159500 2024-05-09T01:29:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.549+0000 7f801659c700 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56067f159500 0x56067f15b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.549+0000 7f80067fc700 10 client.16248 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.549+0000 7f801659c700 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56067f159500 0x56067f15b920 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370682003c6be6e server_cookie=641e3f68eb327ca5 in_seq=0 out_seq=0 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.549+0000 7f80067fc700 1 -- 172.21.15.19:0/613812952 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56067efffa70 con 0x56067f159500 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.549+0000 7f80067fc700 10 client.16248 handle_command_reply: tid=0 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.549+0000 7f8033c02b80 1 client.16248 shutdown 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 -- 172.21.15.19:0/613812952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 msgr2=0x56067f0f1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0f1560 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8010004dd0 tx=0x7f8010010b30 comp rx=0 tx=0).stop 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 -- 172.21.15.19:0/613812952 shutdown_connections 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56067f159500 0x56067f15b920 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56067f0d56a0 0x56067f1576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56067ef584a0 0x56067f0f1560 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 --2- 172.21.15.19:0/613812952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56067f042160 0x56067f0f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 -- 172.21.15.19:0/613812952 >> 172.21.15.19:0/613812952 conn(0x56067efb86a0 msgr2=0x56067f0da330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80067fc700 0 client.16248 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 -- 172.21.15.19:0/613812952 shutdown_connections 2024-05-09T01:29:14.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 1 -- 172.21.15.19:0/613812952 wait complete. 2024-05-09T01:29:14.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f8033c02b80 20 client.16248 trim_cache size 0 max 16384 2024-05-09T01:29:14.552 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:29:14.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 -- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f800c0b2900 msgr2=0x7f800c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f800c0b2900 0x7f800c0b4e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f801000ac50 tx=0x7f801000a380 comp rx=0 tx=0).stop 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 -- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020107df0 msgr2=0x7f802019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020107df0 0x7f802019d570 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f800800d980 tx=0x7f800800dc90 comp rx=0 tx=0).stop 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 -- 172.21.15.19:0/2937907066 shutdown_connections 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8020108740 0x7f80201a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80201034c0 0x7f802019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f800c0b2900 0x7f800c0b4e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 --2- 172.21.15.19:0/2937907066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020107df0 0x7f802019d570 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.550+0000 7f80251ec700 1 -- 172.21.15.19:0/2937907066 >> 172.21.15.19:0/2937907066 conn(0x7f80200fedf0 msgr2=0x7f8020109730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.551+0000 7f80251ec700 1 -- 172.21.15.19:0/2937907066 shutdown_connections 2024-05-09T01:29:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:14.551+0000 7f80251ec700 1 -- 172.21.15.19:0/2937907066 wait complete. 2024-05-09T01:29:14.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:14 smithi059 ceph-mon[34351]: pgmap v489: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:29:14.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:14 smithi196 ceph-mon[35041]: pgmap v489: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:29:16.318 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:29:16.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:29:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:16.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:16 smithi059 ceph-mon[34351]: pgmap v490: 97 pgs: 97 active+clean; 1.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:29:16.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:16 smithi019 ceph-mon[30158]: pgmap v490: 97 pgs: 97 active+clean; 1.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:29:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:16 smithi196 ceph-mon[35041]: pgmap v490: 97 pgs: 97 active+clean; 1.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:29:18.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:18 smithi019 ceph-mon[30158]: pgmap v491: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:29:18.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.429+0000 7f065abb8700 1 -- 172.21.15.19:0/2385072515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 msgr2=0x7f0654108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:18.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.429+0000 7f065abb8700 1 --2- 172.21.15.19:0/2385072515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 0x7f0654108220 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f06400099c0 tx=0x7f0640009cd0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.429+0000 7f065abb8700 1 -- 172.21.15.19:0/2385072515 shutdown_connections 2024-05-09T01:29:18.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.429+0000 7f065abb8700 1 --2- 172.21.15.19:0/2385072515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0654108760 0x7f06541100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.429+0000 7f065abb8700 1 --2- 172.21.15.19:0/2385072515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541034e0 0x7f06541038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.429+0000 7f065abb8700 1 --2- 172.21.15.19:0/2385072515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 0x7f0654108220 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.429+0000 7f065abb8700 1 -- 172.21.15.19:0/2385072515 >> 172.21.15.19:0/2385072515 conn(0x7f06540fef30 msgr2=0x7f0654101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:18.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.429+0000 7f065abb8700 1 -- 172.21.15.19:0/2385072515 shutdown_connections 2024-05-09T01:29:18.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.429+0000 7f065abb8700 1 -- 172.21.15.19:0/2385072515 wait complete. 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f065abb8700 1 Processor -- start 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f065abb8700 1 -- start start 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f065abb8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541034e0 0x7f065419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f065abb8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 0x7f065419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f065abb8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0654108760 0x7f06541a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f065abb8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0654113370 con 0x7f0654107e10 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f065abb8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06541131f0 con 0x7f0654108760 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f065abb8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06541134f0 con 0x7f06541034e0 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f0653fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 0x7f065419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f0653fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 0x7f065419c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48034/0 (socket says 172.21.15.19:48034) 2024-05-09T01:29:18.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f0653fff700 1 -- 172.21.15.19:0/2394173425 learned_addr learned my addr 172.21.15.19:0/2394173425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f0658954700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541034e0 0x7f065419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.430+0000 7f0659155700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0654108760 0x7f06541a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0653fff700 1 -- 172.21.15.19:0/2394173425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541034e0 msgr2=0x7f065419bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0653fff700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541034e0 0x7f065419bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0653fff700 1 -- 172.21.15.19:0/2394173425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0654108760 msgr2=0x7f06541a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0653fff700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0654108760 0x7f06541a6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0653fff700 1 -- 172.21.15.19:0/2394173425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065419cac0 con 0x7f0654107e10 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0659155700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0654108760 0x7f06541a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0653fff700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 0x7f065419c210 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0640006010 tx=0x7f06400055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0651ffb700 1 -- 172.21.15.19:0/2394173425 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640026070 con 0x7f0654107e10 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0651ffb700 1 -- 172.21.15.19:0/2394173425 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f064000f460 con 0x7f0654107e10 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f065abb8700 1 -- 172.21.15.19:0/2394173425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0654199ac0 con 0x7f0654107e10 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f0651ffb700 1 -- 172.21.15.19:0/2394173425 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640009e60 con 0x7f0654107e10 2024-05-09T01:29:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.431+0000 7f065abb8700 1 -- 172.21.15.19:0/2394173425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0654199fa0 con 0x7f0654107e10 2024-05-09T01:29:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.434+0000 7f0651ffb700 1 -- 172.21.15.19:0/2394173425 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0640029070 con 0x7f0654107e10 2024-05-09T01:29:18.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.435+0000 7f0651ffb700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06440b2900 0x7f06440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.435+0000 7f0658954700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06440b2900 0x7f06440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.435+0000 7f0651ffb700 1 -- 172.21.15.19:0/2394173425 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f06400e2810 con 0x7f0654107e10 2024-05-09T01:29:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.435+0000 7f06695ceb80 1 Processor -- start 2024-05-09T01:29:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.435+0000 7f06695ceb80 1 -- start start 2024-05-09T01:29:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.435+0000 7f0658954700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06440b2900 0x7f06440b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0648005950 tx=0x7f064800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541ac570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 0x55ff541a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f06695ceb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff540c9070 con 0x55ff54029520 2024-05-09T01:29:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f06695ceb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ff54067870 con 0x55ff541131e0 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f06695ceb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff540c5030 con 0x55ff541a6720 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0653fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0653fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48040/0 (socket says 172.21.15.19:48040) 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0653fff700 1 -- 172.21.15.19:0/1393654786 learned_addr learned my addr 172.21.15.19:0/1393654786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0659155700 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 0x55ff541a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0658954700 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0653fff700 1 -- 172.21.15.19:0/1393654786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 msgr2=0x55ff541a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0653fff700 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 0x55ff541a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0653fff700 1 -- 172.21.15.19:0/1393654786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 msgr2=0x55ff541ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0653fff700 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541ac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0653fff700 1 -- 172.21.15.19:0/1393654786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5405f3f0 con 0x55ff54029520 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0658954700 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.436+0000 7f0653fff700 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541acab0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f06400a9e30 tx=0x7f06400a97f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8bfd0aa0f4dcba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.437+0000 7f063effd700 1 -- 172.21.15.19:0/1393654786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400aa5a0 con 0x55ff54029520 2024-05-09T01:29:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.437+0000 7f063effd700 1 -- 172.21.15.19:0/1393654786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06400a9ea0 con 0x55ff54029520 2024-05-09T01:29:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.437+0000 7f063effd700 1 -- 172.21.15.19:0/1393654786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400aa320 con 0x55ff54029520 2024-05-09T01:29:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.437+0000 7f06695ceb80 1 -- 172.21.15.19:0/1393654786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 msgr2=0x55ff541acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.437+0000 7f06695ceb80 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541acab0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f06400a9e30 tx=0x7f06400a97f0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.438+0000 7f06695ceb80 1 -- 172.21.15.19:0/1393654786 shutdown_connections 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.438+0000 7f06695ceb80 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 0x55ff541a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.438+0000 7f06695ceb80 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.438+0000 7f06695ceb80 1 --2- 172.21.15.19:0/1393654786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541acab0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.438+0000 7f06695ceb80 1 -- 172.21.15.19:0/1393654786 >> 172.21.15.19:0/1393654786 conn(0x55ff54089720 msgr2=0x55ff541a1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.438+0000 7f06695ceb80 1 -- 172.21.15.19:0/1393654786 shutdown_connections 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.438+0000 7f06695ceb80 1 -- 172.21.15.19:0/1393654786 wait complete. 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.438+0000 7f06695ceb80 1 -- start start 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.438+0000 7f06695ceb80 1 Processor -- start 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541131e0 0x55ff541b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541a6720 0x55ff541b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f06695ceb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff53f44c20 con 0x55ff54029520 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f06695ceb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ff540831d0 con 0x55ff541a6720 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f06695ceb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff541a73e0 con 0x55ff541131e0 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0658954700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f06695ceb80 10 client.0 _ll_register_callbacks cb 0x55ff53f3a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0658954700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48044/0 (socket says 172.21.15.19:48044) 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0658954700 1 -- 172.21.15.19:0/100640676 learned_addr learned my addr 172.21.15.19:0/100640676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0653fff700 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541131e0 0x55ff541b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0659155700 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541a6720 0x55ff541b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0658954700 1 -- 172.21.15.19:0/100640676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541131e0 msgr2=0x55ff541b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0658954700 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541131e0 0x55ff541b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0658954700 1 -- 172.21.15.19:0/100640676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541a6720 msgr2=0x55ff541b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0658954700 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541a6720 0x55ff541b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0658954700 1 -- 172.21.15.19:0/100640676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff53f44510 con 0x55ff54029520 2024-05-09T01:29:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0659155700 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541a6720 0x55ff541b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0658954700 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541b5dc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0648005d90 tx=0x7f06480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb3e752fb610825 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.439+0000 7f0633fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f0633fff700 1 -- 172.21.15.19:0/100640676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648010ea0 con 0x55ff54029520 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f0633fff700 1 -- 172.21.15.19:0/100640676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f064801e830 con 0x55ff54029520 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f06695ceb80 1 -- 172.21.15.19:0/100640676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06400a9ea0 con 0x55ff54029520 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f0633fff700 1 -- 172.21.15.19:0/100640676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f064801ea70 con 0x55ff54029520 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f06695ceb80 10 client.16260 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f06695ceb80 1 -- 172.21.15.19:0/100640676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff541bd9b0 con 0x55ff54029520 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f0633fff700 1 -- 172.21.15.19:0/100640676 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06400a9ea0 con 0x55ff54029520 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f0633fff700 1 -- 172.21.15.19:0/100640676 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0648027ca0 con 0x55ff54029520 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f06695ceb80 10 client.16260 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f06695ceb80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f06695ceb80 20 client.16260 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f06695ceb80 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ff541bfff0 0x55ff541c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f06695ceb80 4 client.16260 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.440+0000 7f06695ceb80 1 -- 172.21.15.19:0/100640676 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff540d6a50 con 0x55ff541bfff0 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.441+0000 7f0653fff700 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ff541bfff0 0x55ff541c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.441+0000 7f0653fff700 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ff541bfff0 0x55ff541c2410 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df967934a18cb89 server_cookie=b162ffaf6f777a60 in_seq=0 out_seq=0 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.441+0000 7f0633fff700 10 client.16260 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f0633fff700 1 -- 172.21.15.19:0/100640676 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff540d6a50 con 0x55ff541bfff0 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f0633fff700 10 client.16260 handle_command_reply: tid=0 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 client.16260 shutdown 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 -- 172.21.15.19:0/100640676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 msgr2=0x55ff541b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541b5dc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0648005d90 tx=0x7f06480047c0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 -- 172.21.15.19:0/100640676 shutdown_connections 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ff541bfff0 0x55ff541c2410 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541a6720 0x55ff541b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54029520 0x55ff541b5dc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 --2- 172.21.15.19:0/100640676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541131e0 0x55ff541b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 -- 172.21.15.19:0/100640676 >> 172.21.15.19:0/100640676 conn(0x55ff54089720 msgr2=0x55ff541a1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f0633fff700 0 client.16260 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 -- 172.21.15.19:0/100640676 shutdown_connections 2024-05-09T01:29:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 1 -- 172.21.15.19:0/100640676 wait complete. 2024-05-09T01:29:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.442+0000 7f06695ceb80 20 client.16260 trim_cache size 0 max 16384 2024-05-09T01:29:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.463+0000 7f06695ceb80 1 Processor -- start 2024-05-09T01:29:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.463+0000 7f06695ceb80 1 -- start start 2024-05-09T01:29:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.463+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff54029520 0x55ff541aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.463+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541131e0 0x55ff541a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 0x55ff541a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f06695ceb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06541acac0 con 0x55ff541131e0 2024-05-09T01:29:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f06695ceb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06541ae600 con 0x55ff54029520 2024-05-09T01:29:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f06695ceb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06541ad190 con 0x55ff541a6720 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0653fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541131e0 0x55ff541a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0653fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541131e0 0x55ff541a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48046/0 (socket says 172.21.15.19:48046) 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0653fff700 1 -- 172.21.15.19:0/1367687976 learned_addr learned my addr 172.21.15.19:0/1367687976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0658954700 1 --2- 172.21.15.19:0/1367687976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff54029520 0x55ff541aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0659155700 1 --2- 172.21.15.19:0/1367687976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 0x55ff541a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0653fff700 1 -- 172.21.15.19:0/1367687976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 msgr2=0x55ff541a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0653fff700 1 --2- 172.21.15.19:0/1367687976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 0x55ff541a8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0653fff700 1 -- 172.21.15.19:0/1367687976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff54029520 msgr2=0x55ff541aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0653fff700 1 --2- 172.21.15.19:0/1367687976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff54029520 0x55ff541aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0653fff700 1 -- 172.21.15.19:0/1367687976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5419afe0 con 0x55ff541131e0 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f0653fff700 1 --2- 172.21.15.19:0/1367687976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541131e0 0x55ff541a5e20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0640009990 tx=0x7f06400095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d62cf428ece3e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f063e7fc700 1 -- 172.21.15.19:0/1367687976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400aa5a0 con 0x55ff541131e0 2024-05-09T01:29:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f063e7fc700 1 -- 172.21.15.19:0/1367687976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06400a6490 con 0x55ff541131e0 2024-05-09T01:29:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.464+0000 7f063e7fc700 1 -- 172.21.15.19:0/1367687976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400b2a40 con 0x55ff541131e0 2024-05-09T01:29:18.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.465+0000 7f06695ceb80 1 -- 172.21.15.19:0/1367687976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541131e0 msgr2=0x55ff541a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:18.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.465+0000 7f06695ceb80 1 --2- 172.21.15.19:0/1367687976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541131e0 0x55ff541a5e20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0640009990 tx=0x7f06400095f0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.465+0000 7f06695ceb80 1 -- 172.21.15.19:0/1367687976 shutdown_connections 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.465+0000 7f06695ceb80 1 --2- 172.21.15.19:0/1367687976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff541a6720 0x55ff541a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.465+0000 7f06695ceb80 1 --2- 172.21.15.19:0/1367687976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff54029520 0x55ff541aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.465+0000 7f06695ceb80 1 --2- 172.21.15.19:0/1367687976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541131e0 0x55ff541a5e20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.465+0000 7f06695ceb80 1 -- 172.21.15.19:0/1367687976 >> 172.21.15.19:0/1367687976 conn(0x55ff54089720 msgr2=0x55ff541a5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.465+0000 7f06695ceb80 1 -- 172.21.15.19:0/1367687976 shutdown_connections 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.465+0000 7f06695ceb80 1 -- 172.21.15.19:0/1367687976 wait complete. 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f06695ceb80 1 -- start start 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f06695ceb80 1 Processor -- start 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff54029520 0x55ff541b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f06695ceb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541a6720 0x55ff541b8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f06695ceb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06541adf30 con 0x55ff541a6720 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f06695ceb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06541ad860 con 0x55ff541131e0 2024-05-09T01:29:18.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f06695ceb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06541aaa60 con 0x55ff54029520 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f06695ceb80 10 client.0 _ll_register_callbacks cb 0x55ff53f3a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f0659155700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541a6720 0x55ff541b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f0659155700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541a6720 0x55ff541b8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48062/0 (socket says 172.21.15.19:48062) 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f0659155700 1 -- 172.21.15.19:0/931305765 learned_addr learned my addr 172.21.15.19:0/931305765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f0653fff700 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.466+0000 7f0658954700 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff54029520 0x55ff541b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0659155700 1 -- 172.21.15.19:0/931305765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff54029520 msgr2=0x55ff541b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0659155700 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff54029520 0x55ff541b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0659155700 1 -- 172.21.15.19:0/931305765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 msgr2=0x55ff541b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0659155700 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0659155700 1 -- 172.21.15.19:0/931305765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff542297a0 con 0x55ff541a6720 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0658954700 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff54029520 0x55ff541b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:18.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0653fff700 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0659155700 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541a6720 0x55ff541b8850 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f064c0077b0 tx=0x7f064c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e7f4f2a567f21a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0633fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0633fff700 1 -- 172.21.15.19:0/931305765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c020070 con 0x55ff541a6720 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0633fff700 1 -- 172.21.15.19:0/931305765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f064c004db0 con 0x55ff541a6720 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f06695ceb80 1 -- 172.21.15.19:0/931305765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06400a6490 con 0x55ff541a6720 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f0633fff700 1 -- 172.21.15.19:0/931305765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f064c02b050 con 0x55ff541a6720 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f06695ceb80 10 client.16266 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.467+0000 7f06695ceb80 1 -- 172.21.15.19:0/931305765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff5422a9e0 con 0x55ff541a6720 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f0633fff700 1 -- 172.21.15.19:0/931305765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06400a6490 con 0x55ff541a6720 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f0633fff700 1 -- 172.21.15.19:0/931305765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f064c005320 con 0x55ff541a6720 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f06695ceb80 10 client.16266 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f06695ceb80 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f06695ceb80 20 client.16266 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:18.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f06695ceb80 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ff5422b0f0 0x55ff5422d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f06695ceb80 4 client.16266 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f06695ceb80 1 -- 172.21.15.19:0/931305765 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ff540d0af0 con 0x55ff5422b0f0 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f0658954700 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ff5422b0f0 0x55ff5422d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f0633fff700 10 client.16266 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.468+0000 7f0658954700 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ff5422b0f0 0x55ff5422d510 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321e561c82d63072 server_cookie=16ec27b3f894ef32 in_seq=0 out_seq=0 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f0633fff700 1 -- 172.21.15.19:0/931305765 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ff540d0af0 con 0x55ff5422b0f0 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f0633fff700 10 client.16266 handle_command_reply: tid=0 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 client.16266 shutdown 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 -- 172.21.15.19:0/931305765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541a6720 msgr2=0x55ff541b8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541a6720 0x55ff541b8850 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f064c0077b0 tx=0x7f064c009920 comp rx=0 tx=0).stop 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 -- 172.21.15.19:0/931305765 shutdown_connections 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ff5422b0f0 0x55ff5422d510 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f0633fff700 0 client.16266 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff541a6720 0x55ff541b8850 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff54029520 0x55ff541b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 --2- 172.21.15.19:0/931305765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff541131e0 0x55ff541b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 -- 172.21.15.19:0/931305765 >> 172.21.15.19:0/931305765 conn(0x55ff54089720 msgr2=0x55ff541a5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 -- 172.21.15.19:0/931305765 shutdown_connections 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 1 -- 172.21.15.19:0/931305765 wait complete. 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.470+0000 7f06695ceb80 20 client.16266 trim_cache size 0 max 16384 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "baa9e0b8-9786-4e98-a108-62ff11bb1db3", 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:29:18.472 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:29:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 -- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06440b2900 msgr2=0x7f06440b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06440b2900 0x7f06440b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0648005950 tx=0x7f064800b410 comp rx=0 tx=0).stop 2024-05-09T01:29:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 -- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 msgr2=0x7f065419c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 0x7f065419c210 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0640006010 tx=0x7f06400055f0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 -- 172.21.15.19:0/2394173425 shutdown_connections 2024-05-09T01:29:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0654108760 0x7f06541a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541034e0 0x7f065419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f06440b2900 0x7f06440b4e10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 --2- 172.21.15.19:0/2394173425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0654107e10 0x7f065419c210 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 -- 172.21.15.19:0/2394173425 >> 172.21.15.19:0/2394173425 conn(0x7f06540fef30 msgr2=0x7f06541096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 -- 172.21.15.19:0/2394173425 shutdown_connections 2024-05-09T01:29:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:18.471+0000 7f065abb8700 1 -- 172.21.15.19:0/2394173425 wait complete. 2024-05-09T01:29:18.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:18 smithi059 ceph-mon[34351]: pgmap v491: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:29:18.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:18 smithi196 ceph-mon[35041]: pgmap v491: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:29:19.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:29:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:19.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:19 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:29:19.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:19 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:29:19.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:19 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:29:19.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:19 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:29:19.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:19 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:29:19.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:19 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:29:19.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:19 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:29:19.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:19 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:29:19.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:19 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:29:19.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:19 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:29:19.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:19 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:29:19.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:19 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:29:20.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:20 smithi059 ceph-mon[34351]: pgmap v492: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:20.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:20 smithi019 ceph-mon[30158]: pgmap v492: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:20.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:20 smithi196 ceph-mon[35041]: pgmap v492: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.304+0000 7fd22ed0a700 1 -- 172.21.15.19:0/4060126080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 msgr2=0x7fd228108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.304+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/4060126080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 0x7fd228108220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd2100099c0 tx=0x7fd210009cd0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.305+0000 7fd22ed0a700 1 -- 172.21.15.19:0/4060126080 shutdown_connections 2024-05-09T01:29:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.305+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/4060126080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd228108760 0x7fd2281100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.305+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/4060126080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2281034e0 0x7fd2281038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.305+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/4060126080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 0x7fd228108220 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.305+0000 7fd22ed0a700 1 -- 172.21.15.19:0/4060126080 >> 172.21.15.19:0/4060126080 conn(0x7fd2280fef50 msgr2=0x7fd228101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:21.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.305+0000 7fd22ed0a700 1 -- 172.21.15.19:0/4060126080 shutdown_connections 2024-05-09T01:29:21.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.305+0000 7fd22ed0a700 1 -- 172.21.15.19:0/4060126080 wait complete. 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.306+0000 7fd22ed0a700 1 Processor -- start 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.306+0000 7fd22ed0a700 1 -- start start 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.306+0000 7fd22ed0a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2281034e0 0x7fd22819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.306+0000 7fd22ed0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 0x7fd22819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.306+0000 7fd22ed0a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd228108760 0x7fd22819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.306+0000 7fd22ed0a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd228113370 con 0x7fd228107e10 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.306+0000 7fd22ed0a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281131f0 con 0x7fd228108760 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.306+0000 7fd22ed0a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281134f0 con 0x7fd2281034e0 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.306+0000 7fd227fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 0x7fd22819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd227fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 0x7fd22819c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42380/0 (socket says 172.21.15.19:42380) 2024-05-09T01:29:21.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd227fff700 1 -- 172.21.15.19:0/3214305038 learned_addr learned my addr 172.21.15.19:0/3214305038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd22d2a7700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd228108760 0x7fd22819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd22caa6700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2281034e0 0x7fd22819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd227fff700 1 -- 172.21.15.19:0/3214305038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2281034e0 msgr2=0x7fd22819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd227fff700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2281034e0 0x7fd22819bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd227fff700 1 -- 172.21.15.19:0/3214305038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd228108760 msgr2=0x7fd22819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd227fff700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd228108760 0x7fd22819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd227fff700 1 -- 172.21.15.19:0/3214305038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2281a6fb0 con 0x7fd228107e10 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd227fff700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 0x7fd22819c230 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd2100098e0 tx=0x7fd2100057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd225ffb700 1 -- 172.21.15.19:0/3214305038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210027070 con 0x7fd228107e10 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd22ed0a700 1 -- 172.21.15.19:0/3214305038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2281a7240 con 0x7fd228107e10 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd225ffb700 1 -- 172.21.15.19:0/3214305038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd210004020 con 0x7fd228107e10 2024-05-09T01:29:21.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.307+0000 7fd225ffb700 1 -- 172.21.15.19:0/3214305038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210022e30 con 0x7fd228107e10 2024-05-09T01:29:21.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.308+0000 7fd22ed0a700 1 -- 172.21.15.19:0/3214305038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2281a76f0 con 0x7fd228107e10 2024-05-09T01:29:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.311+0000 7fd225ffb700 1 -- 172.21.15.19:0/3214305038 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd21002a070 con 0x7fd228107e10 2024-05-09T01:29:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd225ffb700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd2140b2880 0x7fd2140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd22caa6700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd2140b2880 0x7fd2140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd23d720b80 1 Processor -- start 2024-05-09T01:29:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd225ffb700 1 -- 172.21.15.19:0/3214305038 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd2100e2ea0 con 0x7fd228107e10 2024-05-09T01:29:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd23d720b80 1 -- start start 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5017b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa501776a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa50178150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd23d720b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aa50099ff0 con 0x55aa50177c80 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd23d720b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55aa500387f0 con 0x55aa4fffa4a0 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.312+0000 7fd23d720b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aa50095fb0 con 0x55aa500e4160 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd22caa6700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd2140b2880 0x7fd2140b4d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd218005ea0 tx=0x7fd218005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd22d2a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa50178150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd227fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa501776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd22d2a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa50178150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42388/0 (socket says 172.21.15.19:42388) 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd227fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa501776a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49634/0 (socket says 172.21.15.19:49634) 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd227fff700 1 -- 172.21.15.19:0/3152192917 learned_addr learned my addr 172.21.15.19:0/3152192917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd22caa6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5017b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd227fff700 1 -- 172.21.15.19:0/3152192917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 msgr2=0x55aa5017b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd227fff700 1 --2- 172.21.15.19:0/3152192917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5017b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd227fff700 1 -- 172.21.15.19:0/3152192917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 msgr2=0x55aa50178150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd227fff700 1 --2- 172.21.15.19:0/3152192917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa50178150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd227fff700 1 -- 172.21.15.19:0/3152192917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa50030370 con 0x55aa4fffa4a0 2024-05-09T01:29:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd22d2a7700 1 --2- 172.21.15.19:0/3152192917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa50178150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:29:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.313+0000 7fd227fff700 1 --2- 172.21.15.19:0/3152192917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa501776a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd2100a65b0 tx=0x7fd210009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=996683e1cd212e39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.315+0000 7fd21effd700 1 -- 172.21.15.19:0/3152192917 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210092110 con 0x55aa4fffa4a0 2024-05-09T01:29:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.315+0000 7fd21effd700 1 -- 172.21.15.19:0/3152192917 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2100aa530 con 0x55aa4fffa4a0 2024-05-09T01:29:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.315+0000 7fd21effd700 1 -- 172.21.15.19:0/3152192917 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2100aa790 con 0x55aa4fffa4a0 2024-05-09T01:29:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.316+0000 7fd23d720b80 1 -- 172.21.15.19:0/3152192917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 msgr2=0x55aa501776a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.316+0000 7fd23d720b80 1 --2- 172.21.15.19:0/3152192917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa501776a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd2100a65b0 tx=0x7fd210009310 comp rx=0 tx=0).stop 2024-05-09T01:29:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.316+0000 7fd23d720b80 1 -- 172.21.15.19:0/3152192917 shutdown_connections 2024-05-09T01:29:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.316+0000 7fd23d720b80 1 --2- 172.21.15.19:0/3152192917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa50178150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.316+0000 7fd23d720b80 1 --2- 172.21.15.19:0/3152192917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5017b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.316+0000 7fd23d720b80 1 --2- 172.21.15.19:0/3152192917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa501776a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.316+0000 7fd23d720b80 1 -- 172.21.15.19:0/3152192917 >> 172.21.15.19:0/3152192917 conn(0x55aa5005a6a0 msgr2=0x55aa50172400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.316+0000 7fd23d720b80 1 -- 172.21.15.19:0/3152192917 shutdown_connections 2024-05-09T01:29:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.316+0000 7fd23d720b80 1 -- 172.21.15.19:0/3152192917 wait complete. 2024-05-09T01:29:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd23d720b80 1 -- start start 2024-05-09T01:29:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd23d720b80 1 Processor -- start 2024-05-09T01:29:21.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa4fffa4a0 0x55aa50189420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa500e4160 0x55aa50189960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 0x55aa50189ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd23d720b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55aa4ff15ba0 con 0x55aa500e4160 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd23d720b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55aa50054150 con 0x55aa50177c80 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd23d720b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aa5018c610 con 0x55aa4fffa4a0 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd23d720b80 10 client.0 _ll_register_callbacks cb 0x55aa4ff0b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.317+0000 7fd227fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa500e4160 0x55aa50189960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd227fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa500e4160 0x55aa50189960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42402/0 (socket says 172.21.15.19:42402) 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd227fff700 1 -- 172.21.15.19:0/722857604 learned_addr learned my addr 172.21.15.19:0/722857604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd22d2a7700 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 0x55aa50189ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd22caa6700 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa4fffa4a0 0x55aa50189420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd227fff700 1 -- 172.21.15.19:0/722857604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa4fffa4a0 msgr2=0x55aa50189420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd227fff700 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa4fffa4a0 0x55aa50189420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd227fff700 1 -- 172.21.15.19:0/722857604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 msgr2=0x55aa50189ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd227fff700 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 0x55aa50189ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd227fff700 1 -- 172.21.15.19:0/722857604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa4ff15490 con 0x55aa500e4160 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd227fff700 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa500e4160 0x55aa50189960 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd2100e09d0 tx=0x7fd210009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8cd35290dbe0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd207fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd207fff700 1 -- 172.21.15.19:0/722857604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2100c4060 con 0x55aa500e4160 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd207fff700 1 -- 172.21.15.19:0/722857604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd2100a6b20 con 0x55aa500e4160 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd23d720b80 1 -- 172.21.15.19:0/722857604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2100aa530 con 0x55aa500e4160 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd207fff700 1 -- 172.21.15.19:0/722857604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd2100b3300 con 0x55aa500e4160 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd23d720b80 10 client.16278 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.318+0000 7fd23d720b80 1 -- 172.21.15.19:0/722857604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa50197020 con 0x55aa500e4160 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.319+0000 7fd207fff700 1 -- 172.21.15.19:0/722857604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2100aa530 con 0x55aa500e4160 2024-05-09T01:29:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.319+0000 7fd207fff700 1 -- 172.21.15.19:0/722857604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd2100cb200 con 0x55aa500e4160 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.319+0000 7fd23d720b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.319+0000 7fd23d720b80 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.319+0000 7fd23d720b80 20 client.16278 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.319+0000 7fd23d720b80 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa50199320 0x55aa5019b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.319+0000 7fd23d720b80 4 client.16278 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.319+0000 7fd23d720b80 1 -- 172.21.15.19:0/722857604 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa500a79d0 con 0x55aa50199320 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.319+0000 7fd22caa6700 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa50199320 0x55aa5019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.320+0000 7fd22caa6700 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa50199320 0x55aa5019b740 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a9ce669fb7d6bd server_cookie=7c3470593ca80e05 in_seq=0 out_seq=0 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.320+0000 7fd207fff700 10 client.16278 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.320+0000 7fd207fff700 1 -- 172.21.15.19:0/722857604 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa500a79d0 con 0x55aa50199320 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.320+0000 7fd207fff700 10 client.16278 handle_command_reply: tid=0 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.320+0000 7fd23d720b80 1 client.16278 shutdown 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 -- 172.21.15.19:0/722857604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa500e4160 msgr2=0x55aa50189960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa500e4160 0x55aa50189960 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd2100e09d0 tx=0x7fd210009710 comp rx=0 tx=0).stop 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 -- 172.21.15.19:0/722857604 shutdown_connections 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa50199320 0x55aa5019b740 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd207fff700 0 client.16278 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 0x55aa50189ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa4fffa4a0 0x55aa50189420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 --2- 172.21.15.19:0/722857604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa500e4160 0x55aa50189960 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 -- 172.21.15.19:0/722857604 >> 172.21.15.19:0/722857604 conn(0x55aa5005a6a0 msgr2=0x55aa501721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 -- 172.21.15.19:0/722857604 shutdown_connections 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 1 -- 172.21.15.19:0/722857604 wait complete. 2024-05-09T01:29:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.321+0000 7fd23d720b80 20 client.16278 trim_cache size 0 max 16384 2024-05-09T01:29:21.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd23d720b80 1 Processor -- start 2024-05-09T01:29:21.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd23d720b80 1 -- start start 2024-05-09T01:29:21.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa50189990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5018afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa5018b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd23d720b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281aca40 con 0x55aa50177c80 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd23d720b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281ae580 con 0x55aa4fffa4a0 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd23d720b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281ad110 con 0x55aa500e4160 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd22d2a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa5018b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd22d2a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa5018b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42414/0 (socket says 172.21.15.19:42414) 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.342+0000 7fd22d2a7700 1 -- 172.21.15.19:0/4232908975 learned_addr learned my addr 172.21.15.19:0/4232908975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd22caa6700 1 --2- 172.21.15.19:0/4232908975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa50189990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd227fff700 1 --2- 172.21.15.19:0/4232908975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5018afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd22d2a7700 1 -- 172.21.15.19:0/4232908975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 msgr2=0x55aa5018afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd22d2a7700 1 --2- 172.21.15.19:0/4232908975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5018afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd22d2a7700 1 -- 172.21.15.19:0/4232908975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 msgr2=0x55aa50189990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd22d2a7700 1 --2- 172.21.15.19:0/4232908975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa50189990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd22d2a7700 1 -- 172.21.15.19:0/4232908975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa5017b070 con 0x55aa50177c80 2024-05-09T01:29:21.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd22d2a7700 1 --2- 172.21.15.19:0/4232908975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa5018b500 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd22000a970 tx=0x7fd2200078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42532f523d43ee55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:21.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd21e7fc700 1 -- 172.21.15.19:0/4232908975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220015740 con 0x55aa50177c80 2024-05-09T01:29:21.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd21e7fc700 1 -- 172.21.15.19:0/4232908975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd220015d90 con 0x55aa50177c80 2024-05-09T01:29:21.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.343+0000 7fd21e7fc700 1 -- 172.21.15.19:0/4232908975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220020e00 con 0x55aa50177c80 2024-05-09T01:29:21.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.344+0000 7fd23d720b80 1 -- 172.21.15.19:0/4232908975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 msgr2=0x55aa5018b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:21.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.344+0000 7fd23d720b80 1 --2- 172.21.15.19:0/4232908975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa5018b500 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd22000a970 tx=0x7fd2200078d0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.344+0000 7fd23d720b80 1 -- 172.21.15.19:0/4232908975 shutdown_connections 2024-05-09T01:29:21.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.344+0000 7fd23d720b80 1 --2- 172.21.15.19:0/4232908975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa50177c80 0x55aa5018b500 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.344+0000 7fd23d720b80 1 --2- 172.21.15.19:0/4232908975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa4fffa4a0 0x55aa50189990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.344+0000 7fd23d720b80 1 --2- 172.21.15.19:0/4232908975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5018afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.344+0000 7fd23d720b80 1 -- 172.21.15.19:0/4232908975 >> 172.21.15.19:0/4232908975 conn(0x55aa5005a6a0 msgr2=0x55aa5018a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.344+0000 7fd23d720b80 1 -- 172.21.15.19:0/4232908975 shutdown_connections 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.344+0000 7fd23d720b80 1 -- 172.21.15.19:0/4232908975 wait complete. 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd23d720b80 1 -- start start 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd23d720b80 1 Processor -- start 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa4fffa4a0 0x55aa5018db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5018e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd23d720b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 0x55aa50202a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd23d720b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281adeb0 con 0x55aa4fffa4a0 2024-05-09T01:29:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd23d720b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281ad7e0 con 0x55aa50177c80 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd23d720b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281aaa00 con 0x55aa500e4160 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd23d720b80 10 client.0 _ll_register_callbacks cb 0x55aa4ff0b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd22caa6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa4fffa4a0 0x55aa5018db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd22caa6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa4fffa4a0 0x55aa5018db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42424/0 (socket says 172.21.15.19:42424) 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd22caa6700 1 -- 172.21.15.19:0/1787308272 learned_addr learned my addr 172.21.15.19:0/1787308272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd22d2a7700 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 0x55aa50202a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.345+0000 7fd227fff700 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5018e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd22caa6700 1 -- 172.21.15.19:0/1787308272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 msgr2=0x55aa5018e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd22caa6700 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5018e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd22caa6700 1 -- 172.21.15.19:0/1787308272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 msgr2=0x55aa50202a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd22caa6700 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 0x55aa50202a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd22caa6700 1 -- 172.21.15.19:0/1787308272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa50202f50 con 0x55aa4fffa4a0 2024-05-09T01:29:21.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd227fff700 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5018e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:21.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd22caa6700 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa4fffa4a0 0x55aa5018db10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55aa5019be00 tx=0x7fd21800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce25ac9f5e3ce88d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:21.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd207fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:21.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd207fff700 1 -- 172.21.15.19:0/1787308272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2180273a0 con 0x55aa4fffa4a0 2024-05-09T01:29:21.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd207fff700 1 -- 172.21.15.19:0/1787308272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd218027540 con 0x55aa4fffa4a0 2024-05-09T01:29:21.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd23d720b80 1 -- 172.21.15.19:0/1787308272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd220015d90 con 0x55aa4fffa4a0 2024-05-09T01:29:21.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd207fff700 1 -- 172.21.15.19:0/1787308272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd218036070 con 0x55aa4fffa4a0 2024-05-09T01:29:21.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd23d720b80 10 client.16284 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:21.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.346+0000 7fd23d720b80 1 -- 172.21.15.19:0/1787308272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa50204200 con 0x55aa4fffa4a0 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd207fff700 1 -- 172.21.15.19:0/1787308272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd220015d90 con 0x55aa4fffa4a0 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd207fff700 1 -- 172.21.15.19:0/1787308272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd218027ce0 con 0x55aa4fffa4a0 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd23d720b80 10 client.16284 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd23d720b80 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd23d720b80 20 client.16284 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd23d720b80 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa50199320 0x55aa50206920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd23d720b80 4 client.16284 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd23d720b80 1 -- 172.21.15.19:0/1787308272 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa500a1a70 con 0x55aa50199320 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd227fff700 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa50199320 0x55aa50206920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:21.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd207fff700 10 client.16284 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.347+0000 7fd227fff700 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa50199320 0x55aa50206920 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a2e347096320cb5 server_cookie=338e9cf1e189ec23 in_seq=0 out_seq=0 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd207fff700 1 -- 172.21.15.19:0/1787308272 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55aa500a1a70 con 0x55aa50199320 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd207fff700 10 client.16284 handle_command_reply: tid=0 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 client.16284 shutdown 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 -- 172.21.15.19:0/1787308272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa4fffa4a0 msgr2=0x55aa5018db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa4fffa4a0 0x55aa5018db10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55aa5019be00 tx=0x7fd21800af90 comp rx=0 tx=0).stop 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 -- 172.21.15.19:0/1787308272 shutdown_connections 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55aa50199320 0x55aa50206920 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aa50177c80 0x55aa50202a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55aa4fffa4a0 0x55aa5018db10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 --2- 172.21.15.19:0/1787308272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa500e4160 0x55aa5018e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 -- 172.21.15.19:0/1787308272 >> 172.21.15.19:0/1787308272 conn(0x55aa5005a6a0 msgr2=0x55aa5018b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd207fff700 0 client.16284 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:21.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 -- 172.21.15.19:0/1787308272 shutdown_connections 2024-05-09T01:29:21.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 1 -- 172.21.15.19:0/1787308272 wait complete. 2024-05-09T01:29:21.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.348+0000 7fd23d720b80 20 client.16284 trim_cache size 0 max 16384 2024-05-09T01:29:21.352 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:29:21.352 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T01:29:21.352 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T01:29:21.352 INFO:teuthology.orchestra.run.smithi019.stdout: "baa9e0b8-9786-4e98-a108-62ff11bb1db3": { 2024-05-09T01:29:21.352 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T01:29:21.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "baa9e0b8-9786-4e98-a108-62ff11bb1db3", 2024-05-09T01:29:21.353 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T01:29:21.353 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:29:21.353 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:29:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:29:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 -- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd2140b2880 msgr2=0x7fd2140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:21.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd2140b2880 0x7fd2140b4d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd218005ea0 tx=0x7fd218005e30 comp rx=0 tx=0).stop 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 -- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 msgr2=0x7fd22819c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 0x7fd22819c230 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd2100098e0 tx=0x7fd2100057c0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 -- 172.21.15.19:0/3214305038 shutdown_connections 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd228108760 0x7fd22819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2281034e0 0x7fd22819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd2140b2880 0x7fd2140b4d90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 --2- 172.21.15.19:0/3214305038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd228107e10 0x7fd22819c230 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 -- 172.21.15.19:0/3214305038 >> 172.21.15.19:0/3214305038 conn(0x7fd2280fef50 msgr2=0x7fd2281075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 -- 172.21.15.19:0/3214305038 shutdown_connections 2024-05-09T01:29:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:21.349+0000 7fd22ed0a700 1 -- 172.21.15.19:0/3214305038 wait complete. 2024-05-09T01:29:22.074 INFO:tasks.cephfs.filesystem:scrub status for tag:baa9e0b8-9786-4e98-a108-62ff11bb1db3 - {'path': '/', 'tag': 'baa9e0b8-9786-4e98-a108-62ff11bb1db3', 'options': 'recursive,force'} 2024-05-09T01:29:22.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:22 smithi059 ceph-mon[34351]: pgmap v493: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:29:22.597 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:22 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:29:22.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:22 smithi019 ceph-mon[30158]: pgmap v493: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:29:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:22 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:29:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:22 smithi196 ceph-mon[35041]: pgmap v493: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:29:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:22 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:29:24.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:24 smithi059 ceph-mon[34351]: pgmap v494: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:29:24.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:24 smithi019 ceph-mon[30158]: pgmap v494: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:29:24.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:24 smithi196 ceph-mon[35041]: pgmap v494: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.098+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/2212125028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 msgr2=0x7f2ed0103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.098+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/2212125028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 0x7f2ed0103830 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009900 tx=0x7f2ec002a410 comp rx=0 tx=0).stop 2024-05-09T01:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.098+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/2212125028 shutdown_connections 2024-05-09T01:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.098+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/2212125028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed0108a70 0x7f2ed010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.098+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/2212125028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ed0103d70 0x7f2ed0104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.098+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/2212125028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 0x7f2ed0103830 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.098+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/2212125028 >> 172.21.15.19:0/2212125028 conn(0x7f2ed00fdd00 msgr2=0x7f2ed01001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.098+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/2212125028 shutdown_connections 2024-05-09T01:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.098+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/2212125028 wait complete. 2024-05-09T01:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.099+0000 7f2ed5dd5640 1 Processor -- start 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.099+0000 7f2ed5dd5640 1 -- start start 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.099+0000 7f2ed5dd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 0x7f2ed019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.099+0000 7f2ed5dd5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ed0103d70 0x7f2ed019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ed5dd5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed0108a70 0x7f2ed01a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ed5dd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed010e610 con 0x7f2ed0102b30 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ed5dd5640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed010e490 con 0x7f2ed0108a70 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ed5dd5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed010e790 con 0x7f2ed0103d70 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 0x7f2ed019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 0x7f2ed019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42430/0 (socket says 172.21.15.19:42430) 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/490484510 learned_addr learned my addr 172.21.15.19:0/490484510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecffff640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed0108a70 0x7f2ed01a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ec7fff640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ed0103d70 0x7f2ed019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/490484510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ed0103d70 msgr2=0x7f2ed019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ed0103d70 0x7f2ed019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/490484510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed0108a70 msgr2=0x7f2ed01a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed0108a70 0x7f2ed01a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/490484510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed01a1ea0 con 0x7f2ed0102b30 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecffff640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed0108a70 0x7f2ed01a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 0x7f2ed019a360 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0006010 tx=0x7f2ec0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecd7fa640 1 -- 172.21.15.19:0/490484510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec003f030 con 0x7f2ed0102b30 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecd7fa640 1 -- 172.21.15.19:0/490484510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec0003aa0 con 0x7f2ed0102b30 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/490484510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed01a2130 con 0x7f2ed0102b30 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ecd7fa640 1 -- 172.21.15.19:0/490484510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0035070 con 0x7f2ed0102b30 2024-05-09T01:29:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.100+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/490484510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed01a2670 con 0x7f2ed0102b30 2024-05-09T01:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.103+0000 7f2edcd63740 1 Processor -- start 2024-05-09T01:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.103+0000 7f2edcd63740 1 -- start start 2024-05-09T01:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.103+0000 7f2ecd7fa640 1 -- 172.21.15.19:0/490484510 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2ec0042030 con 0x7f2ed0102b30 2024-05-09T01:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.103+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 0x56552161c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.103+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56552124b2f0 0x56552161ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.103+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 0x565521623d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.103+0000 7f2edcd63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56552157c7c0 con 0x56552124b2f0 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.103+0000 7f2edcd63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5655215180e0 con 0x5655212741b0 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.103+0000 7f2edcd63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565521578780 con 0x5655212534d0 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56552124b2f0 0x56552161ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56552124b2f0 0x56552161ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42442/0 (socket says 172.21.15.19:42442) 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/778714651 learned_addr learned my addr 172.21.15.19:0/778714651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecffff640 1 --2- 172.21.15.19:0/778714651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 0x56552161c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ec7fff640 1 --2- 172.21.15.19:0/778714651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 0x565521623d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/778714651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 msgr2=0x565521623d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/778714651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 0x565521623d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/778714651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 msgr2=0x56552161c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/778714651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 0x56552161c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/778714651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655215417c0 con 0x56552124b2f0 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/778714651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56552124b2f0 0x56552161ce90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0108cf0 tx=0x7f2ec0108de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf5ca300f85c181 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecd7fa640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ebc0b2860 0x7f2ebc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ec5ffb640 1 -- 172.21.15.19:0/778714651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00c6470 con 0x56552124b2f0 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ecffff640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ebc0b2860 0x7f2ebc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ec5ffb640 1 -- 172.21.15.19:0/778714651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec010f070 con 0x56552124b2f0 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.104+0000 7f2ec5ffb640 1 -- 172.21.15.19:0/778714651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00c9590 con 0x56552124b2f0 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.105+0000 7f2ecd7fa640 1 -- 172.21.15.19:0/490484510 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2ec0102820 con 0x7f2ed0102b30 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.105+0000 7f2edcd63740 1 -- 172.21.15.19:0/778714651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56552124b2f0 msgr2=0x56552161ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.105+0000 7f2ecffff640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ebc0b2860 0x7f2ebc0b4d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x56552161dae0 tx=0x7f2ec8009790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.105+0000 7f2edcd63740 1 --2- 172.21.15.19:0/778714651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56552124b2f0 0x56552161ce90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0108cf0 tx=0x7f2ec0108de0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.105+0000 7f2edcd63740 1 -- 172.21.15.19:0/778714651 shutdown_connections 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.105+0000 7f2edcd63740 1 --2- 172.21.15.19:0/778714651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 0x565521623d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.105+0000 7f2edcd63740 1 --2- 172.21.15.19:0/778714651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56552124b2f0 0x56552161ce90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.105+0000 7f2edcd63740 1 --2- 172.21.15.19:0/778714651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 0x56552161c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.105+0000 7f2edcd63740 1 -- 172.21.15.19:0/778714651 >> 172.21.15.19:0/778714651 conn(0x565521268300 msgr2=0x565521613e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 -- 172.21.15.19:0/778714651 shutdown_connections 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 -- 172.21.15.19:0/778714651 wait complete. 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 -- start start 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 Processor -- start 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56552124b2f0 0x565521626330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x565521626870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 0x56552162d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5655216245a0 con 0x5655212534d0 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5655215039c0 con 0x5655212741b0 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.106+0000 7f2edcd63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565521624720 con 0x56552124b2f0 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2edcd63740 10 client.0 _ll_register_callbacks cb 0x565521530020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ec7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x565521626870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ec7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x565521626870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42450/0 (socket says 172.21.15.19:42450) 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ec7fff640 1 -- 172.21.15.19:0/2027178753 learned_addr learned my addr 172.21.15.19:0/2027178753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ecffff640 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 0x56552162d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56552124b2f0 0x565521626330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ec7fff640 1 -- 172.21.15.19:0/2027178753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56552124b2f0 msgr2=0x565521626330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ec7fff640 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56552124b2f0 0x565521626330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ec7fff640 1 -- 172.21.15.19:0/2027178753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 msgr2=0x56552162d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ec7fff640 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 0x56552162d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ec7fff640 1 -- 172.21.15.19:0/2027178753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56552152a1f0 con 0x5655212534d0 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2ec7fff640 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x565521626870 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800f6a0 tx=0x7f2eb800faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2e6de1e5a92a36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2eaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2eaeffd640 1 -- 172.21.15.19:0/2027178753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8016070 con 0x5655212534d0 2024-05-09T01:29:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2eaeffd640 1 -- 172.21.15.19:0/2027178753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2eb8005220 con 0x5655212534d0 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.107+0000 7f2edcd63740 1 -- 172.21.15.19:0/2027178753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec010f070 con 0x5655212534d0 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2eaeffd640 1 -- 172.21.15.19:0/2027178753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2eb8005420 con 0x5655212534d0 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2edcd63740 10 client.16296 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2edcd63740 1 -- 172.21.15.19:0/2027178753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5655212df790 con 0x5655212534d0 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2eaeffd640 1 -- 172.21.15.19:0/2027178753 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec010f070 con 0x5655212534d0 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2eaeffd640 1 -- 172.21.15.19:0/2027178753 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2eb8004c90 con 0x5655212534d0 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2edcd63740 10 client.16296 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2edcd63740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2edcd63740 20 client.16296 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x565521232d80 0x565521633d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2edcd63740 4 client.16296 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2edcd63740 1 -- 172.21.15.19:0/2027178753 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56552158a3d0 con 0x565521232d80 2024-05-09T01:29:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.108+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x565521232d80 0x565521633d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.109+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x565521232d80 0x565521633d40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d14e091d70dc56 server_cookie=3e003f330c6a6249 in_seq=0 out_seq=0 2024-05-09T01:29:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.109+0000 7f2eaeffd640 10 client.16296 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2eaeffd640 1 -- 172.21.15.19:0/2027178753 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56552158a3d0 con 0x565521232d80 2024-05-09T01:29:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2eaeffd640 10 client.16296 handle_command_reply: tid=0 2024-05-09T01:29:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 client.16296 shutdown 2024-05-09T01:29:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 -- 172.21.15.19:0/2027178753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 msgr2=0x565521626870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x565521626870 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800f6a0 tx=0x7f2eb800faa0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 -- 172.21.15.19:0/2027178753 shutdown_connections 2024-05-09T01:29:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x565521232d80 0x565521633d40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5655212741b0 0x56552162d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2eaeffd640 0 client.16296 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x565521626870 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2027178753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56552124b2f0 0x565521626330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 -- 172.21.15.19:0/2027178753 >> 172.21.15.19:0/2027178753 conn(0x565521268300 msgr2=0x565521612700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 -- 172.21.15.19:0/2027178753 shutdown_connections 2024-05-09T01:29:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 1 -- 172.21.15.19:0/2027178753 wait complete. 2024-05-09T01:29:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.110+0000 7f2edcd63740 20 client.16296 trim_cache size 0 max 16384 2024-05-09T01:29:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2edcd63740 1 Processor -- start 2024-05-09T01:29:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2edcd63740 1 -- start start 2024-05-09T01:29:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565521232d80 0x5655216137b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 0x565521613cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 0x56552162d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2edcd63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed01a9250 con 0x565521232d80 2024-05-09T01:29:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2edcd63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed01aad90 con 0x56552124b2f0 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2edcd63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed01a9920 con 0x5655212534d0 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565521232d80 0x5655216137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565521232d80 0x5655216137b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42456/0 (socket says 172.21.15.19:42456) 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/2678095694 learned_addr learned my addr 172.21.15.19:0/2678095694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ec7fff640 1 --2- 172.21.15.19:0/2678095694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 0x565521613cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ecffff640 1 --2- 172.21.15.19:0/2678095694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 0x56552162d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/2678095694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 msgr2=0x56552162d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/2678095694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 0x56552162d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/2678095694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 msgr2=0x565521613cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/2678095694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 0x565521613cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.126+0000 7f2ecf7fe640 1 -- 172.21.15.19:0/2678095694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655213565f0 con 0x565521232d80 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.127+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/2678095694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565521232d80 0x5655216137b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0031440 tx=0x7f2ec0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb1c507013a788 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.127+0000 7f2ec57fa640 1 -- 172.21.15.19:0/2678095694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0049040 con 0x565521232d80 2024-05-09T01:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.127+0000 7f2ec57fa640 1 -- 172.21.15.19:0/2678095694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec010f070 con 0x565521232d80 2024-05-09T01:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.127+0000 7f2ec57fa640 1 -- 172.21.15.19:0/2678095694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00c9770 con 0x565521232d80 2024-05-09T01:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.127+0000 7f2edcd63740 1 -- 172.21.15.19:0/2678095694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565521232d80 msgr2=0x5655216137b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.127+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2678095694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565521232d80 0x5655216137b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0031440 tx=0x7f2ec0009260 comp rx=0 tx=0).stop 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 -- 172.21.15.19:0/2678095694 shutdown_connections 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2678095694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655212534d0 0x56552162d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2678095694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 0x565521613cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 --2- 172.21.15.19:0/2678095694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565521232d80 0x5655216137b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 -- 172.21.15.19:0/2678095694 >> 172.21.15.19:0/2678095694 conn(0x565521268300 msgr2=0x565521620a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 -- 172.21.15.19:0/2678095694 shutdown_connections 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 -- 172.21.15.19:0/2678095694 wait complete. 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 -- start start 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 Processor -- start 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.128+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565521232d80 0x56552162b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 0x56552162bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2edcd63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x56552162c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2edcd63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed01aa6c0 con 0x5655212534d0 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2edcd63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed01a9ff0 con 0x56552124b2f0 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2edcd63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed01a7230 con 0x565521232d80 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2edcd63740 10 client.0 _ll_register_callbacks cb 0x7f2eb800d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x56552162c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x56552162c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42468/0 (socket says 172.21.15.19:42468) 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecffff640 1 -- 172.21.15.19:0/3102606768 learned_addr learned my addr 172.21.15.19:0/3102606768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ec7fff640 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 0x56552162bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565521232d80 0x56552162b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecffff640 1 -- 172.21.15.19:0/3102606768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565521232d80 msgr2=0x56552162b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecffff640 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565521232d80 0x56552162b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecffff640 1 -- 172.21.15.19:0/3102606768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 msgr2=0x56552162bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecffff640 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 0x56552162bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecffff640 1 -- 172.21.15.19:0/3102606768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565521540ed0 con 0x5655212534d0 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2ecffff640 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x56552162c9e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8012d10 tx=0x7f2ec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58486d3294fa8f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2eaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2eaeffd640 1 -- 172.21.15.19:0/3102606768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8004b20 con 0x5655212534d0 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2eaeffd640 1 -- 172.21.15.19:0/3102606768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2ec8004cc0 con 0x5655212534d0 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.129+0000 7f2edcd63740 1 -- 172.21.15.19:0/3102606768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec010f070 con 0x5655212534d0 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2eaeffd640 1 -- 172.21.15.19:0/3102606768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ec8005020 con 0x5655212534d0 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2edcd63740 10 client.16302 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2edcd63740 1 -- 172.21.15.19:0/3102606768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56552162e0d0 con 0x5655212534d0 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2eaeffd640 1 -- 172.21.15.19:0/3102606768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec010f070 con 0x5655212534d0 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2eaeffd640 1 -- 172.21.15.19:0/3102606768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2ec800f040 con 0x5655212534d0 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2edcd63740 10 client.16302 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2edcd63740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2edcd63740 20 client.16302 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2edcd63740 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5655212741b0 0x56552169e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2edcd63740 4 client.16302 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:29:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2edcd63740 1 -- 172.21.15.19:0/3102606768 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565521584240 con 0x5655212741b0 2024-05-09T01:29:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.130+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5655212741b0 0x56552169e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.131+0000 7f2eaeffd640 10 client.16302 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.131+0000 7f2ecf7fe640 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5655212741b0 0x56552169e780 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=518be79fa9f8ecd9 server_cookie=460d80f9a8e84313 in_seq=0 out_seq=0 2024-05-09T01:29:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.145+0000 7f2eaeffd640 1 -- 172.21.15.19:0/3102606768 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565521584240 con 0x5655212741b0 2024-05-09T01:29:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.145+0000 7f2eaeffd640 10 client.16302 handle_command_reply: tid=0 2024-05-09T01:29:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.145+0000 7f2edcd63740 1 client.16302 shutdown 2024-05-09T01:29:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 -- 172.21.15.19:0/3102606768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 msgr2=0x56552162c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x56552162c9e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8012d10 tx=0x7f2ec8004a40 comp rx=0 tx=0).stop 2024-05-09T01:29:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 -- 172.21.15.19:0/3102606768 shutdown_connections 2024-05-09T01:29:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5655212741b0 0x56552169e780 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5655212534d0 0x56552162c9e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56552124b2f0 0x56552162bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2eaeffd640 0 client.16302 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 --2- 172.21.15.19:0/3102606768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565521232d80 0x56552162b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 -- 172.21.15.19:0/3102606768 >> 172.21.15.19:0/3102606768 conn(0x565521268300 msgr2=0x5655216137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 -- 172.21.15.19:0/3102606768 shutdown_connections 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 1 -- 172.21.15.19:0/3102606768 wait complete. 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2edcd63740 20 client.16302 trim_cache size 0 max 16384 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:29:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:29:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:29:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ebc0b2860 msgr2=0x7f2ebc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ebc0b2860 0x7f2ebc0b4d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x56552161dae0 tx=0x7f2ec8009790 comp rx=0 tx=0).stop 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 msgr2=0x7f2ed019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 0x7f2ed019a360 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0006010 tx=0x7f2ec0031a50 comp rx=0 tx=0).stop 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/490484510 shutdown_connections 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2ebc0b2860 0x7f2ebc0b4d70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed0108a70 0x7f2ed01a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ed0103d70 0x7f2ed019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 --2- 172.21.15.19:0/490484510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ed0102b30 0x7f2ed019a360 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.146+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/490484510 >> 172.21.15.19:0/490484510 conn(0x7f2ed00fdd00 msgr2=0x7f2ed0109fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.147+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/490484510 shutdown_connections 2024-05-09T01:29:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:26.147+0000 7f2ed5dd5640 1 -- 172.21.15.19:0/490484510 wait complete. 2024-05-09T01:29:26.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:26 smithi059 ceph-mon[34351]: pgmap v495: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-05-09T01:29:26.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:26 smithi019 ceph-mon[30158]: pgmap v495: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-05-09T01:29:26.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:26 smithi196 ceph-mon[35041]: pgmap v495: 97 pgs: 97 active+clean; 1.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-05-09T01:29:28.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:28 smithi059 ceph-mon[34351]: pgmap v496: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:29:28.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:28 smithi019 ceph-mon[30158]: pgmap v496: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:29:28.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:28 smithi196 ceph-mon[35041]: pgmap v496: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:29:30.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:30 smithi059 ceph-mon[34351]: pgmap v497: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-05-09T01:29:30.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:30 smithi019 ceph-mon[30158]: pgmap v497: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-05-09T01:29:30.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:30 smithi196 ceph-mon[35041]: pgmap v497: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-05-09T01:29:32.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:32 smithi059 ceph-mon[34351]: pgmap v498: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:29:32.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:32 smithi019 ceph-mon[30158]: pgmap v498: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:29:32.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:32 smithi196 ceph-mon[35041]: pgmap v498: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:29:34.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:34 smithi059 ceph-mon[34351]: pgmap v499: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:29:34.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:34 smithi019 ceph-mon[30158]: pgmap v499: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:29:34.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:34 smithi196 ceph-mon[35041]: pgmap v499: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:29:36.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:36 smithi059 ceph-mon[34351]: pgmap v500: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:36.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:36 smithi019 ceph-mon[30158]: pgmap v500: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:36.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:36 smithi196 ceph-mon[35041]: pgmap v500: 97 pgs: 97 active+clean; 2.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:38.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:38 smithi059 ceph-mon[34351]: pgmap v501: 97 pgs: 97 active+clean; 2.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:29:38.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:38 smithi019 ceph-mon[30158]: pgmap v501: 97 pgs: 97 active+clean; 2.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:29:38.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:38 smithi196 ceph-mon[35041]: pgmap v501: 97 pgs: 97 active+clean; 2.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:29:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.261+0000 7fac95fc7640 1 -- 172.21.15.19:0/2654551437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 msgr2=0x7fac9010be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.261+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2654551437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 0x7fac9010be10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fac88009ca0 tx=0x7fac8802a8c0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.261+0000 7fac95fc7640 1 -- 172.21.15.19:0/2654551437 shutdown_connections 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.261+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2654551437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 0x7fac9010be10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.261+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2654551437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac901018b0 0x7fac90101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.261+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2654551437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac90105200 0x7fac90105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.261+0000 7fac95fc7640 1 -- 172.21.15.19:0/2654551437 >> 172.21.15.19:0/2654551437 conn(0x7fac900fd4d0 msgr2=0x7fac900ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.261+0000 7fac95fc7640 1 -- 172.21.15.19:0/2654551437 shutdown_connections 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.261+0000 7fac95fc7640 1 -- 172.21.15.19:0/2654551437 wait complete. 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac95fc7640 1 Processor -- start 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac95fc7640 1 -- start start 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac95fc7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac901018b0 0x7fac90104360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac95fc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 0x7fac90102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac95fc7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac90105200 0x7fac90102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac95fc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac9010f790 con 0x7fac901023b0 2024-05-09T01:29:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac95fc7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac9010f610 con 0x7fac90105200 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac95fc7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac9010f910 con 0x7fac901018b0 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 0x7fac90102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac8f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac901018b0 0x7fac90104360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 0x7fac90102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53050/0 (socket says 172.21.15.19:53050) 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac87fff640 1 -- 172.21.15.19:0/2277819272 learned_addr learned my addr 172.21.15.19:0/2277819272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac8ffff640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac90105200 0x7fac90102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac87fff640 1 -- 172.21.15.19:0/2277819272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac901018b0 msgr2=0x7fac90104360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac87fff640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac901018b0 0x7fac90104360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac87fff640 1 -- 172.21.15.19:0/2277819272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac90105200 msgr2=0x7fac90102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac87fff640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac90105200 0x7fac90102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.262+0000 7fac87fff640 1 -- 172.21.15.19:0/2277819272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac90103670 con 0x7fac901023b0 2024-05-09T01:29:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.263+0000 7fac87fff640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 0x7fac90102a50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fac7800e9f0 tx=0x7fac7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.263+0000 7fac8d7fa640 1 -- 172.21.15.19:0/2277819272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7800cd80 con 0x7fac901023b0 2024-05-09T01:29:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.263+0000 7fac8d7fa640 1 -- 172.21.15.19:0/2277819272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac780108e0 con 0x7fac901023b0 2024-05-09T01:29:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.263+0000 7fac8d7fa640 1 -- 172.21.15.19:0/2277819272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78018980 con 0x7fac901023b0 2024-05-09T01:29:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.263+0000 7fac95fc7640 1 -- 172.21.15.19:0/2277819272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac901a4430 con 0x7fac901023b0 2024-05-09T01:29:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.263+0000 7fac95fc7640 1 -- 172.21.15.19:0/2277819272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac901a48f0 con 0x7fac901023b0 2024-05-09T01:29:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac9ce73740 1 Processor -- start 2024-05-09T01:29:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac9ce73740 1 -- start start 2024-05-09T01:29:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac8d7fa640 1 -- 172.21.15.19:0/2277819272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fac78010430 con 0x7fac901023b0 2024-05-09T01:29:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628aa0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aa0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 0x555628aa7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac9ce73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555628a047c0 con 0x5556286fc1b0 2024-05-09T01:29:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac9ce73740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5556289a00e0 con 0x5556286db4d0 2024-05-09T01:29:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac9ce73740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555628a00780 con 0x5556286d32f0 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac8ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628aa0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac8ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628aa0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53062/0 (socket says 172.21.15.19:53062) 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac8ffff640 1 -- 172.21.15.19:0/574338201 learned_addr learned my addr 172.21.15.19:0/574338201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/574338201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aa0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.266+0000 7fac87fff640 1 --2- 172.21.15.19:0/574338201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 0x555628aa7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac8ffff640 1 -- 172.21.15.19:0/574338201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 msgr2=0x555628aa0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac8ffff640 1 --2- 172.21.15.19:0/574338201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aa0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac8ffff640 1 -- 172.21.15.19:0/574338201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 msgr2=0x555628aa7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac8ffff640 1 --2- 172.21.15.19:0/574338201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 0x555628aa7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac8ffff640 1 -- 172.21.15.19:0/574338201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556289c97c0 con 0x5556286fc1b0 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac8d7fa640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac7c0b2860 0x7fac7c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac7c0b2860 0x7fac7c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac8ffff640 1 --2- 172.21.15.19:0/574338201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628aa0500 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fac8802f390 tx=0x7fac8802f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb13ccef35b47f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac8d7fa640 1 -- 172.21.15.19:0/2277819272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fac780e25a0 con 0x7fac901023b0 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac85ffb640 1 -- 172.21.15.19:0/574338201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88031650 con 0x5556286fc1b0 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac85ffb640 1 -- 172.21.15.19:0/574338201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac88036040 con 0x5556286fc1b0 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.267+0000 7fac85ffb640 1 -- 172.21.15.19:0/574338201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac880046e0 con 0x5556286fc1b0 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac9ce73740 1 -- 172.21.15.19:0/574338201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 msgr2=0x555628aa0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac9ce73740 1 --2- 172.21.15.19:0/574338201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628aa0500 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fac8802f390 tx=0x7fac8802f3c0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac7c0b2860 0x7fac7c0b4d70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x555628aa18e0 tx=0x7fac8000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac9ce73740 1 -- 172.21.15.19:0/574338201 shutdown_connections 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac9ce73740 1 --2- 172.21.15.19:0/574338201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 0x555628aa7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac9ce73740 1 --2- 172.21.15.19:0/574338201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aa0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac9ce73740 1 --2- 172.21.15.19:0/574338201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628aa0500 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac9ce73740 1 -- 172.21.15.19:0/574338201 >> 172.21.15.19:0/574338201 conn(0x5556286f0300 msgr2=0x555628a9be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac9ce73740 1 -- 172.21.15.19:0/574338201 shutdown_connections 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.268+0000 7fac9ce73740 1 -- 172.21.15.19:0/574338201 wait complete. 2024-05-09T01:29:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.269+0000 7fac9ce73740 1 -- start start 2024-05-09T01:29:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.269+0000 7fac9ce73740 1 Processor -- start 2024-05-09T01:29:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.269+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286d32f0 0x555628aaa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.269+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286db4d0 0x555628aaa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.269+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 0x555628ab1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.269+0000 7fac9ce73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555628aa8370 con 0x5556286d32f0 2024-05-09T01:29:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.269+0000 7fac9ce73740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55562898b9c0 con 0x5556286fc1b0 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.269+0000 7fac9ce73740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555628aa84f0 con 0x5556286db4d0 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.269+0000 7fac9ce73740 10 client.0 _ll_register_callbacks cb 0x5556289b8020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286d32f0 0x555628aaa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286d32f0 0x555628aaa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53076/0 (socket says 172.21.15.19:53076) 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8f7fe640 1 -- 172.21.15.19:0/1893841837 learned_addr learned my addr 172.21.15.19:0/1893841837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8ffff640 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 0x555628ab1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac87fff640 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286db4d0 0x555628aaa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8f7fe640 1 -- 172.21.15.19:0/1893841837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286db4d0 msgr2=0x555628aaa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286db4d0 0x555628aaa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8f7fe640 1 -- 172.21.15.19:0/1893841837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 msgr2=0x555628ab1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 0x555628ab1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8f7fe640 1 -- 172.21.15.19:0/1893841837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556289b21f0 con 0x5556286d32f0 2024-05-09T01:29:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286d32f0 0x555628aaa2c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fac8000bfc0 tx=0x7fac80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32246a432121286 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac6effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac6effd640 1 -- 172.21.15.19:0/1893841837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8002c060 con 0x5556286d32f0 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac6effd640 1 -- 172.21.15.19:0/1893841837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fac80017030 con 0x5556286d32f0 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.270+0000 7fac9ce73740 1 -- 172.21.15.19:0/1893841837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac88036040 con 0x5556286d32f0 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac6effd640 1 -- 172.21.15.19:0/1893841837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac8001c600 con 0x5556286d32f0 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac9ce73740 10 client.16314 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac9ce73740 1 -- 172.21.15.19:0/1893841837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555628767790 con 0x5556286d32f0 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac6effd640 1 -- 172.21.15.19:0/1893841837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac88036040 con 0x5556286d32f0 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac6effd640 1 -- 172.21.15.19:0/1893841837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac80032890 con 0x5556286d32f0 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac9ce73740 10 client.16314 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac9ce73740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac9ce73740 20 client.16314 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5556286bad80 0x555628ab7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac9ce73740 4 client.16314 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac9ce73740 1 -- 172.21.15.19:0/1893841837 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555628a123d0 con 0x5556286bad80 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.271+0000 7fac87fff640 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5556286bad80 0x555628ab7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.272+0000 7fac87fff640 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5556286bad80 0x555628ab7ce0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a2807166b34e943 server_cookie=6cd275d0a72177a3 in_seq=0 out_seq=0 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.272+0000 7fac6effd640 10 client.16314 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac6effd640 1 -- 172.21.15.19:0/1893841837 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555628a123d0 con 0x5556286bad80 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac6effd640 10 client.16314 handle_command_reply: tid=0 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 client.16314 shutdown 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 -- 172.21.15.19:0/1893841837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286d32f0 msgr2=0x555628aaa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286d32f0 0x555628aaa2c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fac8000bfc0 tx=0x7fac80004730 comp rx=0 tx=0).stop 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 -- 172.21.15.19:0/1893841837 shutdown_connections 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5556286bad80 0x555628ab7ce0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 0x555628ab1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286db4d0 0x555628aaa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1893841837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286d32f0 0x555628aaa2c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac6effd640 0 client.16314 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 -- 172.21.15.19:0/1893841837 >> 172.21.15.19:0/1893841837 conn(0x5556286f0300 msgr2=0x555628a9a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 -- 172.21.15.19:0/1893841837 shutdown_connections 2024-05-09T01:29:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 1 -- 172.21.15.19:0/1893841837 wait complete. 2024-05-09T01:29:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.273+0000 7fac9ce73740 20 client.16314 trim_cache size 0 max 16384 2024-05-09T01:29:39.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac9ce73740 1 Processor -- start 2024-05-09T01:29:39.286 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac9ce73740 1 -- start start 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aa4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 0x555628a9bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628ab16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac9ce73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac901ad6a0 con 0x5556286fc1b0 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac9ce73740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac901af1e0 con 0x5556286db4d0 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac9ce73740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac901add70 con 0x5556286d32f0 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac8ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628ab16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac8ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628ab16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53084/0 (socket says 172.21.15.19:53084) 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.288+0000 7fac8ffff640 1 -- 172.21.15.19:0/1834689966 learned_addr learned my addr 172.21.15.19:0/1834689966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/1834689966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aa4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac87fff640 1 --2- 172.21.15.19:0/1834689966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 0x555628a9bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac8ffff640 1 -- 172.21.15.19:0/1834689966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 msgr2=0x555628aa4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac8ffff640 1 --2- 172.21.15.19:0/1834689966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aa4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac8ffff640 1 -- 172.21.15.19:0/1834689966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 msgr2=0x555628a9bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac8ffff640 1 --2- 172.21.15.19:0/1834689966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 0x555628a9bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac8ffff640 1 -- 172.21.15.19:0/1834689966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556287de5f0 con 0x5556286fc1b0 2024-05-09T01:29:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac8ffff640 1 --2- 172.21.15.19:0/1834689966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628ab16f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fac8802f430 tx=0x7fac88009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596ff654256bd444 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac857fa640 1 -- 172.21.15.19:0/1834689966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88053020 con 0x5556286fc1b0 2024-05-09T01:29:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac857fa640 1 -- 172.21.15.19:0/1834689966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac88036040 con 0x5556286fc1b0 2024-05-09T01:29:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.289+0000 7fac857fa640 1 -- 172.21.15.19:0/1834689966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8804a040 con 0x5556286fc1b0 2024-05-09T01:29:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 -- 172.21.15.19:0/1834689966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 msgr2=0x555628ab16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1834689966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628ab16f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fac8802f430 tx=0x7fac88009940 comp rx=0 tx=0).stop 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 -- 172.21.15.19:0/1834689966 shutdown_connections 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1834689966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286fc1b0 0x555628ab16f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1834689966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286db4d0 0x555628a9bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 --2- 172.21.15.19:0/1834689966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aa4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 -- 172.21.15.19:0/1834689966 >> 172.21.15.19:0/1834689966 conn(0x5556286f0300 msgr2=0x555628a9b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 -- 172.21.15.19:0/1834689966 shutdown_connections 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 -- 172.21.15.19:0/1834689966 wait complete. 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.290+0000 7fac9ce73740 1 -- start start 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac9ce73740 1 Processor -- start 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aaeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286db4d0 0x555628aaf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac9ce73740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 0x555628b1e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac9ce73740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac901aeb10 con 0x5556286db4d0 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac9ce73740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac901ae440 con 0x5556286fc1b0 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac9ce73740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac901ab680 con 0x5556286d32f0 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac9ce73740 10 client.0 _ll_register_callbacks cb 0x7fac8000be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286db4d0 0x555628aaf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac87fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286db4d0 0x555628aaf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53098/0 (socket says 172.21.15.19:53098) 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac87fff640 1 -- 172.21.15.19:0/3432081106 learned_addr learned my addr 172.21.15.19:0/3432081106 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aaeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac8ffff640 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 0x555628b1e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac87fff640 1 -- 172.21.15.19:0/3432081106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 msgr2=0x555628aaeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac87fff640 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aaeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac87fff640 1 -- 172.21.15.19:0/3432081106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 msgr2=0x555628b1e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac87fff640 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 0x555628b1e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.291+0000 7fac87fff640 1 -- 172.21.15.19:0/3432081106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556289c8ed0 con 0x5556286db4d0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac87fff640 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286db4d0 0x555628aaf430 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fac780e0080 tx=0x7fac780cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662dd7defeb93484 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac6effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac6effd640 1 -- 172.21.15.19:0/3432081106 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac780a9310 con 0x5556286db4d0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac6effd640 1 -- 172.21.15.19:0/3432081106 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fac780c41d0 con 0x5556286db4d0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac9ce73740 1 -- 172.21.15.19:0/3432081106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac88036040 con 0x5556286db4d0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac6effd640 1 -- 172.21.15.19:0/3432081106 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac780c9050 con 0x5556286db4d0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac9ce73740 10 client.16320 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac9ce73740 1 -- 172.21.15.19:0/3432081106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555628b1f6e0 con 0x5556286db4d0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac6effd640 1 -- 172.21.15.19:0/3432081106 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac88036040 con 0x5556286db4d0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac6effd640 1 -- 172.21.15.19:0/3432081106 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac780a5e30 con 0x5556286db4d0 2024-05-09T01:29:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac9ce73740 10 client.16320 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac9ce73740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac9ce73740 20 client.16320 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac9ce73740 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5556286901c0 0x555628b22100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac9ce73740 4 client.16320 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:29:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.292+0000 7fac9ce73740 1 -- 172.21.15.19:0/3432081106 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555628a0c240 con 0x5556286901c0 2024-05-09T01:29:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.293+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5556286901c0 0x555628b22100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.293+0000 7fac8f7fe640 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5556286901c0 0x555628b22100 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a834b17f98657a server_cookie=919ea40cc5d073ac in_seq=0 out_seq=0 2024-05-09T01:29:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.293+0000 7fac6effd640 10 client.16320 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.309+0000 7fac6effd640 1 -- 172.21.15.19:0/3432081106 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555628a0c240 con 0x5556286901c0 2024-05-09T01:29:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.309+0000 7fac6effd640 10 client.16320 handle_command_reply: tid=0 2024-05-09T01:29:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 client.16320 shutdown 2024-05-09T01:29:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 -- 172.21.15.19:0/3432081106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286db4d0 msgr2=0x555628aaf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286db4d0 0x555628aaf430 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fac780e0080 tx=0x7fac780cc040 comp rx=0 tx=0).stop 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 -- 172.21.15.19:0/3432081106 shutdown_connections 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5556286901c0 0x555628b22100 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556286fc1b0 0x555628b1e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556286db4d0 0x555628aaf430 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 --2- 172.21.15.19:0/3432081106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5556286d32f0 0x555628aaeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac6effd640 0 client.16320 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 -- 172.21.15.19:0/3432081106 >> 172.21.15.19:0/3432081106 conn(0x5556286f0300 msgr2=0x555628aa9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 -- 172.21.15.19:0/3432081106 shutdown_connections 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 1 -- 172.21.15.19:0/3432081106 wait complete. 2024-05-09T01:29:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac9ce73740 20 client.16320 trim_cache size 0 max 16384 2024-05-09T01:29:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:29:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:29:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac95fc7640 1 -- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac7c0b2860 msgr2=0x7fac7c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac7c0b2860 0x7fac7c0b4d70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x555628aa18e0 tx=0x7fac8000b390 comp rx=0 tx=0).stop 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac95fc7640 1 -- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 msgr2=0x7fac90102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.310+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 0x7fac90102a50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fac7800e9f0 tx=0x7fac7800eeb0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.311+0000 7fac95fc7640 1 -- 172.21.15.19:0/2277819272 shutdown_connections 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.311+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac7c0b2860 0x7fac7c0b4d70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.311+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac90105200 0x7fac90102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.311+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac901023b0 0x7fac90102a50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.311+0000 7fac95fc7640 1 --2- 172.21.15.19:0/2277819272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac901018b0 0x7fac90104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.311+0000 7fac95fc7640 1 -- 172.21.15.19:0/2277819272 >> 172.21.15.19:0/2277819272 conn(0x7fac900fd4d0 msgr2=0x7fac90107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.311+0000 7fac95fc7640 1 -- 172.21.15.19:0/2277819272 shutdown_connections 2024-05-09T01:29:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:39.311+0000 7fac95fc7640 1 -- 172.21.15.19:0/2277819272 wait complete. 2024-05-09T01:29:40.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:40 smithi059 ceph-mon[34351]: pgmap v502: 97 pgs: 97 active+clean; 2.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:29:40.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:40 smithi019 ceph-mon[30158]: pgmap v502: 97 pgs: 97 active+clean; 2.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:29:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:40 smithi196 ceph-mon[35041]: pgmap v502: 97 pgs: 97 active+clean; 2.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:29:42.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:42 smithi059 ceph-mon[34351]: pgmap v503: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:29:42.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:42 smithi019 ceph-mon[30158]: pgmap v503: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:29:42.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:42 smithi196 ceph-mon[35041]: pgmap v503: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:29:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:44 smithi019 ceph-mon[30158]: pgmap v504: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:29:44.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:44 smithi196 ceph-mon[35041]: pgmap v504: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:29:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:44 smithi059 ceph-mon[34351]: pgmap v504: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:29:46.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:46 smithi019 ceph-mon[30158]: pgmap v505: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:46.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:46 smithi196 ceph-mon[35041]: pgmap v505: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:46.805 INFO:tasks.workunit.client.0.smithi019.stdout:64/600: open fd = 3 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:65/600: open fd = 3 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:66/600: open fd = 3 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.806 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:67/600: open fd = 3 2024-05-09T01:29:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:68/600: open fd = 3 2024-05-09T01:29:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:69/600: open fd = 3 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:70/600: open fd = 3 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:71/600: open fd = 3 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:72/600: open fd = 3 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:73/600: open fd = 3 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:74/600: open fd = 3 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:75/600: open fd = 3 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:76/600: open fd = 3 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:77/600: open fd = 3 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.810 INFO:tasks.workunit.client.0.smithi019.stdout:78/600: open fd = 3 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:79/600: open fd = 3 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:80/600: open fd = 3 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:81/600: open fd = 3 2024-05-09T01:29:46.812 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.812 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.812 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.812 INFO:tasks.workunit.client.0.smithi019.stdout:82/600: open fd = 3 2024-05-09T01:29:46.812 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.812 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.812 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.813 INFO:tasks.workunit.client.0.smithi019.stdout:83/600: open fd = 3 2024-05-09T01:29:46.813 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.813 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.813 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.813 INFO:tasks.workunit.client.0.smithi019.stdout:84/600: open fd = 3 2024-05-09T01:29:46.813 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.814 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.814 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.814 INFO:tasks.workunit.client.0.smithi019.stdout:85/600: open fd = 3 2024-05-09T01:29:46.814 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.814 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.814 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.815 INFO:tasks.workunit.client.0.smithi019.stdout:86/600: open fd = 3 2024-05-09T01:29:46.815 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.815 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.815 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.815 INFO:tasks.workunit.client.0.smithi019.stdout:87/600: open fd = 3 2024-05-09T01:29:46.815 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:88/600: open fd = 3 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:89/600: open fd = 3 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:90/600: open fd = 3 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.816 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:91/600: open fd = 3 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:92/600: open fd = 3 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:93/600: open fd = 3 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.817 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:94/600: open fd = 3 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:95/600: open fd = 3 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:96/600: open fd = 3 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.818 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:97/600: open fd = 3 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:98/600: open fd = 3 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:99/600: open fd = 3 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:100/600: open fd = 3 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.820 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.820 INFO:tasks.workunit.client.0.smithi019.stdout:101/600: open fd = 3 2024-05-09T01:29:46.820 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.820 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.820 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.820 INFO:tasks.workunit.client.0.smithi019.stdout:102/600: open fd = 3 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:103/600: open fd = 3 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:104/600: open fd = 3 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.821 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:105/600: open fd = 3 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:106/600: open fd = 3 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:107/600: open fd = 3 2024-05-09T01:29:46.822 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:108/600: open fd = 3 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:109/600: open fd = 3 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:110/600: open fd = 3 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.823 INFO:tasks.workunit.client.0.smithi019.stdout:111/600: open fd = 3 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:112/600: open fd = 3 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:113/600: open fd = 3 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:114/600: open fd = 3 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:115/600: open fd = 3 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:116/600: open fd = 3 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:117/600: open fd = 3 2024-05-09T01:29:46.825 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:118/600: open fd = 3 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:119/600: open fd = 3 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:120/600: open fd = 3 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:121/600: open fd = 3 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:122/600: open fd = 3 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:123/600: open fd = 3 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.827 INFO:tasks.workunit.client.0.smithi019.stdout:124/600: open fd = 3 2024-05-09T01:29:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:125/600: open fd = 3 2024-05-09T01:29:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:29:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:126/600: open fd = 3 2024-05-09T01:29:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:29:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:46 smithi059 ceph-mon[34351]: pgmap v505: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:29:48.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:48 smithi019 ceph-mon[30158]: pgmap v506: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:29:48.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:48 smithi196 ceph-mon[35041]: pgmap v506: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:29:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:48 smithi059 ceph-mon[34351]: pgmap v506: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:29:50.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:50 smithi019 ceph-mon[30158]: pgmap v507: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:29:50.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:50 smithi196 ceph-mon[35041]: pgmap v507: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:29:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:50 smithi059 ceph-mon[34351]: pgmap v507: 97 pgs: 97 active+clean; 2.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:29:52.075 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:29:52.361 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.423+0000 7fb40afb2640 1 -- 172.21.15.19:0/828286369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404109a10 msgr2=0x7fb40410be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.423+0000 7fb40afb2640 1 --2- 172.21.15.19:0/828286369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404109a10 0x7fb40410be30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009ca0 tx=0x7fb3fc02a8c0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.423+0000 7fb40afb2640 1 -- 172.21.15.19:0/828286369 shutdown_connections 2024-05-09T01:29:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.423+0000 7fb40afb2640 1 --2- 172.21.15.19:0/828286369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404109a10 0x7fb40410be30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.423+0000 7fb40afb2640 1 --2- 172.21.15.19:0/828286369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4041070b0 0x7fb4041094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.423+0000 7fb40afb2640 1 --2- 172.21.15.19:0/828286369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb404104750 0x7fb404106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.423+0000 7fb40afb2640 1 -- 172.21.15.19:0/828286369 >> 172.21.15.19:0/828286369 conn(0x7fb4040fdd50 msgr2=0x7fb404100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.423+0000 7fb40afb2640 1 -- 172.21.15.19:0/828286369 shutdown_connections 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.423+0000 7fb40afb2640 1 -- 172.21.15.19:0/828286369 wait complete. 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb40afb2640 1 Processor -- start 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb40afb2640 1 -- start start 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb40afb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404104750 0x7fb40419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb40afb2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4041070b0 0x7fb40419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb40afb2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb404109a10 0x7fb4041a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb40afb2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410f790 con 0x7fb404104750 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb40afb2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410f610 con 0x7fb4041070b0 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb40afb2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410f910 con 0x7fb404109a10 2024-05-09T01:29:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb408d4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404104750 0x7fb40419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb408d4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404104750 0x7fb40419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59570/0 (socket says 172.21.15.19:59570) 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb408d4e640 1 -- 172.21.15.19:0/395925665 learned_addr learned my addr 172.21.15.19:0/395925665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb40954f640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb404109a10 0x7fb4041a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb403fff640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4041070b0 0x7fb40419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb408d4e640 1 -- 172.21.15.19:0/395925665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb404109a10 msgr2=0x7fb4041a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb408d4e640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb404109a10 0x7fb4041a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb408d4e640 1 -- 172.21.15.19:0/395925665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4041070b0 msgr2=0x7fb40419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb408d4e640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4041070b0 0x7fb40419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.424+0000 7fb408d4e640 1 -- 172.21.15.19:0/395925665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4041a1e40 con 0x7fb404104750 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.425+0000 7fb408d4e640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404104750 0x7fb40419a300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400d740 tx=0x7fb3f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.425+0000 7fb401ffb640 1 -- 172.21.15.19:0/395925665 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4017070 con 0x7fb404104750 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.425+0000 7fb401ffb640 1 -- 172.21.15.19:0/395925665 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f400de50 con 0x7fb404104750 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.425+0000 7fb40afb2640 1 -- 172.21.15.19:0/395925665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4041a2130 con 0x7fb404104750 2024-05-09T01:29:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.425+0000 7fb401ffb640 1 -- 172.21.15.19:0/395925665 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4004ce0 con 0x7fb404104750 2024-05-09T01:29:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.425+0000 7fb40afb2640 1 -- 172.21.15.19:0/395925665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a2680 con 0x7fb404104750 2024-05-09T01:29:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb401ffb640 1 -- 172.21.15.19:0/395925665 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb3f4004e80 con 0x7fb404104750 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb411e65740 1 Processor -- start 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb411e65740 1 -- start start 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb411e65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f7249500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb411e65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7249a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb411e65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7250980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb411e65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5607f71ad7c0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb411e65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5607f71490e0 con 0x5607f6e844d0 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb411e65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5607f71a9780 con 0x5607f6ea51b0 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb408d4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7249a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb408d4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7249a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59572/0 (socket says 172.21.15.19:59572) 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb408d4e640 1 -- 172.21.15.19:0/3897891729 learned_addr learned my addr 172.21.15.19:0/3897891729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb40954f640 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f7249500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb403fff640 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7250980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb408d4e640 1 -- 172.21.15.19:0/3897891729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 msgr2=0x5607f7249500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.428+0000 7fb408d4e640 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f7249500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb408d4e640 1 -- 172.21.15.19:0/3897891729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 msgr2=0x5607f7250980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb408d4e640 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7250980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb401ffb640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3ec0b2860 0x7fb3ec0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb408d4e640 1 -- 172.21.15.19:0/3897891729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f71727c0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb403fff640 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7250980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb40954f640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3ec0b2860 0x7fb3ec0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb408d4e640 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7249a80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40e5c60 tx=0x7fb3f40ea590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f23d0f4b4500a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb401ffb640 1 -- 172.21.15.19:0/395925665 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb3f40e1ac0 con 0x7fb404104750 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb3eaffd640 1 -- 172.21.15.19:0/3897891729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4020070 con 0x5607f6e7c2f0 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb3eaffd640 1 -- 172.21.15.19:0/3897891729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40b1f20 con 0x5607f6e7c2f0 2024-05-09T01:29:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.429+0000 7fb3eaffd640 1 -- 172.21.15.19:0/3897891729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40a8ac0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 -- 172.21.15.19:0/3897891729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 msgr2=0x5607f7249a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb40954f640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3ec0b2860 0x7fb3ec0b4d70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc004af0 tx=0x7fb3fc009450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7249a80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40e5c60 tx=0x7fb3f40ea590 comp rx=0 tx=0).stop 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 -- 172.21.15.19:0/3897891729 shutdown_connections 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7250980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7249a80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 --2- 172.21.15.19:0/3897891729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f7249500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 -- 172.21.15.19:0/3897891729 >> 172.21.15.19:0/3897891729 conn(0x5607f6e99300 msgr2=0x5607f7244e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 -- 172.21.15.19:0/3897891729 shutdown_connections 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 -- 172.21.15.19:0/3897891729 wait complete. 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 -- start start 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.430+0000 7fb411e65740 1 Processor -- start 2024-05-09T01:29:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb411e65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e7c2f0 0x5607f7253230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb411e65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e844d0 0x5607f7253770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb411e65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f725a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb411e65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5607f72513d0 con 0x5607f6e844d0 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb411e65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5607f71349c0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb411e65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5607f7251550 con 0x5607f6ea51b0 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb411e65740 10 client.0 _ll_register_callbacks cb 0x5607f7161020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb403fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e844d0 0x5607f7253770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb403fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e844d0 0x5607f7253770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59576/0 (socket says 172.21.15.19:59576) 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb403fff640 1 -- 172.21.15.19:0/1037584399 learned_addr learned my addr 172.21.15.19:0/1037584399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb40954f640 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f725a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb408d4e640 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e7c2f0 0x5607f7253230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb403fff640 1 -- 172.21.15.19:0/1037584399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 msgr2=0x5607f725a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb403fff640 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f725a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb403fff640 1 -- 172.21.15.19:0/1037584399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e7c2f0 msgr2=0x5607f7253230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb403fff640 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e7c2f0 0x5607f7253230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.431+0000 7fb403fff640 1 -- 172.21.15.19:0/1037584399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f715b1f0 con 0x5607f6e844d0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb403fff640 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e844d0 0x5607f7253770 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800beb0 tx=0x7fb3f8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41736fbdfd8dd0a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb3e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb3e3fff640 1 -- 172.21.15.19:0/1037584399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8004220 con 0x5607f6e844d0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb3e3fff640 1 -- 172.21.15.19:0/1037584399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb3f80043c0 con 0x5607f6e844d0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb411e65740 1 -- 172.21.15.19:0/1037584399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5607f71bb3d0 con 0x5607f6e844d0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb3e3fff640 1 -- 172.21.15.19:0/1037584399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3f80107d0 con 0x5607f6e844d0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb411e65740 10 client.16332 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb411e65740 1 -- 172.21.15.19:0/1037584399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607f6f10790 con 0x5607f6e844d0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb3e3fff640 1 -- 172.21.15.19:0/1037584399 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5607f71bb3d0 con 0x5607f6e844d0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb3e3fff640 1 -- 172.21.15.19:0/1037584399 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb3f8019030 con 0x5607f6e844d0 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb411e65740 10 client.16332 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb411e65740 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb411e65740 20 client.16332 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb411e65740 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5607f6e63d80 0x5607f7260c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb411e65740 4 client.16332 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.432+0000 7fb411e65740 1 -- 172.21.15.19:0/1037584399 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607f71b5240 con 0x5607f6e63d80 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.433+0000 7fb408d4e640 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5607f6e63d80 0x5607f7260c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.433+0000 7fb408d4e640 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5607f6e63d80 0x5607f7260c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e564a4eccfa4562 server_cookie=2b87987af839b481 in_seq=0 out_seq=0 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.433+0000 7fb3e3fff640 10 client.16332 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb3e3fff640 1 -- 172.21.15.19:0/1037584399 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607f71b5240 con 0x5607f6e63d80 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb3e3fff640 10 client.16332 handle_command_reply: tid=0 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 client.16332 shutdown 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 -- 172.21.15.19:0/1037584399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e844d0 msgr2=0x5607f7253770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e844d0 0x5607f7253770 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800beb0 tx=0x7fb3f8012820 comp rx=0 tx=0).stop 2024-05-09T01:29:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 -- 172.21.15.19:0/1037584399 shutdown_connections 2024-05-09T01:29:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5607f6e63d80 0x5607f7260c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f725a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e844d0 0x5607f7253770 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 --2- 172.21.15.19:0/1037584399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e7c2f0 0x5607f7253230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb3e3fff640 0 client.16332 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 -- 172.21.15.19:0/1037584399 >> 172.21.15.19:0/1037584399 conn(0x5607f6e99300 msgr2=0x5607f7243700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 -- 172.21.15.19:0/1037584399 shutdown_connections 2024-05-09T01:29:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 1 -- 172.21.15.19:0/1037584399 wait complete. 2024-05-09T01:29:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.434+0000 7fb411e65740 20 client.16332 trim_cache size 0 max 16384 2024-05-09T01:29:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb411e65740 1 Processor -- start 2024-05-09T01:29:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb411e65740 1 -- start start 2024-05-09T01:29:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb411e65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7243700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb411e65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6e844d0 0x5607f7243c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb411e65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6ea51b0 0x5607f725a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb411e65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ab470 con 0x5607f6e7c2f0 2024-05-09T01:29:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb411e65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041acfb0 con 0x5607f6ea51b0 2024-05-09T01:29:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb411e65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041abb40 con 0x5607f6e844d0 2024-05-09T01:29:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb408d4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7243700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb408d4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7243700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59582/0 (socket says 172.21.15.19:59582) 2024-05-09T01:29:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb408d4e640 1 -- 172.21.15.19:0/1812731633 learned_addr learned my addr 172.21.15.19:0/1812731633 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb40954f640 1 --2- 172.21.15.19:0/1812731633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6ea51b0 0x5607f725a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb403fff640 1 --2- 172.21.15.19:0/1812731633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6e844d0 0x5607f7243c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb408d4e640 1 -- 172.21.15.19:0/1812731633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6e844d0 msgr2=0x5607f7243c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb408d4e640 1 --2- 172.21.15.19:0/1812731633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6e844d0 0x5607f7243c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb408d4e640 1 -- 172.21.15.19:0/1812731633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6ea51b0 msgr2=0x5607f725a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb408d4e640 1 --2- 172.21.15.19:0/1812731633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6ea51b0 0x5607f725a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.450+0000 7fb408d4e640 1 -- 172.21.15.19:0/1812731633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f6f875f0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.451+0000 7fb408d4e640 1 --2- 172.21.15.19:0/1812731633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7243700 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40df5f0 tx=0x7fb3f40e55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cdd1047777207f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.451+0000 7fb3ea7fc640 1 -- 172.21.15.19:0/1812731633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4020040 con 0x5607f6e7c2f0 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.451+0000 7fb3ea7fc640 1 -- 172.21.15.19:0/1812731633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40e8070 con 0x5607f6e7c2f0 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.451+0000 7fb3ea7fc640 1 -- 172.21.15.19:0/1812731633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40a88e0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.451+0000 7fb411e65740 1 -- 172.21.15.19:0/1812731633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 msgr2=0x5607f7243700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.451+0000 7fb411e65740 1 --2- 172.21.15.19:0/1812731633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7243700 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40df5f0 tx=0x7fb3f40e55a0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 -- 172.21.15.19:0/1812731633 shutdown_connections 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 --2- 172.21.15.19:0/1812731633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6ea51b0 0x5607f725a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 --2- 172.21.15.19:0/1812731633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6e844d0 0x5607f7243c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 --2- 172.21.15.19:0/1812731633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7243700 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 -- 172.21.15.19:0/1812731633 >> 172.21.15.19:0/1812731633 conn(0x5607f6e99300 msgr2=0x5607f724d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 -- 172.21.15.19:0/1812731633 shutdown_connections 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 -- 172.21.15.19:0/1812731633 wait complete. 2024-05-09T01:29:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 -- start start 2024-05-09T01:29:52.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 Processor -- start 2024-05-09T01:29:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7257ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7258530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f7259460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ac8e0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ac210 con 0x5607f6e844d0 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.452+0000 7fb411e65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041a9450 con 0x5607f6ea51b0 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb411e65740 10 client.0 _ll_register_callbacks cb 0x7fb3f800ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb408d4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7257ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb408d4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7257ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59594/0 (socket says 172.21.15.19:59594) 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb408d4e640 1 -- 172.21.15.19:0/2351243529 learned_addr learned my addr 172.21.15.19:0/2351243529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb40954f640 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f7259460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb403fff640 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7258530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb408d4e640 1 -- 172.21.15.19:0/2351243529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 msgr2=0x5607f7259460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb408d4e640 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f7259460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb408d4e640 1 -- 172.21.15.19:0/2351243529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 msgr2=0x5607f7258530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb408d4e640 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7258530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb408d4e640 1 -- 172.21.15.19:0/2351243529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f7171ed0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb403fff640 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7258530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb408d4e640 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7257ff0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009610 tx=0x7fb3f40e5b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af6a861e3792c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb3e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb3e3fff640 1 -- 172.21.15.19:0/2351243529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4020070 con 0x5607f6e7c2f0 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb3e3fff640 1 -- 172.21.15.19:0/2351243529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb3f40a5400 con 0x5607f6e7c2f0 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb411e65740 1 -- 172.21.15.19:0/2351243529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f40e8070 con 0x5607f6e7c2f0 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb3e3fff640 1 -- 172.21.15.19:0/2351243529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3f40a88e0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb411e65740 10 client.16338 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.453+0000 7fb411e65740 1 -- 172.21.15.19:0/2351243529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607f725aae0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb3e3fff640 1 -- 172.21.15.19:0/2351243529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40e8070 con 0x5607f6e7c2f0 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb3e3fff640 1 -- 172.21.15.19:0/2351243529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb3f40a93d0 con 0x5607f6e7c2f0 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb411e65740 10 client.16338 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb411e65740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb411e65740 20 client.16338 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb411e65740 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5607f6e391c0 0x5607f72cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb411e65740 4 client.16338 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb411e65740 1 -- 172.21.15.19:0/2351243529 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607f7130350 con 0x5607f6e391c0 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb403fff640 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5607f6e391c0 0x5607f72cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb3e3fff640 10 client.16338 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.454+0000 7fb403fff640 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5607f6e391c0 0x5607f72cb000 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81fbee417dc23e4 server_cookie=e1ebba166e60c63b in_seq=0 out_seq=0 2024-05-09T01:29:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.467+0000 7fb3e3fff640 1 -- 172.21.15.19:0/2351243529 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5607f7130350 con 0x5607f6e391c0 2024-05-09T01:29:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.467+0000 7fb3e3fff640 10 client.16338 handle_command_reply: tid=0 2024-05-09T01:29:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.467+0000 7fb411e65740 1 client.16338 shutdown 2024-05-09T01:29:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.467+0000 7fb411e65740 1 -- 172.21.15.19:0/2351243529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 msgr2=0x5607f7257ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.467+0000 7fb411e65740 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7257ff0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009610 tx=0x7fb3f40e5b90 comp rx=0 tx=0).stop 2024-05-09T01:29:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb411e65740 1 -- 172.21.15.19:0/2351243529 shutdown_connections 2024-05-09T01:29:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb411e65740 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5607f6e391c0 0x5607f72cb000 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb411e65740 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607f6ea51b0 0x5607f7259460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb411e65740 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607f6e844d0 0x5607f7258530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb411e65740 1 --2- 172.21.15.19:0/2351243529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5607f6e7c2f0 0x5607f7257ff0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb3e3fff640 0 client.16338 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb411e65740 1 -- 172.21.15.19:0/2351243529 >> 172.21.15.19:0/2351243529 conn(0x5607f6e99300 msgr2=0x5607f7243700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb411e65740 1 -- 172.21.15.19:0/2351243529 shutdown_connections 2024-05-09T01:29:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb411e65740 1 -- 172.21.15.19:0/2351243529 wait complete. 2024-05-09T01:29:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb411e65740 20 client.16338 trim_cache size 0 max 16384 2024-05-09T01:29:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:29:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:29:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:29:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 -- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3ec0b2860 msgr2=0x7fb3ec0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3ec0b2860 0x7fb3ec0b4d70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc004af0 tx=0x7fb3fc009450 comp rx=0 tx=0).stop 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 -- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404104750 msgr2=0x7fb40419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404104750 0x7fb40419a300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400d740 tx=0x7fb3f400dc00 comp rx=0 tx=0).stop 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 -- 172.21.15.19:0/395925665 shutdown_connections 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3ec0b2860 0x7fb3ec0b4d70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb404109a10 0x7fb4041a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4041070b0 0x7fb40419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 --2- 172.21.15.19:0/395925665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb404104750 0x7fb40419a300 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.468+0000 7fb40afb2640 1 -- 172.21.15.19:0/395925665 >> 172.21.15.19:0/395925665 conn(0x7fb4040fdd50 msgr2=0x7fb404105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.469+0000 7fb40afb2640 1 -- 172.21.15.19:0/395925665 shutdown_connections 2024-05-09T01:29:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:29:52.469+0000 7fb40afb2640 1 -- 172.21.15.19:0/395925665 wait complete. 2024-05-09T01:29:52.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:52 smithi019 ceph-mon[30158]: pgmap v508: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:29:52.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:52 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:29:52.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:52 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:29:52.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:52 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:29:52.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:52 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:29:52.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:52 smithi196 ceph-mon[35041]: pgmap v508: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:29:52.960 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:52 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:29:52.961 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:52 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:29:52.961 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:52 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:29:52.961 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:52 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:29:52.962 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:52 smithi059 ceph-mon[34351]: pgmap v508: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:29:52.962 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:52 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:29:52.962 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:52 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:29:52.963 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:52 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:29:52.963 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:52 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:29:54.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.197+0000 7f88e90e0700 1 -- 172.21.15.19:0/2537513168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e410e910 msgr2=0x7f88e4110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:54.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.197+0000 7f88e90e0700 1 --2- 172.21.15.19:0/2537513168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e410e910 0x7f88e4110df0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00b0a0 tx=0x7f88dc00b3b0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.197+0000 7f88e90e0700 1 -- 172.21.15.19:0/2537513168 shutdown_connections 2024-05-09T01:29:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.197+0000 7f88e90e0700 1 --2- 172.21.15.19:0/2537513168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e410e910 0x7f88e4110df0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.197+0000 7f88e90e0700 1 --2- 172.21.15.19:0/2537513168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e41054e0 0x7f88e41058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.197+0000 7f88e90e0700 1 --2- 172.21.15.19:0/2537513168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4105e30 0x7f88e410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.197+0000 7f88e90e0700 1 -- 172.21.15.19:0/2537513168 >> 172.21.15.19:0/2537513168 conn(0x7f88e40fef30 msgr2=0x7f88e4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.197+0000 7f88e90e0700 1 -- 172.21.15.19:0/2537513168 shutdown_connections 2024-05-09T01:29:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.197+0000 7f88e90e0700 1 -- 172.21.15.19:0/2537513168 wait complete. 2024-05-09T01:29:54.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e90e0700 1 Processor -- start 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e90e0700 1 -- start start 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e90e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e41054e0 0x7f88e41031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e90e0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4105e30 0x7f88e4103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e90e0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e410e910 0x7f88e4103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e90e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41144e0 con 0x7f88e41054e0 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e90e0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4114360 con 0x7f88e4105e30 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e90e0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4114660 con 0x7f88e410e910 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e41054e0 0x7f88e41031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e41054e0 0x7f88e41031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59614/0 (socket says 172.21.15.19:59614) 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.198+0000 7f88e2d9d700 1 -- 172.21.15.19:0/1270327253 learned_addr learned my addr 172.21.15.19:0/1270327253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e259c700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4105e30 0x7f88e4103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e359e700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e410e910 0x7f88e4103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e2d9d700 1 -- 172.21.15.19:0/1270327253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e410e910 msgr2=0x7f88e4103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e410e910 0x7f88e4103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e2d9d700 1 -- 172.21.15.19:0/1270327253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4105e30 msgr2=0x7f88e4103710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4105e30 0x7f88e4103710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e2d9d700 1 -- 172.21.15.19:0/1270327253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e41a9100 con 0x7f88e41054e0 2024-05-09T01:29:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e41054e0 0x7f88e41031d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f88d400b530 tx=0x7f88d400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88cbfff700 1 -- 172.21.15.19:0/1270327253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4015020 con 0x7f88e41054e0 2024-05-09T01:29:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88cbfff700 1 -- 172.21.15.19:0/1270327253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d4011ce0 con 0x7f88e41054e0 2024-05-09T01:29:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e90e0700 1 -- 172.21.15.19:0/1270327253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e41a93f0 con 0x7f88e41054e0 2024-05-09T01:29:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88cbfff700 1 -- 172.21.15.19:0/1270327253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4010e40 con 0x7f88e41054e0 2024-05-09T01:29:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.199+0000 7f88e90e0700 1 -- 172.21.15.19:0/1270327253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e41a9930 con 0x7f88e41054e0 2024-05-09T01:29:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.203+0000 7f88cbfff700 1 -- 172.21.15.19:0/1270327253 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f88d4018020 con 0x7f88e41054e0 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.204+0000 7f88cbfff700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f88cc0b2880 0x7f88cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.204+0000 7f88f7af6b80 1 Processor -- start 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.204+0000 7f88f7af6b80 1 -- start start 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.204+0000 7f88e259c700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f88cc0b2880 0x7f88cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.204+0000 7f88cbfff700 1 -- 172.21.15.19:0/1270327253 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f88d40e1ac0 con 0x7f88e41054e0 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.204+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1af6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.204+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1af6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.204+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac1af0400 0x55fac1af0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fac1a130a0 con 0x55fac19734a0 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fac19b89a0 con 0x55fac19edd00 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fac1a0f060 con 0x55fac1af0400 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1af6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1af6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45660/0 (socket says 172.21.15.19:45660) 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e2d9d700 1 -- 172.21.15.19:0/1029758845 learned_addr learned my addr 172.21.15.19:0/1029758845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e359e700 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac1af0400 0x55fac1af0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e259c700 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1af6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e259c700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f88cc0b2880 0x7f88cc0b4d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f88e4102490 tx=0x7f88d800f6a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e359e700 1 -- 172.21.15.19:0/1029758845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 msgr2=0x55fac1af6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e359e700 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1af6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e359e700 1 -- 172.21.15.19:0/1029758845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 msgr2=0x55fac1af6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e359e700 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1af6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e359e700 1 -- 172.21.15.19:0/1029758845 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac19a9500 con 0x55fac1af0400 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e259c700 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1af6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.205+0000 7f88e359e700 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac1af0400 0x55fac1af0810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00b830 tx=0x7f88dc009150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a075401c61baaee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.206+0000 7f88c97fa700 1 -- 172.21.15.19:0/1029758845 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc004a50 con 0x55fac1af0400 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.206+0000 7f88c97fa700 1 -- 172.21.15.19:0/1029758845 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc004bf0 con 0x55fac1af0400 2024-05-09T01:29:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.206+0000 7f88c97fa700 1 -- 172.21.15.19:0/1029758845 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc01c7c0 con 0x55fac1af0400 2024-05-09T01:29:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 -- 172.21.15.19:0/1029758845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac1af0400 msgr2=0x55fac1af0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac1af0400 0x55fac1af0810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00b830 tx=0x7f88dc009150 comp rx=0 tx=0).stop 2024-05-09T01:29:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 -- 172.21.15.19:0/1029758845 shutdown_connections 2024-05-09T01:29:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac1af0400 0x55fac1af0810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1af6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/1029758845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1af6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 -- 172.21.15.19:0/1029758845 >> 172.21.15.19:0/1029758845 conn(0x55fac1976f70 msgr2=0x55fac1aeb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 -- 172.21.15.19:0/1029758845 shutdown_connections 2024-05-09T01:29:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 -- 172.21.15.19:0/1029758845 wait complete. 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 -- start start 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.207+0000 7f88f7af6b80 1 Processor -- start 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 0x55fac1af9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1afa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1afa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fac1af1100 con 0x55fac1af0400 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fac19adb20 con 0x55fac19edd00 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fac1b06200 con 0x55fac19734a0 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88f7af6b80 10 client.0 _ll_register_callbacks cb 0x55fac18846c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1afa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1afa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59640/0 (socket says 172.21.15.19:59640) 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e359e700 1 -- 172.21.15.19:0/3420070257 learned_addr learned my addr 172.21.15.19:0/3420070257 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e259c700 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1afa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 0x55fac1af9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e359e700 1 -- 172.21.15.19:0/3420070257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 msgr2=0x55fac1af9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e359e700 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 0x55fac1af9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e359e700 1 -- 172.21.15.19:0/3420070257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 msgr2=0x55fac1afa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e359e700 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1afa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e359e700 1 -- 172.21.15.19:0/3420070257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac188e490 con 0x55fac1af0400 2024-05-09T01:29:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88e359e700 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1afa9e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00b040 tx=0x7f88dc007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1705c2291a05741e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88c6ff5700 1 -- 172.21.15.19:0/3420070257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc01c640 con 0x55fac1af0400 2024-05-09T01:29:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88c6ff5700 1 -- 172.21.15.19:0/3420070257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f88dc01c7e0 con 0x55fac1af0400 2024-05-09T01:29:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.208+0000 7f88f7af6b80 1 -- 172.21.15.19:0/3420070257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88dc004bf0 con 0x55fac1af0400 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88c6ff5700 1 -- 172.21.15.19:0/3420070257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88dc009ca0 con 0x55fac1af0400 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88f7af6b80 10 client.16350 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88f7af6b80 1 -- 172.21.15.19:0/3420070257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fac1b07a70 con 0x55fac1af0400 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88c6ff5700 1 -- 172.21.15.19:0/3420070257 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc004bf0 con 0x55fac1af0400 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88c6ff5700 1 -- 172.21.15.19:0/3420070257 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f88dc011070 con 0x55fac1af0400 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88f7af6b80 10 client.16350 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88f7af6b80 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88f7af6b80 20 client.16350 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fac1b09f40 0x55fac1b0c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88f7af6b80 4 client.16350 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88f7af6b80 1 -- 172.21.15.19:0/3420070257 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fac1a20a80 con 0x55fac1b09f40 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.209+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fac1b09f40 0x55fac1b0c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.210+0000 7f88c6ff5700 10 client.16350 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.210+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fac1b09f40 0x55fac1b0c360 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6866fbaee0bb50f server_cookie=615ea57514085ef1 in_seq=0 out_seq=0 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88c6ff5700 1 -- 172.21.15.19:0/3420070257 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fac1a20a80 con 0x55fac1b09f40 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88c6ff5700 10 client.16350 handle_command_reply: tid=0 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 client.16350 shutdown 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 -- 172.21.15.19:0/3420070257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 msgr2=0x55fac1afa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1afa9e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00b040 tx=0x7f88dc007cc0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 -- 172.21.15.19:0/3420070257 shutdown_connections 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fac1b09f40 0x55fac1b0c360 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1afa9e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 0x55fac1af9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/3420070257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1afa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 -- 172.21.15.19:0/3420070257 >> 172.21.15.19:0/3420070257 conn(0x55fac1976f70 msgr2=0x55fac1ae4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88c6ff5700 0 client.16350 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 -- 172.21.15.19:0/3420070257 shutdown_connections 2024-05-09T01:29:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 1 -- 172.21.15.19:0/3420070257 wait complete. 2024-05-09T01:29:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.211+0000 7f88f7af6b80 20 client.16350 trim_cache size 0 max 16384 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.231+0000 7f88f7af6b80 1 Processor -- start 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88f7af6b80 1 -- start start 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1ae5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19edd00 0x55fac1afa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac1af0400 0x55fac1b06170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41b0db0 con 0x55fac19734a0 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41b28f0 con 0x55fac1af0400 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41b1480 con 0x55fac19edd00 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1ae5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1ae5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59642/0 (socket says 172.21.15.19:59642) 2024-05-09T01:29:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e2d9d700 1 -- 172.21.15.19:0/340325808 learned_addr learned my addr 172.21.15.19:0/340325808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e259c700 1 --2- 172.21.15.19:0/340325808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19edd00 0x55fac1afa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e359e700 1 --2- 172.21.15.19:0/340325808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac1af0400 0x55fac1b06170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e2d9d700 1 -- 172.21.15.19:0/340325808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19edd00 msgr2=0x55fac1afa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/340325808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19edd00 0x55fac1afa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e2d9d700 1 -- 172.21.15.19:0/340325808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac1af0400 msgr2=0x55fac1b06170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/340325808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac1af0400 0x55fac1b06170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e2d9d700 1 -- 172.21.15.19:0/340325808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac1ae79d0 con 0x55fac19734a0 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.232+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/340325808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1ae5a20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f88d40a8ff0 tx=0x7f88d40e1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2046929b746ca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88c8ff9700 1 -- 172.21.15.19:0/340325808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d406d070 con 0x55fac19734a0 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88c8ff9700 1 -- 172.21.15.19:0/340325808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d40ad410 con 0x55fac19734a0 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88c8ff9700 1 -- 172.21.15.19:0/340325808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d40b1a30 con 0x55fac19734a0 2024-05-09T01:29:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88f7af6b80 1 -- 172.21.15.19:0/340325808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 msgr2=0x55fac1ae5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/340325808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1ae5a20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f88d40a8ff0 tx=0x7f88d40e1040 comp rx=0 tx=0).stop 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88f7af6b80 1 -- 172.21.15.19:0/340325808 shutdown_connections 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/340325808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac1af0400 0x55fac1b06170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/340325808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac19734a0 0x55fac1ae5a20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/340325808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19edd00 0x55fac1afa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.233+0000 7f88f7af6b80 1 -- 172.21.15.19:0/340325808 >> 172.21.15.19:0/340325808 conn(0x55fac1976f70 msgr2=0x55fac1af5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 -- 172.21.15.19:0/340325808 shutdown_connections 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 -- 172.21.15.19:0/340325808 wait complete. 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 -- start start 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 Processor -- start 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 0x55fac1b0d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1b0d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1b73490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41b2220 con 0x55fac1af0400 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41b1b50 con 0x55fac19edd00 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41aed50 con 0x55fac19734a0 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88f7af6b80 10 client.0 _ll_register_callbacks cb 0x55fac18846c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88e359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1b73490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88e359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1b73490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59650/0 (socket says 172.21.15.19:59650) 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.234+0000 7f88e359e700 1 -- 172.21.15.19:0/534877608 learned_addr learned my addr 172.21.15.19:0/534877608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88e259c700 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1b0d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 0x55fac1b0d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88e359e700 1 -- 172.21.15.19:0/534877608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 msgr2=0x55fac1b0d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88e359e700 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 0x55fac1b0d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88e359e700 1 -- 172.21.15.19:0/534877608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 msgr2=0x55fac1b0d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88e359e700 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1b0d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88e359e700 1 -- 172.21.15.19:0/534877608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac1b0e010 con 0x55fac1af0400 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88e259c700 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1b0d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88e359e700 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1b73490 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f88dc03bba0 tx=0x7f88dc00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fda8582477d2405 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88c6ff5700 1 -- 172.21.15.19:0/534877608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc03f060 con 0x55fac1af0400 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88c6ff5700 1 -- 172.21.15.19:0/534877608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f88dc01c440 con 0x55fac1af0400 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88f7af6b80 1 -- 172.21.15.19:0/534877608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d40ad410 con 0x55fac1af0400 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88c6ff5700 1 -- 172.21.15.19:0/534877608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88dc01c640 con 0x55fac1af0400 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88f7af6b80 10 client.16356 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.235+0000 7f88f7af6b80 1 -- 172.21.15.19:0/534877608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fac1b749b0 con 0x55fac1af0400 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.236+0000 7f88c6ff5700 1 -- 172.21.15.19:0/534877608 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d40ad410 con 0x55fac1af0400 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.236+0000 7f88c6ff5700 1 -- 172.21.15.19:0/534877608 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f88dc011040 con 0x55fac1af0400 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.236+0000 7f88f7af6b80 10 client.16356 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.236+0000 7f88f7af6b80 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.236+0000 7f88f7af6b80 20 client.16356 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.236+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fac1b74fa0 0x55fac1b773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.236+0000 7f88f7af6b80 4 client.16356 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:29:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.236+0000 7f88f7af6b80 1 -- 172.21.15.19:0/534877608 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fac1a1ab20 con 0x55fac1b74fa0 2024-05-09T01:29:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.236+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fac1b74fa0 0x55fac1b773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88e2d9d700 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fac1b74fa0 0x55fac1b773c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=311f03c591e5dd46 server_cookie=c265c12c4346b29c in_seq=0 out_seq=0 2024-05-09T01:29:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88c6ff5700 10 client.16356 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88c6ff5700 1 -- 172.21.15.19:0/534877608 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fac1a1ab20 con 0x55fac1b74fa0 2024-05-09T01:29:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88c6ff5700 10 client.16356 handle_command_reply: tid=0 2024-05-09T01:29:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 client.16356 shutdown 2024-05-09T01:29:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 -- 172.21.15.19:0/534877608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 msgr2=0x55fac1b73490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1b73490 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f88dc03bba0 tx=0x7f88dc00b460 comp rx=0 tx=0).stop 2024-05-09T01:29:54.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 -- 172.21.15.19:0/534877608 shutdown_connections 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fac1b74fa0 0x55fac1b773c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1af0400 0x55fac1b73490 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fac19734a0 0x55fac1b0d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 --2- 172.21.15.19:0/534877608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fac19edd00 0x55fac1b0d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 -- 172.21.15.19:0/534877608 >> 172.21.15.19:0/534877608 conn(0x55fac1976f70 msgr2=0x55fac1aef3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88c6ff5700 0 client.16356 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 -- 172.21.15.19:0/534877608 shutdown_connections 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 1 -- 172.21.15.19:0/534877608 wait complete. 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.237+0000 7f88f7af6b80 20 client.16356 trim_cache size 0 max 16384 2024-05-09T01:29:54.239 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:29:54.240 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:29:54.240 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:29:54.240 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:29:54.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 -- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f88cc0b2880 msgr2=0x7f88cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:54.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f88cc0b2880 0x7f88cc0b4d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f88e4102490 tx=0x7f88d800f6a0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 -- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e41054e0 msgr2=0x7f88e41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:54.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e41054e0 0x7f88e41031d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f88d400b530 tx=0x7f88d400b840 comp rx=0 tx=0).stop 2024-05-09T01:29:54.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 -- 172.21.15.19:0/1270327253 shutdown_connections 2024-05-09T01:29:54.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e410e910 0x7f88e4103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f88cc0b2880 0x7f88cc0b4d90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e41054e0 0x7f88e41031d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 --2- 172.21.15.19:0/1270327253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4105e30 0x7f88e4103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 -- 172.21.15.19:0/1270327253 >> 172.21.15.19:0/1270327253 conn(0x7f88e40fef30 msgr2=0x7f88e40ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 -- 172.21.15.19:0/1270327253 shutdown_connections 2024-05-09T01:29:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:54.238+0000 7f88e90e0700 1 -- 172.21.15.19:0/1270327253 wait complete. 2024-05-09T01:29:54.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:54 smithi019 ceph-mon[30158]: pgmap v509: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:29:54.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:54 smithi196 ceph-mon[35041]: pgmap v509: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:29:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:54 smithi059 ceph-mon[34351]: pgmap v509: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:29:54.956 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:29:54.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:29:55.209 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:56.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:56 smithi019 ceph-mon[30158]: pgmap v510: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:29:56.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:56 smithi196 ceph-mon[35041]: pgmap v510: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:29:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:56 smithi059 ceph-mon[34351]: pgmap v510: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:29:56.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.875+0000 7f4428f7f700 1 -- 172.21.15.19:0/425840803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 msgr2=0x7f44241100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:56.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.875+0000 7f4428f7f700 1 --2- 172.21.15.19:0/425840803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 0x7f44241100a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f441c00b0d0 tx=0x7f441c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:29:56.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.875+0000 7f4428f7f700 1 -- 172.21.15.19:0/425840803 shutdown_connections 2024-05-09T01:29:56.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.875+0000 7f4428f7f700 1 --2- 172.21.15.19:0/425840803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 0x7f44241100a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:56.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.875+0000 7f4428f7f700 1 --2- 172.21.15.19:0/425840803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44241034c0 0x7f44241038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:56.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.875+0000 7f4428f7f700 1 --2- 172.21.15.19:0/425840803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424107df0 0x7f4424108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:56.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.875+0000 7f4428f7f700 1 -- 172.21.15.19:0/425840803 >> 172.21.15.19:0/425840803 conn(0x7f44240fedc0 msgr2=0x7f44241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:56.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.876+0000 7f4428f7f700 1 -- 172.21.15.19:0/425840803 shutdown_connections 2024-05-09T01:29:56.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.876+0000 7f4428f7f700 1 -- 172.21.15.19:0/425840803 wait complete. 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.876+0000 7f4428f7f700 1 Processor -- start 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.876+0000 7f4428f7f700 1 -- start start 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4428f7f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44241034c0 0x7f44240ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4428f7f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424107df0 0x7f44240ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4428f7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 0x7f44240ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4428f7f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4424113370 con 0x7f4424108740 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4428f7f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f44241131f0 con 0x7f4424107df0 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4428f7f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44241134f0 con 0x7f44241034c0 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4422d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 0x7f44240ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4422d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 0x7f44240ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59658/0 (socket says 172.21.15.19:59658) 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f442259c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44241034c0 0x7f44240ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:56.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4422d9d700 1 -- 172.21.15.19:0/856848888 learned_addr learned my addr 172.21.15.19:0/856848888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4421d9b700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424107df0 0x7f44240ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4422d9d700 1 -- 172.21.15.19:0/856848888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44241034c0 msgr2=0x7f44240ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4422d9d700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44241034c0 0x7f44240ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4422d9d700 1 -- 172.21.15.19:0/856848888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424107df0 msgr2=0x7f44240ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4422d9d700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424107df0 0x7f44240ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4422d9d700 1 -- 172.21.15.19:0/856848888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44241a9170 con 0x7f4424108740 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.877+0000 7f4422d9d700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 0x7f44240ffd20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f441c0095c0 tx=0x7f441c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.878+0000 7f44137fe700 1 -- 172.21.15.19:0/856848888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c01f070 con 0x7f4424108740 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.878+0000 7f44137fe700 1 -- 172.21.15.19:0/856848888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f441c003c00 con 0x7f4424108740 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.878+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44241a9400 con 0x7f4424108740 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.878+0000 7f44137fe700 1 -- 172.21.15.19:0/856848888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c01b6a0 con 0x7f4424108740 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.878+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44241a9850 con 0x7f4424108740 2024-05-09T01:29:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.879+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44240512c0 con 0x7f4424108740 2024-05-09T01:29:56.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.881+0000 7f44137fe700 1 -- 172.21.15.19:0/856848888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f441c022070 con 0x7f4424108740 2024-05-09T01:29:56.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.882+0000 7f44137fe700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f440c0b2880 0x7f440c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:56.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.882+0000 7f442259c700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f440c0b2880 0x7f440c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:56.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.882+0000 7f44137fe700 1 -- 172.21.15.19:0/856848888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f441c00f020 con 0x7f4424108740 2024-05-09T01:29:56.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.883+0000 7f442259c700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f440c0b2880 0x7f440c0b4d90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4414005950 tx=0x7f441400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:56.886+0000 7f44137fe700 1 -- 172.21.15.19:0/856848888 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f441c029020 con 0x7f4424108740 2024-05-09T01:29:57.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.147+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f442402e5f0 con 0x7f4424108740 2024-05-09T01:29:57.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.148+0000 7f44137fe700 1 -- 172.21.15.19:0/856848888 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f441c0ab6a0 con 0x7f4424108740 2024-05-09T01:29:57.146 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:29:57.147 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:29:57.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.150+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f440c0b2880 msgr2=0x7f440c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.150+0000 7f4428f7f700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f440c0b2880 0x7f440c0b4d90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4414005950 tx=0x7f441400b410 comp rx=0 tx=0).stop 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.150+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 msgr2=0x7f44240ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.150+0000 7f4428f7f700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 0x7f44240ffd20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f441c0095c0 tx=0x7f441c007a20 comp rx=0 tx=0).stop 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.151+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 shutdown_connections 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.151+0000 7f4428f7f700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f440c0b2880 0x7f440c0b4d90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.151+0000 7f4428f7f700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424108740 0x7f44240ffd20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.151+0000 7f4428f7f700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44241034c0 0x7f44240ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.151+0000 7f4428f7f700 1 --2- 172.21.15.19:0/856848888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424107df0 0x7f44240ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.151+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 >> 172.21.15.19:0/856848888 conn(0x7f44240fedc0 msgr2=0x7f442406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.151+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 shutdown_connections 2024-05-09T01:29:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:57.151+0000 7f4428f7f700 1 -- 172.21.15.19:0/856848888 wait complete. 2024-05-09T01:29:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:29:57.540 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:57 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/856848888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:29:57.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:57 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/856848888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:29:57.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:57 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/856848888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:29:57.849 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:29:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:29:58.479 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:29:58 smithi019 ceph-mon[30158]: pgmap v511: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:29:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:29:58 smithi196 ceph-mon[35041]: pgmap v511: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:29:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:29:58 smithi059 ceph-mon[34351]: pgmap v511: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:29:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.924+0000 7f2442382700 1 -- 172.21.15.19:0/1380744600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c1045e0 msgr2=0x7f243c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.924+0000 7f2442382700 1 --2- 172.21.15.19:0/1380744600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c1045e0 0x7f243c1049f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f242c0099c0 tx=0x7f242c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.924+0000 7f2442382700 1 -- 172.21.15.19:0/1380744600 shutdown_connections 2024-05-09T01:29:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.924+0000 7f2442382700 1 --2- 172.21.15.19:0/1380744600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243c109860 0x7f243c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.924+0000 7f2442382700 1 --2- 172.21.15.19:0/1380744600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c1045e0 0x7f243c1049f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.924+0000 7f2442382700 1 --2- 172.21.15.19:0/1380744600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c108f10 0x7f243c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.924+0000 7f2442382700 1 -- 172.21.15.19:0/1380744600 >> 172.21.15.19:0/1380744600 conn(0x7f243c0ffef0 msgr2=0x7f243c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.924+0000 7f2442382700 1 -- 172.21.15.19:0/1380744600 shutdown_connections 2024-05-09T01:29:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.924+0000 7f2442382700 1 -- 172.21.15.19:0/1380744600 wait complete. 2024-05-09T01:29:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.925+0000 7f2442382700 1 Processor -- start 2024-05-09T01:29:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.925+0000 7f2442382700 1 -- start start 2024-05-09T01:29:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.925+0000 7f2442382700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c1045e0 0x7f243c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.925+0000 7f2442382700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c108f10 0x7f243c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.925+0000 7f2442382700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243c109860 0x7f243c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.925+0000 7f2442382700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1144e0 con 0x7f243c108f10 2024-05-09T01:29:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.925+0000 7f2442382700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f243c114360 con 0x7f243c1045e0 2024-05-09T01:29:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.925+0000 7f2442382700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f243c114660 con 0x7f243c109860 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2433fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c108f10 0x7f243c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f243bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c1045e0 0x7f243c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f243bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c1045e0 0x7f243c19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:54018/0 (socket says 172.21.15.19:54018) 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f243bfff700 1 -- 172.21.15.19:0/684758424 learned_addr learned my addr 172.21.15.19:0/684758424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2433fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c108f10 0x7f243c19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40714/0 (socket says 172.21.15.19:40714) 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f244091f700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243c109860 0x7f243c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2433fff700 1 -- 172.21.15.19:0/684758424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243c109860 msgr2=0x7f243c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2433fff700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243c109860 0x7f243c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2433fff700 1 -- 172.21.15.19:0/684758424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c1045e0 msgr2=0x7f243c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2433fff700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c1045e0 0x7f243c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2433fff700 1 -- 172.21.15.19:0/684758424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c19fd00 con 0x7f243c108f10 2024-05-09T01:29:59.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2433fff700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c108f10 0x7f243c19b3d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f243c06c1b0 tx=0x7f242400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:59.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2439ffb700 1 -- 172.21.15.19:0/684758424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424016070 con 0x7f243c108f10 2024-05-09T01:29:59.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2442382700 1 -- 172.21.15.19:0/684758424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c19fff0 con 0x7f243c108f10 2024-05-09T01:29:59.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2439ffb700 1 -- 172.21.15.19:0/684758424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242400fd40 con 0x7f243c108f10 2024-05-09T01:29:59.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.926+0000 7f2439ffb700 1 -- 172.21.15.19:0/684758424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242400e3b0 con 0x7f243c108f10 2024-05-09T01:29:59.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.927+0000 7f2442382700 1 -- 172.21.15.19:0/684758424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c076b00 con 0x7f243c108f10 2024-05-09T01:29:59.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.929+0000 7f2439ffb700 1 -- 172.21.15.19:0/684758424 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2424019070 con 0x7f243c108f10 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.930+0000 7f2439ffb700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24280b2880 0x7f24280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.930+0000 7f2450d98b80 1 Processor -- start 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.930+0000 7f243bfff700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24280b2880 0x7f24280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.930+0000 7f2450d98b80 1 -- start start 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.930+0000 7f2439ffb700 1 -- 172.21.15.19:0/684758424 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f24240e1c90 con 0x7f243c108f10 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fac3160 0x56339fb64cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb651f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb5f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2450d98b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56339fa78ff0 con 0x56339f9d94a0 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2450d98b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56339fa177f0 con 0x56339fb5eeb0 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2450d98b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56339fa74fb0 con 0x56339fac3160 2024-05-09T01:29:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2433fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f243bfff700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24280b2880 0x7f24280b4d90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f242c003800 tx=0x7f242c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2433fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40722/0 (socket says 172.21.15.19:40722) 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2433fff700 1 -- 172.21.15.19:0/2949569904 learned_addr learned my addr 172.21.15.19:0/2949569904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f243bfff700 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fac3160 0x56339fb64cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f244091f700 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb5f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2433fff700 1 -- 172.21.15.19:0/2949569904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fac3160 msgr2=0x56339fb64cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2433fff700 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fac3160 0x56339fb64cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2433fff700 1 -- 172.21.15.19:0/2949569904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 msgr2=0x56339fb5f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2433fff700 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb5f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2433fff700 1 -- 172.21.15.19:0/2949569904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56339fa0f370 con 0x56339f9d94a0 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f244091f700 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb5f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f2433fff700 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb651f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f24240a91c0 tx=0x7f242400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b25966614d657d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.931+0000 7f24327fc700 1 -- 172.21.15.19:0/2949569904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424020040 con 0x56339f9d94a0 2024-05-09T01:29:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.932+0000 7f24327fc700 1 -- 172.21.15.19:0/2949569904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24240a9230 con 0x56339f9d94a0 2024-05-09T01:29:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.932+0000 7f24327fc700 1 -- 172.21.15.19:0/2949569904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24240dfe30 con 0x56339f9d94a0 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 -- 172.21.15.19:0/2949569904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 msgr2=0x56339fb651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb651f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f24240a91c0 tx=0x7f242400bf90 comp rx=0 tx=0).stop 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 -- 172.21.15.19:0/2949569904 shutdown_connections 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb5f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fac3160 0x56339fb64cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2949569904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb651f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 -- 172.21.15.19:0/2949569904 >> 172.21.15.19:0/2949569904 conn(0x56339fa396a0 msgr2=0x56339fb51400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 -- 172.21.15.19:0/2949569904 shutdown_connections 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 -- 172.21.15.19:0/2949569904 wait complete. 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.933+0000 7f2450d98b80 1 -- start start 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2450d98b80 1 Processor -- start 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 0x56339fb689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb68ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb69430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2450d98b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56339f8f4ba0 con 0x56339fac3160 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2450d98b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56339fa33150 con 0x56339fb5eeb0 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2450d98b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56339fb5fc90 con 0x56339f9d94a0 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2433fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb68ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2450d98b80 10 client.0 _ll_register_callbacks cb 0x56339f8ea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2433fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb68ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40726/0 (socket says 172.21.15.19:40726) 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f244091f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb69430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f2433fff700 1 -- 172.21.15.19:0/679997734 learned_addr learned my addr 172.21.15.19:0/679997734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.934+0000 7f243bfff700 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 0x56339fb689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f244091f700 1 -- 172.21.15.19:0/679997734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 msgr2=0x56339fb689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f244091f700 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 0x56339fb689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f244091f700 1 -- 172.21.15.19:0/679997734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 msgr2=0x56339fb68ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f244091f700 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb68ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f244091f700 1 -- 172.21.15.19:0/679997734 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56339f8f4490 con 0x56339fb5eeb0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f2433fff700 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb68ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f244091f700 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb69430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2434009fd0 tx=0x7f2434004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477e9aaf73c1d52a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f241b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f241b7fe700 1 -- 172.21.15.19:0/679997734 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243401a020 con 0x56339fb5eeb0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f241b7fe700 1 -- 172.21.15.19:0/679997734 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2434007910 con 0x56339fb5eeb0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f2450d98b80 1 -- 172.21.15.19:0/679997734 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24240a9230 con 0x56339fb5eeb0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f241b7fe700 1 -- 172.21.15.19:0/679997734 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2434004bb0 con 0x56339fb5eeb0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f2450d98b80 10 client.25943 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.935+0000 7f2450d98b80 1 -- 172.21.15.19:0/679997734 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56339fb76300 con 0x56339fb5eeb0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.936+0000 7f241b7fe700 1 -- 172.21.15.19:0/679997734 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24240a9230 con 0x56339fb5eeb0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.936+0000 7f241b7fe700 1 -- 172.21.15.19:0/679997734 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f24340169c0 con 0x56339fb5eeb0 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.936+0000 7f2450d98b80 10 client.25943 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.936+0000 7f2450d98b80 10 client.25943 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:29:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.936+0000 7f2450d98b80 20 client.25943 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.936+0000 7f2450d98b80 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56339fb78990 0x56339fb7adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.936+0000 7f2450d98b80 4 client.25943 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.936+0000 7f2450d98b80 1 -- 172.21.15.19:0/679997734 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56339fa869d0 con 0x56339fb78990 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.936+0000 7f243bfff700 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56339fb78990 0x56339fb7adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.937+0000 7f241b7fe700 10 client.25943 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.937+0000 7f243bfff700 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56339fb78990 0x56339fb7adf0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f64731d99f1a8420 server_cookie=9a76a058da5a4eb6 in_seq=0 out_seq=0 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f241b7fe700 1 -- 172.21.15.19:0/679997734 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56339fa869d0 con 0x56339fb78990 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f241b7fe700 10 client.25943 handle_command_reply: tid=0 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 client.25943 shutdown 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 -- 172.21.15.19:0/679997734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 msgr2=0x56339fb69430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb69430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2434009fd0 tx=0x7f2434004320 comp rx=0 tx=0).stop 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 -- 172.21.15.19:0/679997734 shutdown_connections 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56339fb78990 0x56339fb7adf0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb69430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f241b7fe700 0 client.25943 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:29:59.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 0x56339fb689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 --2- 172.21.15.19:0/679997734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb68ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 -- 172.21.15.19:0/679997734 >> 172.21.15.19:0/679997734 conn(0x56339fa396a0 msgr2=0x56339fb511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:59.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 -- 172.21.15.19:0/679997734 shutdown_connections 2024-05-09T01:29:59.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 1 -- 172.21.15.19:0/679997734 wait complete. 2024-05-09T01:29:59.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.938+0000 7f2450d98b80 20 client.25943 trim_cache size 0 max 16384 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.953+0000 7f2450d98b80 1 Processor -- start 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.953+0000 7f2450d98b80 1 -- start start 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb66630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fac3160 0x56339fb69270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fb5eeb0 0x56339fb679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f2450d98b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1acbe0 con 0x56339f9d94a0 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f2450d98b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1ae720 con 0x56339fac3160 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f2450d98b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1ad2b0 con 0x56339fb5eeb0 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f243bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb66630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f243bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb66630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40734/0 (socket says 172.21.15.19:40734) 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f243bfff700 1 -- 172.21.15.19:0/3391529423 learned_addr learned my addr 172.21.15.19:0/3391529423 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:59.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f244091f700 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fb5eeb0 0x56339fb679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f243bfff700 1 -- 172.21.15.19:0/3391529423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fb5eeb0 msgr2=0x56339fb679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f243bfff700 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fb5eeb0 0x56339fb679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f243bfff700 1 -- 172.21.15.19:0/3391529423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fac3160 msgr2=0x56339fb69270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f2433fff700 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fac3160 0x56339fb69270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f243bfff700 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fac3160 0x56339fb69270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f243bfff700 1 -- 172.21.15.19:0/3391529423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56339fb523c0 con 0x56339f9d94a0 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f2433fff700 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fac3160 0x56339fb69270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f243bfff700 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb66630 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f242c009990 tx=0x7f242c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23f3fe8e4007ef9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.954+0000 7f2431ffb700 1 -- 172.21.15.19:0/3391529423 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c037070 con 0x56339f9d94a0 2024-05-09T01:29:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2431ffb700 1 -- 172.21.15.19:0/3391529423 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242c030470 con 0x56339f9d94a0 2024-05-09T01:29:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2431ffb700 1 -- 172.21.15.19:0/3391529423 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c032330 con 0x56339f9d94a0 2024-05-09T01:29:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2450d98b80 1 -- 172.21.15.19:0/3391529423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 msgr2=0x56339fb66630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2450d98b80 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb66630 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f242c009990 tx=0x7f242c01e000 comp rx=0 tx=0).stop 2024-05-09T01:29:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2450d98b80 1 -- 172.21.15.19:0/3391529423 shutdown_connections 2024-05-09T01:29:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2450d98b80 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339fb5eeb0 0x56339fb679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2450d98b80 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339f9d94a0 0x56339fb66630 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2450d98b80 1 --2- 172.21.15.19:0/3391529423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fac3160 0x56339fb69270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2450d98b80 1 -- 172.21.15.19:0/3391529423 >> 172.21.15.19:0/3391529423 conn(0x56339fa396a0 msgr2=0x56339fb672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2450d98b80 1 -- 172.21.15.19:0/3391529423 shutdown_connections 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.955+0000 7f2450d98b80 1 -- 172.21.15.19:0/3391529423 wait complete. 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.956+0000 7f2450d98b80 1 -- start start 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.956+0000 7f2450d98b80 1 Processor -- start 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.956+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 0x56339fb70c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.956+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.956+0000 7f2450d98b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb6c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.956+0000 7f2450d98b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1ae050 con 0x56339fac3160 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.956+0000 7f2450d98b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1ad980 con 0x56339fb5eeb0 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.956+0000 7f2450d98b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1aaba0 con 0x56339f9d94a0 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.956+0000 7f2450d98b80 10 client.0 _ll_register_callbacks cb 0x56339f8ea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2433fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2433fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40748/0 (socket says 172.21.15.19:40748) 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2433fff700 1 -- 172.21.15.19:0/2433172362 learned_addr learned my addr 172.21.15.19:0/2433172362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f243bfff700 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 0x56339fb70c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f244091f700 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb6c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2433fff700 1 -- 172.21.15.19:0/2433172362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 msgr2=0x56339fb70c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2433fff700 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 0x56339fb70c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2433fff700 1 -- 172.21.15.19:0/2433172362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 msgr2=0x56339fb6c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2433fff700 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb6c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2433fff700 1 -- 172.21.15.19:0/2433172362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56339fb6c990 con 0x56339fac3160 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f244091f700 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb6c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2433fff700 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb711c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f24240a8f30 tx=0x7f24240b1e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d75d770314fec6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f241b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f241b7fe700 1 -- 172.21.15.19:0/2433172362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424020070 con 0x56339fac3160 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f241b7fe700 1 -- 172.21.15.19:0/2433172362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f24240dfde0 con 0x56339fac3160 2024-05-09T01:29:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2450d98b80 1 -- 172.21.15.19:0/2433172362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f242c030470 con 0x56339fac3160 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f241b7fe700 1 -- 172.21.15.19:0/2433172362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f242406e070 con 0x56339fac3160 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2450d98b80 10 client.16380 fetch_fsmap learned FSMap version 28 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.957+0000 7f2450d98b80 1 -- 172.21.15.19:0/2433172362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56339fbe2c10 con 0x56339fac3160 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.958+0000 7f241b7fe700 1 -- 172.21.15.19:0/2433172362 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242c030470 con 0x56339fac3160 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.958+0000 7f241b7fe700 1 -- 172.21.15.19:0/2433172362 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f24240ad240 con 0x56339fac3160 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.958+0000 7f2450d98b80 10 client.16380 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.958+0000 7f2450d98b80 10 client.16380 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.958+0000 7f2450d98b80 20 client.16380 populate_metadata read hostname 'smithi019' 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.958+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56339fbe2ef0 0x56339fbe5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.958+0000 7f2450d98b80 4 client.16380 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.958+0000 7f2450d98b80 1 -- 172.21.15.19:0/2433172362 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56339fa80a70 con 0x56339fbe2ef0 2024-05-09T01:29:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.958+0000 7f243bfff700 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56339fbe2ef0 0x56339fbe5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f241b7fe700 10 client.16380 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f243bfff700 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56339fbe2ef0 0x56339fbe5310 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=687cda5297d106c5 server_cookie=12df12861af641be in_seq=0 out_seq=0 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f241b7fe700 1 -- 172.21.15.19:0/2433172362 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56339fa80a70 con 0x56339fbe2ef0 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f241b7fe700 10 client.16380 handle_command_reply: tid=0 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 client.16380 shutdown 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 -- 172.21.15.19:0/2433172362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 msgr2=0x56339fb711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb711c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f24240a8f30 tx=0x7f24240b1e30 comp rx=0 tx=0).stop 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 -- 172.21.15.19:0/2433172362 shutdown_connections 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56339fbe2ef0 0x56339fbe5310 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56339fb5eeb0 0x56339fb6c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56339f9d94a0 0x56339fb70c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 --2- 172.21.15.19:0/2433172362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56339fac3160 0x56339fb711c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 -- 172.21.15.19:0/2433172362 >> 172.21.15.19:0/2433172362 conn(0x56339fa396a0 msgr2=0x56339fb642a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f241b7fe700 0 client.16380 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:29:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 -- 172.21.15.19:0/2433172362 shutdown_connections 2024-05-09T01:29:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 1 -- 172.21.15.19:0/2433172362 wait complete. 2024-05-09T01:29:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.959+0000 7f2450d98b80 20 client.16380 trim_cache size 0 max 16384 2024-05-09T01:29:59.963 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:29:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 -- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24280b2880 msgr2=0x7f24280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24280b2880 0x7f24280b4d90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f242c003800 tx=0x7f242c005fb0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 -- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c108f10 msgr2=0x7f243c19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:29:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c108f10 0x7f243c19b3d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f243c06c1b0 tx=0x7f242400dc30 comp rx=0 tx=0).stop 2024-05-09T01:29:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 -- 172.21.15.19:0/684758424 shutdown_connections 2024-05-09T01:29:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243c109860 0x7f243c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c1045e0 0x7f243c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24280b2880 0x7f24280b4d90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 --2- 172.21.15.19:0/684758424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f243c108f10 0x7f243c19b3d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:29:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 -- 172.21.15.19:0/684758424 >> 172.21.15.19:0/684758424 conn(0x7f243c0ffef0 msgr2=0x7f243c108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:29:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 -- 172.21.15.19:0/684758424 shutdown_connections 2024-05-09T01:29:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:29:59.960+0000 7f2442382700 1 -- 172.21.15.19:0/684758424 wait complete. 2024-05-09T01:30:00.522 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:00 smithi019 ceph-mon[30158]: pgmap v512: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:30:00.522 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:00 smithi019 ceph-mon[30158]: overall HEALTH_OK 2024-05-09T01:30:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:00 smithi196 ceph-mon[35041]: pgmap v512: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:30:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:00 smithi196 ceph-mon[35041]: overall HEALTH_OK 2024-05-09T01:30:00.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:30:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:00 smithi059 ceph-mon[34351]: pgmap v512: 97 pgs: 97 active+clean; 2.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:30:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:00 smithi059 ceph-mon[34351]: overall HEALTH_OK 2024-05-09T01:30:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:02.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:02 smithi019 ceph-mon[30158]: pgmap v513: 97 pgs: 97 active+clean; 2.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:30:02.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:02 smithi059 ceph-mon[34351]: pgmap v513: 97 pgs: 97 active+clean; 2.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:30:02.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:02 smithi196 ceph-mon[35041]: pgmap v513: 97 pgs: 97 active+clean; 2.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:30:02.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.752+0000 7feec80f0700 1 -- 172.21.15.19:0/2912155727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec0103080 msgr2=0x7feec0103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:02.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.752+0000 7feec80f0700 1 --2- 172.21.15.19:0/2912155727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec0103080 0x7feec0103490 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7feebc009990 tx=0x7feebc009ca0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.753+0000 7feec80f0700 1 -- 172.21.15.19:0/2912155727 shutdown_connections 2024-05-09T01:30:02.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.753+0000 7feec80f0700 1 --2- 172.21.15.19:0/2912155727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec0110c30 0x7feec0113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.753+0000 7feec80f0700 1 --2- 172.21.15.19:0/2912155727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec0103080 0x7feec0103490 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.753+0000 7feec80f0700 1 --2- 172.21.15.19:0/2912155727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec01039d0 0x7feec01106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.753+0000 7feec80f0700 1 -- 172.21.15.19:0/2912155727 >> 172.21.15.19:0/2912155727 conn(0x7feec00fedf0 msgr2=0x7feec0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:02.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.753+0000 7feec80f0700 1 -- 172.21.15.19:0/2912155727 shutdown_connections 2024-05-09T01:30:02.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.753+0000 7feec80f0700 1 -- 172.21.15.19:0/2912155727 wait complete. 2024-05-09T01:30:02.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.753+0000 7feec80f0700 1 Processor -- start 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.753+0000 7feec80f0700 1 -- start start 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec80f0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec0103080 0x7feec019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec80f0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec01039d0 0x7feec019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec80f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec0110c30 0x7feec01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec80f0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feec01166f0 con 0x7feec01039d0 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec80f0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feec0116570 con 0x7feec0110c30 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec80f0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feec0116870 con 0x7feec0103080 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec568b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec01039d0 0x7feec019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec568b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec01039d0 0x7feec019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40772/0 (socket says 172.21.15.19:40772) 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec568b700 1 -- 172.21.15.19:0/102792305 learned_addr learned my addr 172.21.15.19:0/102792305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec5e8c700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec0103080 0x7feec019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec668d700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec0110c30 0x7feec01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec568b700 1 -- 172.21.15.19:0/102792305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec0103080 msgr2=0x7feec019d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec568b700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec0103080 0x7feec019d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec568b700 1 -- 172.21.15.19:0/102792305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec0110c30 msgr2=0x7feec01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec568b700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec0110c30 0x7feec01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec568b700 1 -- 172.21.15.19:0/102792305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec01a1f20 con 0x7feec01039d0 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec668d700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec0110c30 0x7feec01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.754+0000 7feec568b700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec01039d0 0x7feec019d640 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7feeb000d950 tx=0x7feeb000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.755+0000 7feeb6ffd700 1 -- 172.21.15.19:0/102792305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0017070 con 0x7feec01039d0 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.755+0000 7feeb6ffd700 1 -- 172.21.15.19:0/102792305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb0009de0 con 0x7feec01039d0 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.755+0000 7feec80f0700 1 -- 172.21.15.19:0/102792305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec01a2210 con 0x7feec01039d0 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.755+0000 7feeb6ffd700 1 -- 172.21.15.19:0/102792305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb000f3c0 con 0x7feec01039d0 2024-05-09T01:30:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.755+0000 7feec80f0700 1 -- 172.21.15.19:0/102792305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec01a9b30 con 0x7feec01039d0 2024-05-09T01:30:02.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.757+0000 7feeb6ffd700 1 -- 172.21.15.19:0/102792305 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7feeb001a070 con 0x7feec01039d0 2024-05-09T01:30:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.758+0000 7feeb6ffd700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2880 0x7feeac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.758+0000 7feed6b06b80 1 Processor -- start 2024-05-09T01:30:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feec5e8c700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2880 0x7feeac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feeb6ffd700 1 -- 172.21.15.19:0/102792305 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7feeb00e1b40 con 0x7feec01039d0 2024-05-09T01:30:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feed6b06b80 1 -- start start 2024-05-09T01:30:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feec5e8c700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2880 0x7feeac0b4d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feebc003800 tx=0x7feebc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a67240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a67780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a61810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feed6b06b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f0919840a0 con 0x55f091a61400 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feed6b06b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f0919299a0 con 0x55f09195ed00 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feed6b06b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f091980060 con 0x55f0918e44a0 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feec668d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a61810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feec668d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a61810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40788/0 (socket says 172.21.15.19:40788) 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feec668d700 1 -- 172.21.15.19:0/3266750008 learned_addr learned my addr 172.21.15.19:0/3266750008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.759+0000 7feec5e8c700 1 --2- 172.21.15.19:0/3266750008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a67240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feec568b700 1 --2- 172.21.15.19:0/3266750008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a67780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feec668d700 1 -- 172.21.15.19:0/3266750008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 msgr2=0x55f091a67780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feec668d700 1 --2- 172.21.15.19:0/3266750008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a67780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feec668d700 1 -- 172.21.15.19:0/3266750008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 msgr2=0x55f091a67240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feec668d700 1 --2- 172.21.15.19:0/3266750008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a67240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feec668d700 1 -- 172.21.15.19:0/3266750008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09191a500 con 0x55f091a61400 2024-05-09T01:30:02.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feec668d700 1 --2- 172.21.15.19:0/3266750008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a61810 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009cd0 tx=0x7feeb8007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aba49838d805e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:02.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feeabfff700 1 -- 172.21.15.19:0/3266750008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8019070 con 0x55f091a61400 2024-05-09T01:30:02.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feeabfff700 1 -- 172.21.15.19:0/3266750008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb80148b0 con 0x55f091a61400 2024-05-09T01:30:02.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.760+0000 7feeabfff700 1 -- 172.21.15.19:0/3266750008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8013880 con 0x55f091a61400 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.761+0000 7feed6b06b80 1 -- 172.21.15.19:0/3266750008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 msgr2=0x55f091a61810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.761+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3266750008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a61810 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009cd0 tx=0x7feeb8007500 comp rx=0 tx=0).stop 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.762+0000 7feed6b06b80 1 -- 172.21.15.19:0/3266750008 shutdown_connections 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.762+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3266750008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a61810 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.762+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3266750008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a67240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.762+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3266750008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a67780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.762+0000 7feed6b06b80 1 -- 172.21.15.19:0/3266750008 >> 172.21.15.19:0/3266750008 conn(0x55f0918e7f70 msgr2=0x55f091a5c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.762+0000 7feed6b06b80 1 -- 172.21.15.19:0/3266750008 shutdown_connections 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.762+0000 7feed6b06b80 1 -- 172.21.15.19:0/3266750008 wait complete. 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.762+0000 7feed6b06b80 1 -- start start 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.762+0000 7feed6b06b80 1 Processor -- start 2024-05-09T01:30:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a6b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a6b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a6bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feed6b06b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f091a62160 con 0x55f091a61400 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feed6b06b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f09191eb20 con 0x55f09195ed00 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feed6b06b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f091a6e020 con 0x55f0918e44a0 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feed6b06b80 10 client.0 _ll_register_callbacks cb 0x55f0917f56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec668d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a6bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec668d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a6bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40802/0 (socket says 172.21.15.19:40802) 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec668d700 1 -- 172.21.15.19:0/3847937319 learned_addr learned my addr 172.21.15.19:0/3847937319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec5e8c700 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a6b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec568b700 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a6b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec668d700 1 -- 172.21.15.19:0/3847937319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 msgr2=0x55f091a6b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec668d700 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a6b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec668d700 1 -- 172.21.15.19:0/3847937319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 msgr2=0x55f091a6b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec668d700 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a6b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec668d700 1 -- 172.21.15.19:0/3847937319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0917ff490 con 0x55f091a61400 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.763+0000 7feec568b700 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a6b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feec668d700 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a6bac0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7feeb8017040 tx=0x7feeb80097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781871f4b9631341 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feea97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feea97fa700 1 -- 172.21.15.19:0/3847937319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8021020 con 0x55f091a61400 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feed6b06b80 1 -- 172.21.15.19:0/3847937319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb80148b0 con 0x55f091a61400 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feea97fa700 1 -- 172.21.15.19:0/3847937319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7feeb8004ad0 con 0x55f091a61400 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feea97fa700 1 -- 172.21.15.19:0/3847937319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feeb802c050 con 0x55f091a61400 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feed6b06b80 10 client.16392 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feed6b06b80 1 -- 172.21.15.19:0/3847937319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f091a78a60 con 0x55f091a61400 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feea97fa700 1 -- 172.21.15.19:0/3847937319 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb80148b0 con 0x55f091a61400 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feea97fa700 1 -- 172.21.15.19:0/3847937319 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feeb801c050 con 0x55f091a61400 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feed6b06b80 10 client.16392 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.764+0000 7feed6b06b80 10 client.16392 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:30:02.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.765+0000 7feed6b06b80 20 client.16392 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:02.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.765+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55f091a7b040 0x55f091a7d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.765+0000 7feed6b06b80 4 client.16392 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:02.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.765+0000 7feed6b06b80 1 -- 172.21.15.19:0/3847937319 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f091991a80 con 0x55f091a7b040 2024-05-09T01:30:02.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.765+0000 7feec5e8c700 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55f091a7b040 0x55f091a7d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.765+0000 7feea97fa700 10 client.16392 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.765+0000 7feec5e8c700 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55f091a7b040 0x55f091a7d460 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=168daed08569928e server_cookie=af7c98dbfc3f84ca in_seq=0 out_seq=0 2024-05-09T01:30:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feea97fa700 1 -- 172.21.15.19:0/3847937319 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f091991a80 con 0x55f091a7b040 2024-05-09T01:30:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feea97fa700 10 client.16392 handle_command_reply: tid=0 2024-05-09T01:30:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feed6b06b80 1 client.16392 shutdown 2024-05-09T01:30:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feed6b06b80 1 -- 172.21.15.19:0/3847937319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 msgr2=0x55f091a6bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a6bac0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7feeb8017040 tx=0x7feeb80097e0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feed6b06b80 1 -- 172.21.15.19:0/3847937319 shutdown_connections 2024-05-09T01:30:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55f091a7b040 0x55f091a7d460 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f091a61400 0x55f091a6bac0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feea97fa700 0 client.16392 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a6b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3847937319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a6b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.766+0000 7feed6b06b80 1 -- 172.21.15.19:0/3847937319 >> 172.21.15.19:0/3847937319 conn(0x55f0918e7f70 msgr2=0x55f091a55ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.767+0000 7feed6b06b80 1 -- 172.21.15.19:0/3847937319 shutdown_connections 2024-05-09T01:30:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.767+0000 7feed6b06b80 1 -- 172.21.15.19:0/3847937319 wait complete. 2024-05-09T01:30:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.767+0000 7feed6b06b80 20 client.16392 trim_cache size 0 max 16384 2024-05-09T01:30:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feed6b06b80 1 Processor -- start 2024-05-09T01:30:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feed6b06b80 1 -- start start 2024-05-09T01:30:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0918e7f70 0x55f091a56a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a65310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feed6b06b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b0fa0 con 0x55f0918e7f70 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feed6b06b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b2ae0 con 0x55f09195ed00 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feed6b06b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b1670 con 0x55f0918e44a0 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feec5e8c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0918e7f70 0x55f091a56a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feec5e8c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0918e7f70 0x55f091a56a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40812/0 (socket says 172.21.15.19:40812) 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feec5e8c700 1 -- 172.21.15.19:0/402245960 learned_addr learned my addr 172.21.15.19:0/402245960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feec568b700 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feec668d700 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a65310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feec5e8c700 1 -- 172.21.15.19:0/402245960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 msgr2=0x55f091a667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.782+0000 7feec5e8c700 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.783+0000 7feec5e8c700 1 -- 172.21.15.19:0/402245960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 msgr2=0x55f091a65310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.783+0000 7feec5e8c700 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a65310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.783+0000 7feec5e8c700 1 -- 172.21.15.19:0/402245960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f091a589d0 con 0x55f0918e7f70 2024-05-09T01:30:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.783+0000 7feec568b700 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.783+0000 7feec668d700 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.783+0000 7feec5e8c700 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0918e7f70 0x55f091a56a20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7feebc009990 tx=0x7feebc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f64c0b4e1bd17a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.783+0000 7feeab7fe700 1 -- 172.21.15.19:0/402245960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc00b9d0 con 0x55f0918e7f70 2024-05-09T01:30:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.783+0000 7feeab7fe700 1 -- 172.21.15.19:0/402245960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feebc03a070 con 0x55f0918e7f70 2024-05-09T01:30:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.783+0000 7feeab7fe700 1 -- 172.21.15.19:0/402245960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc035a60 con 0x55f0918e7f70 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 -- 172.21.15.19:0/402245960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0918e7f70 msgr2=0x55f091a56a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0918e7f70 0x55f091a56a20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7feebc009990 tx=0x7feebc01e000 comp rx=0 tx=0).stop 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 -- 172.21.15.19:0/402245960 shutdown_connections 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f09195ed00 0x55f091a65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f0918e7f70 0x55f091a56a20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 --2- 172.21.15.19:0/402245960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 -- 172.21.15.19:0/402245960 >> 172.21.15.19:0/402245960 conn(0x55f091a6bc10 msgr2=0x55f091a64c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 -- 172.21.15.19:0/402245960 shutdown_connections 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 -- 172.21.15.19:0/402245960 wait complete. 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 -- start start 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.784+0000 7feed6b06b80 1 Processor -- start 2024-05-09T01:30:02.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a73580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0918e7f70 0x55f091a70900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feed6b06b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f09195ed00 0x55f091a70e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feed6b06b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b2410 con 0x55f09195ed00 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feed6b06b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b1d40 con 0x55f0918e7f70 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feed6b06b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feec01aef40 con 0x55f0918e44a0 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feed6b06b80 10 client.0 _ll_register_callbacks cb 0x55f0917f56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec668d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f09195ed00 0x55f091a70e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec668d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f09195ed00 0x55f091a70e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40814/0 (socket says 172.21.15.19:40814) 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec668d700 1 -- 172.21.15.19:0/3021514969 learned_addr learned my addr 172.21.15.19:0/3021514969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec5e8c700 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a73580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec568b700 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0918e7f70 0x55f091a70900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec668d700 1 -- 172.21.15.19:0/3021514969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 msgr2=0x55f091a73580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec668d700 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a73580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec668d700 1 -- 172.21.15.19:0/3021514969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0918e7f70 msgr2=0x55f091a70900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec668d700 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0918e7f70 0x55f091a70900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec668d700 1 -- 172.21.15.19:0/3021514969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f091a62600 con 0x55f09195ed00 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feec668d700 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f09195ed00 0x55f091a70e40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feeb8007540 tx=0x7feeb8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3e93b36fac4d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.785+0000 7feea97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feea97fa700 1 -- 172.21.15.19:0/3021514969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8021040 con 0x55f09195ed00 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feea97fa700 1 -- 172.21.15.19:0/3021514969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7feeb8004a30 con 0x55f09195ed00 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feed6b06b80 1 -- 172.21.15.19:0/3021514969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feebc03a070 con 0x55f09195ed00 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feea97fa700 1 -- 172.21.15.19:0/3021514969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feeb8019040 con 0x55f09195ed00 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feed6b06b80 10 client.16398 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feed6b06b80 1 -- 172.21.15.19:0/3021514969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f091ae5280 con 0x55f09195ed00 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feea97fa700 1 -- 172.21.15.19:0/3021514969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feebc03a070 con 0x55f09195ed00 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feea97fa700 1 -- 172.21.15.19:0/3021514969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feeb801c070 con 0x55f09195ed00 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feed6b06b80 10 client.16398 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feed6b06b80 10 client.16398 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:30:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feed6b06b80 20 client.16398 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55f091a61400 0x55f091ae7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feed6b06b80 4 client.16398 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.786+0000 7feed6b06b80 1 -- 172.21.15.19:0/3021514969 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f09198bb20 con 0x55f091a61400 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.787+0000 7feec5e8c700 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55f091a61400 0x55f091ae7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.787+0000 7feec5e8c700 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55f091a61400 0x55f091ae7880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4cf6f969210628e5 server_cookie=440d59a200bb1b3b in_seq=0 out_seq=0 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.787+0000 7feea97fa700 10 client.16398 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.787+0000 7feea97fa700 1 -- 172.21.15.19:0/3021514969 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f09198bb20 con 0x55f091a61400 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feea97fa700 10 client.16398 handle_command_reply: tid=0 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 client.16398 shutdown 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 -- 172.21.15.19:0/3021514969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f09195ed00 msgr2=0x55f091a70e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f09195ed00 0x55f091a70e40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feeb8007540 tx=0x7feeb8009040 comp rx=0 tx=0).stop 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 -- 172.21.15.19:0/3021514969 shutdown_connections 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55f091a61400 0x55f091ae7880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f09195ed00 0x55f091a70e40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0918e44a0 0x55f091a73580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feea97fa700 0 client.16398 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:02.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 --2- 172.21.15.19:0/3021514969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0918e7f70 0x55f091a70900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 -- 172.21.15.19:0/3021514969 >> 172.21.15.19:0/3021514969 conn(0x55f091a6bc10 msgr2=0x55f091a6c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:02.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 -- 172.21.15.19:0/3021514969 shutdown_connections 2024-05-09T01:30:02.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 1 -- 172.21.15.19:0/3021514969 wait complete. 2024-05-09T01:30:02.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.788+0000 7feed6b06b80 20 client.16398 trim_cache size 0 max 16384 2024-05-09T01:30:02.790 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 -- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2880 msgr2=0x7feeac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2880 0x7feeac0b4d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feebc003800 tx=0x7feebc005fd0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 -- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec01039d0 msgr2=0x7feec019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec01039d0 0x7feec019d640 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7feeb000d950 tx=0x7feeb000dc60 comp rx=0 tx=0).stop 2024-05-09T01:30:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 -- 172.21.15.19:0/102792305 shutdown_connections 2024-05-09T01:30:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec0110c30 0x7feec01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec0103080 0x7feec019d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feeac0b2880 0x7feeac0b4d90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 --2- 172.21.15.19:0/102792305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feec01039d0 0x7feec019d640 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 -- 172.21.15.19:0/102792305 >> 172.21.15.19:0/102792305 conn(0x7feec00fedf0 msgr2=0x7feec0112830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 -- 172.21.15.19:0/102792305 shutdown_connections 2024-05-09T01:30:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:02.789+0000 7feec80f0700 1 -- 172.21.15.19:0/102792305 wait complete. 2024-05-09T01:30:03.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:30:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:04.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:04 smithi019 ceph-mon[30158]: pgmap v514: 97 pgs: 97 active+clean; 2.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:30:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:04 smithi196 ceph-mon[35041]: pgmap v514: 97 pgs: 97 active+clean; 2.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:30:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:04 smithi059 ceph-mon[34351]: pgmap v514: 97 pgs: 97 active+clean; 2.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:30:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.573+0000 7f891c526640 1 -- 172.21.15.19:0/3829691577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 msgr2=0x7f8914101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.573+0000 7f891c526640 1 --2- 172.21.15.19:0/3829691577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 0x7f8914101d30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8904009900 tx=0x7f890402a410 comp rx=0 tx=0).stop 2024-05-09T01:30:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.573+0000 7f891c526640 1 -- 172.21.15.19:0/3829691577 shutdown_connections 2024-05-09T01:30:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.573+0000 7f891c526640 1 --2- 172.21.15.19:0/3829691577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89141023c0 0x7f891410be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.573+0000 7f891c526640 1 --2- 172.21.15.19:0/3829691577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 0x7f8914101d30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.573+0000 7f891c526640 1 --2- 172.21.15.19:0/3829691577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8914105210 0x7f8914105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.573+0000 7f891c526640 1 -- 172.21.15.19:0/3829691577 >> 172.21.15.19:0/3829691577 conn(0x7f89140fd4c0 msgr2=0x7f89140ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.573+0000 7f891c526640 1 -- 172.21.15.19:0/3829691577 shutdown_connections 2024-05-09T01:30:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.573+0000 7f891c526640 1 -- 172.21.15.19:0/3829691577 wait complete. 2024-05-09T01:30:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891c526640 1 Processor -- start 2024-05-09T01:30:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891c526640 1 -- start start 2024-05-09T01:30:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891c526640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 0x7f891419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891c526640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89141023c0 0x7f891419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891c526640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914105210 0x7f89141a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891c526640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f891410f780 con 0x7f89141018c0 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891c526640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f891410f600 con 0x7f89141023c0 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891c526640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f891410f900 con 0x7f8914105210 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891a2c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 0x7f891419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891a2c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 0x7f891419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40834/0 (socket says 172.21.15.19:40834) 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f891a2c2640 1 -- 172.21.15.19:0/1460018164 learned_addr learned my addr 172.21.15.19:0/1460018164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.574+0000 7f8919ac1640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89141023c0 0x7f891419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891aac3640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914105210 0x7f89141a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891a2c2640 1 -- 172.21.15.19:0/1460018164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914105210 msgr2=0x7f89141a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891a2c2640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914105210 0x7f89141a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891a2c2640 1 -- 172.21.15.19:0/1460018164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89141023c0 msgr2=0x7f891419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891a2c2640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89141023c0 0x7f891419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891a2c2640 1 -- 172.21.15.19:0/1460018164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89141a1ed0 con 0x7f89141018c0 2024-05-09T01:30:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f8919ac1640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89141023c0 0x7f891419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891aac3640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914105210 0x7f89141a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891a2c2640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 0x7f891419a390 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8910005fd0 tx=0x7f891000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f890b7fe640 1 -- 172.21.15.19:0/1460018164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910017070 con 0x7f89141018c0 2024-05-09T01:30:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f890b7fe640 1 -- 172.21.15.19:0/1460018164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8910009c30 con 0x7f89141018c0 2024-05-09T01:30:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891c526640 1 -- 172.21.15.19:0/1460018164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89141a21c0 con 0x7f89141018c0 2024-05-09T01:30:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f890b7fe640 1 -- 172.21.15.19:0/1460018164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910003d10 con 0x7f89141018c0 2024-05-09T01:30:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.575+0000 7f891c526640 1 -- 172.21.15.19:0/1460018164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89141a2710 con 0x7f89141018c0 2024-05-09T01:30:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.578+0000 7f89233d8740 1 Processor -- start 2024-05-09T01:30:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.578+0000 7f89233d8740 1 -- start start 2024-05-09T01:30:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f890b7fe640 1 -- 172.21.15.19:0/1460018164 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8910003eb0 con 0x7f89141018c0 2024-05-09T01:30:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f89233d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f022b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f89233d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f022be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f89233d8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0232d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f89233d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c3f018b7c0 con 0x55c3efe831b0 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f89233d8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c3f01270e0 con 0x55c3efe624d0 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f89233d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c3f0187780 con 0x55c3efe5a2f0 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f891aac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f022b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f891aac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f022b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40836/0 (socket says 172.21.15.19:40836) 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f891aac3640 1 -- 172.21.15.19:0/3222715803 learned_addr learned my addr 172.21.15.19:0/3222715803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f891a2c2640 1 --2- 172.21.15.19:0/3222715803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f022be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f8919ac1640 1 --2- 172.21.15.19:0/3222715803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0232d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f891aac3640 1 -- 172.21.15.19:0/3222715803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 msgr2=0x55c3f022be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f891aac3640 1 --2- 172.21.15.19:0/3222715803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f022be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f891aac3640 1 -- 172.21.15.19:0/3222715803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 msgr2=0x55c3f0232d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f891aac3640 1 --2- 172.21.15.19:0/3222715803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0232d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.579+0000 7f891aac3640 1 -- 172.21.15.19:0/3222715803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3f01507c0 con 0x55c3efe831b0 2024-05-09T01:30:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.580+0000 7f890b7fe640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f89000b27e0 0x7f89000b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.580+0000 7f891aac3640 1 --2- 172.21.15.19:0/3222715803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f022b910 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f890c00f430 tx=0x7f890c00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2641a193a4a7d9ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.580+0000 7f891a2c2640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f89000b27e0 0x7f89000b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.580+0000 7f890b7fe640 1 -- 172.21.15.19:0/1460018164 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f89100e1ed0 con 0x7f89141018c0 2024-05-09T01:30:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.580+0000 7f8908bfe640 1 -- 172.21.15.19:0/3222715803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c015070 con 0x55c3efe831b0 2024-05-09T01:30:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.580+0000 7f8908bfe640 1 -- 172.21.15.19:0/3222715803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f890c011bb0 con 0x55c3efe831b0 2024-05-09T01:30:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.580+0000 7f8908bfe640 1 -- 172.21.15.19:0/3222715803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c010cd0 con 0x55c3efe831b0 2024-05-09T01:30:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f89233d8740 1 -- 172.21.15.19:0/3222715803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 msgr2=0x55c3f022b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f89233d8740 1 --2- 172.21.15.19:0/3222715803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f022b910 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f890c00f430 tx=0x7f890c00f8f0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f891a2c2640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f89000b27e0 0x7f89000b4cf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55c3f022ccf0 tx=0x7f89100e59b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f89233d8740 1 -- 172.21.15.19:0/3222715803 shutdown_connections 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f89233d8740 1 --2- 172.21.15.19:0/3222715803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0232d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f89233d8740 1 --2- 172.21.15.19:0/3222715803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f022be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f89233d8740 1 --2- 172.21.15.19:0/3222715803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f022b910 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f89233d8740 1 -- 172.21.15.19:0/3222715803 >> 172.21.15.19:0/3222715803 conn(0x55c3efe77300 msgr2=0x55c3f0222e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f89233d8740 1 -- 172.21.15.19:0/3222715803 shutdown_connections 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.581+0000 7f89233d8740 1 -- 172.21.15.19:0/3222715803 wait complete. 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f89233d8740 1 -- start start 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f89233d8740 1 Processor -- start 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f89233d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f89233d8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0235b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f89233d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe831b0 0x55c3f023cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f89233d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c3f02337e0 con 0x55c3efe5a2f0 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f89233d8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c3f01129c0 con 0x55c3efe624d0 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f89233d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c3f0233960 con 0x55c3efe831b0 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f89233d8740 10 client.0 _ll_register_callbacks cb 0x55c3f013f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f891aac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f891aac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40852/0 (socket says 172.21.15.19:40852) 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f891aac3640 1 -- 172.21.15.19:0/3391209714 learned_addr learned my addr 172.21.15.19:0/3391209714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f891aac3640 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe831b0 0x55c3f023cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f8919ac1640 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0235b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f891aac3640 1 -- 172.21.15.19:0/3391209714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe831b0 msgr2=0x55c3f023cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f891aac3640 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe831b0 0x55c3f023cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f891aac3640 1 -- 172.21.15.19:0/3391209714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 msgr2=0x55c3f0235b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f891aac3640 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0235b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.582+0000 7f891aac3640 1 -- 172.21.15.19:0/3391209714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3f01391f0 con 0x55c3efe5a2f0 2024-05-09T01:30:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f891aac3640 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235640 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f890c009590 tx=0x7f890c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ba18d2cc7f7ef5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f88fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f88fdffb640 1 -- 172.21.15.19:0/3391209714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c022040 con 0x55c3efe5a2f0 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f88fdffb640 1 -- 172.21.15.19:0/3391209714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f890c004a40 con 0x55c3efe5a2f0 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f89233d8740 1 -- 172.21.15.19:0/3391209714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f890c011bb0 con 0x55c3efe5a2f0 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f88fdffb640 1 -- 172.21.15.19:0/3391209714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f890c015040 con 0x55c3efe5a2f0 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f89233d8740 10 client.16410 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f89233d8740 1 -- 172.21.15.19:0/3391209714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3efeee790 con 0x55c3efe5a2f0 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f88fdffb640 1 -- 172.21.15.19:0/3391209714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f890c011bb0 con 0x55c3efe5a2f0 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f88fdffb640 1 -- 172.21.15.19:0/3391209714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f890c0050d0 con 0x55c3efe5a2f0 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f89233d8740 10 client.16410 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f89233d8740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f89233d8740 20 client.16410 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f89233d8740 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c3efe41d80 0x55c3f0242c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f89233d8740 4 client.16410 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.583+0000 7f89233d8740 1 -- 172.21.15.19:0/3391209714 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3f01993d0 con 0x55c3efe41d80 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.584+0000 7f8919ac1640 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c3efe41d80 0x55c3f0242c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.584+0000 7f8919ac1640 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c3efe41d80 0x55c3f0242c60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88275358fb06f9ee server_cookie=6c1e654dcea1a80c in_seq=0 out_seq=0 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.584+0000 7f88fdffb640 10 client.16410 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f88fdffb640 1 -- 172.21.15.19:0/3391209714 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c3f01993d0 con 0x55c3efe41d80 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f88fdffb640 10 client.16410 handle_command_reply: tid=0 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 client.16410 shutdown 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 -- 172.21.15.19:0/3391209714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe5a2f0 msgr2=0x55c3f0235640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235640 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f890c009590 tx=0x7f890c009520 comp rx=0 tx=0).stop 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 -- 172.21.15.19:0/3391209714 shutdown_connections 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c3efe41d80 0x55c3f0242c60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe831b0 0x55c3f023cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0235b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f88fdffb640 0 client.16410 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 --2- 172.21.15.19:0/3391209714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235640 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 -- 172.21.15.19:0/3391209714 >> 172.21.15.19:0/3391209714 conn(0x55c3efe77300 msgr2=0x55c3f0221700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.585+0000 7f89233d8740 1 -- 172.21.15.19:0/3391209714 shutdown_connections 2024-05-09T01:30:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.586+0000 7f89233d8740 1 -- 172.21.15.19:0/3391209714 wait complete. 2024-05-09T01:30:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.586+0000 7f89233d8740 20 client.16410 trim_cache size 0 max 16384 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.601+0000 7f89233d8740 1 Processor -- start 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.601+0000 7f89233d8740 1 -- start start 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.601+0000 7f89233d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.601+0000 7f89233d8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0247410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.601+0000 7f89233d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f0247950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.601+0000 7f89233d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89141a93b0 con 0x55c3efe831b0 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.601+0000 7f89233d8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f89141aaef0 con 0x55c3efe624d0 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.601+0000 7f89233d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89141a9a80 con 0x55c3efe5a2f0 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f0247950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f0247950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40858/0 (socket says 172.21.15.19:40858) 2024-05-09T01:30:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 -- 172.21.15.19:0/2767712756 learned_addr learned my addr 172.21.15.19:0/2767712756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f8919ac1640 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0247410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 -- 172.21.15.19:0/2767712756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 msgr2=0x55c3f0235920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 -- 172.21.15.19:0/2767712756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 msgr2=0x55c3f0247410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0247410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 -- 172.21.15.19:0/2767712756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3eff655f0 con 0x55c3efe831b0 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f8919ac1640 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0247410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f891aac3640 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f0247950 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f890c005820 tx=0x7f890c00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca760a316d59a6f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f88fffff640 1 -- 172.21.15.19:0/2767712756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c010810 con 0x55c3efe831b0 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f88fffff640 1 -- 172.21.15.19:0/2767712756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f890c010c60 con 0x55c3efe831b0 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.602+0000 7f88fffff640 1 -- 172.21.15.19:0/2767712756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c011500 con 0x55c3efe831b0 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.603+0000 7f89233d8740 1 -- 172.21.15.19:0/2767712756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 msgr2=0x55c3f0247950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.605 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.603+0000 7f89233d8740 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f0247950 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f890c005820 tx=0x7f890c00f8f0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.603+0000 7f89233d8740 1 -- 172.21.15.19:0/2767712756 shutdown_connections 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.603+0000 7f89233d8740 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f0247950 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.603+0000 7f89233d8740 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f0247410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.603+0000 7f89233d8740 1 --2- 172.21.15.19:0/2767712756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f0235920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.603+0000 7f89233d8740 1 -- 172.21.15.19:0/2767712756 >> 172.21.15.19:0/2767712756 conn(0x55c3efe77300 msgr2=0x55c3f0235210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.603+0000 7f89233d8740 1 -- 172.21.15.19:0/2767712756 shutdown_connections 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.603+0000 7f89233d8740 1 -- 172.21.15.19:0/2767712756 wait complete. 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f89233d8740 1 -- start start 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f89233d8740 1 Processor -- start 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f89233d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f02394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f89233d8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f02399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f89233d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f023a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f89233d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89141aa820 con 0x55c3efe831b0 2024-05-09T01:30:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f89233d8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f89141aa150 con 0x55c3efe624d0 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f89233d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89141a7390 con 0x55c3efe5a2f0 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f89233d8740 10 client.0 _ll_register_callbacks cb 0x7f8904009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f891aac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f023a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f8919ac1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f02399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f891aac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f023a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40866/0 (socket says 172.21.15.19:40866) 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.604+0000 7f891aac3640 1 -- 172.21.15.19:0/3165106188 learned_addr learned my addr 172.21.15.19:0/3165106188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f891aac3640 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f02394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f891aac3640 1 -- 172.21.15.19:0/3165106188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 msgr2=0x55c3f02394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f891aac3640 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f02394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f891aac3640 1 -- 172.21.15.19:0/3165106188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 msgr2=0x55c3f02399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f891aac3640 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f02399e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f891aac3640 1 -- 172.21.15.19:0/3165106188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3f014fed0 con 0x55c3efe831b0 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f8919ac1640 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f02399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f891aac3640 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f023a910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f890c009da0 tx=0x7f890c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10860d655f9b2271 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f88fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f88fdffb640 1 -- 172.21.15.19:0/3165106188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c0049e0 con 0x55c3efe831b0 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f89233d8740 1 -- 172.21.15.19:0/3165106188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f890c010c60 con 0x55c3efe831b0 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f88fdffb640 1 -- 172.21.15.19:0/3165106188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f890c004b80 con 0x55c3efe831b0 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f88fdffb640 1 -- 172.21.15.19:0/3165106188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f890c049070 con 0x55c3efe831b0 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f89233d8740 10 client.16416 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.605+0000 7f89233d8740 1 -- 172.21.15.19:0/3165106188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3f023bf90 con 0x55c3efe831b0 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.606+0000 7f88fdffb640 1 -- 172.21.15.19:0/3165106188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f890c010c60 con 0x55c3efe831b0 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.606+0000 7f88fdffb640 1 -- 172.21.15.19:0/3165106188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f890c0454d0 con 0x55c3efe831b0 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.606+0000 7f89233d8740 10 client.16416 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.606+0000 7f89233d8740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.606+0000 7f89233d8740 20 client.16416 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.606+0000 7f89233d8740 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c3efe171c0 0x55c3f02ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.606+0000 7f89233d8740 4 client.16416 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:30:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.606+0000 7f89233d8740 1 -- 172.21.15.19:0/3165106188 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3f0193240 con 0x55c3efe171c0 2024-05-09T01:30:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.606+0000 7f8919ac1640 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c3efe171c0 0x55c3f02ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.607+0000 7f88fdffb640 10 client.16416 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.607+0000 7f8919ac1640 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c3efe171c0 0x55c3f02ad1b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01c28e67114da07 server_cookie=cfb7923afae7db9c in_seq=0 out_seq=0 2024-05-09T01:30:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.619+0000 7f9fc025c700 1 -- 172.21.15.19:0/32695963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb8109870 msgr2=0x7f9fb81111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.619+0000 7f9fc025c700 1 --2- 172.21.15.19:0/32695963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb8109870 0x7f9fb81111d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b0d0 tx=0x7f9fb000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.619+0000 7f9fc025c700 1 -- 172.21.15.19:0/32695963 shutdown_connections 2024-05-09T01:30:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.619+0000 7f9fc025c700 1 --2- 172.21.15.19:0/32695963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb8109870 0x7f9fb81111d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.619+0000 7f9fc025c700 1 --2- 172.21.15.19:0/32695963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb81045f0 0x7f9fb8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.619+0000 7f9fc025c700 1 --2- 172.21.15.19:0/32695963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb8108f20 0x7f9fb8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.619+0000 7f9fc025c700 1 -- 172.21.15.19:0/32695963 >> 172.21.15.19:0/32695963 conn(0x7f9fb8100040 msgr2=0x7f9fb8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.619+0000 7f9fc025c700 1 -- 172.21.15.19:0/32695963 shutdown_connections 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.619+0000 7f9fc025c700 1 -- 172.21.15.19:0/32695963 wait complete. 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fc025c700 1 Processor -- start 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fc025c700 1 -- start start 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fc025c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb81045f0 0x7f9fb819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fc025c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb8108f20 0x7f9fb819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fc025c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb8109870 0x7f9fb819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fc025c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81144a0 con 0x7f9fb81045f0 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fc025c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8114320 con 0x7f9fb8108f20 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fc025c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8114620 con 0x7f9fb8109870 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fbdff8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb81045f0 0x7f9fb819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fbdff8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb81045f0 0x7f9fb819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40884/0 (socket says 172.21.15.19:40884) 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.620+0000 7f9fbdff8700 1 -- 172.21.15.19:0/2412611879 learned_addr learned my addr 172.21.15.19:0/2412611879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fbe7f9700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb8109870 0x7f9fb819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb8108f20 0x7f9fb819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fbdff8700 1 -- 172.21.15.19:0/2412611879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb8109870 msgr2=0x7f9fb819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb8109870 0x7f9fb819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fbdff8700 1 -- 172.21.15.19:0/2412611879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb8108f20 msgr2=0x7f9fb819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb8108f20 0x7f9fb819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fbdff8700 1 -- 172.21.15.19:0/2412611879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb819fc80 con 0x7f9fb81045f0 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb81045f0 0x7f9fb819adb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400e9a0 tx=0x7f9fb400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9faeffd700 1 -- 172.21.15.19:0/2412611879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400ca10 con 0x7f9fb81045f0 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9faeffd700 1 -- 172.21.15.19:0/2412611879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb400cbb0 con 0x7f9fb81045f0 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fc025c700 1 -- 172.21.15.19:0/2412611879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb819ff70 con 0x7f9fb81045f0 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9faeffd700 1 -- 172.21.15.19:0/2412611879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4018680 con 0x7f9fb81045f0 2024-05-09T01:30:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.621+0000 7f9fc025c700 1 -- 172.21.15.19:0/2412611879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb81a7890 con 0x7f9fb81045f0 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f88fdffb640 1 -- 172.21.15.19:0/3165106188 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c3f0193240 con 0x55c3efe171c0 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f88fdffb640 10 client.16416 handle_command_reply: tid=0 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 client.16416 shutdown 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 -- 172.21.15.19:0/3165106188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 msgr2=0x55c3f023a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f023a910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f890c009da0 tx=0x7f890c009f20 comp rx=0 tx=0).stop 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 -- 172.21.15.19:0/3165106188 shutdown_connections 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55c3efe171c0 0x55c3f02ad1b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c3efe831b0 0x55c3f023a910 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f88fdffb640 0 client.16416 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c3efe624d0 0x55c3f02399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 --2- 172.21.15.19:0/3165106188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3efe5a2f0 0x55c3f02394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 -- 172.21.15.19:0/3165106188 >> 172.21.15.19:0/3165106188 conn(0x55c3efe77300 msgr2=0x55c3f0225c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 -- 172.21.15.19:0/3165106188 shutdown_connections 2024-05-09T01:30:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.623+0000 7f89233d8740 1 -- 172.21.15.19:0/3165106188 wait complete. 2024-05-09T01:30:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f89233d8740 20 client.16416 trim_cache size 0 max 16384 2024-05-09T01:30:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:30:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:30:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:30:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 -- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f89000b27e0 msgr2=0x7f89000b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f89000b27e0 0x7f89000b4cf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55c3f022ccf0 tx=0x7f89100e59b0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 -- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 msgr2=0x7f891419a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 0x7f891419a390 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8910005fd0 tx=0x7f891000dde0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f9faeffd700 1 -- 172.21.15.19:0/2412611879 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9fb4010430 con 0x7f9fb81045f0 2024-05-09T01:30:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 -- 172.21.15.19:0/1460018164 shutdown_connections 2024-05-09T01:30:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f89000b27e0 0x7f89000b4cf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8914105210 0x7f89141a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89141023c0 0x7f891419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 --2- 172.21.15.19:0/1460018164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89141018c0 0x7f891419a390 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 -- 172.21.15.19:0/1460018164 >> 172.21.15.19:0/1460018164 conn(0x7f89140fd4c0 msgr2=0x7f8914105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 -- 172.21.15.19:0/1460018164 shutdown_connections 2024-05-09T01:30:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:05.624+0000 7f891c526640 1 -- 172.21.15.19:0/1460018164 wait complete. 2024-05-09T01:30:05.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.625+0000 7f9faeffd700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fa40b2880 0x7f9fa40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.625+0000 7f9faeffd700 1 -- 172.21.15.19:0/2412611879 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9fb40e2380 con 0x7f9fb81045f0 2024-05-09T01:30:05.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.625+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fa40b2880 0x7f9fa40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.625+0000 7f9fcec72b80 1 Processor -- start 2024-05-09T01:30:05.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fcec72b80 1 -- start start 2024-05-09T01:30:05.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fa40b2880 0x7f9fa40b4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8103a40 tx=0x7f9fa8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x5615177fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615176754a0 0x5615177fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615177f6ab0 0x5615177f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561517714ff0 con 0x5615177f6ab0 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5615176b37f0 con 0x5615176754a0 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561517710fb0 con 0x56151775f160 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fbdff8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x5615177fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fbdff8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x5615177fc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:44738/0 (socket says 172.21.15.19:44738) 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fbe7f9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615177f6ab0 0x5615177f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fbdff8700 1 -- 172.21.15.19:0/912281129 learned_addr learned my addr 172.21.15.19:0/912281129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.626+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/912281129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615176754a0 0x5615177fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fbdff8700 1 -- 172.21.15.19:0/912281129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615176754a0 msgr2=0x5615177fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/912281129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615176754a0 0x5615177fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fbdff8700 1 -- 172.21.15.19:0/912281129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615177f6ab0 msgr2=0x5615177f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/912281129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615177f6ab0 0x5615177f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fbdff8700 1 -- 172.21.15.19:0/912281129 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615176ab370 con 0x56151775f160 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/912281129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615176754a0 0x5615177fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/912281129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x5615177fc8b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40a5a00 tx=0x7f9fb40a9400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f122f91c99ceb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fa3fff700 1 -- 172.21.15.19:0/912281129 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40a9980 con 0x56151775f160 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fa3fff700 1 -- 172.21.15.19:0/912281129 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb40add30 con 0x56151775f160 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.627+0000 7f9fa3fff700 1 -- 172.21.15.19:0/912281129 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40b2350 con 0x56151775f160 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.628+0000 7f9fcec72b80 1 -- 172.21.15.19:0/912281129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 msgr2=0x5615177fc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.628+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/912281129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x5615177fc8b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40a5a00 tx=0x7f9fb40a9400 comp rx=0 tx=0).stop 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.628+0000 7f9fcec72b80 1 -- 172.21.15.19:0/912281129 shutdown_connections 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.628+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/912281129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615177f6ab0 0x5615177f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.628+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/912281129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x5615177fc8b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.628+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/912281129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615176754a0 0x5615177fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.628+0000 7f9fcec72b80 1 -- 172.21.15.19:0/912281129 >> 172.21.15.19:0/912281129 conn(0x5615176d56a0 msgr2=0x5615177ed400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.629+0000 7f9fcec72b80 1 -- 172.21.15.19:0/912281129 shutdown_connections 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.629+0000 7f9fcec72b80 1 -- 172.21.15.19:0/912281129 wait complete. 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.629+0000 7f9fcec72b80 1 -- start start 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.629+0000 7f9fcec72b80 1 Processor -- start 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615176754a0 0x561517800640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x561517800b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x5615178010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561517590ba0 con 0x5615176754a0 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5615176cf150 con 0x5615177f6ab0 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615177f7890 con 0x56151775f160 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fcec72b80 10 client.0 _ll_register_callbacks cb 0x5615175866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbdff8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615176754a0 0x561517800640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbdff8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615176754a0 0x561517800640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40896/0 (socket says 172.21.15.19:40896) 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbdff8700 1 -- 172.21.15.19:0/2359730480 learned_addr learned my addr 172.21.15.19:0/2359730480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbe7f9700 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x5615178010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x561517800b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbdff8700 1 -- 172.21.15.19:0/2359730480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 msgr2=0x561517800b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x561517800b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbdff8700 1 -- 172.21.15.19:0/2359730480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 msgr2=0x5615178010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x5615178010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbdff8700 1 -- 172.21.15.19:0/2359730480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561517590490 con 0x5615176754a0 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615176754a0 0x561517800640 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40a6bf0 tx=0x7f9fb40a5120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9951db0334be8253 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.630+0000 7f9fa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fa17fa700 1 -- 172.21.15.19:0/2359730480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb406e060 con 0x5615176754a0 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fa17fa700 1 -- 172.21.15.19:0/2359730480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9fb40b2560 con 0x5615176754a0 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2359730480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb40add30 con 0x5615176754a0 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fa17fa700 1 -- 172.21.15.19:0/2359730480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9fb406f250 con 0x5615176754a0 2024-05-09T01:30:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fcec72b80 10 client.16428 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2359730480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56151780df20 con 0x5615176754a0 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fa17fa700 1 -- 172.21.15.19:0/2359730480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb406f3f0 con 0x5615176754a0 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fa17fa700 1 -- 172.21.15.19:0/2359730480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9fb40a8650 con 0x5615176754a0 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fcec72b80 10 client.16428 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fcec72b80 10 client.16428 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fcec72b80 20 client.16428 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5615178103f0 0x561517812810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fcec72b80 4 client.16428 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.631+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2359730480 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9fb40add30 con 0x5615178103f0 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.632+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5615178103f0 0x561517812810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.632+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5615178103f0 0x561517812810 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a896b7ffdbbba879 server_cookie=eda97e5b6eab1ede in_seq=0 out_seq=0 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.632+0000 7f9fa17fa700 10 client.16428 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.633+0000 7f9fa17fa700 1 -- 172.21.15.19:0/2359730480 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9fb40add30 con 0x5615178103f0 2024-05-09T01:30:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.633+0000 7f9fa17fa700 10 client.16428 handle_command_reply: tid=0 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.633+0000 7f9fcec72b80 1 client.16428 shutdown 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.633+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2359730480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615176754a0 msgr2=0x561517800640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.633+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615176754a0 0x561517800640 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40a6bf0 tx=0x7f9fb40a5120 comp rx=0 tx=0).stop 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.633+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2359730480 shutdown_connections 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.633+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5615178103f0 0x561517812810 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.633+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x5615178010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.634+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615176754a0 0x561517800640 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.634+0000 7f9fa17fa700 0 client.16428 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.634+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2359730480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56151775f160 0x561517800b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.634+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2359730480 >> 172.21.15.19:0/2359730480 conn(0x5615176d56a0 msgr2=0x5615177ed1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.634+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2359730480 shutdown_connections 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.634+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2359730480 wait complete. 2024-05-09T01:30:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.634+0000 7f9fcec72b80 20 client.16428 trim_cache size 0 max 16384 2024-05-09T01:30:05.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.650+0000 7f9fcec72b80 1 Processor -- start 2024-05-09T01:30:05.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.650+0000 7f9fcec72b80 1 -- start start 2024-05-09T01:30:05.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.650+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 0x56151780c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.650+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517800f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.650+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x5615177f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.650+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81acb60 con 0x56151775f160 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.650+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81ae6a0 con 0x5615177f6ab0 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.650+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81ad230 con 0x5615176754a0 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbd7f7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517800f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbd7f7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517800f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40912/0 (socket says 172.21.15.19:40912) 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbd7f7700 1 -- 172.21.15.19:0/2791214593 learned_addr learned my addr 172.21.15.19:0/2791214593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 0x56151780c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbe7f9700 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x5615177f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbd7f7700 1 -- 172.21.15.19:0/2791214593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 msgr2=0x56151780c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 0x56151780c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbd7f7700 1 -- 172.21.15.19:0/2791214593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 msgr2=0x5615177f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x5615177f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbd7f7700 1 -- 172.21.15.19:0/2791214593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561517801440 con 0x56151775f160 2024-05-09T01:30:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 0x56151780c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517800f00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009fd0 tx=0x7f9fa8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c972167f9c93539 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fa37fe700 1 -- 172.21.15.19:0/2791214593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa802e070 con 0x56151775f160 2024-05-09T01:30:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fa37fe700 1 -- 172.21.15.19:0/2791214593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa80044e0 con 0x56151775f160 2024-05-09T01:30:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.651+0000 7f9fa37fe700 1 -- 172.21.15.19:0/2791214593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8003b20 con 0x56151775f160 2024-05-09T01:30:05.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.652+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2791214593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 msgr2=0x561517800f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.652+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517800f00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009fd0 tx=0x7f9fa8014000 comp rx=0 tx=0).stop 2024-05-09T01:30:05.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.652+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2791214593 shutdown_connections 2024-05-09T01:30:05.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.652+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x5615177f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.652+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 0x56151780c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.652+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/2791214593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517800f00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.652+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2791214593 >> 172.21.15.19:0/2791214593 conn(0x5615176d56a0 msgr2=0x5615177ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.652+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2791214593 shutdown_connections 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.652+0000 7f9fcec72b80 1 -- 172.21.15.19:0/2791214593 wait complete. 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fcec72b80 1 -- start start 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fcec72b80 1 Processor -- start 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 0x5615177f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517815840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fcec72b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x561517815d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81adfd0 con 0x56151775f160 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81ad900 con 0x5615177f6ab0 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fcec72b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81aab20 con 0x5615176754a0 2024-05-09T01:30:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fcec72b80 10 client.0 _ll_register_callbacks cb 0x5615175866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.653+0000 7f9fbd7f7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517815840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbd7f7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517815840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40922/0 (socket says 172.21.15.19:40922) 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbd7f7700 1 -- 172.21.15.19:0/1224079177 learned_addr learned my addr 172.21.15.19:0/1224079177 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 0x5615177f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbe7f9700 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x561517815d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbd7f7700 1 -- 172.21.15.19:0/1224079177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 msgr2=0x5615177f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 0x5615177f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbd7f7700 1 -- 172.21.15.19:0/1224079177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 msgr2=0x561517815d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x561517815d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbd7f7700 1 -- 172.21.15.19:0/1224079177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615177f3540 con 0x56151775f160 2024-05-09T01:30:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbe7f9700 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x561517815d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fbd7f7700 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517815840 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8005950 tx=0x7f9fa80090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c788184e54ac56 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fa17fa700 1 -- 172.21.15.19:0/1224079177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa802e070 con 0x56151775f160 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fa17fa700 1 -- 172.21.15.19:0/1224079177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9fa8020d30 con 0x56151775f160 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.654+0000 7f9fcec72b80 1 -- 172.21.15.19:0/1224079177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa80044e0 con 0x56151775f160 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fa17fa700 1 -- 172.21.15.19:0/1224079177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9fa8005040 con 0x56151775f160 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fcec72b80 10 client.16434 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fcec72b80 1 -- 172.21.15.19:0/1224079177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56151787f180 con 0x56151775f160 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fa17fa700 1 -- 172.21.15.19:0/1224079177 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa80044e0 con 0x56151775f160 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fa17fa700 1 -- 172.21.15.19:0/1224079177 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9fa8028ce0 con 0x56151775f160 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fcec72b80 10 client.16434 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fcec72b80 10 client.16434 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:30:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fcec72b80 20 client.16434 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5615178103f0 0x561517881760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fcec72b80 4 client.16434 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fcec72b80 1 -- 172.21.15.19:0/1224079177 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615177229d0 con 0x5615178103f0 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.655+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5615178103f0 0x561517881760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fa17fa700 10 client.16434 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fbdff8700 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5615178103f0 0x561517881760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45d58cc176c302aa server_cookie=cc98defbde4ecb33 in_seq=0 out_seq=0 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fa17fa700 1 -- 172.21.15.19:0/1224079177 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615177229d0 con 0x5615178103f0 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fa17fa700 10 client.16434 handle_command_reply: tid=0 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 client.16434 shutdown 2024-05-09T01:30:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 -- 172.21.15.19:0/1224079177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 msgr2=0x561517815840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517815840 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8005950 tx=0x7f9fa80090d0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 -- 172.21.15.19:0/1224079177 shutdown_connections 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5615178103f0 0x561517881760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5615177f6ab0 0x561517815d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615176754a0 0x5615177f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 --2- 172.21.15.19:0/1224079177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56151775f160 0x561517815840 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fa17fa700 0 client.16434 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 -- 172.21.15.19:0/1224079177 >> 172.21.15.19:0/1224079177 conn(0x5615176d56a0 msgr2=0x5615178023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 -- 172.21.15.19:0/1224079177 shutdown_connections 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 1 -- 172.21.15.19:0/1224079177 wait complete. 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.656+0000 7f9fcec72b80 20 client.16434 trim_cache size 0 max 16384 2024-05-09T01:30:05.660 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 -- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fa40b2880 msgr2=0x7f9fa40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fa40b2880 0x7f9fa40b4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8103a40 tx=0x7f9fa8009260 comp rx=0 tx=0).stop 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 -- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb81045f0 msgr2=0x7f9fb819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb81045f0 0x7f9fb819adb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400e9a0 tx=0x7f9fb400ecb0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 -- 172.21.15.19:0/2412611879 shutdown_connections 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb8109870 0x7f9fb819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9fa40b2880 0x7f9fa40b4d90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb81045f0 0x7f9fb819adb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 --2- 172.21.15.19:0/2412611879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb8108f20 0x7f9fb819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 -- 172.21.15.19:0/2412611879 >> 172.21.15.19:0/2412611879 conn(0x7f9fb8100040 msgr2=0x7f9fb8108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:05.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 -- 172.21.15.19:0/2412611879 shutdown_connections 2024-05-09T01:30:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:05.657+0000 7f9fc025c700 1 -- 172.21.15.19:0/2412611879 wait complete. 2024-05-09T01:30:06.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:30:06.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:06 smithi019 ceph-mon[30158]: pgmap v515: 97 pgs: 97 active+clean; 2.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:30:06.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:06 smithi196 ceph-mon[35041]: pgmap v515: 97 pgs: 97 active+clean; 2.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:30:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:06 smithi059 ceph-mon[34351]: pgmap v515: 97 pgs: 97 active+clean; 2.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:30:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.411+0000 7f9145af9700 1 -- 172.21.15.19:0/1035586572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 msgr2=0x7f91401111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.411+0000 7f9145af9700 1 --2- 172.21.15.19:0/1035586572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 0x7f91401111e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f913800b0d0 tx=0x7f913800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.409 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:08 smithi019 ceph-mon[30158]: pgmap v516: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.411+0000 7f9145af9700 1 -- 172.21.15.19:0/1035586572 shutdown_connections 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.411+0000 7f9145af9700 1 --2- 172.21.15.19:0/1035586572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 0x7f91401111e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.411+0000 7f9145af9700 1 --2- 172.21.15.19:0/1035586572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9140104650 0x7f9140104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.411+0000 7f9145af9700 1 --2- 172.21.15.19:0/1035586572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9140108f30 0x7f9140109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.411+0000 7f9145af9700 1 -- 172.21.15.19:0/1035586572 >> 172.21.15.19:0/1035586572 conn(0x7f9140079f00 msgr2=0x7f914007a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.411+0000 7f9145af9700 1 -- 172.21.15.19:0/1035586572 shutdown_connections 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.411+0000 7f9145af9700 1 -- 172.21.15.19:0/1035586572 wait complete. 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.412+0000 7f9145af9700 1 Processor -- start 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.412+0000 7f9145af9700 1 -- start start 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.412+0000 7f9145af9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9140104650 0x7f914019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.412+0000 7f9145af9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9140108f30 0x7f914019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.412+0000 7f9145af9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 0x7f914019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f9145af9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91401144b0 con 0x7f9140109880 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f9145af9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9140114330 con 0x7f9140108f30 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f9145af9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9140114630 con 0x7f9140104650 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 0x7f914019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9140104650 0x7f914019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 0x7f914019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40946/0 (socket says 172.21.15.19:40946) 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913ffff700 1 -- 172.21.15.19:0/2447281802 learned_addr learned my addr 172.21.15.19:0/2447281802 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913effd700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9140108f30 0x7f914019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913f7fe700 1 -- 172.21.15.19:0/2447281802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9140108f30 msgr2=0x7f914019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913f7fe700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9140108f30 0x7f914019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913f7fe700 1 -- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 msgr2=0x7f914019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913f7fe700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 0x7f914019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913f7fe700 1 -- 172.21.15.19:0/2447281802 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914019fbe0 con 0x7f9140104650 2024-05-09T01:30:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913ffff700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 0x7f914019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.413+0000 7f913f7fe700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9140104650 0x7f914019adc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f913000d950 tx=0x7f913000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.414+0000 7f913cff9700 1 -- 172.21.15.19:0/2447281802 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130016070 con 0x7f9140104650 2024-05-09T01:30:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.414+0000 7f9145af9700 1 -- 172.21.15.19:0/2447281802 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914019fed0 con 0x7f9140104650 2024-05-09T01:30:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.414+0000 7f913cff9700 1 -- 172.21.15.19:0/2447281802 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f913000fd40 con 0x7f9140104650 2024-05-09T01:30:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.414+0000 7f913cff9700 1 -- 172.21.15.19:0/2447281802 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913000e3b0 con 0x7f9140104650 2024-05-09T01:30:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.414+0000 7f9145af9700 1 -- 172.21.15.19:0/2447281802 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401a0410 con 0x7f9140104650 2024-05-09T01:30:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.416+0000 7f913cff9700 1 -- 172.21.15.19:0/2447281802 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9130019070 con 0x7f9140104650 2024-05-09T01:30:08.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.417+0000 7f913cff9700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f912c0b2880 0x7f912c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.417+0000 7f913effd700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f912c0b2880 0x7f912c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.417+0000 7f913cff9700 1 -- 172.21.15.19:0/2447281802 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f91300e1c90 con 0x7f9140104650 2024-05-09T01:30:08.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f913effd700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f912c0b2880 0x7f912c0b4d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9140109810 tx=0x7f912800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:08.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f915450fb80 1 Processor -- start 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f915450fb80 1 -- start start 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f915450fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f915450fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 0x5600371f6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f915450fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 0x5600371f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f915450fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5600371130a0 con 0x5600370edd00 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f915450fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5600370b89a0 con 0x5600370734a0 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f915450fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56003710f060 con 0x5600371f0400 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f913f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f913f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40950/0 (socket says 172.21.15.19:40950) 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f913f7fe700 1 -- 172.21.15.19:0/4217775123 learned_addr learned my addr 172.21.15.19:0/4217775123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.418+0000 7f913effd700 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 0x5600371f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f913ffff700 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 0x5600371f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f913f7fe700 1 -- 172.21.15.19:0/4217775123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 msgr2=0x5600371f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f913f7fe700 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 0x5600371f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f913f7fe700 1 -- 172.21.15.19:0/4217775123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 msgr2=0x5600371f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f913f7fe700 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 0x5600371f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f913f7fe700 1 -- 172.21.15.19:0/4217775123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600370a9500 con 0x5600370edd00 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f913ffff700 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 0x5600371f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f913f7fe700 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371f6240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9130005950 tx=0x7f91300a89a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce831c3b495c0005 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f9135ffb700 1 -- 172.21.15.19:0/4217775123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130020040 con 0x5600370edd00 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f9135ffb700 1 -- 172.21.15.19:0/4217775123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91300a9930 con 0x5600370edd00 2024-05-09T01:30:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.419+0000 7f9135ffb700 1 -- 172.21.15.19:0/4217775123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91300dfe30 con 0x5600370edd00 2024-05-09T01:30:08.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 -- 172.21.15.19:0/4217775123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 msgr2=0x5600371f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:08.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371f6240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9130005950 tx=0x7f91300a89a0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 -- 172.21.15.19:0/4217775123 shutdown_connections 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 0x5600371f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371f6240 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 --2- 172.21.15.19:0/4217775123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 0x5600371f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 -- 172.21.15.19:0/4217775123 >> 172.21.15.19:0/4217775123 conn(0x560037076f70 msgr2=0x5600371eb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 -- 172.21.15.19:0/4217775123 shutdown_connections 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 -- 172.21.15.19:0/4217775123 wait complete. 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.420+0000 7f915450fb80 1 -- start start 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f915450fb80 1 Processor -- start 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f915450fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 0x5600371fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f915450fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f915450fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 0x5600371faac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f915450fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5600371f10a0 con 0x5600370edd00 2024-05-09T01:30:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f915450fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5600370adb20 con 0x5600370734a0 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f915450fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5600371f1220 con 0x5600371f0400 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f915450fb80 10 client.0 _ll_register_callbacks cb 0x560036f846c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f913effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f913effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40952/0 (socket says 172.21.15.19:40952) 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f913effd700 1 -- 172.21.15.19:0/3207900199 learned_addr learned my addr 172.21.15.19:0/3207900199 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f913f7fe700 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 0x5600371fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.421+0000 7f913ffff700 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 0x5600371faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f913effd700 1 -- 172.21.15.19:0/3207900199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 msgr2=0x5600371faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f913effd700 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 0x5600371faac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f913effd700 1 -- 172.21.15.19:0/3207900199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 msgr2=0x5600371fa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f913effd700 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 0x5600371fa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f913effd700 1 -- 172.21.15.19:0/3207900199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560036f8e490 con 0x5600370edd00 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f913f7fe700 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 0x5600371fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f913effd700 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fa580 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9128003680 tx=0x7f91280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514f48c393286d03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f911effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f911effd700 1 -- 172.21.15.19:0/3207900199 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128016070 con 0x5600370edd00 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f911effd700 1 -- 172.21.15.19:0/3207900199 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9128010840 con 0x5600370edd00 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f915450fb80 1 -- 172.21.15.19:0/3207900199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91300a9930 con 0x5600370edd00 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f911effd700 1 -- 172.21.15.19:0/3207900199 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f912802d070 con 0x5600370edd00 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f915450fb80 10 client.16446 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.422+0000 7f915450fb80 1 -- 172.21.15.19:0/3207900199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560037207930 con 0x5600370edd00 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.423+0000 7f911effd700 1 -- 172.21.15.19:0/3207900199 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91300a9930 con 0x5600370edd00 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.423+0000 7f911effd700 1 -- 172.21.15.19:0/3207900199 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9128027990 con 0x5600370edd00 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.423+0000 7f915450fb80 10 client.16446 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.423+0000 7f915450fb80 10 client.16446 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.423+0000 7f915450fb80 20 client.16446 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.423+0000 7f915450fb80 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560037209f70 0x56003720c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.423+0000 7f915450fb80 4 client.16446 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.423+0000 7f915450fb80 1 -- 172.21.15.19:0/3207900199 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560037120a80 con 0x560037209f70 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.423+0000 7f913f7fe700 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560037209f70 0x56003720c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.424+0000 7f913f7fe700 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560037209f70 0x56003720c390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6137bcae9fab81e8 server_cookie=8306b6924f61167 in_seq=0 out_seq=0 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.424+0000 7f911effd700 10 client.16446 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.424+0000 7f911effd700 1 -- 172.21.15.19:0/3207900199 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560037120a80 con 0x560037209f70 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.424+0000 7f911effd700 10 client.16446 handle_command_reply: tid=0 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.424+0000 7f915450fb80 1 client.16446 shutdown 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 -- 172.21.15.19:0/3207900199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 msgr2=0x5600371fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fa580 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9128003680 tx=0x7f91280047c0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 -- 172.21.15.19:0/3207900199 shutdown_connections 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560037209f70 0x56003720c390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600371f0400 0x5600371faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600370734a0 0x5600371fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 --2- 172.21.15.19:0/3207900199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fa580 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 -- 172.21.15.19:0/3207900199 >> 172.21.15.19:0/3207900199 conn(0x560037076f70 msgr2=0x5600371e4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:08.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f911effd700 0 client.16446 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:08.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 -- 172.21.15.19:0/3207900199 shutdown_connections 2024-05-09T01:30:08.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 1 -- 172.21.15.19:0/3207900199 wait complete. 2024-05-09T01:30:08.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.425+0000 7f915450fb80 20 client.16446 trim_cache size 0 max 16384 2024-05-09T01:30:08.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f915450fb80 1 Processor -- start 2024-05-09T01:30:08.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f915450fb80 1 -- start start 2024-05-09T01:30:08.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f915450fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 0x5600371f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f915450fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f915450fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 0x560037205d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f915450fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91401aed10 con 0x5600370edd00 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f915450fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91401b0850 con 0x5600371f0400 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f915450fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91401af3e0 con 0x5600370734a0 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f913effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.440+0000 7f913effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40964/0 (socket says 172.21.15.19:40964) 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913effd700 1 -- 172.21.15.19:0/1402392706 learned_addr learned my addr 172.21.15.19:0/1402392706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913f7fe700 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 0x5600371f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913ffff700 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 0x560037205d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913effd700 1 -- 172.21.15.19:0/1402392706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 msgr2=0x5600371f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913effd700 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 0x5600371f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913effd700 1 -- 172.21.15.19:0/1402392706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 msgr2=0x560037205d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913effd700 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 0x560037205d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913effd700 1 -- 172.21.15.19:0/1402392706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600371e7510 con 0x5600370edd00 2024-05-09T01:30:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913f7fe700 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 0x5600371f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:08.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f913effd700 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fc4e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9128027e20 tx=0x7f9128000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1768867743791c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:08.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f91357fa700 1 -- 172.21.15.19:0/1402392706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128027620 con 0x5600370edd00 2024-05-09T01:30:08.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f91357fa700 1 -- 172.21.15.19:0/1402392706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9128022030 con 0x5600370edd00 2024-05-09T01:30:08.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.441+0000 7f91357fa700 1 -- 172.21.15.19:0/1402392706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128032480 con 0x5600370edd00 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.442+0000 7f915450fb80 1 -- 172.21.15.19:0/1402392706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 msgr2=0x5600371fc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.442+0000 7f915450fb80 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fc4e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9128027e20 tx=0x7f9128000bc0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.442+0000 7f915450fb80 1 -- 172.21.15.19:0/1402392706 shutdown_connections 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.442+0000 7f915450fb80 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 0x560037205d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.442+0000 7f915450fb80 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 0x5600371f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.442+0000 7f915450fb80 1 --2- 172.21.15.19:0/1402392706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600371fc4e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.442+0000 7f915450fb80 1 -- 172.21.15.19:0/1402392706 >> 172.21.15.19:0/1402392706 conn(0x560037076f70 msgr2=0x5600371f51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.442+0000 7f915450fb80 1 -- 172.21.15.19:0/1402392706 shutdown_connections 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.442+0000 7f915450fb80 1 -- 172.21.15.19:0/1402392706 wait complete. 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f915450fb80 1 -- start start 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f915450fb80 1 Processor -- start 2024-05-09T01:30:08.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f915450fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 0x560037211e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f915450fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600372020f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f915450fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 0x560037202630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f915450fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91401b0180 con 0x5600370edd00 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f915450fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91401afab0 con 0x5600371f0400 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f915450fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91401accb0 con 0x5600370734a0 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f915450fb80 10 client.0 _ll_register_callbacks cb 0x560036f846c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f913effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600372020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f913effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600372020f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40980/0 (socket says 172.21.15.19:40980) 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.443+0000 7f913effd700 1 -- 172.21.15.19:0/1943561370 learned_addr learned my addr 172.21.15.19:0/1943561370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f913ffff700 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 0x560037202630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f913f7fe700 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 0x560037211e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f913effd700 1 -- 172.21.15.19:0/1943561370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 msgr2=0x560037211e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f913effd700 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 0x560037211e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f913effd700 1 -- 172.21.15.19:0/1943561370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 msgr2=0x560037202630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f913effd700 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 0x560037202630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f913effd700 1 -- 172.21.15.19:0/1943561370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560037210b20 con 0x5600370edd00 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f913ffff700 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 0x560037202630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f913effd700 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600372020f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f912800a9b0 tx=0x7f9128011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2760d3b26e5234ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f911effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f911effd700 1 -- 172.21.15.19:0/1943561370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128016070 con 0x5600370edd00 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f911effd700 1 -- 172.21.15.19:0/1943561370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9128005240 con 0x5600370edd00 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f915450fb80 1 -- 172.21.15.19:0/1943561370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9128022030 con 0x5600370edd00 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f911effd700 1 -- 172.21.15.19:0/1943561370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9128005440 con 0x5600370edd00 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f915450fb80 10 client.16452 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f915450fb80 1 -- 172.21.15.19:0/1943561370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560037277af0 con 0x5600370edd00 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.444+0000 7f911effd700 1 -- 172.21.15.19:0/1943561370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9128022030 con 0x5600370edd00 2024-05-09T01:30:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f911effd700 1 -- 172.21.15.19:0/1943561370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9128039070 con 0x5600370edd00 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f915450fb80 10 client.16452 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f915450fb80 10 client.16452 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f915450fb80 20 client.16452 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f915450fb80 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560037278060 0x56003727a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f915450fb80 4 client.16452 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f915450fb80 1 -- 172.21.15.19:0/1943561370 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56003711ab20 con 0x560037278060 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f913f7fe700 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560037278060 0x56003727a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f911effd700 10 client.16452 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.445+0000 7f913f7fe700 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560037278060 0x56003727a480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb938db2ba2cdfb server_cookie=57d07f4293653165 in_seq=0 out_seq=0 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f911effd700 1 -- 172.21.15.19:0/1943561370 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56003711ab20 con 0x560037278060 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f911effd700 10 client.16452 handle_command_reply: tid=0 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 client.16452 shutdown 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 -- 172.21.15.19:0/1943561370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 msgr2=0x5600372020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600372020f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f912800a9b0 tx=0x7f9128011e20 comp rx=0 tx=0).stop 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 -- 172.21.15.19:0/1943561370 shutdown_connections 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560037278060 0x56003727a480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600371f0400 0x560037202630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f911effd700 0 client.16452 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600370734a0 0x560037211e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 --2- 172.21.15.19:0/1943561370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5600370edd00 0x5600372020f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 -- 172.21.15.19:0/1943561370 >> 172.21.15.19:0/1943561370 conn(0x560037076f70 msgr2=0x5600371e4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 -- 172.21.15.19:0/1943561370 shutdown_connections 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 1 -- 172.21.15.19:0/1943561370 wait complete. 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.446+0000 7f915450fb80 20 client.16452 trim_cache size 0 max 16384 2024-05-09T01:30:08.448 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:08.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 -- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f912c0b2880 msgr2=0x7f912c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:08.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f912c0b2880 0x7f912c0b4d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9140109810 tx=0x7f912800a380 comp rx=0 tx=0).stop 2024-05-09T01:30:08.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 -- 172.21.15.19:0/2447281802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9140104650 msgr2=0x7f914019adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:08.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9140104650 0x7f914019adc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f913000d950 tx=0x7f913000dc60 comp rx=0 tx=0).stop 2024-05-09T01:30:08.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 -- 172.21.15.19:0/2447281802 shutdown_connections 2024-05-09T01:30:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f912c0b2880 0x7f912c0b4d90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9140109880 0x7f914019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9140104650 0x7f914019adc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 --2- 172.21.15.19:0/2447281802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9140108f30 0x7f914019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 -- 172.21.15.19:0/2447281802 >> 172.21.15.19:0/2447281802 conn(0x7f9140079f00 msgr2=0x7f9140102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 -- 172.21.15.19:0/2447281802 shutdown_connections 2024-05-09T01:30:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:08.447+0000 7f9145af9700 1 -- 172.21.15.19:0/2447281802 wait complete. 2024-05-09T01:30:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:08 smithi196 ceph-mon[35041]: pgmap v516: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:30:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:08 smithi059 ceph-mon[34351]: pgmap v516: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:30:09.341 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:30:09.593 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:10.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:10 smithi019 ceph-mon[30158]: pgmap v517: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:30:10.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:10 smithi196 ceph-mon[35041]: pgmap v517: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:30:10.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:10 smithi059 ceph-mon[34351]: pgmap v517: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:30:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.370+0000 7fc4aeb34700 1 -- 172.21.15.19:0/465019774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a8109300 msgr2=0x7fc4a8109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.370+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/465019774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a8109300 0x7fc4a8109710 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009990 tx=0x7fc49c009ca0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.370+0000 7fc4aeb34700 1 -- 172.21.15.19:0/465019774 shutdown_connections 2024-05-09T01:30:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.370+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/465019774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4a81030e0 0x7fc4a8111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.370+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/465019774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a8109300 0x7fc4a8109710 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.370+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/465019774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4a8102790 0x7fc4a8102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.370+0000 7fc4aeb34700 1 -- 172.21.15.19:0/465019774 >> 172.21.15.19:0/465019774 conn(0x7fc4a80fe4e0 msgr2=0x7fc4a8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.370+0000 7fc4aeb34700 1 -- 172.21.15.19:0/465019774 shutdown_connections 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.370+0000 7fc4aeb34700 1 -- 172.21.15.19:0/465019774 wait complete. 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4aeb34700 1 Processor -- start 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4aeb34700 1 -- start start 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4aeb34700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4a8102790 0x7fc4a8103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4aeb34700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a81030e0 0x7fc4a8103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4aeb34700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4a8109300 0x7fc4a81043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4aeb34700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8114600 con 0x7fc4a81030e0 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4aeb34700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8114480 con 0x7fc4a8109300 2024-05-09T01:30:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4aeb34700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8114780 con 0x7fc4a8102790 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a81030e0 0x7fc4a8103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a81030e0 0x7fc4a8103eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57250/0 (socket says 172.21.15.19:57250) 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.371+0000 7fc4a7fff700 1 -- 172.21.15.19:0/3493310796 learned_addr learned my addr 172.21.15.19:0/3493310796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4ad0d1700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4a8109300 0x7fc4a81043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4a8102790 0x7fc4a8103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4a7fff700 1 -- 172.21.15.19:0/3493310796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4a8102790 msgr2=0x7fc4a8103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4a8102790 0x7fc4a8103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4a7fff700 1 -- 172.21.15.19:0/3493310796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4a8109300 msgr2=0x7fc4a81043f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4a8109300 0x7fc4a81043f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4a7fff700 1 -- 172.21.15.19:0/3493310796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a81a9370 con 0x7fc4a81030e0 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a81030e0 0x7fc4a8103eb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc49400b500 tx=0x7fc49400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4a5ffb700 1 -- 172.21.15.19:0/3493310796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494015020 con 0x7fc4a81030e0 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4aeb34700 1 -- 172.21.15.19:0/3493310796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a81a9660 con 0x7fc4a81030e0 2024-05-09T01:30:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4a5ffb700 1 -- 172.21.15.19:0/3493310796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc494011ce0 con 0x7fc4a81030e0 2024-05-09T01:30:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4a5ffb700 1 -- 172.21.15.19:0/3493310796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494010d40 con 0x7fc4a81030e0 2024-05-09T01:30:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.372+0000 7fc4aeb34700 1 -- 172.21.15.19:0/3493310796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a81a9ba0 con 0x7fc4a81030e0 2024-05-09T01:30:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.375+0000 7fc4a5ffb700 1 -- 172.21.15.19:0/3493310796 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc494018020 con 0x7fc4a81030e0 2024-05-09T01:30:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.376+0000 7fc4a5ffb700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4980b2880 0x7fc4980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.376+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4980b2880 0x7fc4980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.376+0000 7fc4a5ffb700 1 -- 172.21.15.19:0/3493310796 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc4940e1940 con 0x7fc4a81030e0 2024-05-09T01:30:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.376+0000 7fc4bd54ab80 1 Processor -- start 2024-05-09T01:30:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.376+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4980b2880 0x7fc4980b4d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc49c006010 tx=0x7fc49c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.376+0000 7fc4bd54ab80 1 -- start start 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59f8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 0x5559c59f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5559c590bff0 con 0x5559c5956160 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5559c58aa7f0 con 0x5559c59f1eb0 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5559c5907fb0 con 0x5559c586c4a0 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ac8d0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ac8d0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59f7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57258/0 (socket says 172.21.15.19:57258) 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ac8d0700 1 -- 172.21.15.19:0/3409922197 learned_addr learned my addr 172.21.15.19:0/3409922197 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ad0d1700 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 0x5559c59f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ac8d0700 1 -- 172.21.15.19:0/3409922197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 msgr2=0x5559c59f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ac8d0700 1 -- 172.21.15.19:0/3409922197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 msgr2=0x5559c59f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 0x5559c59f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ac8d0700 1 -- 172.21.15.19:0/3409922197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c58a2370 con 0x5559c5956160 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.377+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59f7d00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0059d0 tx=0x7fc49c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8ce004155d2c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.378+0000 7fc492ffd700 1 -- 172.21.15.19:0/3409922197 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00fa40 con 0x5559c5956160 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.378+0000 7fc492ffd700 1 -- 172.21.15.19:0/3409922197 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc49c031070 con 0x5559c5956160 2024-05-09T01:30:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.378+0000 7fc492ffd700 1 -- 172.21.15.19:0/3409922197 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c022ce0 con 0x5559c5956160 2024-05-09T01:30:11.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.378+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3409922197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 msgr2=0x5559c59f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:11.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59f7d00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0059d0 tx=0x7fc49c01d000 comp rx=0 tx=0).stop 2024-05-09T01:30:11.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3409922197 shutdown_connections 2024-05-09T01:30:11.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 0x5559c59f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59f7d00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3409922197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3409922197 >> 172.21.15.19:0/3409922197 conn(0x5559c58cc6a0 msgr2=0x5559c59e4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3409922197 shutdown_connections 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3409922197 wait complete. 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 -- start start 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.379+0000 7fc4bd54ab80 1 Processor -- start 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 0x5559c59fc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5559c5787ba0 con 0x5559c5956160 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5559c58c6150 con 0x5559c59f1eb0 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5559c59fe9e0 con 0x5559c586c4a0 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4bd54ab80 10 client.0 _ll_register_callbacks cb 0x5559c577d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57262/0 (socket says 172.21.15.19:57262) 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4a7fff700 1 -- 172.21.15.19:0/599873769 learned_addr learned my addr 172.21.15.19:0/599873769 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4ad0d1700 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 0x5559c59fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4a7fff700 1 -- 172.21.15.19:0/599873769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 msgr2=0x5559c59fb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59fb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4a7fff700 1 -- 172.21.15.19:0/599873769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 msgr2=0x5559c59fc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 0x5559c59fc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4a7fff700 1 -- 172.21.15.19:0/599873769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c5787490 con 0x5559c5956160 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4ad0d1700 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 0x5559c59fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59fbf20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc4940a4fc0 tx=0x7fc4940a8a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd18b721cedf2b94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.380+0000 7fc487fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc487fff700 1 -- 172.21.15.19:0/599873769 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940a8f40 con 0x5559c5956160 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc487fff700 1 -- 172.21.15.19:0/599873769 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc4940b1910 con 0x5559c5956160 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/599873769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49c031070 con 0x5559c5956160 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc487fff700 1 -- 172.21.15.19:0/599873769 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4940c9070 con 0x5559c5956160 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc4bd54ab80 10 client.16464 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/599873769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5559c5a093d0 con 0x5559c5956160 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc487fff700 1 -- 172.21.15.19:0/599873769 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc49c031070 con 0x5559c5956160 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc487fff700 1 -- 172.21.15.19:0/599873769 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc4940a8330 con 0x5559c5956160 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc4bd54ab80 10 client.16464 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc4bd54ab80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc4bd54ab80 20 client.16464 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5559c5a0ba10 0x5559c5a0de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc4bd54ab80 4 client.16464 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.381+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/599873769 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559c59199d0 con 0x5559c5a0ba10 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.382+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5559c5a0ba10 0x5559c5a0de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.382+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5559c5a0ba10 0x5559c5a0de50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176c315922876a0d server_cookie=f0bf002e5451a206 in_seq=0 out_seq=0 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.382+0000 7fc487fff700 10 client.16464 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc487fff700 1 -- 172.21.15.19:0/599873769 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5559c59199d0 con 0x5559c5a0ba10 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc487fff700 10 client.16464 handle_command_reply: tid=0 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 client.16464 shutdown 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/599873769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 msgr2=0x5559c59fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59fbf20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc4940a4fc0 tx=0x7fc4940a8a50 comp rx=0 tx=0).stop 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/599873769 shutdown_connections 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5559c5a0ba10 0x5559c5a0de50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c59f1eb0 0x5559c59fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/599873769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c59fbf20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/599873769 >> 172.21.15.19:0/599873769 conn(0x5559c58cc6a0 msgr2=0x5559c59e41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:11.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc487fff700 0 client.16464 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:11.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/599873769 shutdown_connections 2024-05-09T01:30:11.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/599873769 wait complete. 2024-05-09T01:30:11.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.383+0000 7fc4bd54ab80 20 client.16464 trim_cache size 0 max 16384 2024-05-09T01:30:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.400+0000 7fc4bd54ab80 1 Processor -- start 2024-05-09T01:30:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.400+0000 7fc4bd54ab80 1 -- start start 2024-05-09T01:30:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.400+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59f6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.400+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c5956160 0x5559c59f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.400+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c59f1eb0 0x5559c59fa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.400+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81acb10 con 0x5559c59f1eb0 2024-05-09T01:30:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.400+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ae650 con 0x5559c5956160 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.400+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ad1e0 con 0x5559c586c4a0 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4ad0d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c59f1eb0 0x5559c59fa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c5956160 0x5559c59f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4ad0d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c59f1eb0 0x5559c59fa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57270/0 (socket says 172.21.15.19:57270) 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4ad0d1700 1 -- 172.21.15.19:0/2960270533 learned_addr learned my addr 172.21.15.19:0/2960270533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/2960270533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4a7fff700 1 -- 172.21.15.19:0/2960270533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 msgr2=0x5559c59f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/2960270533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4a7fff700 1 -- 172.21.15.19:0/2960270533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c59f1eb0 msgr2=0x5559c59fa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/2960270533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c59f1eb0 0x5559c59fa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4a7fff700 1 -- 172.21.15.19:0/2960270533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c59ddf60 con 0x5559c5956160 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4ad0d1700 1 --2- 172.21.15.19:0/2960270533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c59f1eb0 0x5559c59fa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/2960270533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c5956160 0x5559c59f71d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4940a8ed0 tx=0x7fc4940107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9eafae2fc571ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.401+0000 7fc4927fc700 1 -- 172.21.15.19:0/2960270533 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940ad2f0 con 0x5559c5956160 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.402+0000 7fc4927fc700 1 -- 172.21.15.19:0/2960270533 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4940b1da0 con 0x5559c5956160 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.402+0000 7fc4927fc700 1 -- 172.21.15.19:0/2960270533 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940a81a0 con 0x5559c5956160 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.402+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/2960270533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c5956160 msgr2=0x5559c59f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.402+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/2960270533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c5956160 0x5559c59f71d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4940a8ed0 tx=0x7fc4940107e0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.402+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/2960270533 shutdown_connections 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.402+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/2960270533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c59f1eb0 0x5559c59fa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.402+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/2960270533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c586c4a0 0x5559c59f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.402+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/2960270533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c5956160 0x5559c59f71d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.403+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/2960270533 >> 172.21.15.19:0/2960270533 conn(0x5559c58cc6a0 msgr2=0x5559c59f6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.403+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/2960270533 shutdown_connections 2024-05-09T01:30:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.403+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/2960270533 wait complete. 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.403+0000 7fc4bd54ab80 1 -- start start 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.403+0000 7fc4bd54ab80 1 Processor -- start 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c586c4a0 0x5559c5a12600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c5a12b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4bd54ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c59f1eb0 0x5559c5a13080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81adf80 con 0x5559c5956160 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ad8b0 con 0x5559c586c4a0 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4bd54ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a819eb80 con 0x5559c59f1eb0 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4bd54ab80 10 client.0 _ll_register_callbacks cb 0x5559c577d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c5a12b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c5a12b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57278/0 (socket says 172.21.15.19:57278) 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4a7fff700 1 -- 172.21.15.19:0/3350652593 learned_addr learned my addr 172.21.15.19:0/3350652593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c586c4a0 0x5559c5a12600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4a7fff700 1 -- 172.21.15.19:0/3350652593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c59f1eb0 msgr2=0x5559c5a13080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c59f1eb0 0x5559c5a13080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4a7fff700 1 -- 172.21.15.19:0/3350652593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c586c4a0 msgr2=0x5559c5a12600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c586c4a0 0x5559c5a12600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4a7fff700 1 -- 172.21.15.19:0/3350652593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c5a02b70 con 0x5559c5956160 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4a7fff700 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c5a12b40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc49400bc10 tx=0x7fc4940a4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdba77c6433402af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc487fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc487fff700 1 -- 172.21.15.19:0/3350652593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940ad2f0 con 0x5559c5956160 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc487fff700 1 -- 172.21.15.19:0/3350652593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc4940a5530 con 0x5559c5956160 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.404+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3350652593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4940b1da0 con 0x5559c5956160 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc487fff700 1 -- 172.21.15.19:0/3350652593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc494011830 con 0x5559c5956160 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc4bd54ab80 10 client.16470 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3350652593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5559c5a02b70 con 0x5559c5956160 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc487fff700 1 -- 172.21.15.19:0/3350652593 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4940b1da0 con 0x5559c5956160 2024-05-09T01:30:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc487fff700 1 -- 172.21.15.19:0/3350652593 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc49408b070 con 0x5559c5956160 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc4bd54ab80 10 client.16470 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc4bd54ab80 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc4bd54ab80 20 client.16470 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5559c5a04090 0x5559c5a781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc4bd54ab80 4 client.16470 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3350652593 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559c5913a70 con 0x5559c5a04090 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.405+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5559c5a04090 0x5559c5a781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4ac8d0700 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5559c5a04090 0x5559c5a781d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f682dc7d38abc3c6 server_cookie=645a40c5313d3720 in_seq=0 out_seq=0 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc487fff700 10 client.16470 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc487fff700 1 -- 172.21.15.19:0/3350652593 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559c5913a70 con 0x5559c5a04090 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc487fff700 10 client.16470 handle_command_reply: tid=0 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 client.16470 shutdown 2024-05-09T01:30:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3350652593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 msgr2=0x5559c5a12b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c5a12b40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc49400bc10 tx=0x7fc4940a4c60 comp rx=0 tx=0).stop 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3350652593 shutdown_connections 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5559c5a04090 0x5559c5a781d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559c59f1eb0 0x5559c5a13080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559c586c4a0 0x5559c5a12600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc487fff700 0 client.16470 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 --2- 172.21.15.19:0/3350652593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559c5956160 0x5559c5a12b40 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3350652593 >> 172.21.15.19:0/3350652593 conn(0x5559c58cc6a0 msgr2=0x5559c59fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3350652593 shutdown_connections 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.406+0000 7fc4bd54ab80 1 -- 172.21.15.19:0/3350652593 wait complete. 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4bd54ab80 20 client.16470 trim_cache size 0 max 16384 2024-05-09T01:30:11.408 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:11.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 -- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4980b2880 msgr2=0x7fc4980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:11.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4980b2880 0x7fc4980b4d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc49c006010 tx=0x7fc49c01a040 comp rx=0 tx=0).stop 2024-05-09T01:30:11.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 -- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a81030e0 msgr2=0x7fc4a8103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:11.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a81030e0 0x7fc4a8103eb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc49400b500 tx=0x7fc49400b810 comp rx=0 tx=0).stop 2024-05-09T01:30:11.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 -- 172.21.15.19:0/3493310796 shutdown_connections 2024-05-09T01:30:11.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4a8109300 0x7fc4a81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4a8102790 0x7fc4a8103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc4980b2880 0x7fc4980b4d90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 --2- 172.21.15.19:0/3493310796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4a81030e0 0x7fc4a8103eb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 -- 172.21.15.19:0/3493310796 >> 172.21.15.19:0/3493310796 conn(0x7fc4a80fe4e0 msgr2=0x7fc4a8107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 -- 172.21.15.19:0/3493310796 shutdown_connections 2024-05-09T01:30:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:11.407+0000 7fc4aeb34700 1 -- 172.21.15.19:0/3493310796 wait complete. 2024-05-09T01:30:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:12 smithi196 ceph-mon[35041]: pgmap v518: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:12 smithi059 ceph-mon[34351]: pgmap v518: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:12 smithi019 ceph-mon[30158]: pgmap v518: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:13.109 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:30:13.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:30:13.380 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:14.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:14 smithi019 ceph-mon[30158]: pgmap v519: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:14.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:14 smithi196 ceph-mon[35041]: pgmap v519: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:14 smithi059 ceph-mon[34351]: pgmap v519: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.177+0000 7fd79dcb1700 1 -- 172.21.15.19:0/3224090084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7981034e0 msgr2=0x7fd7981038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.177+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/3224090084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7981034e0 0x7fd7981038f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd7880099c0 tx=0x7fd788009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.177+0000 7fd79dcb1700 1 -- 172.21.15.19:0/3224090084 shutdown_connections 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.177+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/3224090084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd798108760 0x7fd7981100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.178+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/3224090084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7981034e0 0x7fd7981038f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.178+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/3224090084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd798107e10 0x7fd798108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.178+0000 7fd79dcb1700 1 -- 172.21.15.19:0/3224090084 >> 172.21.15.19:0/3224090084 conn(0x7fd7980fef30 msgr2=0x7fd798101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.178+0000 7fd79dcb1700 1 -- 172.21.15.19:0/3224090084 shutdown_connections 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.178+0000 7fd79dcb1700 1 -- 172.21.15.19:0/3224090084 wait complete. 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.178+0000 7fd79dcb1700 1 Processor -- start 2024-05-09T01:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd79dcb1700 1 -- start start 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd79dcb1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7981034e0 0x7fd79819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd79dcb1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd798107e10 0x7fd79819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd79dcb1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd798108760 0x7fd79819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd79dcb1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd798113390 con 0x7fd798108760 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd79dcb1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd798113210 con 0x7fd7981034e0 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd79dcb1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd798113510 con 0x7fd798107e10 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd797fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd798108760 0x7fd79819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd797fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd798108760 0x7fd79819c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57300/0 (socket says 172.21.15.19:57300) 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd797fff700 1 -- 172.21.15.19:0/800049142 learned_addr learned my addr 172.21.15.19:0/800049142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd796ffd700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd798107e10 0x7fd79819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.179+0000 7fd7977fe700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7981034e0 0x7fd79819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd797fff700 1 -- 172.21.15.19:0/800049142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd798107e10 msgr2=0x7fd79819c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd797fff700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd798107e10 0x7fd79819c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd797fff700 1 -- 172.21.15.19:0/800049142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7981034e0 msgr2=0x7fd79819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd797fff700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7981034e0 0x7fd79819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd797fff700 1 -- 172.21.15.19:0/800049142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7981a7000 con 0x7fd798108760 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd797fff700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd798108760 0x7fd79819c750 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd790007ee0 tx=0x7fd79000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd794ff9700 1 -- 172.21.15.19:0/800049142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790004bb0 con 0x7fd798108760 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd79dcb1700 1 -- 172.21.15.19:0/800049142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7981a72f0 con 0x7fd798108760 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd794ff9700 1 -- 172.21.15.19:0/800049142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd790004d50 con 0x7fd798108760 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd794ff9700 1 -- 172.21.15.19:0/800049142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790005920 con 0x7fd798108760 2024-05-09T01:30:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.180+0000 7fd79dcb1700 1 -- 172.21.15.19:0/800049142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7981a77e0 con 0x7fd798108760 2024-05-09T01:30:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.183+0000 7fd794ff9700 1 -- 172.21.15.19:0/800049142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd79001a020 con 0x7fd798108760 2024-05-09T01:30:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.184+0000 7fd794ff9700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7800b2880 0x7fd7800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.184+0000 7fd7977fe700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7800b2880 0x7fd7800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.184+0000 7fd794ff9700 1 -- 172.21.15.19:0/800049142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd7900e1850 con 0x7fd798108760 2024-05-09T01:30:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.184+0000 7fd7977fe700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7800b2880 0x7fd7800b4d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd788009990 tx=0x7fd788000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd7ac6c7b80 1 Processor -- start 2024-05-09T01:30:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd7ac6c7b80 1 -- start start 2024-05-09T01:30:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 0x562e7d5c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e7d4eb070 con 0x562e7d44b520 2024-05-09T01:30:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562e7d489870 con 0x562e7d5c8720 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e7d4e7030 con 0x562e7d5351e0 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd796ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd796ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57314/0 (socket says 172.21.15.19:57314) 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd796ffd700 1 -- 172.21.15.19:0/310856719 learned_addr learned my addr 172.21.15.19:0/310856719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd797fff700 1 --2- 172.21.15.19:0/310856719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 0x562e7d5c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd796ffd700 1 -- 172.21.15.19:0/310856719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 msgr2=0x562e7d5ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd796ffd700 1 --2- 172.21.15.19:0/310856719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd796ffd700 1 -- 172.21.15.19:0/310856719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 msgr2=0x562e7d5c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd796ffd700 1 --2- 172.21.15.19:0/310856719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 0x562e7d5c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.185+0000 7fd796ffd700 1 -- 172.21.15.19:0/310856719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e7d4813f0 con 0x562e7d44b520 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.186+0000 7fd7977fe700 1 --2- 172.21.15.19:0/310856719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.186+0000 7fd796ffd700 1 --2- 172.21.15.19:0/310856719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5ceab0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00bcd0 tx=0x7fd78c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69363ea9abc08da9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.186+0000 7fd77dffb700 1 -- 172.21.15.19:0/310856719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c010b90 con 0x562e7d44b520 2024-05-09T01:30:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.186+0000 7fd77dffb700 1 -- 172.21.15.19:0/310856719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd78c0044e0 con 0x562e7d44b520 2024-05-09T01:30:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.186+0000 7fd77dffb700 1 -- 172.21.15.19:0/310856719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c011440 con 0x562e7d44b520 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/310856719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 msgr2=0x562e7d5ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/310856719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5ceab0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00bcd0 tx=0x7fd78c0103d0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/310856719 shutdown_connections 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/310856719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 0x562e7d5c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/310856719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/310856719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5ceab0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/310856719 >> 172.21.15.19:0/310856719 conn(0x562e7d4ab720 msgr2=0x562e7d5c3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/310856719 shutdown_connections 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/310856719 wait complete. 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.187+0000 7fd7ac6c7b80 1 -- start start 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7ac6c7b80 1 Processor -- start 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 0x562e7d5d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e7d366c20 con 0x562e7d44b520 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562e7d4a51d0 con 0x562e7d5c8720 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e7d5d5250 con 0x562e7d5351e0 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7ac6c7b80 10 client.0 _ll_register_callbacks cb 0x562e7d35c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57322/0 (socket says 172.21.15.19:57322) 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7977fe700 1 -- 172.21.15.19:0/2157850333 learned_addr learned my addr 172.21.15.19:0/2157850333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd797fff700 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 0x562e7d5d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd796ffd700 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7977fe700 1 -- 172.21.15.19:0/2157850333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 msgr2=0x562e7d5d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7977fe700 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7977fe700 1 -- 172.21.15.19:0/2157850333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 msgr2=0x562e7d5d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7977fe700 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 0x562e7d5d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd7977fe700 1 -- 172.21.15.19:0/2157850333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e7d366510 con 0x562e7d44b520 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.188+0000 7fd796ffd700 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd7977fe700 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d22f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd7880099c0 tx=0x7fd78801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213ccce9386f1bbe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd776ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd776ffd700 1 -- 172.21.15.19:0/2157850333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78800f460 con 0x562e7d44b520 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd776ffd700 1 -- 172.21.15.19:0/2157850333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd78800f600 con 0x562e7d44b520 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2157850333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c0044e0 con 0x562e7d44b520 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd776ffd700 1 -- 172.21.15.19:0/2157850333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd78800f800 con 0x562e7d44b520 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd7ac6c7b80 10 client.16482 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2157850333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e7d5dfcc0 con 0x562e7d44b520 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd776ffd700 1 -- 172.21.15.19:0/2157850333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd78c0044e0 con 0x562e7d44b520 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.189+0000 7fd776ffd700 1 -- 172.21.15.19:0/2157850333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd788022070 con 0x562e7d44b520 2024-05-09T01:30:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.190+0000 7fd7ac6c7b80 10 client.16482 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.190+0000 7fd7ac6c7b80 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.190+0000 7fd7ac6c7b80 20 client.16482 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.190+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562e7d5e2350 0x562e7d5e47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.190+0000 7fd7ac6c7b80 4 client.16482 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.190+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2157850333 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e7d4f8a50 con 0x562e7d5e2350 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.190+0000 7fd796ffd700 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562e7d5e2350 0x562e7d5e47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.190+0000 7fd796ffd700 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562e7d5e2350 0x562e7d5e47b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8a586c953e18ef server_cookie=5e77243ac5fc118d in_seq=0 out_seq=0 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.190+0000 7fd776ffd700 10 client.16482 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd776ffd700 1 -- 172.21.15.19:0/2157850333 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e7d4f8a50 con 0x562e7d5e2350 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd776ffd700 10 client.16482 handle_command_reply: tid=0 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd7ac6c7b80 1 client.16482 shutdown 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2157850333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 msgr2=0x562e7d5d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d22f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd7880099c0 tx=0x7fd78801a050 comp rx=0 tx=0).stop 2024-05-09T01:30:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2157850333 shutdown_connections 2024-05-09T01:30:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562e7d5e2350 0x562e7d5e47b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5c8720 0x562e7d5d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d22f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2157850333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5351e0 0x562e7d5d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2157850333 >> 172.21.15.19:0/2157850333 conn(0x562e7d4ab720 msgr2=0x562e7d5c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.191+0000 7fd776ffd700 0 client.16482 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.192+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2157850333 shutdown_connections 2024-05-09T01:30:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.192+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2157850333 wait complete. 2024-05-09T01:30:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.192+0000 7fd7ac6c7b80 20 client.16482 trim_cache size 0 max 16384 2024-05-09T01:30:15.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.212+0000 7fd7ac6c7b80 1 Processor -- start 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7ac6c7b80 1 -- start start 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5351e0 0x562e7d5c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5c8720 0x562e7d5d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981aec80 con 0x562e7d44b520 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981b07c0 con 0x562e7d5351e0 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981af350 con 0x562e7d5c8720 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57334/0 (socket says 172.21.15.19:57334) 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7977fe700 1 -- 172.21.15.19:0/135372442 learned_addr learned my addr 172.21.15.19:0/135372442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7977fe700 1 -- 172.21.15.19:0/135372442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5c8720 msgr2=0x562e7d5d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7977fe700 1 --2- 172.21.15.19:0/135372442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5c8720 0x562e7d5d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7977fe700 1 -- 172.21.15.19:0/135372442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5351e0 msgr2=0x562e7d5c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:15.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7977fe700 1 --2- 172.21.15.19:0/135372442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5351e0 0x562e7d5c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7977fe700 1 -- 172.21.15.19:0/135372442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e7d5c4d80 con 0x562e7d44b520 2024-05-09T01:30:15.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.213+0000 7fd7977fe700 1 --2- 172.21.15.19:0/135372442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d15e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd788023a30 tx=0x7fd78800baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac26f4adb283bd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:15.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd77d7fa700 1 -- 172.21.15.19:0/135372442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788047070 con 0x562e7d44b520 2024-05-09T01:30:15.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd77d7fa700 1 -- 172.21.15.19:0/135372442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd788034070 con 0x562e7d44b520 2024-05-09T01:30:15.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd77d7fa700 1 -- 172.21.15.19:0/135372442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788042680 con 0x562e7d44b520 2024-05-09T01:30:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/135372442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 msgr2=0x562e7d5d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/135372442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d15e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd788023a30 tx=0x7fd78800baf0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/135372442 shutdown_connections 2024-05-09T01:30:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/135372442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5c8720 0x562e7d5d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/135372442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d44b520 0x562e7d5d15e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/135372442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d5351e0 0x562e7d5c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/135372442 >> 172.21.15.19:0/135372442 conn(0x562e7d4ab720 msgr2=0x562e7d5d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/135372442 shutdown_connections 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.214+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/135372442 wait complete. 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.215+0000 7fd7ac6c7b80 1 -- start start 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.215+0000 7fd7ac6c7b80 1 Processor -- start 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.215+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d44b520 0x562e7d5d6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.215+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d5351e0 0x562e7d5d73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.215+0000 7fd7ac6c7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5c8720 0x562e7d5d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.215+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981b00f0 con 0x562e7d5351e0 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.215+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981afa20 con 0x562e7d44b520 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.215+0000 7fd7ac6c7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981acc20 con 0x562e7d5c8720 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd7ac6c7b80 10 client.0 _ll_register_callbacks cb 0x562e7d35c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd796ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d5351e0 0x562e7d5d73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd796ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d5351e0 0x562e7d5d73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57346/0 (socket says 172.21.15.19:57346) 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d44b520 0x562e7d5d6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd796ffd700 1 -- 172.21.15.19:0/2833945307 learned_addr learned my addr 172.21.15.19:0/2833945307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd797fff700 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5c8720 0x562e7d5d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd796ffd700 1 -- 172.21.15.19:0/2833945307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5c8720 msgr2=0x562e7d5d7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd796ffd700 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5c8720 0x562e7d5d7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd796ffd700 1 -- 172.21.15.19:0/2833945307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d44b520 msgr2=0x562e7d5d6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd796ffd700 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d44b520 0x562e7d5d6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd796ffd700 1 -- 172.21.15.19:0/2833945307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e7d64cb30 con 0x562e7d5351e0 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd7977fe700 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d44b520 0x562e7d5d6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd796ffd700 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d5351e0 0x562e7d5d73f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00bcd0 tx=0x7fd78c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7050edc7e44e51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd776ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd776ffd700 1 -- 172.21.15.19:0/2833945307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0039f0 con 0x562e7d5351e0 2024-05-09T01:30:15.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd776ffd700 1 -- 172.21.15.19:0/2833945307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd78c003b90 con 0x562e7d5351e0 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.216+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2833945307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd788034070 con 0x562e7d5351e0 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd776ffd700 1 -- 172.21.15.19:0/2833945307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd78c003d90 con 0x562e7d5351e0 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd7ac6c7b80 10 client.16488 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2833945307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e7d64dcb0 con 0x562e7d5351e0 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd776ffd700 1 -- 172.21.15.19:0/2833945307 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd788034070 con 0x562e7d5351e0 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd776ffd700 1 -- 172.21.15.19:0/2833945307 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd78c0278a0 con 0x562e7d5351e0 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd7ac6c7b80 10 client.16488 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd7ac6c7b80 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd7ac6c7b80 20 client.16488 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562e7d64e300 0x562e7d650720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd7ac6c7b80 4 client.16488 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2833945307 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562e7d4f2af0 con 0x562e7d64e300 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.217+0000 7fd7977fe700 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562e7d64e300 0x562e7d650720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.218+0000 7fd7977fe700 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562e7d64e300 0x562e7d650720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd27dea894819a3 server_cookie=6e529cc62a1c5ca4 in_seq=0 out_seq=0 2024-05-09T01:30:15.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.218+0000 7fd776ffd700 10 client.16488 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:15.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd776ffd700 1 -- 172.21.15.19:0/2833945307 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562e7d4f2af0 con 0x562e7d64e300 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd776ffd700 10 client.16488 handle_command_reply: tid=0 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 client.16488 shutdown 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2833945307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d5351e0 msgr2=0x562e7d5d73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d5351e0 0x562e7d5d73f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00bcd0 tx=0x7fd78c003680 comp rx=0 tx=0).stop 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2833945307 shutdown_connections 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562e7d64e300 0x562e7d650720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e7d5c8720 0x562e7d5d7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e7d44b520 0x562e7d5d6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 --2- 172.21.15.19:0/2833945307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e7d5351e0 0x562e7d5d73f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd776ffd700 0 client.16488 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2833945307 >> 172.21.15.19:0/2833945307 conn(0x562e7d4ab720 msgr2=0x562e7d5c7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2833945307 shutdown_connections 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 1 -- 172.21.15.19:0/2833945307 wait complete. 2024-05-09T01:30:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.219+0000 7fd7ac6c7b80 20 client.16488 trim_cache size 0 max 16384 2024-05-09T01:30:15.221 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:30:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:30:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "a0daacba-b681-4214-8abc-460e61712b0d", 2024-05-09T01:30:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:30:15.221 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 -- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7800b2880 msgr2=0x7fd7800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7800b2880 0x7fd7800b4d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd788009990 tx=0x7fd788000bc0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 -- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd798108760 msgr2=0x7fd79819c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd798108760 0x7fd79819c750 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd790007ee0 tx=0x7fd79000d3b0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 -- 172.21.15.19:0/800049142 shutdown_connections 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7800b2880 0x7fd7800b4d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd798108760 0x7fd79819c750 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7981034e0 0x7fd79819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 --2- 172.21.15.19:0/800049142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd798107e10 0x7fd79819c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 -- 172.21.15.19:0/800049142 >> 172.21.15.19:0/800049142 conn(0x7fd7980fef30 msgr2=0x7fd7981096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 -- 172.21.15.19:0/800049142 shutdown_connections 2024-05-09T01:30:15.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:15.220+0000 7fd79dcb1700 1 -- 172.21.15.19:0/800049142 wait complete. 2024-05-09T01:30:16.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:30:16.349 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:16 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:30:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:16 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:16 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:30:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:16 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:16 smithi196 ceph-mon[35041]: pgmap v520: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:30:16.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:16 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:30:16.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:16 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:16.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:16 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:30:16.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:16 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:16.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:16 smithi019 ceph-mon[30158]: pgmap v520: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:30:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:16 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:30:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:16 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:16 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:30:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:16 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:16 smithi059 ceph-mon[34351]: pgmap v520: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:30:17.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:17 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:30:17.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:17 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:30:17.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:17 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:30:18.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.090+0000 7f0076b14700 1 -- 172.21.15.19:0/615130788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0070107df0 msgr2=0x7f0070108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.090+0000 7f0076b14700 1 --2- 172.21.15.19:0/615130788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0070107df0 0x7f0070108200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f005c0099c0 tx=0x7f005c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.090+0000 7f0076b14700 1 -- 172.21.15.19:0/615130788 shutdown_connections 2024-05-09T01:30:18.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.090+0000 7f0076b14700 1 --2- 172.21.15.19:0/615130788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0070108740 0x7f00701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.090+0000 7f0076b14700 1 --2- 172.21.15.19:0/615130788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00701034c0 0x7f00701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.090+0000 7f0076b14700 1 --2- 172.21.15.19:0/615130788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0070107df0 0x7f0070108200 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.090+0000 7f0076b14700 1 -- 172.21.15.19:0/615130788 >> 172.21.15.19:0/615130788 conn(0x7f00700fedf0 msgr2=0x7f0070101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.090+0000 7f0076b14700 1 -- 172.21.15.19:0/615130788 shutdown_connections 2024-05-09T01:30:18.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.090+0000 7f0076b14700 1 -- 172.21.15.19:0/615130788 wait complete. 2024-05-09T01:30:18.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.091+0000 7f0076b14700 1 Processor -- start 2024-05-09T01:30:18.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.091+0000 7f0076b14700 1 -- start start 2024-05-09T01:30:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.091+0000 7f0076b14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00701034c0 0x7f007019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.091+0000 7f0076b14700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0070107df0 0x7f007019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.091+0000 7f0076b14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0070108740 0x7f007019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.091+0000 7f0076b14700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0070113350 con 0x7f00701034c0 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.091+0000 7f0076b14700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00701131d0 con 0x7f0070108740 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.091+0000 7f0076b14700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00701134d0 con 0x7f0070107df0 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00748b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00701034c0 0x7f007019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00748b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00701034c0 0x7f007019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57370/0 (socket says 172.21.15.19:57370) 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00748b0700 1 -- 172.21.15.19:0/238777440 learned_addr learned my addr 172.21.15.19:0/238777440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00750b1700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0070108740 0x7f007019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f006ffff700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0070107df0 0x7f007019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00748b0700 1 -- 172.21.15.19:0/238777440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0070107df0 msgr2=0x7f007019c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00748b0700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0070107df0 0x7f007019c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00748b0700 1 -- 172.21.15.19:0/238777440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0070108740 msgr2=0x7f007019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00748b0700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0070108740 0x7f007019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00748b0700 1 -- 172.21.15.19:0/238777440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00701a6f70 con 0x7f00701034c0 2024-05-09T01:30:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f006ffff700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0070107df0 0x7f007019c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00750b1700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0070108740 0x7f007019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f00748b0700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00701034c0 0x7f007019bc90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f006400d980 tx=0x7f006400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f006dffb700 1 -- 172.21.15.19:0/238777440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064016070 con 0x7f00701034c0 2024-05-09T01:30:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f0076b14700 1 -- 172.21.15.19:0/238777440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00701a7260 con 0x7f00701034c0 2024-05-09T01:30:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f006dffb700 1 -- 172.21.15.19:0/238777440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00640044d0 con 0x7f00701034c0 2024-05-09T01:30:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.092+0000 7f006dffb700 1 -- 172.21.15.19:0/238777440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064004d00 con 0x7f00701034c0 2024-05-09T01:30:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.093+0000 7f0076b14700 1 -- 172.21.15.19:0/238777440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00701a77a0 con 0x7f00701034c0 2024-05-09T01:30:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.095+0000 7f006dffb700 1 -- 172.21.15.19:0/238777440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0064004020 con 0x7f00701034c0 2024-05-09T01:30:18.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.096+0000 7f006dffb700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f00600b2880 0x7f00600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.096+0000 7f006dffb700 1 -- 172.21.15.19:0/238777440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f00640e2100 con 0x7f00701034c0 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.096+0000 7f006ffff700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f00600b2880 0x7f00600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.096+0000 7f008552ab80 1 Processor -- start 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.096+0000 7f008552ab80 1 -- start start 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f006ffff700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f00600b2880 0x7f00600b4d90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f005c006010 tx=0x7f005c005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f008552ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56bfd00 0x5574d57c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f008552ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56454a0 0x5574d57c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f008552ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 0x5574d57c2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f008552ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5574d56e50a0 con 0x5574d56bfd00 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f008552ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5574d568a9a0 con 0x5574d56454a0 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f008552ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5574d56e1060 con 0x5574d57c2400 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00748b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56bfd00 0x5574d57c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00748b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56bfd00 0x5574d57c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57386/0 (socket says 172.21.15.19:57386) 2024-05-09T01:30:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00748b0700 1 -- 172.21.15.19:0/3609766987 learned_addr learned my addr 172.21.15.19:0/3609766987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f006ffff700 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56454a0 0x5574d57c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00750b1700 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 0x5574d57c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00748b0700 1 -- 172.21.15.19:0/3609766987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 msgr2=0x5574d57c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00748b0700 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 0x5574d57c2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00748b0700 1 -- 172.21.15.19:0/3609766987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56454a0 msgr2=0x5574d57c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00748b0700 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56454a0 0x5574d57c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00748b0700 1 -- 172.21.15.19:0/3609766987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d567b500 con 0x5574d56bfd00 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f006ffff700 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56454a0 0x5574d57c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.097+0000 7f00748b0700 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56bfd00 0x5574d57c8240 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f00640a5780 tx=0x7f00640a8f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86d24b3c957d476 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.098+0000 7f005affd700 1 -- 172.21.15.19:0/3609766987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640a9e00 con 0x5574d56bfd00 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.098+0000 7f005affd700 1 -- 172.21.15.19:0/3609766987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00640a9700 con 0x5574d56bfd00 2024-05-09T01:30:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.098+0000 7f005affd700 1 -- 172.21.15.19:0/3609766987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640a9960 con 0x5574d56bfd00 2024-05-09T01:30:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.098+0000 7f008552ab80 1 -- 172.21.15.19:0/3609766987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56bfd00 msgr2=0x5574d57c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.098+0000 7f008552ab80 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56bfd00 0x5574d57c8240 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f00640a5780 tx=0x7f00640a8f40 comp rx=0 tx=0).stop 2024-05-09T01:30:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.099+0000 7f008552ab80 1 -- 172.21.15.19:0/3609766987 shutdown_connections 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.099+0000 7f008552ab80 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 0x5574d57c2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.099+0000 7f008552ab80 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56bfd00 0x5574d57c8240 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.099+0000 7f008552ab80 1 --2- 172.21.15.19:0/3609766987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56454a0 0x5574d57c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.099+0000 7f008552ab80 1 -- 172.21.15.19:0/3609766987 >> 172.21.15.19:0/3609766987 conn(0x5574d5648f70 msgr2=0x5574d57bd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.099+0000 7f008552ab80 1 -- 172.21.15.19:0/3609766987 shutdown_connections 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.099+0000 7f008552ab80 1 -- 172.21.15.19:0/3609766987 wait complete. 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.099+0000 7f008552ab80 1 -- start start 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.099+0000 7f008552ab80 1 Processor -- start 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f008552ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56454a0 0x5574d57cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f008552ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 0x5574d57cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f008552ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d57c2400 0x5574d57cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f008552ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5574d57c30a0 con 0x5574d57c2400 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f008552ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5574d567fb20 con 0x5574d56bfd00 2024-05-09T01:30:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f008552ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5574d57c3220 con 0x5574d56454a0 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f008552ab80 10 client.0 _ll_register_callbacks cb 0x5574d55566c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00750b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d57c2400 0x5574d57cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00750b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d57c2400 0x5574d57cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57400/0 (socket says 172.21.15.19:57400) 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00750b1700 1 -- 172.21.15.19:0/2038972968 learned_addr learned my addr 172.21.15.19:0/2038972968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00748b0700 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56454a0 0x5574d57cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f006ffff700 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 0x5574d57cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00750b1700 1 -- 172.21.15.19:0/2038972968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56454a0 msgr2=0x5574d57cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00750b1700 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56454a0 0x5574d57cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00750b1700 1 -- 172.21.15.19:0/2038972968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 msgr2=0x5574d57cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00750b1700 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 0x5574d57cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00750b1700 1 -- 172.21.15.19:0/2038972968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d5560490 con 0x5574d57c2400 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f00750b1700 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d57c2400 0x5574d57cca40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0068010b00 tx=0x7f0068010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea97370430e0b5ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.100+0000 7f004ffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f004ffff700 1 -- 172.21.15.19:0/2038972968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006801a030 con 0x5574d57c2400 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f004ffff700 1 -- 172.21.15.19:0/2038972968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0068003a00 con 0x5574d57c2400 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f008552ab80 1 -- 172.21.15.19:0/2038972968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00640a9700 con 0x5574d57c2400 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f004ffff700 1 -- 172.21.15.19:0/2038972968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0068003c00 con 0x5574d57c2400 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f008552ab80 10 client.16500 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f008552ab80 1 -- 172.21.15.19:0/2038972968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574d57d9940 con 0x5574d57c2400 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f004ffff700 1 -- 172.21.15.19:0/2038972968 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00640a9700 con 0x5574d57c2400 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f004ffff700 1 -- 172.21.15.19:0/2038972968 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0068015860 con 0x5574d57c2400 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f008552ab80 10 client.16500 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f008552ab80 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f008552ab80 20 client.16500 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f008552ab80 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574d57dbf80 0x5574d57de3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f008552ab80 4 client.16500 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.101+0000 7f008552ab80 1 -- 172.21.15.19:0/2038972968 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574d56f2a80 con 0x5574d57dbf80 2024-05-09T01:30:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.102+0000 7f00748b0700 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574d57dbf80 0x5574d57de3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.102+0000 7f004ffff700 10 client.16500 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.102+0000 7f00748b0700 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574d57dbf80 0x5574d57de3c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826f38321dea25ca server_cookie=d32e2e5fad17af36 in_seq=0 out_seq=0 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f004ffff700 1 -- 172.21.15.19:0/2038972968 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574d56f2a80 con 0x5574d57dbf80 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f004ffff700 10 client.16500 handle_command_reply: tid=0 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 client.16500 shutdown 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 -- 172.21.15.19:0/2038972968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d57c2400 msgr2=0x5574d57cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d57c2400 0x5574d57cca40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0068010b00 tx=0x7f0068010e10 comp rx=0 tx=0).stop 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 -- 172.21.15.19:0/2038972968 shutdown_connections 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574d57dbf80 0x5574d57de3c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d57c2400 0x5574d57cca40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56454a0 0x5574d57cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 --2- 172.21.15.19:0/2038972968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 0x5574d57cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 -- 172.21.15.19:0/2038972968 >> 172.21.15.19:0/2038972968 conn(0x5574d5648f70 msgr2=0x5574d57b6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f004ffff700 0 client.16500 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.103+0000 7f008552ab80 1 -- 172.21.15.19:0/2038972968 shutdown_connections 2024-05-09T01:30:18.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.104+0000 7f008552ab80 1 -- 172.21.15.19:0/2038972968 wait complete. 2024-05-09T01:30:18.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.104+0000 7f008552ab80 20 client.16500 trim_cache size 0 max 16384 2024-05-09T01:30:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.124+0000 7f008552ab80 1 Processor -- start 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.124+0000 7f008552ab80 1 -- start start 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f008552ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57bd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f008552ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56bfd00 0x5574d57cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f008552ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d57c2400 0x5574d57d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f008552ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00701aedf0 con 0x5574d56454a0 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f008552ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00701b0930 con 0x5574d57c2400 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f008552ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00701af4c0 con 0x5574d56bfd00 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00748b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57bd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00748b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57bd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57402/0 (socket says 172.21.15.19:57402) 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00748b0700 1 -- 172.21.15.19:0/4078009698 learned_addr learned my addr 172.21.15.19:0/4078009698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f006ffff700 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56bfd00 0x5574d57cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00748b0700 1 -- 172.21.15.19:0/4078009698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56bfd00 msgr2=0x5574d57cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00748b0700 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56bfd00 0x5574d57cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00748b0700 1 -- 172.21.15.19:0/4078009698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d57c2400 msgr2=0x5574d57d7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00750b1700 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d57c2400 0x5574d57d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00748b0700 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d57c2400 0x5574d57d7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00748b0700 1 -- 172.21.15.19:0/4078009698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d57cd080 con 0x5574d56454a0 2024-05-09T01:30:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f006ffff700 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56bfd00 0x5574d57cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00750b1700 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d57c2400 0x5574d57d7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f00748b0700 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57bd3f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f00640a9400 tx=0x7f0064091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbb457dc2b40d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f005a7fc700 1 -- 172.21.15.19:0/4078009698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006406a070 con 0x5574d56454a0 2024-05-09T01:30:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f005a7fc700 1 -- 172.21.15.19:0/4078009698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00640a9e00 con 0x5574d56454a0 2024-05-09T01:30:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.125+0000 7f005a7fc700 1 -- 172.21.15.19:0/4078009698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640a9960 con 0x5574d56454a0 2024-05-09T01:30:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.126+0000 7f008552ab80 1 -- 172.21.15.19:0/4078009698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 msgr2=0x5574d57bd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.126+0000 7f008552ab80 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57bd3f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f00640a9400 tx=0x7f0064091040 comp rx=0 tx=0).stop 2024-05-09T01:30:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.126+0000 7f008552ab80 1 -- 172.21.15.19:0/4078009698 shutdown_connections 2024-05-09T01:30:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.126+0000 7f008552ab80 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d57c2400 0x5574d57d7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.126+0000 7f008552ab80 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57bd3f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.126+0000 7f008552ab80 1 --2- 172.21.15.19:0/4078009698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d56bfd00 0x5574d57cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.126+0000 7f008552ab80 1 -- 172.21.15.19:0/4078009698 >> 172.21.15.19:0/4078009698 conn(0x5574d5648f70 msgr2=0x5574d57c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.126+0000 7f008552ab80 1 -- 172.21.15.19:0/4078009698 shutdown_connections 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.126+0000 7f008552ab80 1 -- 172.21.15.19:0/4078009698 wait complete. 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f008552ab80 1 -- start start 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f008552ab80 1 Processor -- start 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f008552ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57d4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f008552ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 0x5574d57e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f008552ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 0x5574d57e3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f008552ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00701b0260 con 0x5574d56454a0 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f008552ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00701afb90 con 0x5574d56bfd00 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f008552ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00701acd90 con 0x5574d57c2400 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f008552ab80 10 client.0 _ll_register_callbacks cb 0x5574d55566c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f00748b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f00748b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57d4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57406/0 (socket says 172.21.15.19:57406) 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f00748b0700 1 -- 172.21.15.19:0/3660377899 learned_addr learned my addr 172.21.15.19:0/3660377899 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.127+0000 7f006ffff700 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 0x5574d57e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f00750b1700 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 0x5574d57e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f00748b0700 1 -- 172.21.15.19:0/3660377899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 msgr2=0x5574d57e3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f00748b0700 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 0x5574d57e3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f00748b0700 1 -- 172.21.15.19:0/3660377899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 msgr2=0x5574d57e3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f00748b0700 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 0x5574d57e3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f00748b0700 1 -- 172.21.15.19:0/3660377899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d57e3e40 con 0x5574d56454a0 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f00750b1700 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 0x5574d57e3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f00748b0700 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57d4710 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f00640a93b0 tx=0x7f0064007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c59cb5fa50955ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f004ffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f004ffff700 1 -- 172.21.15.19:0/3660377899 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006406a040 con 0x5574d56454a0 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f004ffff700 1 -- 172.21.15.19:0/3660377899 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f00640910b0 con 0x5574d56454a0 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f008552ab80 1 -- 172.21.15.19:0/3660377899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00640a9e00 con 0x5574d56454a0 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f004ffff700 1 -- 172.21.15.19:0/3660377899 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00640912b0 con 0x5574d56454a0 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f008552ab80 10 client.16506 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.128+0000 7f008552ab80 1 -- 172.21.15.19:0/3660377899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574d5846570 con 0x5574d56454a0 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f004ffff700 1 -- 172.21.15.19:0/3660377899 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00640a9e00 con 0x5574d56454a0 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f004ffff700 1 -- 172.21.15.19:0/3660377899 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f00640a8960 con 0x5574d56454a0 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f008552ab80 10 client.16506 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f008552ab80 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f008552ab80 20 client.16506 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f008552ab80 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574d58467f0 0x5574d5848c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f008552ab80 4 client.16506 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f008552ab80 1 -- 172.21.15.19:0/3660377899 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574d56ecb20 con 0x5574d58467f0 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f006ffff700 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574d58467f0 0x5574d5848c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f006ffff700 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574d58467f0 0x5574d5848c10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14faa2a1d1ac4000 server_cookie=c2ba0528fa6cd115 in_seq=0 out_seq=0 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.129+0000 7f004ffff700 10 client.16506 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f004ffff700 1 -- 172.21.15.19:0/3660377899 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5574d56ecb20 con 0x5574d58467f0 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f004ffff700 10 client.16506 handle_command_reply: tid=0 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 client.16506 shutdown 2024-05-09T01:30:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 -- 172.21.15.19:0/3660377899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 msgr2=0x5574d57d4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57d4710 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f00640a93b0 tx=0x7f0064007640 comp rx=0 tx=0).stop 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 -- 172.21.15.19:0/3660377899 shutdown_connections 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574d58467f0 0x5574d5848c10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574d57c2400 0x5574d57e3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574d56454a0 0x5574d57d4710 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f004ffff700 0 client.16506 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 --2- 172.21.15.19:0/3660377899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574d56bfd00 0x5574d57e3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 -- 172.21.15.19:0/3660377899 >> 172.21.15.19:0/3660377899 conn(0x5574d5648f70 msgr2=0x5574d57b6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 -- 172.21.15.19:0/3660377899 shutdown_connections 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 1 -- 172.21.15.19:0/3660377899 wait complete. 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.130+0000 7f008552ab80 20 client.16506 trim_cache size 0 max 16384 2024-05-09T01:30:18.132 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:30:18.133 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:30:18.133 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:30:18.133 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:30:18.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 -- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f00600b2880 msgr2=0x7f00600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f00600b2880 0x7f00600b4d90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f005c006010 tx=0x7f005c005ea0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 -- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00701034c0 msgr2=0x7f007019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00701034c0 0x7f007019bc90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f006400d980 tx=0x7f006400dd40 comp rx=0 tx=0).stop 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 -- 172.21.15.19:0/238777440 shutdown_connections 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0070108740 0x7f007019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f00600b2880 0x7f00600b4d90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00701034c0 0x7f007019bc90 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 --2- 172.21.15.19:0/238777440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0070107df0 0x7f007019c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 -- 172.21.15.19:0/238777440 >> 172.21.15.19:0/238777440 conn(0x7f00700fedf0 msgr2=0x7f00701096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 -- 172.21.15.19:0/238777440 shutdown_connections 2024-05-09T01:30:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:18.131+0000 7f0076b14700 1 -- 172.21.15.19:0/238777440 wait complete. 2024-05-09T01:30:18.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:18 smithi196 ceph-mon[35041]: pgmap v521: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.731+0000 7fbdbc973640 1 -- 172.21.15.19:0/1423931749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 msgr2=0x7fbdb8104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.731+0000 7fbdbc973640 1 --2- 172.21.15.19:0/1423931749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 0x7fbdb8104980 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009900 tx=0x7fbdb002a410 comp rx=0 tx=0).stop 2024-05-09T01:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.731+0000 7fbdbc973640 1 -- 172.21.15.19:0/1423931749 shutdown_connections 2024-05-09T01:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.731+0000 7fbdbc973640 1 --2- 172.21.15.19:0/1423931749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdb8109bc0 0x7fbdb810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.731+0000 7fbdbc973640 1 --2- 172.21.15.19:0/1423931749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8104ec0 0x7fbdb81052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.731+0000 7fbdbc973640 1 --2- 172.21.15.19:0/1423931749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 0x7fbdb8104980 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.731+0000 7fbdbc973640 1 -- 172.21.15.19:0/1423931749 >> 172.21.15.19:0/1423931749 conn(0x7fbdb80fee50 msgr2=0x7fbdb8101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.732+0000 7fbdbc973640 1 -- 172.21.15.19:0/1423931749 shutdown_connections 2024-05-09T01:30:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.732+0000 7fbdbc973640 1 -- 172.21.15.19:0/1423931749 wait complete. 2024-05-09T01:30:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.732+0000 7fbdbc973640 1 Processor -- start 2024-05-09T01:30:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.732+0000 7fbdbc973640 1 -- start start 2024-05-09T01:30:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdbc973640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 0x7fbdb819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdbc973640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8104ec0 0x7fbdb819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdbc973640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdb8109bc0 0x7fbdb81a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdbc973640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb810f760 con 0x7fbdb8103c80 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdbc973640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb810f5e0 con 0x7fbdb8104ec0 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdbc973640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb810f8e0 con 0x7fbdb8109bc0 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 0x7fbdb819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 0x7fbdb819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57418/0 (socket says 172.21.15.19:57418) 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb659c640 1 -- 172.21.15.19:0/2595997573 learned_addr learned my addr 172.21.15.19:0/2595997573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8104ec0 0x7fbdb819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdb8109bc0 0x7fbdb81a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb659c640 1 -- 172.21.15.19:0/2595997573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdb8109bc0 msgr2=0x7fbdb81a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb659c640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdb8109bc0 0x7fbdb81a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb659c640 1 -- 172.21.15.19:0/2595997573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8104ec0 msgr2=0x7fbdb819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb659c640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8104ec0 0x7fbdb819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb659c640 1 -- 172.21.15.19:0/2595997573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb81a1ee0 con 0x7fbdb8103c80 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.733+0000 7fbdb659c640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 0x7fbdb819a3a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0006010 tx=0x7fbdb0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.734+0000 7fbdaf7fe640 1 -- 172.21.15.19:0/2595997573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb003f030 con 0x7fbdb8103c80 2024-05-09T01:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.734+0000 7fbdaf7fe640 1 -- 172.21.15.19:0/2595997573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdb0031910 con 0x7fbdb8103c80 2024-05-09T01:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.734+0000 7fbdbc973640 1 -- 172.21.15.19:0/2595997573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb81a2170 con 0x7fbdb8103c80 2024-05-09T01:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.734+0000 7fbdaf7fe640 1 -- 172.21.15.19:0/2595997573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0035070 con 0x7fbdb8103c80 2024-05-09T01:30:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.734+0000 7fbdbc973640 1 -- 172.21.15.19:0/2595997573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb81a2660 con 0x7fbdb8103c80 2024-05-09T01:30:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.736+0000 7fbdc381f740 1 Processor -- start 2024-05-09T01:30:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.736+0000 7fbdc381f740 1 -- start start 2024-05-09T01:30:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdaf7fe640 1 -- 172.21.15.19:0/2595997573 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbdb0042030 con 0x7fbdb8103c80 2024-05-09T01:30:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f2d1b0 0x5641252d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 0x5641252d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252d8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdc381f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5641252357c0 con 0x564124f2d1b0 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdc381f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5641251d10e0 con 0x564124f042f0 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdc381f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564125231780 con 0x564124f0c4d0 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f2d1b0 0x5641252d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f2d1b0 0x5641252d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57428/0 (socket says 172.21.15.19:57428) 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/2246448383 learned_addr learned my addr 172.21.15.19:0/2246448383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252d8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb659c640 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 0x5641252d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/2246448383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 msgr2=0x5641252d8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252d8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/2246448383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 msgr2=0x5641252d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 0x5641252d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/2246448383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641251fa7c0 con 0x564124f2d1b0 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb659c640 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 0x5641252d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.737+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252d8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.738+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f2d1b0 0x5641252d1500 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009fd0 tx=0x7fbda80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c24a3f8e2a55e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.738+0000 7fbdaf7fe640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbda40b27e0 0x7fbda40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.738+0000 7fbdacff9640 1 -- 172.21.15.19:0/2246448383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8004d20 con 0x564124f2d1b0 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.738+0000 7fbdacff9640 1 -- 172.21.15.19:0/2246448383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbda8004050 con 0x564124f2d1b0 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.738+0000 7fbdacff9640 1 -- 172.21.15.19:0/2246448383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8005240 con 0x564124f2d1b0 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.738+0000 7fbdb659c640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbda40b27e0 0x7fbda40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.738+0000 7fbdaf7fe640 1 -- 172.21.15.19:0/2595997573 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbdb0102560 con 0x7fbdb8103c80 2024-05-09T01:30:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 -- 172.21.15.19:0/2246448383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f2d1b0 msgr2=0x5641252d1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f2d1b0 0x5641252d1500 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009fd0 tx=0x7fbda80118d0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdb659c640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbda40b27e0 0x7fbda40b4cf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5641252d28e0 tx=0x7fbdb002a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 -- 172.21.15.19:0/2246448383 shutdown_connections 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252d8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 0x5641252d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2246448383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f2d1b0 0x5641252d1500 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 -- 172.21.15.19:0/2246448383 >> 172.21.15.19:0/2246448383 conn(0x564124f21300 msgr2=0x5641252cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 -- 172.21.15.19:0/2246448383 shutdown_connections 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 -- 172.21.15.19:0/2246448383 wait complete. 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 -- start start 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.739+0000 7fbdc381f740 1 Processor -- start 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252db750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f2d1b0 0x5641252e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdc381f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5641252d93d0 con 0x564124f042f0 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdc381f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5641251bc9c0 con 0x564124f2d1b0 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdc381f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5641252d9550 con 0x564124f0c4d0 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdc381f740 10 client.0 _ll_register_callbacks cb 0x5641251e9020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57438/0 (socket says 172.21.15.19:57438) 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/1926607593 learned_addr learned my addr 172.21.15.19:0/1926607593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252db750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/1926607593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 msgr2=0x5641252db750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252db750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/1926607593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f2d1b0 msgr2=0x5641252e26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f2d1b0 0x5641252e26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/1926607593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641251e31f0 con 0x564124f042f0 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252db750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db210 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbda801b040 tx=0x7fbda8004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464464574f1ed4d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbd9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.740+0000 7fbd9dffb640 1 -- 172.21.15.19:0/1926607593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8028020 con 0x564124f042f0 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbd9dffb640 1 -- 172.21.15.19:0/1926607593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbda8005040 con 0x564124f042f0 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbdc381f740 1 -- 172.21.15.19:0/1926607593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbda8004050 con 0x564124f042f0 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbd9dffb640 1 -- 172.21.15.19:0/1926607593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbda8033050 con 0x564124f042f0 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbdc381f740 10 client.16518 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbdc381f740 1 -- 172.21.15.19:0/1926607593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564124f98790 con 0x564124f042f0 2024-05-09T01:30:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbd9dffb640 1 -- 172.21.15.19:0/1926607593 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbda8004050 con 0x564124f042f0 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbd9dffb640 1 -- 172.21.15.19:0/1926607593 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbda800f680 con 0x564124f042f0 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbdc381f740 10 client.16518 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbdc381f740 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbdc381f740 20 client.16518 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564124eebd80 0x5641252e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbdc381f740 4 client.16518 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.741+0000 7fbdc381f740 1 -- 172.21.15.19:0/1926607593 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641252433d0 con 0x564124eebd80 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.742+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564124eebd80 0x5641252e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.742+0000 7fbd9dffb640 10 client.16518 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.742+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564124eebd80 0x5641252e89a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c29cfca896c152b server_cookie=122e3cc86a787fc2 in_seq=0 out_seq=0 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbd9dffb640 1 -- 172.21.15.19:0/1926607593 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5641252433d0 con 0x564124eebd80 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbd9dffb640 10 client.16518 handle_command_reply: tid=0 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 client.16518 shutdown 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 -- 172.21.15.19:0/1926607593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 msgr2=0x5641252db210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db210 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbda801b040 tx=0x7fbda8004460 comp rx=0 tx=0).stop 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 -- 172.21.15.19:0/1926607593 shutdown_connections 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564124eebd80 0x5641252e89a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f2d1b0 0x5641252e26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f0c4d0 0x5641252db750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1926607593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db210 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 -- 172.21.15.19:0/1926607593 >> 172.21.15.19:0/1926607593 conn(0x564124f21300 msgr2=0x5641252cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbd9dffb640 0 client.16518 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 -- 172.21.15.19:0/1926607593 shutdown_connections 2024-05-09T01:30:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 1 -- 172.21.15.19:0/1926607593 wait complete. 2024-05-09T01:30:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.743+0000 7fbdc381f740 20 client.16518 trim_cache size 0 max 16384 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.758+0000 7fbdc381f740 1 Processor -- start 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.758+0000 7fbdc381f740 1 -- start start 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f0c4d0 0x5641252e2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 0x5641252e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdc381f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb81ab500 con 0x564124f042f0 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdc381f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb81ad040 con 0x564124f0c4d0 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdc381f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb81abbd0 con 0x564124f2d1b0 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57440/0 (socket says 172.21.15.19:57440) 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/1686518725 learned_addr learned my addr 172.21.15.19:0/1686518725 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/1686518725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 msgr2=0x5641252e2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/1686518725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 0x5641252e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/1686518725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f0c4d0 0x5641252e2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/1686518725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f0c4d0 msgr2=0x5641252e2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/1686518725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f0c4d0 0x5641252e2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb6d9d640 1 -- 172.21.15.19:0/1686518725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56412500f5f0 con 0x564124f042f0 2024-05-09T01:30:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/1686518725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db4f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbda8011910 tx=0x7fbda8003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485ae8c7ef66f2a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.759+0000 7fbd9ffff640 1 -- 172.21.15.19:0/1686518725 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda803c070 con 0x564124f042f0 2024-05-09T01:30:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbd9ffff640 1 -- 172.21.15.19:0/1686518725 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbda80044d0 con 0x564124f042f0 2024-05-09T01:30:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbd9ffff640 1 -- 172.21.15.19:0/1686518725 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8028040 con 0x564124f042f0 2024-05-09T01:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbdc381f740 1 -- 172.21.15.19:0/1686518725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 msgr2=0x5641252db4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1686518725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db4f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbda8011910 tx=0x7fbda8003ee0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbdc381f740 1 -- 172.21.15.19:0/1686518725 shutdown_connections 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1686518725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 0x5641252e2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1686518725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f0c4d0 0x5641252e2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbdc381f740 1 --2- 172.21.15.19:0/1686518725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f042f0 0x5641252db4f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbdc381f740 1 -- 172.21.15.19:0/1686518725 >> 172.21.15.19:0/1686518725 conn(0x564124f21300 msgr2=0x5641252dade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbdc381f740 1 -- 172.21.15.19:0/1686518725 shutdown_connections 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.760+0000 7fbdc381f740 1 -- 172.21.15.19:0/1686518725 wait complete. 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdc381f740 1 -- start start 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdc381f740 1 Processor -- start 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 0x5641252e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f0c4d0 0x5641252e0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdc381f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 0x5641252e1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdc381f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb81ac970 con 0x564124f0c4d0 2024-05-09T01:30:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdc381f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb81ac2a0 con 0x564124f042f0 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdc381f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb81a94e0 con 0x564124f2d1b0 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdc381f740 10 client.0 _ll_register_callbacks cb 0x7fbda0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f0c4d0 0x5641252e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f0c4d0 0x5641252e0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57450/0 (socket says 172.21.15.19:57450) 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb5d9b640 1 -- 172.21.15.19:0/2748613923 learned_addr learned my addr 172.21.15.19:0/2748613923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 0x5641252e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb5d9b640 1 -- 172.21.15.19:0/2748613923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 msgr2=0x5641252e1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 0x5641252e1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 0x5641252e1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb5d9b640 1 -- 172.21.15.19:0/2748613923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 msgr2=0x5641252e03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 0x5641252e03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb5d9b640 1 -- 172.21.15.19:0/2748613923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641251f9ed0 con 0x564124f0c4d0 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.761+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 0x5641252e1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdb5d9b640 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f0c4d0 0x5641252e0930 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009c40 tx=0x7fbda0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d49df010479c20c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbd9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbd9dffb640 1 -- 172.21.15.19:0/2748613923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda001de60 con 0x564124f0c4d0 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbd9dffb640 1 -- 172.21.15.19:0/2748613923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbda0004460 con 0x564124f0c4d0 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdc381f740 1 -- 172.21.15.19:0/2748613923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbda80044d0 con 0x564124f0c4d0 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbd9dffb640 1 -- 172.21.15.19:0/2748613923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbda001d460 con 0x564124f0c4d0 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdc381f740 10 client.16524 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdc381f740 1 -- 172.21.15.19:0/2748613923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641252e2fe0 con 0x564124f0c4d0 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbd9dffb640 1 -- 172.21.15.19:0/2748613923 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbda80044d0 con 0x564124f0c4d0 2024-05-09T01:30:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbd9dffb640 1 -- 172.21.15.19:0/2748613923 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbda0003c30 con 0x564124f0c4d0 2024-05-09T01:30:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdc381f740 10 client.16524 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdc381f740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdc381f740 20 client.16524 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564124ec11c0 0x5641253533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdc381f740 4 client.16524 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:30:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.762+0000 7fbdc381f740 1 -- 172.21.15.19:0/2748613923 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56412523d240 con 0x564124ec11c0 2024-05-09T01:30:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.763+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564124ec11c0 0x5641253533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.763+0000 7fbd9dffb640 10 client.16524 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.763+0000 7fbdb6d9d640 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564124ec11c0 0x5641253533e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97c902c33c873e40 server_cookie=d9172d199ccac6b0 in_seq=0 out_seq=0 2024-05-09T01:30:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbd9dffb640 1 -- 172.21.15.19:0/2748613923 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56412523d240 con 0x564124ec11c0 2024-05-09T01:30:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbd9dffb640 10 client.16524 handle_command_reply: tid=0 2024-05-09T01:30:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 client.16524 shutdown 2024-05-09T01:30:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 -- 172.21.15.19:0/2748613923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f0c4d0 msgr2=0x5641252e0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f0c4d0 0x5641252e0930 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009c40 tx=0x7fbda0010800 comp rx=0 tx=0).stop 2024-05-09T01:30:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 -- 172.21.15.19:0/2748613923 shutdown_connections 2024-05-09T01:30:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564124ec11c0 0x5641253533e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564124f2d1b0 0x5641252e1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbd9dffb640 0 client.16524 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564124f0c4d0 0x5641252e0930 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 --2- 172.21.15.19:0/2748613923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564124f042f0 0x5641252e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 -- 172.21.15.19:0/2748613923 >> 172.21.15.19:0/2748613923 conn(0x564124f21300 msgr2=0x5641252cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 -- 172.21.15.19:0/2748613923 shutdown_connections 2024-05-09T01:30:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 1 -- 172.21.15.19:0/2748613923 wait complete. 2024-05-09T01:30:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.779+0000 7fbdc381f740 20 client.16524 trim_cache size 0 max 16384 2024-05-09T01:30:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:30:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:30:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:30:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 -- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbda40b27e0 msgr2=0x7fbda40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbda40b27e0 0x7fbda40b4cf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5641252d28e0 tx=0x7fbdb002a410 comp rx=0 tx=0).stop 2024-05-09T01:30:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 -- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 msgr2=0x7fbdb819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 0x7fbdb819a3a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0006010 tx=0x7fbdb0004270 comp rx=0 tx=0).stop 2024-05-09T01:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 -- 172.21.15.19:0/2595997573 shutdown_connections 2024-05-09T01:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbda40b27e0 0x7fbda40b4cf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdb8109bc0 0x7fbdb81a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8104ec0 0x7fbdb819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 --2- 172.21.15.19:0/2595997573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb8103c80 0x7fbdb819a3a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 -- 172.21.15.19:0/2595997573 >> 172.21.15.19:0/2595997573 conn(0x7fbdb80fee50 msgr2=0x7fbdb8105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 -- 172.21.15.19:0/2595997573 shutdown_connections 2024-05-09T01:30:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:18.780+0000 7fbdbc973640 1 -- 172.21.15.19:0/2595997573 wait complete. 2024-05-09T01:30:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:18 smithi059 ceph-mon[34351]: pgmap v521: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:30:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:18 smithi019 ceph-mon[30158]: pgmap v521: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-05-09T01:30:18.972 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:30:18.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:30:19.223 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:20.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:19 smithi019 ceph-mon[30158]: pgmap v522: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:30:20.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:19 smithi059 ceph-mon[34351]: pgmap v522: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:30:20.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:19 smithi196 ceph-mon[35041]: pgmap v522: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:30:21.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.021+0000 7f748a6a3700 1 -- 172.21.15.19:0/2116587047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484108740 msgr2=0x7f74841100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.021+0000 7f748a6a3700 1 --2- 172.21.15.19:0/2116587047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484108740 0x7f74841100a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f747c00b0d0 tx=0x7f747c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.021+0000 7f748a6a3700 1 -- 172.21.15.19:0/2116587047 shutdown_connections 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.021+0000 7f748a6a3700 1 --2- 172.21.15.19:0/2116587047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484108740 0x7f74841100a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.021+0000 7f748a6a3700 1 --2- 172.21.15.19:0/2116587047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74841034c0 0x7f74841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.021+0000 7f748a6a3700 1 --2- 172.21.15.19:0/2116587047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7484107df0 0x7f7484108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.021+0000 7f748a6a3700 1 -- 172.21.15.19:0/2116587047 >> 172.21.15.19:0/2116587047 conn(0x7f74840fedf0 msgr2=0x7f7484101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.021+0000 7f748a6a3700 1 -- 172.21.15.19:0/2116587047 shutdown_connections 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.021+0000 7f748a6a3700 1 -- 172.21.15.19:0/2116587047 wait complete. 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f748a6a3700 1 Processor -- start 2024-05-09T01:30:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f748a6a3700 1 -- start start 2024-05-09T01:30:21.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f748a6a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74841034c0 0x7f748419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:21.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f748a6a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484107df0 0x7f748419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:21.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f748a6a3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484108740 0x7f748419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:21.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f748a6a3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7484113370 con 0x7f7484107df0 2024-05-09T01:30:21.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f748a6a3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74841131f0 con 0x7f74841034c0 2024-05-09T01:30:21.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f748a6a3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74841134f0 con 0x7f7484108740 2024-05-09T01:30:21.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f74837fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484107df0 0x7f748419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:21.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f7483fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74841034c0 0x7f748419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f74837fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484107df0 0x7f748419c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40070/0 (socket says 172.21.15.19:40070) 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f74837fe700 1 -- 172.21.15.19:0/99247500 learned_addr learned my addr 172.21.15.19:0/99247500 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.022+0000 7f7488c40700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484108740 0x7f748419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f74837fe700 1 -- 172.21.15.19:0/99247500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484108740 msgr2=0x7f748419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f74837fe700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484108740 0x7f748419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f74837fe700 1 -- 172.21.15.19:0/99247500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74841034c0 msgr2=0x7f748419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f74837fe700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74841034c0 0x7f748419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f74837fe700 1 -- 172.21.15.19:0/99247500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74841a6f80 con 0x7f7484107df0 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f7483fff700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74841034c0 0x7f748419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f74837fe700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484107df0 0x7f748419c150 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f747800d950 tx=0x7f747800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f74817fa700 1 -- 172.21.15.19:0/99247500 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478016070 con 0x7f7484107df0 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74841a7270 con 0x7f7484107df0 2024-05-09T01:30:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f74817fa700 1 -- 172.21.15.19:0/99247500 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7478004830 con 0x7f7484107df0 2024-05-09T01:30:21.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f74817fa700 1 -- 172.21.15.19:0/99247500 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478005520 con 0x7f7484107df0 2024-05-09T01:30:21.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.023+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74841a77b0 con 0x7f7484107df0 2024-05-09T01:30:21.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.024+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74840512c0 con 0x7f7484107df0 2024-05-09T01:30:21.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.026+0000 7f74817fa700 1 -- 172.21.15.19:0/99247500 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7478004220 con 0x7f7484107df0 2024-05-09T01:30:21.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.027+0000 7f74817fa700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2880 0x7f746c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:21.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.027+0000 7f7483fff700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2880 0x7f746c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:21.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.027+0000 7f74817fa700 1 -- 172.21.15.19:0/99247500 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f74780e23b0 con 0x7f7484107df0 2024-05-09T01:30:21.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.028+0000 7f7483fff700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2880 0x7f746c0b4d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7474005950 tx=0x7f747400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:21.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.031+0000 7f74817fa700 1 -- 172.21.15.19:0/99247500 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f747801f020 con 0x7f7484107df0 2024-05-09T01:30:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.294+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7484076000 con 0x7f7484107df0 2024-05-09T01:30:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.295+0000 7f74817fa700 1 -- 172.21.15.19:0/99247500 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f74780aa0b0 con 0x7f7484107df0 2024-05-09T01:30:21.292 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:30:21.292 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.297+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2880 msgr2=0x7f746c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.297+0000 7f748a6a3700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2880 0x7f746c0b4d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7474005950 tx=0x7f747400a300 comp rx=0 tx=0).stop 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.297+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484107df0 msgr2=0x7f748419c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.297+0000 7f748a6a3700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484107df0 0x7f748419c150 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f747800d950 tx=0x7f747800dc60 comp rx=0 tx=0).stop 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.298+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 shutdown_connections 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.298+0000 7f748a6a3700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484108740 0x7f748419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.298+0000 7f748a6a3700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74841034c0 0x7f748419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.298+0000 7f748a6a3700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f746c0b2880 0x7f746c0b4d90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.298+0000 7f748a6a3700 1 --2- 172.21.15.19:0/99247500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7484107df0 0x7f748419c150 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.298+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 >> 172.21.15.19:0/99247500 conn(0x7f74840fedf0 msgr2=0x7f7484107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.298+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 shutdown_connections 2024-05-09T01:30:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:21.298+0000 7f748a6a3700 1 -- 172.21.15.19:0/99247500 wait complete. 2024-05-09T01:30:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:30:21.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:21 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/99247500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:30:21.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:21 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/99247500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:30:21.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:21 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/99247500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:30:22.001 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:30:22.255 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:22.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:22 smithi019 ceph-mon[30158]: pgmap v523: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:30:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:22 smithi196 ceph-mon[35041]: pgmap v523: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:30:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:22 smithi059 ceph-mon[34351]: pgmap v523: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:30:24.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.135+0000 7f6fa708e700 1 -- 172.21.15.19:0/249166807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 msgr2=0x7f6fa01111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.135+0000 7f6fa708e700 1 --2- 172.21.15.19:0/249166807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 0x7f6fa01111c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800b0d0 tx=0x7f6f9800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.136+0000 7f6fa708e700 1 -- 172.21.15.19:0/249166807 shutdown_connections 2024-05-09T01:30:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.136+0000 7f6fa708e700 1 --2- 172.21.15.19:0/249166807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 0x7f6fa01111c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.136+0000 7f6fa708e700 1 --2- 172.21.15.19:0/249166807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fa01045e0 0x7f6fa01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.136+0000 7f6fa708e700 1 --2- 172.21.15.19:0/249166807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fa0108f10 0x7f6fa0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.136+0000 7f6fa708e700 1 -- 172.21.15.19:0/249166807 >> 172.21.15.19:0/249166807 conn(0x7f6fa00ffed0 msgr2=0x7f6fa0102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.136+0000 7f6fa708e700 1 -- 172.21.15.19:0/249166807 shutdown_connections 2024-05-09T01:30:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.136+0000 7f6fa708e700 1 -- 172.21.15.19:0/249166807 wait complete. 2024-05-09T01:30:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.136+0000 7f6fa708e700 1 Processor -- start 2024-05-09T01:30:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa708e700 1 -- start start 2024-05-09T01:30:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa708e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fa01045e0 0x7f6fa019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa708e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fa0108f10 0x7f6fa019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa708e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 0x7f6fa01a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa708e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0114490 con 0x7f6fa0109860 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa708e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0114310 con 0x7f6fa0108f10 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa708e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0114610 con 0x7f6fa01045e0 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa562b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 0x7f6fa01a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6f9ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fa0108f10 0x7f6fa019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6f9ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fa0108f10 0x7f6fa019d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:34428/0 (socket says 172.21.15.19:34428) 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa562b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 0x7f6fa01a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40078/0 (socket says 172.21.15.19:40078) 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6f9ffff700 1 -- 172.21.15.19:0/1943619060 learned_addr learned my addr 172.21.15.19:0/1943619060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.137+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fa01045e0 0x7f6fa019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6f9ffff700 1 -- 172.21.15.19:0/1943619060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fa01045e0 msgr2=0x7f6fa019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fa01045e0 0x7f6fa019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6f9ffff700 1 -- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 msgr2=0x7f6fa01a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 0x7f6fa01a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6f9ffff700 1 -- 172.21.15.19:0/1943619060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa01a1e50 con 0x7f6fa0108f10 2024-05-09T01:30:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6fa562b700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 0x7f6fa01a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fa0108f10 0x7f6fa019d520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e970 tx=0x7f6f8800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6f9dffb700 1 -- 172.21.15.19:0/1943619060 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88004a30 con 0x7f6fa0108f10 2024-05-09T01:30:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6fa708e700 1 -- 172.21.15.19:0/1943619060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa01a2140 con 0x7f6fa0108f10 2024-05-09T01:30:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6f9dffb700 1 -- 172.21.15.19:0/1943619060 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f8800cc90 con 0x7f6fa0108f10 2024-05-09T01:30:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.138+0000 7f6f9dffb700 1 -- 172.21.15.19:0/1943619060 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88005610 con 0x7f6fa0108f10 2024-05-09T01:30:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.139+0000 7f6fa708e700 1 -- 172.21.15.19:0/1943619060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa01a9a60 con 0x7f6fa0108f10 2024-05-09T01:30:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.141+0000 7f6f9dffb700 1 -- 172.21.15.19:0/1943619060 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6f8800c7e0 con 0x7f6fa0108f10 2024-05-09T01:30:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.142+0000 7f6f9dffb700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f8c0b2880 0x7f6f8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.142+0000 7f6fb5aa4b80 1 Processor -- start 2024-05-09T01:30:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.142+0000 7f6fb5aa4b80 1 -- start start 2024-05-09T01:30:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.142+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f8c0b2880 0x7f6f8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.142+0000 7f6f9dffb700 1 -- 172.21.15.19:0/1943619060 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6f880e23b0 con 0x7f6fa0108f10 2024-05-09T01:30:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6484a0 0x557e7d7cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e7d6e80a0 con 0x557e7d6c2d00 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557e7d68d9a0 con 0x557e7d6484a0 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e7d6e4060 con 0x557e7d7c5400 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40094/0 (socket says 172.21.15.19:40094) 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/2806959465 learned_addr learned my addr 172.21.15.19:0/2806959465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6484a0 0x557e7d7cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f8c0b2880 0x7f6f8c0b4d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6f90005950 tx=0x7f6f900106b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa562b700 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/2806959465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 msgr2=0x557e7d7c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/2806959465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6484a0 msgr2=0x557e7d7cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6484a0 0x557e7d7cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/2806959465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e7d67e500 con 0x557e7d6c2d00 2024-05-09T01:30:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6484a0 0x557e7d7cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa562b700 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7cb240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000bec0 tx=0x7f6f9001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f824ad12807274 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.143+0000 7f6f96ffd700 1 -- 172.21.15.19:0/2806959465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90013110 con 0x557e7d6c2d00 2024-05-09T01:30:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.144+0000 7f6f96ffd700 1 -- 172.21.15.19:0/2806959465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9001a040 con 0x557e7d6c2d00 2024-05-09T01:30:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.144+0000 7f6f96ffd700 1 -- 172.21.15.19:0/2806959465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90011660 con 0x557e7d6c2d00 2024-05-09T01:30:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.144+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2806959465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 msgr2=0x557e7d7cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.144+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7cb240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000bec0 tx=0x7f6f9001e000 comp rx=0 tx=0).stop 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.144+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2806959465 shutdown_connections 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.144+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.144+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7cb240 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.144+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2806959465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6484a0 0x557e7d7cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.144+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2806959465 >> 172.21.15.19:0/2806959465 conn(0x557e7d64bf70 msgr2=0x557e7d7c0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.145+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2806959465 shutdown_connections 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.145+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2806959465 wait complete. 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.145+0000 7f6fb5aa4b80 1 -- start start 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.145+0000 7f6fb5aa4b80 1 Processor -- start 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6c2d00 0x557e7d7cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e7d7c6160 con 0x557e7d6484a0 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557e7d682b20 con 0x557e7d6c2d00 2024-05-09T01:30:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e7d7d2020 con 0x557e7d7c5400 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fb5aa4b80 10 client.0 _ll_register_callbacks cb 0x557e7d5596c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa4e2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa4e2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40102/0 (socket says 172.21.15.19:40102) 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/2923259081 learned_addr learned my addr 172.21.15.19:0/2923259081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6c2d00 0x557e7d7cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa562b700 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/2923259081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 msgr2=0x557e7d7cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/2923259081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6c2d00 msgr2=0x557e7d7cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6c2d00 0x557e7d7cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/2923259081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e7d563490 con 0x557e7d6484a0 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa562b700 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7cf080 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6f900133f0 tx=0x7f6f90005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612ddf15906c28a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.146+0000 7f6f7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2923259081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90011440 con 0x557e7d6484a0 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2923259081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9001a040 con 0x557e7d6484a0 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2923259081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6f900115e0 con 0x557e7d6484a0 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2923259081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f900117e0 con 0x557e7d6484a0 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6fb5aa4b80 10 client.16542 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2923259081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e7d7dc8f0 con 0x557e7d6484a0 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2923259081 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9001a040 con 0x557e7d6484a0 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2923259081 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6f90029a00 con 0x557e7d6484a0 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6fb5aa4b80 10 client.16542 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6fb5aa4b80 10 client.16542 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6fb5aa4b80 20 client.16542 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x557e7d7dee70 0x557e7d7e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6fb5aa4b80 4 client.16542 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:24.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.147+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2923259081 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e7d6f5a80 con 0x557e7d7dee70 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.148+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x557e7d7dee70 0x557e7d7e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.148+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x557e7d7dee70 0x557e7d7e1290 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a15c10d3c13bca57 server_cookie=db51e410cd64115f in_seq=0 out_seq=0 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.148+0000 7f6f7ffff700 10 client.16542 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2923259081 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e7d6f5a80 con 0x557e7d7dee70 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6f7ffff700 10 client.16542 handle_command_reply: tid=0 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 client.16542 shutdown 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2923259081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 msgr2=0x557e7d7cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7cf080 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6f900133f0 tx=0x7f6f90005860 comp rx=0 tx=0).stop 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2923259081 shutdown_connections 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x557e7d7dee70 0x557e7d7e1290 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d7c5400 0x557e7d7cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7cf080 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2923259081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d6c2d00 0x557e7d7cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2923259081 >> 172.21.15.19:0/2923259081 conn(0x557e7d64bf70 msgr2=0x557e7d7b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6f7ffff700 0 client.16542 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:30:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2923259081 shutdown_connections 2024-05-09T01:30:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2923259081 wait complete. 2024-05-09T01:30:24.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.149+0000 7f6fb5aa4b80 20 client.16542 trim_cache size 0 max 16384 2024-05-09T01:30:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fb5aa4b80 1 Processor -- start 2024-05-09T01:30:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fb5aa4b80 1 -- start start 2024-05-09T01:30:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6c2d00 0x557e7d7db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 0x557e7d7ba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01b0f00 con 0x557e7d6484a0 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01b2a40 con 0x557e7d7c5400 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01b15d0 con 0x557e7d6c2d00 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fa4e2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fa4e2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40116/0 (socket says 172.21.15.19:40116) 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.165+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/959717295 learned_addr learned my addr 172.21.15.19:0/959717295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6c2d00 0x557e7d7db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6fa562b700 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 0x557e7d7ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/959717295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6c2d00 msgr2=0x557e7d7db190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6c2d00 0x557e7d7db190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/959717295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 msgr2=0x557e7d7ba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 0x557e7d7ba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6fa4e2a700 1 -- 172.21.15.19:0/959717295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e7d7bc9d0 con 0x557e7d6484a0 2024-05-09T01:30:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6fa562b700 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 0x557e7d7ba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7d14d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6f90029ef0 tx=0x7f6f90013380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba7f7c6925bf6bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6f967fc700 1 -- 172.21.15.19:0/959717295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90020990 con 0x557e7d6484a0 2024-05-09T01:30:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6f967fc700 1 -- 172.21.15.19:0/959717295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f90040070 con 0x557e7d6484a0 2024-05-09T01:30:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.166+0000 7f6f967fc700 1 -- 172.21.15.19:0/959717295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90029a70 con 0x557e7d6484a0 2024-05-09T01:30:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/959717295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 msgr2=0x557e7d7d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7d14d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6f90029ef0 tx=0x7f6f90013380 comp rx=0 tx=0).stop 2024-05-09T01:30:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/959717295 shutdown_connections 2024-05-09T01:30:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 0x557e7d7ba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6484a0 0x557e7d7d14d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/959717295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6c2d00 0x557e7d7db190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/959717295 >> 172.21.15.19:0/959717295 conn(0x557e7d64bf70 msgr2=0x557e7d7b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/959717295 shutdown_connections 2024-05-09T01:30:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/959717295 wait complete. 2024-05-09T01:30:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.167+0000 7f6fb5aa4b80 1 -- start start 2024-05-09T01:30:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6fb5aa4b80 1 Processor -- start 2024-05-09T01:30:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6484a0 0x557e7d7e0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7e13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6fb5aa4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 0x557e7d7e1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01b2370 con 0x557e7d6c2d00 2024-05-09T01:30:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01b1ca0 con 0x557e7d7c5400 2024-05-09T01:30:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6fb5aa4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01aeea0 con 0x557e7d6484a0 2024-05-09T01:30:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6fb5aa4b80 10 client.0 _ll_register_callbacks cb 0x557e7d5596c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6f9ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7e13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6f9ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7e13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40132/0 (socket says 172.21.15.19:40132) 2024-05-09T01:30:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6f9ffff700 1 -- 172.21.15.19:0/2992325432 learned_addr learned my addr 172.21.15.19:0/2992325432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.168+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6484a0 0x557e7d7e0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6fa562b700 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 0x557e7d7e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f9ffff700 1 -- 172.21.15.19:0/2992325432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6484a0 msgr2=0x557e7d7e0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6484a0 0x557e7d7e0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f9ffff700 1 -- 172.21.15.19:0/2992325432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 msgr2=0x557e7d7e1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 0x557e7d7e1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f9ffff700 1 -- 172.21.15.19:0/2992325432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e7d7b9cc0 con 0x557e7d6c2d00 2024-05-09T01:30:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f9ffff700 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7e13c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6f880a98e0 tx=0x7f6f88091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a673267d0b55f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2992325432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f880c3070 con 0x557e7d6c2d00 2024-05-09T01:30:24.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2992325432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6f880cc0b0 con 0x557e7d6c2d00 2024-05-09T01:30:24.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2992325432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f90040070 con 0x557e7d6c2d00 2024-05-09T01:30:24.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2992325432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f880cc2b0 con 0x557e7d6c2d00 2024-05-09T01:30:24.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6fb5aa4b80 10 client.16548 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:24.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.169+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2992325432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e7d848370 con 0x557e7d6c2d00 2024-05-09T01:30:24.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.170+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2992325432 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f90040070 con 0x557e7d6c2d00 2024-05-09T01:30:24.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.170+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2992325432 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6f880e1040 con 0x557e7d6c2d00 2024-05-09T01:30:24.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.170+0000 7f6fb5aa4b80 10 client.16548 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:24.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.170+0000 7f6fb5aa4b80 10 client.16548 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:30:24.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.170+0000 7f6fb5aa4b80 20 client.16548 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:24.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.170+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x557e7d848a10 0x557e7d84ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:24.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.170+0000 7f6fb5aa4b80 4 client.16548 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:24.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.170+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2992325432 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e7d6efb20 con 0x557e7d848a10 2024-05-09T01:30:24.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.170+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x557e7d848a10 0x557e7d84ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:24.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6f7ffff700 10 client.16548 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:30:24.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fa4e2a700 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x557e7d848a10 0x557e7d84ae30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=769eac3b7c01fd9e server_cookie=df1b40eef01e876d in_seq=0 out_seq=0 2024-05-09T01:30:24.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6f7ffff700 1 -- 172.21.15.19:0/2992325432 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e7d6efb20 con 0x557e7d848a10 2024-05-09T01:30:24.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6f7ffff700 10 client.16548 handle_command_reply: tid=0 2024-05-09T01:30:24.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 client.16548 shutdown 2024-05-09T01:30:24.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2992325432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 msgr2=0x557e7d7e13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:24.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7e13c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6f880a98e0 tx=0x7f6f88091040 comp rx=0 tx=0).stop 2024-05-09T01:30:24.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2992325432 shutdown_connections 2024-05-09T01:30:24.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x557e7d848a10 0x557e7d84ae30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e7d7c5400 0x557e7d7e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e7d6484a0 0x557e7d7e0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 --2- 172.21.15.19:0/2992325432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e7d6c2d00 0x557e7d7e13c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2992325432 >> 172.21.15.19:0/2992325432 conn(0x557e7d64bf70 msgr2=0x557e7d7c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:24.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6f7ffff700 0 client.16548 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:30:24.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2992325432 shutdown_connections 2024-05-09T01:30:24.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 1 -- 172.21.15.19:0/2992325432 wait complete. 2024-05-09T01:30:24.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.171+0000 7f6fb5aa4b80 20 client.16548 trim_cache size 0 max 16384 2024-05-09T01:30:24.178 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:24.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 -- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f8c0b2880 msgr2=0x7f6f8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:24.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f8c0b2880 0x7f6f8c0b4d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6f90005950 tx=0x7f6f900106b0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 -- 172.21.15.19:0/1943619060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fa0108f10 msgr2=0x7f6fa019d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:24.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fa0108f10 0x7f6fa019d520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e970 tx=0x7f6f8800ec80 comp rx=0 tx=0).stop 2024-05-09T01:30:24.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 -- 172.21.15.19:0/1943619060 shutdown_connections 2024-05-09T01:30:24.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6f8c0b2880 0x7f6f8c0b4d90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0109860 0x7f6fa01a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fa01045e0 0x7f6fa019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 --2- 172.21.15.19:0/1943619060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fa0108f10 0x7f6fa019d520 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:24.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 -- 172.21.15.19:0/1943619060 >> 172.21.15.19:0/1943619060 conn(0x7f6fa00ffed0 msgr2=0x7f6fa0110e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:24.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 -- 172.21.15.19:0/1943619060 shutdown_connections 2024-05-09T01:30:24.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:24.172+0000 7f6fa708e700 1 -- 172.21.15.19:0/1943619060 wait complete. 2024-05-09T01:30:24.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:24 smithi019 ceph-mon[30158]: pgmap v524: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:24.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:24 smithi196 ceph-mon[35041]: pgmap v524: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:24.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:30:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:24 smithi059 ceph-mon[34351]: pgmap v524: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:25.093 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:26.601 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:26 smithi019 ceph-mon[30158]: pgmap v525: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:26.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:26 smithi196 ceph-mon[35041]: pgmap v525: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:26 smithi059 ceph-mon[34351]: pgmap v525: 97 pgs: 97 active+clean; 2.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.858+0000 7fd3e5df7700 1 -- 172.21.15.19:0/44299924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e0108740 msgr2=0x7fd3e01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.858+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/44299924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e0108740 0x7fd3e01100a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80099c0 tx=0x7fd3d8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.858+0000 7fd3e5df7700 1 -- 172.21.15.19:0/44299924 shutdown_connections 2024-05-09T01:30:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.858+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/44299924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e0108740 0x7fd3e01100a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.858+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/44299924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e01034c0 0x7fd3e01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.858+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/44299924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e0107df0 0x7fd3e0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.858+0000 7fd3e5df7700 1 -- 172.21.15.19:0/44299924 >> 172.21.15.19:0/44299924 conn(0x7fd3e00fedc0 msgr2=0x7fd3e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.858+0000 7fd3e5df7700 1 -- 172.21.15.19:0/44299924 shutdown_connections 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.858+0000 7fd3e5df7700 1 -- 172.21.15.19:0/44299924 wait complete. 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3e5df7700 1 Processor -- start 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3e5df7700 1 -- start start 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3e5df7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e01034c0 0x7fd3e019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3e5df7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e0107df0 0x7fd3e019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3e5df7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e0108740 0x7fd3e01a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3e5df7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e0113370 con 0x7fd3e01034c0 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3e5df7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01131f0 con 0x7fd3e0107df0 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3e5df7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01134f0 con 0x7fd3e0108740 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e01034c0 0x7fd3e019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e01034c0 0x7fd3e019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40144/0 (socket says 172.21.15.19:40144) 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.859+0000 7fd3df7fe700 1 -- 172.21.15.19:0/3798385276 learned_addr learned my addr 172.21.15.19:0/3798385276 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3deffd700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e0107df0 0x7fd3e019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3dffff700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e0108740 0x7fd3e01a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3df7fe700 1 -- 172.21.15.19:0/3798385276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e0108740 msgr2=0x7fd3e01a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e0108740 0x7fd3e01a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3df7fe700 1 -- 172.21.15.19:0/3798385276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e0107df0 msgr2=0x7fd3e019d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e0107df0 0x7fd3e019d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3df7fe700 1 -- 172.21.15.19:0/3798385276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e01a1e30 con 0x7fd3e01034c0 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3dffff700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e0108740 0x7fd3e01a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3deffd700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e0107df0 0x7fd3e019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e01034c0 0x7fd3e019cf60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00060b0 tx=0x7fd3d0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3dcff9700 1 -- 172.21.15.19:0/3798385276 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d001a070 con 0x7fd3e01034c0 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3e5df7700 1 -- 172.21.15.19:0/3798385276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e01a2120 con 0x7fd3e01034c0 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3dcff9700 1 -- 172.21.15.19:0/3798385276 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d000de20 con 0x7fd3e01034c0 2024-05-09T01:30:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3dcff9700 1 -- 172.21.15.19:0/3798385276 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0014720 con 0x7fd3e01034c0 2024-05-09T01:30:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.860+0000 7fd3e5df7700 1 -- 172.21.15.19:0/3798385276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e01a9a40 con 0x7fd3e01034c0 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.863+0000 7fd3dcff9700 1 -- 172.21.15.19:0/3798385276 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd3d001d070 con 0x7fd3e01034c0 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.864+0000 7fd3dcff9700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3cc0b2880 0x7fd3cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.864+0000 7fd3deffd700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3cc0b2880 0x7fd3cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.864+0000 7fd3dcff9700 1 -- 172.21.15.19:0/3798385276 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd3d00e1bb0 con 0x7fd3e01034c0 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.864+0000 7fd3deffd700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3cc0b2880 0x7fd3cc0b4d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005e10 tx=0x7fd3c8005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.864+0000 7fd3f480db80 1 Processor -- start 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3f480db80 1 -- start start 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e57753a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e57753a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577534810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3f480db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e5774570a0 con 0x55e577534400 2024-05-09T01:30:26.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3f480db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e5773fc9a0 con 0x55e5773b74a0 2024-05-09T01:30:26.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3f480db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e577453060 con 0x55e577431d00 2024-05-09T01:30:26.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577534810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577534810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40150/0 (socket says 172.21.15.19:40150) 2024-05-09T01:30:26.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3dffff700 1 -- 172.21.15.19:0/171498820 learned_addr learned my addr 172.21.15.19:0/171498820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:26.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/171498820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e57753a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3deffd700 1 --2- 172.21.15.19:0/171498820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e57753a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3dffff700 1 -- 172.21.15.19:0/171498820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 msgr2=0x55e57753a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3dffff700 1 --2- 172.21.15.19:0/171498820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e57753a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3dffff700 1 -- 172.21.15.19:0/171498820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 msgr2=0x55e57753a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3dffff700 1 --2- 172.21.15.19:0/171498820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e57753a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3dffff700 1 -- 172.21.15.19:0/171498820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5773ed500 con 0x55e577534400 2024-05-09T01:30:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.865+0000 7fd3dffff700 1 --2- 172.21.15.19:0/171498820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577534810 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009fb0 tx=0x7fd3d8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ec08b15cb0b665 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.866+0000 7fd3d5ffb700 1 -- 172.21.15.19:0/171498820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8003860 con 0x55e577534400 2024-05-09T01:30:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.866+0000 7fd3d5ffb700 1 -- 172.21.15.19:0/171498820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d80050d0 con 0x55e577534400 2024-05-09T01:30:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.866+0000 7fd3d5ffb700 1 -- 172.21.15.19:0/171498820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800bb20 con 0x55e577534400 2024-05-09T01:30:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 -- 172.21.15.19:0/171498820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 msgr2=0x55e577534810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 --2- 172.21.15.19:0/171498820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577534810 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009fb0 tx=0x7fd3d8005e80 comp rx=0 tx=0).stop 2024-05-09T01:30:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 -- 172.21.15.19:0/171498820 shutdown_connections 2024-05-09T01:30:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 --2- 172.21.15.19:0/171498820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577534810 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 --2- 172.21.15.19:0/171498820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e57753a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 --2- 172.21.15.19:0/171498820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e57753a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 -- 172.21.15.19:0/171498820 >> 172.21.15.19:0/171498820 conn(0x55e5773baf70 msgr2=0x55e57752f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 -- 172.21.15.19:0/171498820 shutdown_connections 2024-05-09T01:30:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 -- 172.21.15.19:0/171498820 wait complete. 2024-05-09T01:30:26.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 -- start start 2024-05-09T01:30:26.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.867+0000 7fd3f480db80 1 Processor -- start 2024-05-09T01:30:26.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5773b74a0 0x55e57753e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577431d00 0x55e57753e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e577534400 0x55e57753eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3f480db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e577535100 con 0x55e577431d00 2024-05-09T01:30:26.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3f480db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e5773f1b20 con 0x55e577534400 2024-05-09T01:30:26.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3f480db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e57754a200 con 0x55e5773b74a0 2024-05-09T01:30:26.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3f480db80 10 client.0 _ll_register_callbacks cb 0x55e5772c86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:26.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577431d00 0x55e57753e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577431d00 0x55e57753e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40162/0 (socket says 172.21.15.19:40162) 2024-05-09T01:30:26.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3deffd700 1 -- 172.21.15.19:0/2929471355 learned_addr learned my addr 172.21.15.19:0/2929471355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5773b74a0 0x55e57753e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3dffff700 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e577534400 0x55e57753eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3deffd700 1 -- 172.21.15.19:0/2929471355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5773b74a0 msgr2=0x55e57753e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3deffd700 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5773b74a0 0x55e57753e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3deffd700 1 -- 172.21.15.19:0/2929471355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e577534400 msgr2=0x55e57753eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3deffd700 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e577534400 0x55e57753eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3deffd700 1 -- 172.21.15.19:0/2929471355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5772d2490 con 0x55e577431d00 2024-05-09T01:30:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3deffd700 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577431d00 0x55e57753e650 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0076ab0 tx=0x7fd3c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc8f820b31472ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.868+0000 7fd3beffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.869+0000 7fd3beffd700 1 -- 172.21.15.19:0/2929471355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8024020 con 0x55e577431d00 2024-05-09T01:30:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.869+0000 7fd3beffd700 1 -- 172.21.15.19:0/2929471355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd3c8003b90 con 0x55e577431d00 2024-05-09T01:30:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.869+0000 7fd3f480db80 1 -- 172.21.15.19:0/2929471355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d80050d0 con 0x55e577431d00 2024-05-09T01:30:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.869+0000 7fd3beffd700 1 -- 172.21.15.19:0/2929471355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd3c802f050 con 0x55e577431d00 2024-05-09T01:30:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.869+0000 7fd3f480db80 10 client.16560 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.869+0000 7fd3f480db80 1 -- 172.21.15.19:0/2929471355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e57754bc00 con 0x55e577431d00 2024-05-09T01:30:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.869+0000 7fd3beffd700 1 -- 172.21.15.19:0/2929471355 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d80050d0 con 0x55e577431d00 2024-05-09T01:30:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.869+0000 7fd3beffd700 1 -- 172.21.15.19:0/2929471355 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd3c8011440 con 0x55e577431d00 2024-05-09T01:30:26.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.870+0000 7fd3f480db80 10 client.16560 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:26.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.870+0000 7fd3f480db80 10 client.16560 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:30:26.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.870+0000 7fd3f480db80 20 client.16560 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:26.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.870+0000 7fd3f480db80 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e57754e2e0 0x55e577550700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.870+0000 7fd3f480db80 4 client.16560 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:26.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.870+0000 7fd3f480db80 1 -- 172.21.15.19:0/2929471355 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e577464a80 con 0x55e57754e2e0 2024-05-09T01:30:26.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.870+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e57754e2e0 0x55e577550700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.870+0000 7fd3beffd700 10 client.16560 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:26.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.870+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e57754e2e0 0x55e577550700 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=131786cb0ec72cea server_cookie=6f32d637a8520cd9 in_seq=0 out_seq=0 2024-05-09T01:30:26.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.871+0000 7fd3beffd700 1 -- 172.21.15.19:0/2929471355 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e577464a80 con 0x55e57754e2e0 2024-05-09T01:30:26.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.871+0000 7fd3beffd700 10 client.16560 handle_command_reply: tid=0 2024-05-09T01:30:26.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.871+0000 7fd3f480db80 1 client.16560 shutdown 2024-05-09T01:30:26.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.871+0000 7fd3f480db80 1 -- 172.21.15.19:0/2929471355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577431d00 msgr2=0x55e57753e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:26.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.871+0000 7fd3f480db80 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577431d00 0x55e57753e650 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0076ab0 tx=0x7fd3c8004750 comp rx=0 tx=0).stop 2024-05-09T01:30:26.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.871+0000 7fd3f480db80 1 -- 172.21.15.19:0/2929471355 shutdown_connections 2024-05-09T01:30:26.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.872+0000 7fd3f480db80 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e57754e2e0 0x55e577550700 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.872+0000 7fd3f480db80 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e577534400 0x55e57753eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.872+0000 7fd3f480db80 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5773b74a0 0x55e57753e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.872+0000 7fd3f480db80 1 --2- 172.21.15.19:0/2929471355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577431d00 0x55e57753e650 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.872+0000 7fd3f480db80 1 -- 172.21.15.19:0/2929471355 >> 172.21.15.19:0/2929471355 conn(0x55e5773baf70 msgr2=0x55e577528ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:26.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.872+0000 7fd3beffd700 0 client.16560 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:26.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.872+0000 7fd3f480db80 1 -- 172.21.15.19:0/2929471355 shutdown_connections 2024-05-09T01:30:26.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.872+0000 7fd3f480db80 1 -- 172.21.15.19:0/2929471355 wait complete. 2024-05-09T01:30:26.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.872+0000 7fd3f480db80 20 client.16560 trim_cache size 0 max 16384 2024-05-09T01:30:26.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3f480db80 1 Processor -- start 2024-05-09T01:30:26.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3f480db80 1 -- start start 2024-05-09T01:30:26.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e57753e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e5775391b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e5775396f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3f480db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01b0c90 con 0x55e577534400 2024-05-09T01:30:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3f480db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01b27d0 con 0x55e5773b74a0 2024-05-09T01:30:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3f480db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01b1360 con 0x55e577431d00 2024-05-09T01:30:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e5775396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e5775396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40168/0 (socket says 172.21.15.19:40168) 2024-05-09T01:30:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3dffff700 1 -- 172.21.15.19:0/4270554062 learned_addr learned my addr 172.21.15.19:0/4270554062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.887+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/4270554062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e57753e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3deffd700 1 --2- 172.21.15.19:0/4270554062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e5775391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3dffff700 1 -- 172.21.15.19:0/4270554062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 msgr2=0x55e5775391b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3dffff700 1 --2- 172.21.15.19:0/4270554062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e5775391b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3dffff700 1 -- 172.21.15.19:0/4270554062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 msgr2=0x55e57753e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3dffff700 1 --2- 172.21.15.19:0/4270554062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e57753e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3dffff700 1 -- 172.21.15.19:0/4270554062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e57753ef10 con 0x55e577534400 2024-05-09T01:30:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3dffff700 1 --2- 172.21.15.19:0/4270554062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e5775396f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80095c0 tx=0x7fd3d8009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e179bf627a34dcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3d57fa700 1 -- 172.21.15.19:0/4270554062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8030070 con 0x55e577534400 2024-05-09T01:30:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3d57fa700 1 -- 172.21.15.19:0/4270554062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d8003d10 con 0x55e577534400 2024-05-09T01:30:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.888+0000 7fd3d57fa700 1 -- 172.21.15.19:0/4270554062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8027040 con 0x55e577534400 2024-05-09T01:30:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 -- 172.21.15.19:0/4270554062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 msgr2=0x55e5775396f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 --2- 172.21.15.19:0/4270554062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e5775396f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80095c0 tx=0x7fd3d8009290 comp rx=0 tx=0).stop 2024-05-09T01:30:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 -- 172.21.15.19:0/4270554062 shutdown_connections 2024-05-09T01:30:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 --2- 172.21.15.19:0/4270554062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e5775396f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 --2- 172.21.15.19:0/4270554062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e57753e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 --2- 172.21.15.19:0/4270554062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e5775391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 -- 172.21.15.19:0/4270554062 >> 172.21.15.19:0/4270554062 conn(0x55e5773baf70 msgr2=0x55e57753a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 -- 172.21.15.19:0/4270554062 shutdown_connections 2024-05-09T01:30:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 -- 172.21.15.19:0/4270554062 wait complete. 2024-05-09T01:30:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.889+0000 7fd3f480db80 1 -- start start 2024-05-09T01:30:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3f480db80 1 Processor -- start 2024-05-09T01:30:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e577546130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e5775466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3f480db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577554ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3f480db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01b2100 con 0x55e577534400 2024-05-09T01:30:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3f480db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01b1a30 con 0x55e5773b74a0 2024-05-09T01:30:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3f480db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e01aec30 con 0x55e577431d00 2024-05-09T01:30:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3f480db80 10 client.0 _ll_register_callbacks cb 0x55e577534dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577554ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577554ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40174/0 (socket says 172.21.15.19:40174) 2024-05-09T01:30:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3dffff700 1 -- 172.21.15.19:0/1363089634 learned_addr learned my addr 172.21.15.19:0/1363089634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e577546130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3deffd700 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e5775466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3dffff700 1 -- 172.21.15.19:0/1363089634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 msgr2=0x55e5775466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3dffff700 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e5775466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3dffff700 1 -- 172.21.15.19:0/1363089634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 msgr2=0x55e577546130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3dffff700 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e577546130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.890+0000 7fd3dffff700 1 -- 172.21.15.19:0/1363089634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e577555690 con 0x55e577534400 2024-05-09T01:30:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3dffff700 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577554ef0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005b40 tx=0x7fd3d80054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62c626c5f35c783 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3beffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3beffd700 1 -- 172.21.15.19:0/1363089634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d803b060 con 0x55e577534400 2024-05-09T01:30:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3beffd700 1 -- 172.21.15.19:0/1363089634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd3d8003860 con 0x55e577534400 2024-05-09T01:30:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3f480db80 1 -- 172.21.15.19:0/1363089634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d8003d10 con 0x55e577534400 2024-05-09T01:30:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3beffd700 1 -- 172.21.15.19:0/1363089634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd3d8030070 con 0x55e577534400 2024-05-09T01:30:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3f480db80 10 client.16566 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3f480db80 1 -- 172.21.15.19:0/1363089634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5775b8490 con 0x55e577534400 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3beffd700 1 -- 172.21.15.19:0/1363089634 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d8003d10 con 0x55e577534400 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3beffd700 1 -- 172.21.15.19:0/1363089634 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd3d802a070 con 0x55e577534400 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3f480db80 10 client.16566 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3f480db80 10 client.16566 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3f480db80 20 client.16566 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3f480db80 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e5775b86b0 0x55e5775baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.891+0000 7fd3f480db80 4 client.16566 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.892+0000 7fd3f480db80 1 -- 172.21.15.19:0/1363089634 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e57745eb20 con 0x55e5775b86b0 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.892+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e5775b86b0 0x55e5775baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.892+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e5775b86b0 0x55e5775baad0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e503d36d28744d5 server_cookie=f34c721d7281e0ba in_seq=0 out_seq=0 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.892+0000 7fd3beffd700 10 client.16566 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.892+0000 7fd3beffd700 1 -- 172.21.15.19:0/1363089634 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e57745eb20 con 0x55e5775b86b0 2024-05-09T01:30:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.892+0000 7fd3beffd700 10 client.16566 handle_command_reply: tid=0 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.892+0000 7fd3f480db80 1 client.16566 shutdown 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 -- 172.21.15.19:0/1363089634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 msgr2=0x55e577554ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577554ef0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005b40 tx=0x7fd3d80054b0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 -- 172.21.15.19:0/1363089634 shutdown_connections 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e5775b86b0 0x55e5775baad0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e577534400 0x55e577554ef0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5773b74a0 0x55e577546130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 --2- 172.21.15.19:0/1363089634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e577431d00 0x55e5775466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3beffd700 0 client.16566 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 -- 172.21.15.19:0/1363089634 >> 172.21.15.19:0/1363089634 conn(0x55e5773baf70 msgr2=0x55e57753a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 -- 172.21.15.19:0/1363089634 shutdown_connections 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 1 -- 172.21.15.19:0/1363089634 wait complete. 2024-05-09T01:30:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3f480db80 20 client.16566 trim_cache size 0 max 16384 2024-05-09T01:30:26.904 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3e5df7700 1 -- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3cc0b2880 msgr2=0x7fd3cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3cc0b2880 0x7fd3cc0b4d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005e10 tx=0x7fd3c8005d60 comp rx=0 tx=0).stop 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3e5df7700 1 -- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e01034c0 msgr2=0x7fd3e019cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.893+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e01034c0 0x7fd3e019cf60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00060b0 tx=0x7fd3d0007f90 comp rx=0 tx=0).stop 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.894+0000 7fd3e5df7700 1 -- 172.21.15.19:0/3798385276 shutdown_connections 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.894+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e0108740 0x7fd3e01a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.894+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3cc0b2880 0x7fd3cc0b4d90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.894+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e01034c0 0x7fd3e019cf60 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.894+0000 7fd3e5df7700 1 --2- 172.21.15.19:0/3798385276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e0107df0 0x7fd3e019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.894+0000 7fd3e5df7700 1 -- 172.21.15.19:0/3798385276 >> 172.21.15.19:0/3798385276 conn(0x7fd3e00fedc0 msgr2=0x7fd3e010fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.894+0000 7fd3e5df7700 1 -- 172.21.15.19:0/3798385276 shutdown_connections 2024-05-09T01:30:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:26.894+0000 7fd3e5df7700 1 -- 172.21.15.19:0/3798385276 wait complete. 2024-05-09T01:30:27.616 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:30:27.868 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:28.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:28 smithi019 ceph-mon[30158]: pgmap v526: 97 pgs: 97 active+clean; 2.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:28.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:28 smithi196 ceph-mon[35041]: pgmap v526: 97 pgs: 97 active+clean; 2.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:28.852 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:28 smithi059 ceph-mon[34351]: pgmap v526: 97 pgs: 97 active+clean; 2.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.619+0000 7f6d4639e700 1 -- 172.21.15.19:0/3024040885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d401045d0 msgr2=0x7f6d401049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.619+0000 7f6d4639e700 1 --2- 172.21.15.19:0/3024040885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d401045d0 0x7f6d401049e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6d300099c0 tx=0x7f6d30009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.619+0000 7f6d4639e700 1 -- 172.21.15.19:0/3024040885 shutdown_connections 2024-05-09T01:30:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.619+0000 7f6d4639e700 1 --2- 172.21.15.19:0/3024040885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d40109850 0x7f6d401111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.619+0000 7f6d4639e700 1 --2- 172.21.15.19:0/3024040885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d401045d0 0x7f6d401049e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.619+0000 7f6d4639e700 1 --2- 172.21.15.19:0/3024040885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d40108f00 0x7f6d40109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.619+0000 7f6d4639e700 1 -- 172.21.15.19:0/3024040885 >> 172.21.15.19:0/3024040885 conn(0x7f6d400ffea0 msgr2=0x7f6d401022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.619+0000 7f6d4639e700 1 -- 172.21.15.19:0/3024040885 shutdown_connections 2024-05-09T01:30:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.619+0000 7f6d4639e700 1 -- 172.21.15.19:0/3024040885 wait complete. 2024-05-09T01:30:29.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.620+0000 7f6d4639e700 1 Processor -- start 2024-05-09T01:30:29.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.620+0000 7f6d4639e700 1 -- start start 2024-05-09T01:30:29.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.620+0000 7f6d4639e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d401045d0 0x7f6d4019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d4639e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d40108f00 0x7f6d4019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d4639e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d40109850 0x7f6d4019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d4639e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40114480 con 0x7f6d40108f00 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d4639e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40114300 con 0x7f6d40109850 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d4639e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40114600 con 0x7f6d401045d0 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d40108f00 0x7f6d4019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d40108f00 0x7f6d4019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40202/0 (socket says 172.21.15.19:40202) 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d37fff700 1 -- 172.21.15.19:0/2044707589 learned_addr learned my addr 172.21.15.19:0/2044707589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d4493b700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d40109850 0x7f6d4019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d401045d0 0x7f6d4019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d37fff700 1 -- 172.21.15.19:0/2044707589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d401045d0 msgr2=0x7f6d4019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d37fff700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d401045d0 0x7f6d4019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d37fff700 1 -- 172.21.15.19:0/2044707589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d40109850 msgr2=0x7f6d4019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d37fff700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d40109850 0x7f6d4019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d37fff700 1 -- 172.21.15.19:0/2044707589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4019fc50 con 0x7f6d40108f00 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d401045d0 0x7f6d4019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d37fff700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d40108f00 0x7f6d4019b3d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800c9e0 tx=0x7f6d2800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.621+0000 7f6d3dffb700 1 -- 172.21.15.19:0/2044707589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28007aa0 con 0x7f6d40108f00 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.622+0000 7f6d3dffb700 1 -- 172.21.15.19:0/2044707589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d28007c40 con 0x7f6d40108f00 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.622+0000 7f6d4639e700 1 -- 172.21.15.19:0/2044707589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4019ff40 con 0x7f6d40108f00 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.622+0000 7f6d3dffb700 1 -- 172.21.15.19:0/2044707589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28018660 con 0x7f6d40108f00 2024-05-09T01:30:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.622+0000 7f6d4639e700 1 -- 172.21.15.19:0/2044707589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d401a7910 con 0x7f6d40108f00 2024-05-09T01:30:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.625+0000 7f6d3dffb700 1 -- 172.21.15.19:0/2044707589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6d28018800 con 0x7f6d40108f00 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.625+0000 7f6d3dffb700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6d2c0b2900 0x7f6d2c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d54db4b80 1 Processor -- start 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6d2c0b2900 0x7f6d2c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3dffb700 1 -- 172.21.15.19:0/2044707589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6d280e2170 con 0x7f6d40108f00 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d54db4b80 1 -- start start 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863935d00 0x55b863a3cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 0x55b863a3d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863a411e0 0x55b863a415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b86395b0a0 con 0x55b863935d00 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b8639009a0 con 0x55b8638bb4a0 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b863957060 con 0x55b863a411e0 2024-05-09T01:30:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863935d00 0x55b863a3cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863935d00 0x55b863a3cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40216/0 (socket says 172.21.15.19:40216) 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 -- 172.21.15.19:0/2224670172 learned_addr learned my addr 172.21.15.19:0/2224670172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6d2c0b2900 0x7f6d2c0b4e10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6d30003800 tx=0x7f6d3001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d4493b700 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863a411e0 0x55b863a415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d37fff700 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 0x55b863a3d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 -- 172.21.15.19:0/2224670172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863a411e0 msgr2=0x55b863a415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863a411e0 0x55b863a415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 -- 172.21.15.19:0/2224670172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 msgr2=0x55b863a3d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 0x55b863a3d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d3ffff700 1 -- 172.21.15.19:0/2224670172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8638f1500 con 0x55b863935d00 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.626+0000 7f6d4493b700 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863a411e0 0x55b863a415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.627+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863935d00 0x55b863a3cad0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6d300099c0 tx=0x7f6d3001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e05704ad62d4f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.627+0000 7f6d367fc700 1 -- 172.21.15.19:0/2224670172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30031070 con 0x55b863935d00 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.627+0000 7f6d367fc700 1 -- 172.21.15.19:0/2224670172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d30022400 con 0x55b863935d00 2024-05-09T01:30:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.627+0000 7f6d367fc700 1 -- 172.21.15.19:0/2224670172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30018600 con 0x55b863935d00 2024-05-09T01:30:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2224670172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863935d00 msgr2=0x55b863a3cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863935d00 0x55b863a3cad0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6d300099c0 tx=0x7f6d3001d000 comp rx=0 tx=0).stop 2024-05-09T01:30:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2224670172 shutdown_connections 2024-05-09T01:30:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863a411e0 0x55b863a415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863935d00 0x55b863a3cad0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2224670172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 0x55b863a3d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2224670172 >> 172.21.15.19:0/2224670172 conn(0x55b8638bef70 msgr2=0x55b863a33220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2224670172 shutdown_connections 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2224670172 wait complete. 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.628+0000 7f6d54db4b80 1 -- start start 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d54db4b80 1 Processor -- start 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 0x55b863a46490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863935d00 0x55b863a469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b863a41ea0 con 0x55b863a411e0 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b8638f5b20 con 0x55b8638bb4a0 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b863a42020 con 0x55b863935d00 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d54db4b80 10 client.0 _ll_register_callbacks cb 0x55b8637cc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d4493b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d4493b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40230/0 (socket says 172.21.15.19:40230) 2024-05-09T01:30:29.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d4493b700 1 -- 172.21.15.19:0/2200045718 learned_addr learned my addr 172.21.15.19:0/2200045718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d37fff700 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863935d00 0x55b863a469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 0x55b863a46490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d4493b700 1 -- 172.21.15.19:0/2200045718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863935d00 msgr2=0x55b863a469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d4493b700 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863935d00 0x55b863a469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d4493b700 1 -- 172.21.15.19:0/2200045718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 msgr2=0x55b863a46490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.629+0000 7f6d4493b700 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 0x55b863a46490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d4493b700 1 -- 172.21.15.19:0/2200045718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8637d6490 con 0x55b863a411e0 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d4493b700 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46f10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6d38008e60 tx=0x7f6d380114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3626a613ee28f8a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/2200045718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38011e10 con 0x55b863a411e0 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/2200045718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6d38016480 con 0x55b863a411e0 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2200045718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d30022400 con 0x55b863a411e0 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/2200045718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d38016680 con 0x55b863a411e0 2024-05-09T01:30:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d54db4b80 10 client.16578 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2200045718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b863a53d60 con 0x55b863a411e0 2024-05-09T01:30:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.630+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/2200045718 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d30022400 con 0x55b863a411e0 2024-05-09T01:30:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.631+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/2200045718 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d38016820 con 0x55b863a411e0 2024-05-09T01:30:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.631+0000 7f6d54db4b80 10 client.16578 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.631+0000 7f6d54db4b80 10 client.16578 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.631+0000 7f6d54db4b80 20 client.16578 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.631+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b863a563a0 0x55b863a587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.631+0000 7f6d54db4b80 4 client.16578 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.632+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2200045718 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b863968a80 con 0x55b863a563a0 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.632+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b863a563a0 0x55b863a587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.632+0000 7f6d1f7fe700 10 client.16578 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.632+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b863a563a0 0x55b863a587c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=349a36e21eaccd2 server_cookie=47b8f1efdf44cde5 in_seq=0 out_seq=0 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/2200045718 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b863968a80 con 0x55b863a563a0 2024-05-09T01:30:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d1f7fe700 10 client.16578 handle_command_reply: tid=0 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 client.16578 shutdown 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2200045718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 msgr2=0x55b863a46f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46f10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6d38008e60 tx=0x7f6d380114a0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2200045718 shutdown_connections 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46f10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b863a563a0 0x55b863a587c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d1f7fe700 0 client.16578 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8638bb4a0 0x55b863a46490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/2200045718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b863935d00 0x55b863a469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2200045718 >> 172.21.15.19:0/2200045718 conn(0x55b8638bef70 msgr2=0x55b863a2cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.633+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2200045718 shutdown_connections 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.634+0000 7f6d54db4b80 1 -- 172.21.15.19:0/2200045718 wait complete. 2024-05-09T01:30:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.634+0000 7f6d54db4b80 20 client.16578 trim_cache size 0 max 16384 2024-05-09T01:30:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d54db4b80 1 Processor -- start 2024-05-09T01:30:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d54db4b80 1 -- start start 2024-05-09T01:30:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 0x55b863a457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 0x55b863a33290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401acad0 con 0x55b863a411e0 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ae610 con 0x55b863935d00 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ad1a0 con 0x55b8638bb4a0 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d4493b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d4493b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41618/0 (socket says 172.21.15.19:41618) 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.649+0000 7f6d4493b700 1 -- 172.21.15.19:0/176056225 learned_addr learned my addr 172.21.15.19:0/176056225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 0x55b863a457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d37fff700 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 0x55b863a33290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d4493b700 1 -- 172.21.15.19:0/176056225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 msgr2=0x55b863a457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d4493b700 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 0x55b863a457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d4493b700 1 -- 172.21.15.19:0/176056225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 msgr2=0x55b863a33290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d4493b700 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 0x55b863a33290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d4493b700 1 -- 172.21.15.19:0/176056225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b863a3d8d0 con 0x55b863a411e0 2024-05-09T01:30:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 0x55b863a457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d37fff700 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 0x55b863a33290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d4493b700 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46d50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6d38007440 tx=0x7f6d38007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f8fd156f7dee83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d35ffb700 1 -- 172.21.15.19:0/176056225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3802f020 con 0x55b863a411e0 2024-05-09T01:30:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d35ffb700 1 -- 172.21.15.19:0/176056225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d3801b070 con 0x55b863a411e0 2024-05-09T01:30:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.650+0000 7f6d35ffb700 1 -- 172.21.15.19:0/176056225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3802b920 con 0x55b863a411e0 2024-05-09T01:30:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 -- 172.21.15.19:0/176056225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 msgr2=0x55b863a46d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46d50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6d38007440 tx=0x7f6d38007d60 comp rx=0 tx=0).stop 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 -- 172.21.15.19:0/176056225 shutdown_connections 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a46d50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 0x55b863a457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/176056225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 0x55b863a33290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 -- 172.21.15.19:0/176056225 >> 172.21.15.19:0/176056225 conn(0x55b8638bef70 msgr2=0x55b863a2cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 -- 172.21.15.19:0/176056225 shutdown_connections 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 -- 172.21.15.19:0/176056225 wait complete. 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.651+0000 7f6d54db4b80 1 -- start start 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d54db4b80 1 Processor -- start 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 0x55b863a4e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 0x55b863a49c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d54db4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a4a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401adf40 con 0x55b863a411e0 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ad870 con 0x55b863935d00 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d54db4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401aaa70 con 0x55b8638bb4a0 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d54db4b80 10 client.0 _ll_register_callbacks cb 0x55b8637cc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d4493b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a4a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d4493b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a4a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41626/0 (socket says 172.21.15.19:41626) 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d4493b700 1 -- 172.21.15.19:0/1846752074 learned_addr learned my addr 172.21.15.19:0/1846752074 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 0x55b863a4e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.652+0000 7f6d37fff700 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 0x55b863a49c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d4493b700 1 -- 172.21.15.19:0/1846752074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 msgr2=0x55b863a4e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d4493b700 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 0x55b863a4e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d4493b700 1 -- 172.21.15.19:0/1846752074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 msgr2=0x55b863a49c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d4493b700 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 0x55b863a49c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d4493b700 1 -- 172.21.15.19:0/1846752074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b863a4a760 con 0x55b863a411e0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d4493b700 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a4a1a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6d38002230 tx=0x7f6d38005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edd85c2277e54d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/1846752074 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3802f030 con 0x55b863a411e0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/1846752074 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6d38011e20 con 0x55b863a411e0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d54db4b80 1 -- 172.21.15.19:0/1846752074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3801b070 con 0x55b863a411e0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/1846752074 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d38043050 con 0x55b863a411e0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d54db4b80 10 client.16584 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d54db4b80 1 -- 172.21.15.19:0/1846752074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b863ac0740 con 0x55b863a411e0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.653+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/1846752074 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d3801b070 con 0x55b863a411e0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/1846752074 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d380056f0 con 0x55b863a411e0 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d54db4b80 10 client.16584 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d54db4b80 10 client.16584 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:30:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d54db4b80 20 client.16584 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b863a563a0 0x55b863ac2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d54db4b80 4 client.16584 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d54db4b80 1 -- 172.21.15.19:0/1846752074 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b863962b20 con 0x55b863a563a0 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b863a563a0 0x55b863ac2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d1f7fe700 10 client.16584 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.654+0000 7f6d3ffff700 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b863a563a0 0x55b863ac2d30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f203101f06265057 server_cookie=4827786c8e09a579 in_seq=0 out_seq=0 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d1f7fe700 1 -- 172.21.15.19:0/1846752074 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b863962b20 con 0x55b863a563a0 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d1f7fe700 10 client.16584 handle_command_reply: tid=0 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 client.16584 shutdown 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 -- 172.21.15.19:0/1846752074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 msgr2=0x55b863a4a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a4a1a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6d38002230 tx=0x7f6d38005e90 comp rx=0 tx=0).stop 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 -- 172.21.15.19:0/1846752074 shutdown_connections 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b863a411e0 0x55b863a4a1a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8638bb4a0 0x55b863a4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b863a563a0 0x55b863ac2d30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 --2- 172.21.15.19:0/1846752074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b863935d00 0x55b863a49c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 -- 172.21.15.19:0/1846752074 >> 172.21.15.19:0/1846752074 conn(0x55b8638bef70 msgr2=0x55b863a45510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d1f7fe700 0 client.16584 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 -- 172.21.15.19:0/1846752074 shutdown_connections 2024-05-09T01:30:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 1 -- 172.21.15.19:0/1846752074 wait complete. 2024-05-09T01:30:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.655+0000 7f6d54db4b80 20 client.16584 trim_cache size 0 max 16384 2024-05-09T01:30:29.657 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 -- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6d2c0b2900 msgr2=0x7f6d2c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6d2c0b2900 0x7f6d2c0b4e10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6d30003800 tx=0x7f6d3001a040 comp rx=0 tx=0).stop 2024-05-09T01:30:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 -- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d40108f00 msgr2=0x7f6d4019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d40108f00 0x7f6d4019b3d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800c9e0 tx=0x7f6d2800ccf0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 -- 172.21.15.19:0/2044707589 shutdown_connections 2024-05-09T01:30:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d40109850 0x7f6d4019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d401045d0 0x7f6d4019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6d2c0b2900 0x7f6d2c0b4e10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 --2- 172.21.15.19:0/2044707589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d40108f00 0x7f6d4019b3d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:29.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 -- 172.21.15.19:0/2044707589 >> 172.21.15.19:0/2044707589 conn(0x7f6d400ffea0 msgr2=0x7f6d40102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:29.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 -- 172.21.15.19:0/2044707589 shutdown_connections 2024-05-09T01:30:29.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:29.656+0000 7f6d4639e700 1 -- 172.21.15.19:0/2044707589 wait complete. 2024-05-09T01:30:30.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:30:30.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:30 smithi019 ceph-mon[30158]: pgmap v527: 97 pgs: 97 active+clean; 2.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:30:30.651 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:30.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:30 smithi196 ceph-mon[35041]: pgmap v527: 97 pgs: 97 active+clean; 2.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:30:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:30 smithi059 ceph-mon[34351]: pgmap v527: 97 pgs: 97 active+clean; 2.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:30:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.896+0000 7fd884fb2640 1 -- 172.21.15.19:0/2591409471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 msgr2=0x7fd880109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.896+0000 7fd884fb2640 1 --2- 172.21.15.19:0/2591409471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 0x7fd880109640 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd874009900 tx=0x7fd87402a410 comp rx=0 tx=0).stop 2024-05-09T01:30:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.896+0000 7fd884fb2640 1 -- 172.21.15.19:0/2591409471 shutdown_connections 2024-05-09T01:30:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.896+0000 7fd884fb2640 1 --2- 172.21.15.19:0/2591409471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880109b80 0x7fd88010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.896+0000 7fd884fb2640 1 --2- 172.21.15.19:0/2591409471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 0x7fd880109640 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.896+0000 7fd884fb2640 1 --2- 172.21.15.19:0/2591409471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd880102070 0x7fd880106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.896+0000 7fd884fb2640 1 -- 172.21.15.19:0/2591409471 >> 172.21.15.19:0/2591409471 conn(0x7fd8800fd4d0 msgr2=0x7fd8800ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.896+0000 7fd884fb2640 1 -- 172.21.15.19:0/2591409471 shutdown_connections 2024-05-09T01:30:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.896+0000 7fd884fb2640 1 -- 172.21.15.19:0/2591409471 wait complete. 2024-05-09T01:30:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.897+0000 7fd884fb2640 1 Processor -- start 2024-05-09T01:30:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.897+0000 7fd884fb2640 1 -- start start 2024-05-09T01:30:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.897+0000 7fd884fb2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880102070 0x7fd88019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.897+0000 7fd884fb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 0x7fd88019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.897+0000 7fd884fb2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd880109b80 0x7fd8801a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.897+0000 7fd884fb2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd88010f820 con 0x7fd880107220 2024-05-09T01:30:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.897+0000 7fd884fb2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd88010f6a0 con 0x7fd880109b80 2024-05-09T01:30:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.897+0000 7fd884fb2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd88010f9a0 con 0x7fd880102070 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 0x7fd88019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 0x7fd88019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41648/0 (socket says 172.21.15.19:41648) 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87dd9b640 1 -- 172.21.15.19:0/1442673997 learned_addr learned my addr 172.21.15.19:0/1442673997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87ed9d640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd880109b80 0x7fd8801a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87e59c640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880102070 0x7fd88019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87dd9b640 1 -- 172.21.15.19:0/1442673997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880102070 msgr2=0x7fd88019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880102070 0x7fd88019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87dd9b640 1 -- 172.21.15.19:0/1442673997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd880109b80 msgr2=0x7fd8801a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd880109b80 0x7fd8801a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87dd9b640 1 -- 172.21.15.19:0/1442673997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8801a1f30 con 0x7fd880107220 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 0x7fd88019a930 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd874006010 tx=0x7fd874004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd86f7fe640 1 -- 172.21.15.19:0/1442673997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87403f030 con 0x7fd880107220 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd86f7fe640 1 -- 172.21.15.19:0/1442673997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd874003cb0 con 0x7fd880107220 2024-05-09T01:30:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd884fb2640 1 -- 172.21.15.19:0/1442673997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8801a21c0 con 0x7fd880107220 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd86f7fe640 1 -- 172.21.15.19:0/1442673997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874035070 con 0x7fd880107220 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.898+0000 7fd884fb2640 1 -- 172.21.15.19:0/1442673997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8801a26b0 con 0x7fd880107220 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.901+0000 7fd88be65740 1 Processor -- start 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd88be65740 1 -- start start 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd86f7fe640 1 -- 172.21.15.19:0/1442673997 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd874042030 con 0x7fd880107220 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd88be65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 0x55f6a8b65500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd88be65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b65a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd88be65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b6c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd88be65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8ac97c0 con 0x55f6a87982f0 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd88be65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8a650e0 con 0x55f6a87a04d0 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd88be65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8ac5780 con 0x55f6a87c11b0 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b65a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b65a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41664/0 (socket says 172.21.15.19:41664) 2024-05-09T01:30:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87e59c640 1 -- 172.21.15.19:0/3508984691 learned_addr learned my addr 172.21.15.19:0/3508984691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/3508984691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b6c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87ed9d640 1 --2- 172.21.15.19:0/3508984691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 0x55f6a8b65500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87e59c640 1 -- 172.21.15.19:0/3508984691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 msgr2=0x55f6a8b65500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87e59c640 1 --2- 172.21.15.19:0/3508984691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 0x55f6a8b65500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87e59c640 1 -- 172.21.15.19:0/3508984691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 msgr2=0x55f6a8b6c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87e59c640 1 --2- 172.21.15.19:0/3508984691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b6c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.902+0000 7fd87e59c640 1 -- 172.21.15.19:0/3508984691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a8a8e7c0 con 0x55f6a87982f0 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.903+0000 7fd87e59c640 1 --2- 172.21.15.19:0/3508984691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b65a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd870003af0 tx=0x7fd870009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8417a9604a8568c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.903+0000 7fd86cff9640 1 -- 172.21.15.19:0/3508984691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870011440 con 0x55f6a87982f0 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.903+0000 7fd86cff9640 1 -- 172.21.15.19:0/3508984691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd870015030 con 0x55f6a87982f0 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.903+0000 7fd86cff9640 1 -- 172.21.15.19:0/3508984691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870010aa0 con 0x55f6a87982f0 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.903+0000 7fd86f7fe640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8680b27e0 0x7fd8680b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.903+0000 7fd87ed9d640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8680b27e0 0x7fd8680b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.903+0000 7fd86f7fe640 1 -- 172.21.15.19:0/1442673997 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd874102830 con 0x7fd880107220 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd87ed9d640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8680b27e0 0x7fd8680b4cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55f6a8b666d0 tx=0x7fd87800b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 -- 172.21.15.19:0/3508984691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 msgr2=0x55f6a8b65a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 --2- 172.21.15.19:0/3508984691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b65a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd870003af0 tx=0x7fd870009f70 comp rx=0 tx=0).stop 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 -- 172.21.15.19:0/3508984691 shutdown_connections 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 --2- 172.21.15.19:0/3508984691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b6c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 --2- 172.21.15.19:0/3508984691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b65a80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 --2- 172.21.15.19:0/3508984691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 0x55f6a8b65500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 -- 172.21.15.19:0/3508984691 >> 172.21.15.19:0/3508984691 conn(0x55f6a87b5300 msgr2=0x55f6a8b60e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 -- 172.21.15.19:0/3508984691 shutdown_connections 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 -- 172.21.15.19:0/3508984691 wait complete. 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.904+0000 7fd88be65740 1 -- start start 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd88be65740 1 Processor -- start 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd88be65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b6f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd88be65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b6f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd88be65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 0x55f6a8b766b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd88be65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8b6d3d0 con 0x55f6a87982f0 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd88be65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8a509c0 con 0x55f6a87a04d0 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd88be65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8b6d550 con 0x55f6a87c11b0 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd88be65740 10 client.0 _ll_register_callbacks cb 0x55f6a8a7d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b6f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b6f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41668/0 (socket says 172.21.15.19:41668) 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87e59c640 1 -- 172.21.15.19:0/949229871 learned_addr learned my addr 172.21.15.19:0/949229871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87ed9d640 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 0x55f6a8b766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b6f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87e59c640 1 -- 172.21.15.19:0/949229871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 msgr2=0x55f6a8b766b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87e59c640 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 0x55f6a8b766b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87e59c640 1 -- 172.21.15.19:0/949229871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 msgr2=0x55f6a8b6f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87e59c640 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b6f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.905+0000 7fd87e59c640 1 -- 172.21.15.19:0/949229871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a8a771f0 con 0x55f6a87982f0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd87e59c640 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b6f210 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd870006010 tx=0x7fd870010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffc50a83c37a7cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd865ffb640 1 -- 172.21.15.19:0/949229871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870029060 con 0x55f6a87982f0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd865ffb640 1 -- 172.21.15.19:0/949229871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd8700109f0 con 0x55f6a87982f0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd88be65740 1 -- 172.21.15.19:0/949229871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd870015030 con 0x55f6a87982f0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd865ffb640 1 -- 172.21.15.19:0/949229871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd870010bf0 con 0x55f6a87982f0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd88be65740 10 client.16596 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd88be65740 1 -- 172.21.15.19:0/949229871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6a882c790 con 0x55f6a87982f0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd865ffb640 1 -- 172.21.15.19:0/949229871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd870015030 con 0x55f6a87982f0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd865ffb640 1 -- 172.21.15.19:0/949229871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd870023070 con 0x55f6a87982f0 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd88be65740 10 client.16596 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd88be65740 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd88be65740 20 client.16596 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd88be65740 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f6a877fd80 0x55f6a8b7c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd88be65740 4 client.16596 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.906+0000 7fd88be65740 1 -- 172.21.15.19:0/949229871 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6a8ad73d0 con 0x55f6a877fd80 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.907+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f6a877fd80 0x55f6a8b7c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.907+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f6a877fd80 0x55f6a8b7c880 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41cfd673e0d261fd server_cookie=9c27f964182ef046 in_seq=0 out_seq=0 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.907+0000 7fd865ffb640 10 client.16596 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd865ffb640 1 -- 172.21.15.19:0/949229871 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f6a8ad73d0 con 0x55f6a877fd80 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd865ffb640 10 client.16596 handle_command_reply: tid=0 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd88be65740 1 client.16596 shutdown 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd88be65740 1 -- 172.21.15.19:0/949229871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 msgr2=0x55f6a8b6f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd88be65740 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b6f210 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd870006010 tx=0x7fd870010440 comp rx=0 tx=0).stop 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd88be65740 1 -- 172.21.15.19:0/949229871 shutdown_connections 2024-05-09T01:30:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd88be65740 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f6a877fd80 0x55f6a8b7c880 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd88be65740 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87c11b0 0x55f6a8b766b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd88be65740 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b6f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd865ffb640 0 client.16596 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd88be65740 1 --2- 172.21.15.19:0/949229871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b6f210 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.908+0000 7fd88be65740 1 -- 172.21.15.19:0/949229871 >> 172.21.15.19:0/949229871 conn(0x55f6a87b5300 msgr2=0x55f6a8b5f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.909+0000 7fd88be65740 1 -- 172.21.15.19:0/949229871 shutdown_connections 2024-05-09T01:30:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.909+0000 7fd88be65740 1 -- 172.21.15.19:0/949229871 wait complete. 2024-05-09T01:30:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.909+0000 7fd88be65740 20 client.16596 trim_cache size 0 max 16384 2024-05-09T01:30:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd88be65740 1 Processor -- start 2024-05-09T01:30:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd88be65740 1 -- start start 2024-05-09T01:30:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd88be65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a877fd80 0x55f6a8b643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd88be65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd88be65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87a04d0 0x55f6a8b81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd88be65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801ab410 con 0x55f6a87982f0 2024-05-09T01:30:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd88be65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801acf50 con 0x55f6a877fd80 2024-05-09T01:30:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd88be65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801abae0 con 0x55f6a87a04d0 2024-05-09T01:30:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd87dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd87dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41672/0 (socket says 172.21.15.19:41672) 2024-05-09T01:30:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd87dd9b640 1 -- 172.21.15.19:0/1532566596 learned_addr learned my addr 172.21.15.19:0/1532566596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.924+0000 7fd87e59c640 1 --2- 172.21.15.19:0/1532566596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a877fd80 0x55f6a8b643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.925+0000 7fd87dd9b640 1 -- 172.21.15.19:0/1532566596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87a04d0 msgr2=0x55f6a8b81f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.925+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/1532566596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87a04d0 0x55f6a8b81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.925+0000 7fd87dd9b640 1 -- 172.21.15.19:0/1532566596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a877fd80 msgr2=0x55f6a8b643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.925+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/1532566596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a877fd80 0x55f6a8b643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.925+0000 7fd87dd9b640 1 -- 172.21.15.19:0/1532566596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a88a35f0 con 0x55f6a87982f0 2024-05-09T01:30:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.925+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/1532566596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b648e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd874004650 tx=0x7fd874005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962997ebb0c613cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.925+0000 7fd867fff640 1 -- 172.21.15.19:0/1532566596 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874049040 con 0x55f6a87982f0 2024-05-09T01:30:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.925+0000 7fd867fff640 1 -- 172.21.15.19:0/1532566596 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8740c99f0 con 0x55f6a87982f0 2024-05-09T01:30:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.925+0000 7fd867fff640 1 -- 172.21.15.19:0/1532566596 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87403b460 con 0x55f6a87982f0 2024-05-09T01:30:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 -- 172.21.15.19:0/1532566596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 msgr2=0x55f6a8b648e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 --2- 172.21.15.19:0/1532566596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b648e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd874004650 tx=0x7fd874005d50 comp rx=0 tx=0).stop 2024-05-09T01:30:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 -- 172.21.15.19:0/1532566596 shutdown_connections 2024-05-09T01:30:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 --2- 172.21.15.19:0/1532566596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87a04d0 0x55f6a8b81f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 --2- 172.21.15.19:0/1532566596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a87982f0 0x55f6a8b648e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 --2- 172.21.15.19:0/1532566596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a877fd80 0x55f6a8b643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 -- 172.21.15.19:0/1532566596 >> 172.21.15.19:0/1532566596 conn(0x55f6a87b5300 msgr2=0x55f6a8b63c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 -- 172.21.15.19:0/1532566596 shutdown_connections 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 -- 172.21.15.19:0/1532566596 wait complete. 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 -- start start 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 Processor -- start 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a877fd80 0x55f6a8b743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87982f0 0x55f6a8b748f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801ac880 con 0x55f6a877fd80 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801ac1b0 con 0x55f6a87a04d0 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801a9410 con 0x55f6a87982f0 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd88be65740 10 client.0 _ll_register_callbacks cb 0x7fd874108420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.926+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a877fd80 0x55f6a8b743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a877fd80 0x55f6a8b743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41688/0 (socket says 172.21.15.19:41688) 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87e59c640 1 -- 172.21.15.19:0/476409572 learned_addr learned my addr 172.21.15.19:0/476409572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87ed9d640 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87982f0 0x55f6a8b748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87e59c640 1 -- 172.21.15.19:0/476409572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87982f0 msgr2=0x55f6a8b748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87e59c640 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87982f0 0x55f6a8b748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87e59c640 1 -- 172.21.15.19:0/476409572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 msgr2=0x55f6a8b757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87e59c640 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87e59c640 1 -- 172.21.15.19:0/476409572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a8a8ded0 con 0x55f6a877fd80 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87982f0 0x55f6a8b748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd87e59c640 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a877fd80 0x55f6a8b743b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd870005b40 tx=0x7fd870011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5cf88f2411a9ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd865ffb640 1 -- 172.21.15.19:0/476409572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870029040 con 0x55f6a877fd80 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd865ffb640 1 -- 172.21.15.19:0/476409572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd870010d90 con 0x55f6a877fd80 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd88be65740 1 -- 172.21.15.19:0/476409572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8740c99f0 con 0x55f6a877fd80 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd865ffb640 1 -- 172.21.15.19:0/476409572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd870003970 con 0x55f6a877fd80 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd88be65740 10 client.16602 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.927+0000 7fd88be65740 1 -- 172.21.15.19:0/476409572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6a8b76eb0 con 0x55f6a877fd80 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd865ffb640 1 -- 172.21.15.19:0/476409572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8740c99f0 con 0x55f6a877fd80 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd865ffb640 1 -- 172.21.15.19:0/476409572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd870023070 con 0x55f6a877fd80 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd88be65740 10 client.16602 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd88be65740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd88be65740 20 client.16602 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd88be65740 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f6a87c11b0 0x55f6a8be7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd88be65740 4 client.16602 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:30:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd88be65740 1 -- 172.21.15.19:0/476409572 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6a8ad1240 con 0x55f6a87c11b0 2024-05-09T01:30:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f6a87c11b0 0x55f6a8be7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd865ffb640 10 client.16602 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.928+0000 7fd87dd9b640 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f6a87c11b0 0x55f6a8be7050 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5901e032b04649a4 server_cookie=b8ca583675a42dfe in_seq=0 out_seq=0 2024-05-09T01:30:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd865ffb640 1 -- 172.21.15.19:0/476409572 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f6a8ad1240 con 0x55f6a87c11b0 2024-05-09T01:30:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd865ffb640 10 client.16602 handle_command_reply: tid=0 2024-05-09T01:30:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 client.16602 shutdown 2024-05-09T01:30:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 -- 172.21.15.19:0/476409572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a877fd80 msgr2=0x55f6a8b743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a877fd80 0x55f6a8b743b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd870005b40 tx=0x7fd870011b70 comp rx=0 tx=0).stop 2024-05-09T01:30:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 -- 172.21.15.19:0/476409572 shutdown_connections 2024-05-09T01:30:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f6a87c11b0 0x55f6a8be7050 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f6a87a04d0 0x55f6a8b757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f6a87982f0 0x55f6a8b748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd865ffb640 0 client.16602 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 --2- 172.21.15.19:0/476409572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6a877fd80 0x55f6a8b743b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 -- 172.21.15.19:0/476409572 >> 172.21.15.19:0/476409572 conn(0x55f6a87b5300 msgr2=0x55f6a8b646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 -- 172.21.15.19:0/476409572 shutdown_connections 2024-05-09T01:30:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 1 -- 172.21.15.19:0/476409572 wait complete. 2024-05-09T01:30:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.945+0000 7fd88be65740 20 client.16602 trim_cache size 0 max 16384 2024-05-09T01:30:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:30:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:30:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:30:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 -- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8680b27e0 msgr2=0x7fd8680b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8680b27e0 0x7fd8680b4cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55f6a8b666d0 tx=0x7fd87800b2e0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 -- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 msgr2=0x7fd88019a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 0x7fd88019a930 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd874006010 tx=0x7fd874004340 comp rx=0 tx=0).stop 2024-05-09T01:30:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 -- 172.21.15.19:0/1442673997 shutdown_connections 2024-05-09T01:30:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd8680b27e0 0x7fd8680b4cf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd880109b80 0x7fd8801a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd880107220 0x7fd88019a930 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 --2- 172.21.15.19:0/1442673997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd880102070 0x7fd88019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 -- 172.21.15.19:0/1442673997 >> 172.21.15.19:0/1442673997 conn(0x7fd8800fd4d0 msgr2=0x7fd880105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 -- 172.21.15.19:0/1442673997 shutdown_connections 2024-05-09T01:30:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:31.946+0000 7fd884fb2640 1 -- 172.21.15.19:0/1442673997 wait complete. 2024-05-09T01:30:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.487+0000 7f6a5552f700 1 -- 172.21.15.19:0/1613065715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50108740 msgr2=0x7f6a501100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.487+0000 7f6a5552f700 1 --2- 172.21.15.19:0/1613065715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50108740 0x7f6a501100a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b0d0 tx=0x7f6a4800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.487+0000 7f6a5552f700 1 -- 172.21.15.19:0/1613065715 shutdown_connections 2024-05-09T01:30:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.487+0000 7f6a5552f700 1 --2- 172.21.15.19:0/1613065715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50108740 0x7f6a501100a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.487+0000 7f6a5552f700 1 --2- 172.21.15.19:0/1613065715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a501034c0 0x7f6a501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.487+0000 7f6a5552f700 1 --2- 172.21.15.19:0/1613065715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a50107df0 0x7f6a50108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.487+0000 7f6a5552f700 1 -- 172.21.15.19:0/1613065715 >> 172.21.15.19:0/1613065715 conn(0x7f6a500fedc0 msgr2=0x7f6a501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.487+0000 7f6a5552f700 1 -- 172.21.15.19:0/1613065715 shutdown_connections 2024-05-09T01:30:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.487+0000 7f6a5552f700 1 -- 172.21.15.19:0/1613065715 wait complete. 2024-05-09T01:30:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.488+0000 7f6a5552f700 1 Processor -- start 2024-05-09T01:30:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.488+0000 7f6a5552f700 1 -- start start 2024-05-09T01:30:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a5552f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a501034c0 0x7f6a5019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a5552f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107df0 0x7f6a5019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a5552f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50108740 0x7f6a5019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a5552f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a50113370 con 0x7f6a50107df0 2024-05-09T01:30:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a5552f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501131f0 con 0x7f6a501034c0 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a5552f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501134f0 con 0x7f6a50108740 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107df0 0x7f6a5019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107df0 0x7f6a5019c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41700/0 (socket says 172.21.15.19:41700) 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/43968647 learned_addr learned my addr 172.21.15.19:0/43968647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4effd700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a501034c0 0x7f6a5019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50108740 0x7f6a5019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/43968647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50108740 msgr2=0x7f6a5019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50108740 0x7f6a5019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/43968647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a501034c0 msgr2=0x7f6a5019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a501034c0 0x7f6a5019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/43968647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a501a7000 con 0x7f6a50107df0 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.489+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107df0 0x7f6a5019c1f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400d980 tx=0x7f6a4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.490+0000 7f6a37fff700 1 -- 172.21.15.19:0/43968647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44017070 con 0x7f6a50107df0 2024-05-09T01:30:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.490+0000 7f6a5552f700 1 -- 172.21.15.19:0/43968647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a501a72f0 con 0x7f6a50107df0 2024-05-09T01:30:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.490+0000 7f6a37fff700 1 -- 172.21.15.19:0/43968647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a44009de0 con 0x7f6a50107df0 2024-05-09T01:30:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.490+0000 7f6a37fff700 1 -- 172.21.15.19:0/43968647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4400f3c0 con 0x7f6a50107df0 2024-05-09T01:30:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.490+0000 7f6a5552f700 1 -- 172.21.15.19:0/43968647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a501a7880 con 0x7f6a50107df0 2024-05-09T01:30:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.493+0000 7f6a37fff700 1 -- 172.21.15.19:0/43968647 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6a4400f560 con 0x7f6a50107df0 2024-05-09T01:30:32.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:32 smithi019 ceph-mon[30158]: pgmap v528: 97 pgs: 97 active+clean; 2.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:30:32.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.494+0000 7f6a37fff700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a380b2900 0x7f6a380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.494+0000 7f6a63f45b80 1 Processor -- start 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.494+0000 7f6a63f45b80 1 -- start start 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.494+0000 7f6a4effd700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a380b2900 0x7f6a380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.494+0000 7f6a37fff700 1 -- 172.21.15.19:0/43968647 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6a440e2080 con 0x7f6a50107df0 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c48ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c49030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd30b670a0 con 0x55dd30b41d00 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd30b0c9a0 con 0x55dd30ac74a0 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dd30b63060 con 0x55dd30c4d1e0 2024-05-09T01:30:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c48ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c48ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41708/0 (socket says 172.21.15.19:41708) 2024-05-09T01:30:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c49030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4effd700 1 -- 172.21.15.19:0/490880885 learned_addr learned my addr 172.21.15.19:0/490880885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/490880885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4effd700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a380b2900 0x7f6a380b4e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a40005950 tx=0x7f6a40014750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/490880885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 msgr2=0x55dd30c4d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/490880885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/490880885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 msgr2=0x55dd30c48ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/490880885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c48ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.495+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/490880885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd30afd500 con 0x55dd30ac74a0 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.496+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/490880885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c49030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400d950 tx=0x7f6a440a91b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=547a8bb020e8ce20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.496+0000 7f6a357fa700 1 -- 172.21.15.19:0/490880885 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44020070 con 0x55dd30ac74a0 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.496+0000 7f6a357fa700 1 -- 172.21.15.19:0/490880885 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a440ad9d0 con 0x55dd30ac74a0 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.496+0000 7f6a357fa700 1 -- 172.21.15.19:0/490880885 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a440a9a80 con 0x55dd30ac74a0 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 -- 172.21.15.19:0/490880885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 msgr2=0x55dd30c49030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/490880885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c49030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400d950 tx=0x7f6a440a91b0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 -- 172.21.15.19:0/490880885 shutdown_connections 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/490880885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/490880885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c48ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/490880885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c49030 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 -- 172.21.15.19:0/490880885 >> 172.21.15.19:0/490880885 conn(0x55dd30acaf70 msgr2=0x55dd30c3f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 -- 172.21.15.19:0/490880885 shutdown_connections 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 -- 172.21.15.19:0/490880885 wait complete. 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 -- start start 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.497+0000 7f6a63f45b80 1 Processor -- start 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30ac74a0 0x55dd30c52420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 0x55dd30c52960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c52ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd30c4de40 con 0x55dd30ac74a0 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd30b01b20 con 0x55dd30b41d00 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dd30c4dfc0 con 0x55dd30c4d1e0 2024-05-09T01:30:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30ac74a0 0x55dd30c52420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a63f45b80 10 client.0 _ll_register_callbacks cb 0x55dd309d86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30ac74a0 0x55dd30c52420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41712/0 (socket says 172.21.15.19:41712) 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4effd700 1 -- 172.21.15.19:0/3680260487 learned_addr learned my addr 172.21.15.19:0/3680260487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 0x55dd30c52960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c52ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4effd700 1 -- 172.21.15.19:0/3680260487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 msgr2=0x55dd30c52ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4effd700 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c52ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4effd700 1 -- 172.21.15.19:0/3680260487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 msgr2=0x55dd30c52960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4effd700 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 0x55dd30c52960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4effd700 1 -- 172.21.15.19:0/3680260487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd309e2490 con 0x55dd30ac74a0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a4effd700 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30ac74a0 0x55dd30c52420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6a40005d90 tx=0x7f6a400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca764d2b0feb1b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.498+0000 7f6a32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.499+0000 7f6a32ff5700 1 -- 172.21.15.19:0/3680260487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40028640 con 0x55dd30ac74a0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.499+0000 7f6a32ff5700 1 -- 172.21.15.19:0/3680260487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6a4001e420 con 0x55dd30ac74a0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.499+0000 7f6a63f45b80 1 -- 172.21.15.19:0/3680260487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a440ad9d0 con 0x55dd30ac74a0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.499+0000 7f6a32ff5700 1 -- 172.21.15.19:0/3680260487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a40011e30 con 0x55dd30ac74a0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.499+0000 7f6a63f45b80 10 client.16614 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.499+0000 7f6a63f45b80 1 -- 172.21.15.19:0/3680260487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dd30c5fd70 con 0x55dd30ac74a0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.499+0000 7f6a32ff5700 1 -- 172.21.15.19:0/3680260487 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a440ad9d0 con 0x55dd30ac74a0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.499+0000 7f6a32ff5700 1 -- 172.21.15.19:0/3680260487 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6a40026070 con 0x55dd30ac74a0 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.500+0000 7f6a63f45b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.500+0000 7f6a63f45b80 10 client.16614 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.500+0000 7f6a63f45b80 20 client.16614 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.500+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55dd30c62300 0x55dd30c64760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.500+0000 7f6a63f45b80 4 client.16614 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.500+0000 7f6a63f45b80 1 -- 172.21.15.19:0/3680260487 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd30b74a80 con 0x55dd30c62300 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.500+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55dd30c62300 0x55dd30c64760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.501+0000 7f6a32ff5700 10 client.16614 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.501+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55dd30c62300 0x55dd30c64760 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b22b189c0e0fb50a server_cookie=b18fc929e7836777 in_seq=0 out_seq=0 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.501+0000 7f6a32ff5700 1 -- 172.21.15.19:0/3680260487 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd30b74a80 con 0x55dd30c62300 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.501+0000 7f6a32ff5700 10 client.16614 handle_command_reply: tid=0 2024-05-09T01:30:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.501+0000 7f6a63f45b80 1 client.16614 shutdown 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 -- 172.21.15.19:0/3680260487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30ac74a0 msgr2=0x55dd30c52420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30ac74a0 0x55dd30c52420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6a40005d90 tx=0x7f6a400047c0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 -- 172.21.15.19:0/3680260487 shutdown_connections 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55dd30c62300 0x55dd30c64760 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c52ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30ac74a0 0x55dd30c52420 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/3680260487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 0x55dd30c52960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 -- 172.21.15.19:0/3680260487 >> 172.21.15.19:0/3680260487 conn(0x55dd30acaf70 msgr2=0x55dd30c38ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a32ff5700 0 client.16614 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 -- 172.21.15.19:0/3680260487 shutdown_connections 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 1 -- 172.21.15.19:0/3680260487 wait complete. 2024-05-09T01:30:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.502+0000 7f6a63f45b80 20 client.16614 trim_cache size 0 max 16384 2024-05-09T01:30:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.517+0000 7f6a63f45b80 1 Processor -- start 2024-05-09T01:30:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.517+0000 7f6a63f45b80 1 -- start start 2024-05-09T01:30:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.517+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c3f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.517+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c52ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c39700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501acbb0 con 0x55dd30b41d00 2024-05-09T01:30:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501ae6f0 con 0x55dd30ac74a0 2024-05-09T01:30:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501ad280 con 0x55dd30c4d1e0 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c52ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c52ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41728/0 (socket says 172.21.15.19:41728) 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/2091694234 learned_addr learned my addr 172.21.15.19:0/2091694234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4effd700 1 --2- 172.21.15.19:0/2091694234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c3f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/2091694234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 msgr2=0x55dd30c39700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/2091694234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/2091694234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 msgr2=0x55dd30c3f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/2091694234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c3f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4e7fc700 1 -- 172.21.15.19:0/2091694234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd30c40a30 con 0x55dd30b41d00 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4effd700 1 --2- 172.21.15.19:0/2091694234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c3f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/2091694234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c52ce0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6a440a97e0 tx=0x7f6a44010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6674eb8174bd9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a34ff9700 1 -- 172.21.15.19:0/2091694234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44014070 con 0x55dd30b41d00 2024-05-09T01:30:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a34ff9700 1 -- 172.21.15.19:0/2091694234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a44010460 con 0x55dd30b41d00 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.518+0000 7f6a34ff9700 1 -- 172.21.15.19:0/2091694234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44020040 con 0x55dd30b41d00 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.519+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2091694234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 msgr2=0x55dd30c52ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.519+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2091694234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c52ce0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6a440a97e0 tx=0x7f6a44010a70 comp rx=0 tx=0).stop 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.519+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2091694234 shutdown_connections 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.519+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2091694234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30c4d1e0 0x55dd30c39700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.519+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2091694234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30ac74a0 0x55dd30c3f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.519+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2091694234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30b41d00 0x55dd30c52ce0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.519+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2091694234 >> 172.21.15.19:0/2091694234 conn(0x55dd30acaf70 msgr2=0x55dd30c38ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.519+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2091694234 shutdown_connections 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.519+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2091694234 wait complete. 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a63f45b80 1 -- start start 2024-05-09T01:30:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a63f45b80 1 Processor -- start 2024-05-09T01:30:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30ac74a0 0x55dd30c5a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 0x55dd30c5aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a63f45b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501ae020 con 0x55dd30c4d1e0 2024-05-09T01:30:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501ad950 con 0x55dd30b41d00 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a63f45b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501aab70 con 0x55dd30ac74a0 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a63f45b80 10 client.0 _ll_register_callbacks cb 0x55dd309d86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41730/0 (socket says 172.21.15.19:41730) 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/2322356016 learned_addr learned my addr 172.21.15.19:0/2322356016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a4e7fc700 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 0x55dd30c5aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.520+0000 7f6a4effd700 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30ac74a0 0x55dd30c5a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/2322356016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30ac74a0 msgr2=0x55dd30c5a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30ac74a0 0x55dd30c5a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/2322356016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 msgr2=0x55dd30c5aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 0x55dd30c5aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a4f7fe700 1 -- 172.21.15.19:0/2322356016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd30c4a540 con 0x55dd30c4d1e0 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a4f7fe700 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4a000 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b820 tx=0x7f6a48004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d18fe3b337ad3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a32ff5700 1 -- 172.21.15.19:0/2322356016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a480075c0 con 0x55dd30c4d1e0 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a32ff5700 1 -- 172.21.15.19:0/2322356016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6a48007760 con 0x55dd30c4d1e0 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2322356016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a44010460 con 0x55dd30c4d1e0 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a32ff5700 1 -- 172.21.15.19:0/2322356016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a48007960 con 0x55dd30c4d1e0 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a63f45b80 10 client.16620 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2322356016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dd30ccc080 con 0x55dd30c4d1e0 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.521+0000 7f6a32ff5700 1 -- 172.21.15.19:0/2322356016 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a44010460 con 0x55dd30c4d1e0 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a32ff5700 1 -- 172.21.15.19:0/2322356016 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6a48024070 con 0x55dd30c4d1e0 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a63f45b80 10 client.16620 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a63f45b80 10 client.16620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a63f45b80 20 client.16620 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55dd30ccc790 0x55dd30ccebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a63f45b80 4 client.16620 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2322356016 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd30b6eb20 con 0x55dd30ccc790 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a4effd700 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55dd30ccc790 0x55dd30ccebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a4effd700 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55dd30ccc790 0x55dd30ccebb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3015e35319a7e39d server_cookie=b4191233e392a4f5 in_seq=0 out_seq=0 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.522+0000 7f6a32ff5700 10 client.16620 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a32ff5700 1 -- 172.21.15.19:0/2322356016 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd30b6eb20 con 0x55dd30ccc790 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a32ff5700 10 client.16620 handle_command_reply: tid=0 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 client.16620 shutdown 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2322356016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30c4d1e0 msgr2=0x55dd30c4a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4a000 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b820 tx=0x7f6a48004a50 comp rx=0 tx=0).stop 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2322356016 shutdown_connections 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55dd30ccc790 0x55dd30ccebb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd30c4d1e0 0x55dd30c4a000 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd30ac74a0 0x55dd30c5a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a32ff5700 0 client.16620 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 --2- 172.21.15.19:0/2322356016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd30b41d00 0x55dd30c5aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2322356016 >> 172.21.15.19:0/2322356016 conn(0x55dd30acaf70 msgr2=0x55dd30c50c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2322356016 shutdown_connections 2024-05-09T01:30:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.523+0000 7f6a63f45b80 1 -- 172.21.15.19:0/2322356016 wait complete. 2024-05-09T01:30:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a63f45b80 20 client.16620 trim_cache size 0 max 16384 2024-05-09T01:30:32.526 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 -- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a380b2900 msgr2=0x7f6a380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a380b2900 0x7f6a380b4e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a40005950 tx=0x7f6a40014750 comp rx=0 tx=0).stop 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 -- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107df0 msgr2=0x7f6a5019c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107df0 0x7f6a5019c1f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400d980 tx=0x7f6a4400dc90 comp rx=0 tx=0).stop 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 -- 172.21.15.19:0/43968647 shutdown_connections 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a50108740 0x7f6a5019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a501034c0 0x7f6a5019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6a380b2900 0x7f6a380b4e10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 --2- 172.21.15.19:0/43968647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a50107df0 0x7f6a5019c1f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 -- 172.21.15.19:0/43968647 >> 172.21.15.19:0/43968647 conn(0x7f6a500fedc0 msgr2=0x7f6a50107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 -- 172.21.15.19:0/43968647 shutdown_connections 2024-05-09T01:30:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:32.524+0000 7f6a5552f700 1 -- 172.21.15.19:0/43968647 wait complete. 2024-05-09T01:30:32.647 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:32 smithi059 ceph-mon[34351]: pgmap v528: 97 pgs: 97 active+clean; 2.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:30:32.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:32 smithi196 ceph-mon[35041]: pgmap v528: 97 pgs: 97 active+clean; 2.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:30:33.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:30:33.629 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:34.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:34 smithi019 ceph-mon[30158]: pgmap v529: 97 pgs: 97 active+clean; 2.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-09T01:30:34.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:34 smithi196 ceph-mon[35041]: pgmap v529: 97 pgs: 97 active+clean; 2.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-09T01:30:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:34 smithi059 ceph-mon[34351]: pgmap v529: 97 pgs: 97 active+clean; 2.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-09T01:30:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.429+0000 7f0e717e0700 1 -- 172.21.15.19:0/3359740514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 msgr2=0x7f0e6c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.429+0000 7f0e717e0700 1 --2- 172.21.15.19:0/3359740514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 0x7f0e6c1100a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400b0d0 tx=0x7f0e6400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.429+0000 7f0e717e0700 1 -- 172.21.15.19:0/3359740514 shutdown_connections 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.429+0000 7f0e717e0700 1 --2- 172.21.15.19:0/3359740514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 0x7f0e6c1100a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.429+0000 7f0e717e0700 1 --2- 172.21.15.19:0/3359740514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c1034c0 0x7f0e6c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.429+0000 7f0e717e0700 1 --2- 172.21.15.19:0/3359740514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c107df0 0x7f0e6c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.429+0000 7f0e717e0700 1 -- 172.21.15.19:0/3359740514 >> 172.21.15.19:0/3359740514 conn(0x7f0e6c0fedf0 msgr2=0x7f0e6c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.429+0000 7f0e717e0700 1 -- 172.21.15.19:0/3359740514 shutdown_connections 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.429+0000 7f0e717e0700 1 -- 172.21.15.19:0/3359740514 wait complete. 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.430+0000 7f0e717e0700 1 Processor -- start 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.430+0000 7f0e717e0700 1 -- start start 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.430+0000 7f0e717e0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c1034c0 0x7f0e6c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.430+0000 7f0e717e0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c107df0 0x7f0e6c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.430+0000 7f0e717e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 0x7f0e6c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.430+0000 7f0e717e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c113370 con 0x7f0e6c108740 2024-05-09T01:30:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.430+0000 7f0e717e0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1131f0 con 0x7f0e6c107df0 2024-05-09T01:30:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.430+0000 7f0e717e0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1134f0 con 0x7f0e6c1034c0 2024-05-09T01:30:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 0x7f0e6c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 0x7f0e6c0ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41752/0 (socket says 172.21.15.19:41752) 2024-05-09T01:30:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6b7fe700 1 -- 172.21.15.19:0/2587893546 learned_addr learned my addr 172.21.15.19:0/2587893546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6affd700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c1034c0 0x7f0e6c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c107df0 0x7f0e6c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6b7fe700 1 -- 172.21.15.19:0/2587893546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c1034c0 msgr2=0x7f0e6c0ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:35.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c1034c0 0x7f0e6c0ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6b7fe700 1 -- 172.21.15.19:0/2587893546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c107df0 msgr2=0x7f0e6c0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:35.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c107df0 0x7f0e6c0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6b7fe700 1 -- 172.21.15.19:0/2587893546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6c1a91e0 con 0x7f0e6c108740 2024-05-09T01:30:35.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 0x7f0e6c0ffd50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009fd0 tx=0x7f0e64007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:35.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e53fff700 1 -- 172.21.15.19:0/2587893546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6401f070 con 0x7f0e6c108740 2024-05-09T01:30:35.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e717e0700 1 -- 172.21.15.19:0/2587893546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6c1a9470 con 0x7f0e6c108740 2024-05-09T01:30:35.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e53fff700 1 -- 172.21.15.19:0/2587893546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e64007e50 con 0x7f0e6c108740 2024-05-09T01:30:35.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.431+0000 7f0e53fff700 1 -- 172.21.15.19:0/2587893546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6401b6a0 con 0x7f0e6c108740 2024-05-09T01:30:35.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.432+0000 7f0e717e0700 1 -- 172.21.15.19:0/2587893546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6c1a9920 con 0x7f0e6c108740 2024-05-09T01:30:35.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.435+0000 7f0e53fff700 1 -- 172.21.15.19:0/2587893546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0e64022070 con 0x7f0e6c108740 2024-05-09T01:30:35.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e801f6b80 1 Processor -- start 2024-05-09T01:30:35.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e53fff700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e540b2880 0x7f0e540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e801f6b80 1 -- start start 2024-05-09T01:30:35.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e540b2880 0x7f0e540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e53fff700 1 -- 172.21.15.19:0/2587893546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0e6400f020 con 0x7f0e6c108740 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c9619240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9619780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c9613810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d3c95360a0 con 0x55d3c94964a0 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d3c94db9a0 con 0x55d3c9510d00 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.436+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3c9532060 con 0x55d3c9613400 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9619780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9619780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41766/0 (socket says 172.21.15.19:41766) 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6a7fc700 1 -- 172.21.15.19:0/4130269580 learned_addr learned my addr 172.21.15.19:0/4130269580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c9619240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/4130269580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c9613810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e540b2880 0x7f0e540b4d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e600134b0 tx=0x7f0e60013870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:35.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6affd700 1 -- 172.21.15.19:0/4130269580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 msgr2=0x55d3c9613810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:35.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6affd700 1 --2- 172.21.15.19:0/4130269580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c9613810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6affd700 1 -- 172.21.15.19:0/4130269580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 msgr2=0x55d3c9619780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:35.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6affd700 1 --2- 172.21.15.19:0/4130269580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9619780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6affd700 1 -- 172.21.15.19:0/4130269580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c94cc500 con 0x55d3c9510d00 2024-05-09T01:30:35.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/4130269580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9619780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:35.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e6affd700 1 --2- 172.21.15.19:0/4130269580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c9619240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00ac60 tx=0x7f0e5c00af70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86d5a84bcee39e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:35.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e517fa700 1 -- 172.21.15.19:0/4130269580 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c00fd20 con 0x55d3c9510d00 2024-05-09T01:30:35.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e517fa700 1 -- 172.21.15.19:0/4130269580 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c0044e0 con 0x55d3c9510d00 2024-05-09T01:30:35.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.437+0000 7f0e517fa700 1 -- 172.21.15.19:0/4130269580 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c005280 con 0x55d3c9510d00 2024-05-09T01:30:35.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.438+0000 7f0e801f6b80 1 -- 172.21.15.19:0/4130269580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 msgr2=0x55d3c9619240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:35.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.438+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/4130269580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c9619240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00ac60 tx=0x7f0e5c00af70 comp rx=0 tx=0).stop 2024-05-09T01:30:35.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.438+0000 7f0e801f6b80 1 -- 172.21.15.19:0/4130269580 shutdown_connections 2024-05-09T01:30:35.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.438+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/4130269580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c9613810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.438+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/4130269580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c9619240 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.438+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/4130269580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9619780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.438+0000 7f0e801f6b80 1 -- 172.21.15.19:0/4130269580 >> 172.21.15.19:0/4130269580 conn(0x55d3c9499f70 msgr2=0x55d3c960e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.438+0000 7f0e801f6b80 1 -- 172.21.15.19:0/4130269580 shutdown_connections 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.438+0000 7f0e801f6b80 1 -- 172.21.15.19:0/4130269580 wait complete. 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.439+0000 7f0e801f6b80 1 -- start start 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.439+0000 7f0e801f6b80 1 Processor -- start 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.439+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c94964a0 0x55d3c961cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.439+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9510d00 0x55d3c961d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.439+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c9613400 0x55d3c961da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.439+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d3c9614160 con 0x55d3c9613400 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.439+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d3c94d0b20 con 0x55d3c94964a0 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.439+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3c9620020 con 0x55d3c9510d00 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e801f6b80 10 client.0 _ll_register_callbacks cb 0x55d3c93a76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:35.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c9613400 0x55d3c961da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c9613400 0x55d3c961da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41774/0 (socket says 172.21.15.19:41774) 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6b7fe700 1 -- 172.21.15.19:0/815710194 learned_addr learned my addr 172.21.15.19:0/815710194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9510d00 0x55d3c961d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6affd700 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c94964a0 0x55d3c961cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6b7fe700 1 -- 172.21.15.19:0/815710194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9510d00 msgr2=0x55d3c961d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9510d00 0x55d3c961d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6b7fe700 1 -- 172.21.15.19:0/815710194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c94964a0 msgr2=0x55d3c961cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c94964a0 0x55d3c961cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6b7fe700 1 -- 172.21.15.19:0/815710194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c93b1490 con 0x55d3c9613400 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c9613400 0x55d3c961da70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0e64007a90 tx=0x7f0e6400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdf9d179c0a3360 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e4eff5700 1 -- 172.21.15.19:0/815710194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e640e3320 con 0x55d3c9613400 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e4eff5700 1 -- 172.21.15.19:0/815710194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0e640aadc0 con 0x55d3c9613400 2024-05-09T01:30:35.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.440+0000 7f0e801f6b80 1 -- 172.21.15.19:0/815710194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e5c0044e0 con 0x55d3c9613400 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.441+0000 7f0e4eff5700 1 -- 172.21.15.19:0/815710194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e640aa670 con 0x55d3c9613400 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.441+0000 7f0e801f6b80 10 client.16632 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.441+0000 7f0e801f6b80 1 -- 172.21.15.19:0/815710194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d3c962aa00 con 0x55d3c9613400 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.441+0000 7f0e4eff5700 1 -- 172.21.15.19:0/815710194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c0044e0 con 0x55d3c9613400 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e4eff5700 1 -- 172.21.15.19:0/815710194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0e64011070 con 0x55d3c9613400 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e801f6b80 10 client.16632 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e801f6b80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e801f6b80 20 client.16632 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d3c962ced0 0x55d3c962f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e801f6b80 4 client.16632 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e801f6b80 1 -- 172.21.15.19:0/815710194 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3c9543a80 con 0x55d3c962ced0 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e6affd700 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d3c962ced0 0x55d3c962f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e4eff5700 10 client.16632 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:35.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.442+0000 7f0e6affd700 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d3c962ced0 0x55d3c962f340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9eb8fc748e82b9 server_cookie=643e849377eb7c35 in_seq=0 out_seq=0 2024-05-09T01:30:35.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e4eff5700 1 -- 172.21.15.19:0/815710194 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d3c9543a80 con 0x55d3c962ced0 2024-05-09T01:30:35.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e4eff5700 10 client.16632 handle_command_reply: tid=0 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e801f6b80 1 client.16632 shutdown 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e801f6b80 1 -- 172.21.15.19:0/815710194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c9613400 msgr2=0x55d3c961da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c9613400 0x55d3c961da70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0e64007a90 tx=0x7f0e6400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e801f6b80 1 -- 172.21.15.19:0/815710194 shutdown_connections 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d3c962ced0 0x55d3c962f340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c9613400 0x55d3c961da70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c94964a0 0x55d3c961cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/815710194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9510d00 0x55d3c961d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e801f6b80 1 -- 172.21.15.19:0/815710194 >> 172.21.15.19:0/815710194 conn(0x55d3c9499f70 msgr2=0x55d3c9607ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.443+0000 7f0e4eff5700 0 client.16632 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.444+0000 7f0e801f6b80 1 -- 172.21.15.19:0/815710194 shutdown_connections 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.444+0000 7f0e801f6b80 1 -- 172.21.15.19:0/815710194 wait complete. 2024-05-09T01:30:35.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.444+0000 7f0e801f6b80 20 client.16632 trim_cache size 0 max 16384 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e801f6b80 1 Processor -- start 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e801f6b80 1 -- start start 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9617700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c961bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c9628e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1b0d90 con 0x55d3c94964a0 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1b28d0 con 0x55d3c9510d00 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1b1460 con 0x55d3c9613400 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9617700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9617700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41788/0 (socket says 172.21.15.19:41788) 2024-05-09T01:30:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e6affd700 1 -- 172.21.15.19:0/2479688987 learned_addr learned my addr 172.21.15.19:0/2479688987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c961bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.459+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c9628e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e6affd700 1 -- 172.21.15.19:0/2479688987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 msgr2=0x55d3c9628e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e6affd700 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c9628e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e6affd700 1 -- 172.21.15.19:0/2479688987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 msgr2=0x55d3c961bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e6affd700 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c961bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e6affd700 1 -- 172.21.15.19:0/2479688987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c961cdc0 con 0x55d3c94964a0 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c961bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c9628e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e6affd700 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9617700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00a630 tx=0x7f0e5c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519783e39e4ae48d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e50ff9700 1 -- 172.21.15.19:0/2479688987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c01d040 con 0x55d3c94964a0 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e50ff9700 1 -- 172.21.15.19:0/2479688987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c02f070 con 0x55d3c94964a0 2024-05-09T01:30:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.460+0000 7f0e50ff9700 1 -- 172.21.15.19:0/2479688987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0186a0 con 0x55d3c94964a0 2024-05-09T01:30:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 -- 172.21.15.19:0/2479688987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 msgr2=0x55d3c9617700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9617700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00a630 tx=0x7f0e5c004790 comp rx=0 tx=0).stop 2024-05-09T01:30:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 -- 172.21.15.19:0/2479688987 shutdown_connections 2024-05-09T01:30:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c9628e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c9617700 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/2479688987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c961bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 -- 172.21.15.19:0/2479688987 >> 172.21.15.19:0/2479688987 conn(0x55d3c9499f70 msgr2=0x55d3c961b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 -- 172.21.15.19:0/2479688987 shutdown_connections 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 -- 172.21.15.19:0/2479688987 wait complete. 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 -- start start 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.461+0000 7f0e801f6b80 1 Processor -- start 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c968f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c968ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e801f6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c96206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1b2200 con 0x55d3c94964a0 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1b1b30 con 0x55d3c9510d00 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e801f6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1aed50 con 0x55d3c9613400 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e801f6b80 10 client.0 _ll_register_callbacks cb 0x55d3c93a76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c968f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c968f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41796/0 (socket says 172.21.15.19:41796) 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6affd700 1 -- 172.21.15.19:0/831368347 learned_addr learned my addr 172.21.15.19:0/831368347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c968ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6b7fe700 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c96206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6affd700 1 -- 172.21.15.19:0/831368347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 msgr2=0x55d3c96206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6affd700 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c96206f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6affd700 1 -- 172.21.15.19:0/831368347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 msgr2=0x55d3c968ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6affd700 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c968ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.462+0000 7f0e6affd700 1 -- 172.21.15.19:0/831368347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c9620d30 con 0x55d3c94964a0 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e6affd700 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c968f9e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00ac30 tx=0x7f0e5c029b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87eefdb196e980c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e4eff5700 1 -- 172.21.15.19:0/831368347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c00fe60 con 0x55d3c94964a0 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e4eff5700 1 -- 172.21.15.19:0/831368347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0e5c029170 con 0x55d3c94964a0 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e801f6b80 1 -- 172.21.15.19:0/831368347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e5c02f070 con 0x55d3c94964a0 2024-05-09T01:30:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e4eff5700 1 -- 172.21.15.19:0/831368347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e5c029370 con 0x55d3c94964a0 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e801f6b80 10 client.16638 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e801f6b80 1 -- 172.21.15.19:0/831368347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d3c9697020 con 0x55d3c94964a0 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e4eff5700 1 -- 172.21.15.19:0/831368347 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c02f070 con 0x55d3c94964a0 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e4eff5700 1 -- 172.21.15.19:0/831368347 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0e5c033450 con 0x55d3c94964a0 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e801f6b80 10 client.16638 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e801f6b80 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e801f6b80 20 client.16638 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d3c9697550 0x55d3c9699970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e801f6b80 4 client.16638 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.463+0000 7f0e801f6b80 1 -- 172.21.15.19:0/831368347 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3c953db20 con 0x55d3c9697550 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.464+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d3c9697550 0x55d3c9699970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.464+0000 7f0e6a7fc700 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d3c9697550 0x55d3c9699970 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f788259c38c2b23 server_cookie=d0be755dcd22fe5 in_seq=0 out_seq=0 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.464+0000 7f0e4eff5700 10 client.16638 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.464+0000 7f0e4eff5700 1 -- 172.21.15.19:0/831368347 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3c953db20 con 0x55d3c9697550 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.464+0000 7f0e4eff5700 10 client.16638 handle_command_reply: tid=0 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.464+0000 7f0e801f6b80 1 client.16638 shutdown 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.464+0000 7f0e801f6b80 1 -- 172.21.15.19:0/831368347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 msgr2=0x55d3c968f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.464+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c968f9e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00ac30 tx=0x7f0e5c029b30 comp rx=0 tx=0).stop 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.464+0000 7f0e801f6b80 1 -- 172.21.15.19:0/831368347 shutdown_connections 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55d3c9697550 0x55d3c9699970 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3c9613400 0x55d3c96206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3c94964a0 0x55d3c968f9e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e801f6b80 1 --2- 172.21.15.19:0/831368347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3c9510d00 0x55d3c968ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e801f6b80 1 -- 172.21.15.19:0/831368347 >> 172.21.15.19:0/831368347 conn(0x55d3c9499f70 msgr2=0x55d3c9617460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e4eff5700 0 client.16638 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e801f6b80 1 -- 172.21.15.19:0/831368347 shutdown_connections 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e801f6b80 1 -- 172.21.15.19:0/831368347 wait complete. 2024-05-09T01:30:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e801f6b80 20 client.16638 trim_cache size 0 max 16384 2024-05-09T01:30:35.467 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:35.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 -- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e540b2880 msgr2=0x7f0e540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:35.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e540b2880 0x7f0e540b4d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e600134b0 tx=0x7f0e60013870 comp rx=0 tx=0).stop 2024-05-09T01:30:35.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 -- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 msgr2=0x7f0e6c0ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 0x7f0e6c0ffd50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009fd0 tx=0x7f0e64007a20 comp rx=0 tx=0).stop 2024-05-09T01:30:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 -- 172.21.15.19:0/2587893546 shutdown_connections 2024-05-09T01:30:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0e540b2880 0x7f0e540b4d90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c108740 0x7f0e6c0ffd50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e6c1034c0 0x7f0e6c0ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 --2- 172.21.15.19:0/2587893546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c107df0 0x7f0e6c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.465+0000 7f0e717e0700 1 -- 172.21.15.19:0/2587893546 >> 172.21.15.19:0/2587893546 conn(0x7f0e6c0fedf0 msgr2=0x7f0e6c075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.466+0000 7f0e717e0700 1 -- 172.21.15.19:0/2587893546 shutdown_connections 2024-05-09T01:30:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:35.466+0000 7f0e717e0700 1 -- 172.21.15.19:0/2587893546 wait complete. 2024-05-09T01:30:36.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:36 smithi019 ceph-mon[30158]: pgmap v530: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:30:36.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:36 smithi196 ceph-mon[35041]: pgmap v530: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:30:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:36 smithi059 ceph-mon[34351]: pgmap v530: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:30:37.231 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:30:37.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:30:37.500 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:38.636 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:38 smithi019 ceph-mon[30158]: pgmap v531: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:30:38.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:38 smithi196 ceph-mon[35041]: pgmap v531: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:30:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:38 smithi059 ceph-mon[34351]: pgmap v531: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:30:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.191+0000 7faceb364700 1 -- 172.21.15.19:0/1738383077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4109870 msgr2=0x7face41111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.191+0000 7faceb364700 1 --2- 172.21.15.19:0/1738383077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4109870 0x7face41111d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7facdc0099c0 tx=0x7facdc009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.191+0000 7faceb364700 1 -- 172.21.15.19:0/1738383077 shutdown_connections 2024-05-09T01:30:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.191+0000 7faceb364700 1 --2- 172.21.15.19:0/1738383077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4109870 0x7face41111d0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.191+0000 7faceb364700 1 --2- 172.21.15.19:0/1738383077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face41045f0 0x7face4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.191+0000 7faceb364700 1 --2- 172.21.15.19:0/1738383077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7face4108f20 0x7face4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.192+0000 7faceb364700 1 -- 172.21.15.19:0/1738383077 >> 172.21.15.19:0/1738383077 conn(0x7face4100040 msgr2=0x7face4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.192+0000 7faceb364700 1 -- 172.21.15.19:0/1738383077 shutdown_connections 2024-05-09T01:30:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.192+0000 7faceb364700 1 -- 172.21.15.19:0/1738383077 wait complete. 2024-05-09T01:30:39.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.192+0000 7faceb364700 1 Processor -- start 2024-05-09T01:30:39.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7faceb364700 1 -- start start 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7faceb364700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7face41045f0 0x7face419d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7faceb364700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4108f20 0x7face419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7faceb364700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face4109870 0x7face41a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7faceb364700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7face41144f0 con 0x7face4108f20 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7faceb364700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7face4114370 con 0x7face41045f0 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7faceb364700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7face4114670 con 0x7face4109870 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face88ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4108f20 0x7face419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face88ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4108f20 0x7face419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41818/0 (socket says 172.21.15.19:41818) 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face88ff700 1 -- 172.21.15.19:0/3041918643 learned_addr learned my addr 172.21.15.19:0/3041918643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face9901700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face4109870 0x7face41a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face9100700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7face41045f0 0x7face419d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face88ff700 1 -- 172.21.15.19:0/3041918643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face4109870 msgr2=0x7face41a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face88ff700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face4109870 0x7face41a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face88ff700 1 -- 172.21.15.19:0/3041918643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7face41045f0 msgr2=0x7face419d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face88ff700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7face41045f0 0x7face419d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.193+0000 7face88ff700 1 -- 172.21.15.19:0/3041918643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face41a1f40 con 0x7face4108f20 2024-05-09T01:30:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.194+0000 7face88ff700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4108f20 0x7face419d660 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7facd800e970 tx=0x7facd800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.194+0000 7face27fc700 1 -- 172.21.15.19:0/3041918643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8004a30 con 0x7face4108f20 2024-05-09T01:30:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.194+0000 7faceb364700 1 -- 172.21.15.19:0/3041918643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face41a2230 con 0x7face4108f20 2024-05-09T01:30:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.194+0000 7faceb364700 1 -- 172.21.15.19:0/3041918643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face41a9b50 con 0x7face4108f20 2024-05-09T01:30:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.194+0000 7face27fc700 1 -- 172.21.15.19:0/3041918643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd800cc90 con 0x7face4108f20 2024-05-09T01:30:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.194+0000 7face27fc700 1 -- 172.21.15.19:0/3041918643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8005610 con 0x7face4108f20 2024-05-09T01:30:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.197+0000 7face27fc700 1 -- 172.21.15.19:0/3041918643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7facd8021440 con 0x7face4108f20 2024-05-09T01:30:39.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7face27fc700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faccc0b2880 0x7faccc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7facf9d7ab80 1 Processor -- start 2024-05-09T01:30:39.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7face9100700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faccc0b2880 0x7faccc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7face27fc700 1 -- 172.21.15.19:0/3041918643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7facd80e2330 con 0x7face4108f20 2024-05-09T01:30:39.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7facf9d7ab80 1 -- start start 2024-05-09T01:30:39.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51bcb1e0 0x556f51c64570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c64ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 0x556f51c5eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f51b81070 con 0x556f51bcb1e0 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556f51b1f870 con 0x556f51ae1520 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556f51b7d030 con 0x556f51c5e720 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7face9100700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51bcb1e0 0x556f51c64570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7face9100700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51bcb1e0 0x556f51c64570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41828/0 (socket says 172.21.15.19:41828) 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7face9100700 1 -- 172.21.15.19:0/3036567982 learned_addr learned my addr 172.21.15.19:0/3036567982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7face88ff700 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c64ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7face9901700 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 0x556f51c5eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.198+0000 7face9100700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faccc0b2880 0x7faccc0b4d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7facd4009550 tx=0x7facd40091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7face9100700 1 -- 172.21.15.19:0/3036567982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 msgr2=0x556f51c5eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7face9100700 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 0x556f51c5eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7face9100700 1 -- 172.21.15.19:0/3036567982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 msgr2=0x556f51c64ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7face9100700 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c64ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7face9100700 1 -- 172.21.15.19:0/3036567982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f51b173f0 con 0x556f51bcb1e0 2024-05-09T01:30:39.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7face88ff700 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c64ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:39.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7face9100700 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51bcb1e0 0x556f51c64570 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7facd40046d0 tx=0x7facd4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75766fc10f6dd185 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:39.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7faccb7fe700 1 -- 172.21.15.19:0/3036567982 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4026070 con 0x556f51bcb1e0 2024-05-09T01:30:39.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7faccb7fe700 1 -- 172.21.15.19:0/3036567982 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd401d040 con 0x556f51bcb1e0 2024-05-09T01:30:39.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.199+0000 7faccb7fe700 1 -- 172.21.15.19:0/3036567982 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd401c050 con 0x556f51bcb1e0 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.200+0000 7facf9d7ab80 1 -- 172.21.15.19:0/3036567982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51bcb1e0 msgr2=0x556f51c64570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.200+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51bcb1e0 0x556f51c64570 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7facd40046d0 tx=0x7facd4013000 comp rx=0 tx=0).stop 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.200+0000 7facf9d7ab80 1 -- 172.21.15.19:0/3036567982 shutdown_connections 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.200+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 0x556f51c5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.200+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51bcb1e0 0x556f51c64570 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.200+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/3036567982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c64ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.200+0000 7facf9d7ab80 1 -- 172.21.15.19:0/3036567982 >> 172.21.15.19:0/3036567982 conn(0x556f51b41720 msgr2=0x556f51c59480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.200+0000 7facf9d7ab80 1 -- 172.21.15.19:0/3036567982 shutdown_connections 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.200+0000 7facf9d7ab80 1 -- 172.21.15.19:0/3036567982 wait complete. 2024-05-09T01:30:39.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7facf9d7ab80 1 -- start start 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7facf9d7ab80 1 Processor -- start 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c682b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 0x556f51c687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c68d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f519fcc20 con 0x556f51c5e720 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556f51b3b1d0 con 0x556f51ae1520 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556f51c6b250 con 0x556f51bcb1e0 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7facf9d7ab80 10 client.0 _ll_register_callbacks cb 0x556f519f2740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7face9901700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c68d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7face9901700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c68d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41840/0 (socket says 172.21.15.19:41840) 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.201+0000 7face9901700 1 -- 172.21.15.19:0/2827262525 learned_addr learned my addr 172.21.15.19:0/2827262525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7face9100700 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7face88ff700 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 0x556f51c687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7face9901700 1 -- 172.21.15.19:0/2827262525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 msgr2=0x556f51c687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7face9901700 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 0x556f51c687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7face9901700 1 -- 172.21.15.19:0/2827262525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 msgr2=0x556f51c682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7face9901700 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c682b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7face9901700 1 -- 172.21.15.19:0/2827262525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f519fc510 con 0x556f51c5e720 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7face9901700 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c68d30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7facdc015000 tx=0x7facdc005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc1596a16a6cd6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7facc8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7facc8ff9700 1 -- 172.21.15.19:0/2827262525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc003bd0 con 0x556f51c5e720 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7facc8ff9700 1 -- 172.21.15.19:0/2827262525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7facdc003d70 con 0x556f51c5e720 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2827262525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd401d040 con 0x556f51c5e720 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7facc8ff9700 1 -- 172.21.15.19:0/2827262525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7facdc009590 con 0x556f51c5e720 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7facf9d7ab80 10 client.16650 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:39.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.202+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2827262525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f51c75c90 con 0x556f51c5e720 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.203+0000 7facc8ff9700 1 -- 172.21.15.19:0/2827262525 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd401d040 con 0x556f51c5e720 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.203+0000 7facc8ff9700 1 -- 172.21.15.19:0/2827262525 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7facdc00f950 con 0x556f51c5e720 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.203+0000 7facf9d7ab80 10 client.16650 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.203+0000 7facf9d7ab80 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.203+0000 7facf9d7ab80 20 client.16650 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.203+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556f51c78320 0x556f51c7a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.203+0000 7facf9d7ab80 4 client.16650 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.203+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2827262525 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f51b8ea50 con 0x556f51c78320 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.203+0000 7face9100700 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556f51c78320 0x556f51c7a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.204+0000 7face9100700 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556f51c78320 0x556f51c7a740 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba00fe7bc685cfd4 server_cookie=27365c7527074457 in_seq=0 out_seq=0 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.204+0000 7facc8ff9700 10 client.16650 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.204+0000 7facc8ff9700 1 -- 172.21.15.19:0/2827262525 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f51b8ea50 con 0x556f51c78320 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.204+0000 7facc8ff9700 10 client.16650 handle_command_reply: tid=0 2024-05-09T01:30:39.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 client.16650 shutdown 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2827262525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 msgr2=0x556f51c68d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c68d30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7facdc015000 tx=0x7facdc005790 comp rx=0 tx=0).stop 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2827262525 shutdown_connections 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556f51c78320 0x556f51c7a740 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c68d30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2827262525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 0x556f51c687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2827262525 >> 172.21.15.19:0/2827262525 conn(0x556f51b41720 msgr2=0x556f51c59240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facc8ff9700 0 client.16650 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2827262525 shutdown_connections 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2827262525 wait complete. 2024-05-09T01:30:39.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.205+0000 7facf9d7ab80 20 client.16650 trim_cache size 0 max 16384 2024-05-09T01:30:39.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7facf9d7ab80 1 Processor -- start 2024-05-09T01:30:39.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7facf9d7ab80 1 -- start start 2024-05-09T01:30:39.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c62590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 0x556f51c68270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7face41b0e90 con 0x556f51c5e720 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7face41b29d0 con 0x556f51ae1520 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7face41b1560 con 0x556f51bcb1e0 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7face9901700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7face9901700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c687b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41846/0 (socket says 172.21.15.19:41846) 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.226+0000 7face9901700 1 -- 172.21.15.19:0/2287723617 learned_addr learned my addr 172.21.15.19:0/2287723617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7face88ff700 1 --2- 172.21.15.19:0/2287723617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 0x556f51c68270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7face9100700 1 --2- 172.21.15.19:0/2287723617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c62590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7face9901700 1 -- 172.21.15.19:0/2287723617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 msgr2=0x556f51c68270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7face9901700 1 --2- 172.21.15.19:0/2287723617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 0x556f51c68270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7face9901700 1 -- 172.21.15.19:0/2287723617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 msgr2=0x556f51c62590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7face9901700 1 --2- 172.21.15.19:0/2287723617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c62590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7face9901700 1 -- 172.21.15.19:0/2287723617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f51c66850 con 0x556f51c5e720 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7face9901700 1 --2- 172.21.15.19:0/2287723617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c687b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7facdc005f50 tx=0x7facdc023670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6357528ee67e76d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7faccaffd700 1 -- 172.21.15.19:0/2287723617 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc003bd0 con 0x556f51c5e720 2024-05-09T01:30:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7faccaffd700 1 -- 172.21.15.19:0/2287723617 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facdc003d70 con 0x556f51c5e720 2024-05-09T01:30:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.227+0000 7faccaffd700 1 -- 172.21.15.19:0/2287723617 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc023d80 con 0x556f51c5e720 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2287723617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 msgr2=0x556f51c687b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2287723617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c687b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7facdc005f50 tx=0x7facdc023670 comp rx=0 tx=0).stop 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2287723617 shutdown_connections 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2287723617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51c5e720 0x556f51c687b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2287723617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51ae1520 0x556f51c62590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2287723617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51bcb1e0 0x556f51c68270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2287723617 >> 172.21.15.19:0/2287723617 conn(0x556f51b41720 msgr2=0x556f51c61e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2287723617 shutdown_connections 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2287723617 wait complete. 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 -- start start 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.228+0000 7facf9d7ab80 1 Processor -- start 2024-05-09T01:30:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51ae1520 0x556f51c70330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51bcb1e0 0x556f51c70870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7facf9d7ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 0x556f51c6d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7face41b2300 con 0x556f51ae1520 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7face41b1c30 con 0x556f51bcb1e0 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7facf9d7ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7face41aee30 con 0x556f51c5e720 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7facf9d7ab80 10 client.0 _ll_register_callbacks cb 0x556f519f2740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face9100700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51ae1520 0x556f51c70330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face9100700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51ae1520 0x556f51c70330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41856/0 (socket says 172.21.15.19:41856) 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face9100700 1 -- 172.21.15.19:0/2968664141 learned_addr learned my addr 172.21.15.19:0/2968664141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face88ff700 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51bcb1e0 0x556f51c70870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face9901700 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 0x556f51c6d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face9100700 1 -- 172.21.15.19:0/2968664141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 msgr2=0x556f51c6d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face9100700 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 0x556f51c6d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face9100700 1 -- 172.21.15.19:0/2968664141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51bcb1e0 msgr2=0x556f51c70870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face9100700 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51bcb1e0 0x556f51c70870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.229+0000 7face9100700 1 -- 172.21.15.19:0/2968664141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f51c6d9b0 con 0x556f51ae1520 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.230+0000 7face9100700 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51ae1520 0x556f51c70330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7facd400b010 tx=0x7facd4007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a01a4599bb085 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.230+0000 7facc8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.230+0000 7facc8ff9700 1 -- 172.21.15.19:0/2968664141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4032070 con 0x556f51ae1520 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.230+0000 7facc8ff9700 1 -- 172.21.15.19:0/2968664141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7facd4004be0 con 0x556f51ae1520 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.230+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2968664141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facdc003d70 con 0x556f51ae1520 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.230+0000 7facc8ff9700 1 -- 172.21.15.19:0/2968664141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7facd403e050 con 0x556f51ae1520 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.230+0000 7facf9d7ab80 10 client.16656 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.230+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2968664141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f51ce3440 con 0x556f51ae1520 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.230+0000 7facc8ff9700 1 -- 172.21.15.19:0/2968664141 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facdc003d70 con 0x556f51ae1520 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7facc8ff9700 1 -- 172.21.15.19:0/2968664141 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7facd40050d0 con 0x556f51ae1520 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7facf9d7ab80 10 client.16656 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:39.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7facf9d7ab80 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7facf9d7ab80 20 client.16656 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556f51ce38a0 0x556f51ce5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7facf9d7ab80 4 client.16656 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:30:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2968664141 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f51b88af0 con 0x556f51ce38a0 2024-05-09T01:30:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7face88ff700 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556f51ce38a0 0x556f51ce5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7face88ff700 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556f51ce38a0 0x556f51ce5cc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910e332032468795 server_cookie=f92eb202ec9acf9a in_seq=0 out_seq=0 2024-05-09T01:30:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.231+0000 7facc8ff9700 10 client.16656 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.232+0000 7facc8ff9700 1 -- 172.21.15.19:0/2968664141 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f51b88af0 con 0x556f51ce38a0 2024-05-09T01:30:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.232+0000 7facc8ff9700 10 client.16656 handle_command_reply: tid=0 2024-05-09T01:30:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.232+0000 7facf9d7ab80 1 client.16656 shutdown 2024-05-09T01:30:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2968664141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51ae1520 msgr2=0x556f51c70330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51ae1520 0x556f51c70330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7facd400b010 tx=0x7facd4007830 comp rx=0 tx=0).stop 2024-05-09T01:30:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2968664141 shutdown_connections 2024-05-09T01:30:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556f51ce38a0 0x556f51ce5cc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f51c5e720 0x556f51c6d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f51ae1520 0x556f51c70330 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facc8ff9700 0 client.16656 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 --2- 172.21.15.19:0/2968664141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f51bcb1e0 0x556f51c70870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2968664141 >> 172.21.15.19:0/2968664141 conn(0x556f51b41720 msgr2=0x556f51c68270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2968664141 shutdown_connections 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 1 -- 172.21.15.19:0/2968664141 wait complete. 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.233+0000 7facf9d7ab80 20 client.16656 trim_cache size 0 max 16384 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "914188bc-a02f-4ca1-8531-190c94cfc0b9", 2024-05-09T01:30:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:30:39.235 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:30:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 -- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faccc0b2880 msgr2=0x7faccc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faccc0b2880 0x7faccc0b4d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7facd4009550 tx=0x7facd40091f0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 -- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4108f20 msgr2=0x7face419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4108f20 0x7face419d660 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7facd800e970 tx=0x7facd800ec80 comp rx=0 tx=0).stop 2024-05-09T01:30:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 -- 172.21.15.19:0/3041918643 shutdown_connections 2024-05-09T01:30:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face4109870 0x7face41a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7face41045f0 0x7face419d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7faccc0b2880 0x7faccc0b4d90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 --2- 172.21.15.19:0/3041918643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face4108f20 0x7face419d660 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 -- 172.21.15.19:0/3041918643 >> 172.21.15.19:0/3041918643 conn(0x7face4100040 msgr2=0x7face410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 -- 172.21.15.19:0/3041918643 shutdown_connections 2024-05-09T01:30:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:39.234+0000 7faceb364700 1 -- 172.21.15.19:0/3041918643 wait complete. 2024-05-09T01:30:40.058 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:30:40.310 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:40.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:40 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:30:40.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:40 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:40.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:40 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:30:40.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:40 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:40.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:40 smithi019 ceph-mon[30158]: pgmap v532: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:30:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:40 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:30:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:40 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:40 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:30:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:40 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:40 smithi196 ceph-mon[35041]: pgmap v532: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:30:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:40 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:30:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:40 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:40 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:30:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:40 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:30:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:40 smithi059 ceph-mon[34351]: pgmap v532: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:30:42.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.158+0000 7ff8cc978700 1 -- 172.21.15.19:0/680985755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8108f00 msgr2=0x7ff8c8109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:42.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.158+0000 7ff8cc978700 1 --2- 172.21.15.19:0/680985755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8108f00 0x7ff8c8109310 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00099c0 tx=0x7ff8b0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.158+0000 7ff8cc978700 1 -- 172.21.15.19:0/680985755 shutdown_connections 2024-05-09T01:30:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.158+0000 7ff8cc978700 1 --2- 172.21.15.19:0/680985755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c8109850 0x7ff8c81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.158+0000 7ff8cc978700 1 --2- 172.21.15.19:0/680985755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8c81045d0 0x7ff8c81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.158+0000 7ff8cc978700 1 --2- 172.21.15.19:0/680985755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8108f00 0x7ff8c8109310 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.158+0000 7ff8cc978700 1 -- 172.21.15.19:0/680985755 >> 172.21.15.19:0/680985755 conn(0x7ff8c80ffea0 msgr2=0x7ff8c81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.158+0000 7ff8cc978700 1 -- 172.21.15.19:0/680985755 shutdown_connections 2024-05-09T01:30:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.158+0000 7ff8cc978700 1 -- 172.21.15.19:0/680985755 wait complete. 2024-05-09T01:30:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8cc978700 1 Processor -- start 2024-05-09T01:30:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8cc978700 1 -- start start 2024-05-09T01:30:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8cc978700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8c81045d0 0x7ff8c819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8cc978700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c8108f00 0x7ff8c819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8cc978700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8109850 0x7ff8c819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8cc978700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8114460 con 0x7ff8c8109850 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8cc978700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81142e0 con 0x7ff8c81045d0 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8cc978700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81145e0 con 0x7ff8c8108f00 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8c6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8109850 0x7ff8c819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8c6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8109850 0x7ff8c819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43954/0 (socket says 172.21.15.19:43954) 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.159+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/2320824082 learned_addr learned my addr 172.21.15.19:0/2320824082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8c659c700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8c81045d0 0x7ff8c819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8bdd9b700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c8108f00 0x7ff8c819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/2320824082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c8108f00 msgr2=0x7ff8c819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c8108f00 0x7ff8c819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/2320824082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8c81045d0 msgr2=0x7ff8c819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8c81045d0 0x7ff8c819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/2320824082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c819fc80 con 0x7ff8c8109850 2024-05-09T01:30:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8bdd9b700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c8108f00 0x7ff8c819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8109850 0x7ff8c819f5e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000d980 tx=0x7ff8c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8bffff700 1 -- 172.21.15.19:0/2320824082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0017070 con 0x7ff8c8109850 2024-05-09T01:30:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.160+0000 7ff8cc978700 1 -- 172.21.15.19:0/2320824082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c819ff70 con 0x7ff8c8109850 2024-05-09T01:30:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.161+0000 7ff8bffff700 1 -- 172.21.15.19:0/2320824082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c0005020 con 0x7ff8c8109850 2024-05-09T01:30:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.161+0000 7ff8cc978700 1 -- 172.21.15.19:0/2320824082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c81a7890 con 0x7ff8c8109850 2024-05-09T01:30:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.161+0000 7ff8bffff700 1 -- 172.21.15.19:0/2320824082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0009990 con 0x7ff8c8109850 2024-05-09T01:30:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.164+0000 7ff8bffff700 1 -- 172.21.15.19:0/2320824082 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff8c0009b30 con 0x7ff8c8109850 2024-05-09T01:30:42.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.165+0000 7ff8bffff700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff8b40b2880 0x7ff8b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.165+0000 7ff8c659c700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff8b40b2880 0x7ff8b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.165+0000 7ff8bffff700 1 -- 172.21.15.19:0/2320824082 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff8c00e1f70 con 0x7ff8c8109850 2024-05-09T01:30:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.165+0000 7ff8db38eb80 1 Processor -- start 2024-05-09T01:30:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8c659c700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff8b40b2880 0x7ff8b40b4d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009550 tx=0x7ff8b80091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8db38eb80 1 -- start start 2024-05-09T01:30:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 0x55a1e10f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0f724a0 0x55a1e10f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a1e10120a0 con 0x55a1e10f3810 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0fb79a0 con 0x55a1e0fecd00 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a1e100e060 con 0x55a1e0f724a0 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8c6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8c6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43968/0 (socket says 172.21.15.19:43968) 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 0x55a1e10f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/1512742290 learned_addr learned my addr 172.21.15.19:0/1512742290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.166+0000 7ff8bdd9b700 1 --2- 172.21.15.19:0/1512742290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0f724a0 0x55a1e10f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/1512742290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0f724a0 msgr2=0x55a1e10f9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/1512742290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0f724a0 0x55a1e10f9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/1512742290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 msgr2=0x55a1e10f9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/1512742290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 0x55a1e10f9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/1512742290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e0fa8500 con 0x55a1e10f3810 2024-05-09T01:30:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8c659c700 1 --2- 172.21.15.19:0/1512742290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 0x55a1e10f9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/1512742290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10f3c20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00a55d0 tx=0x7ff8c00a9c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2578d919afaeb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8bcd99700 1 -- 172.21.15.19:0/1512742290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00a9550 con 0x55a1e10f3810 2024-05-09T01:30:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8bcd99700 1 -- 172.21.15.19:0/1512742290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c00ad900 con 0x55a1e10f3810 2024-05-09T01:30:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.167+0000 7ff8bcd99700 1 -- 172.21.15.19:0/1512742290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00a8a10 con 0x55a1e10f3810 2024-05-09T01:30:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.168+0000 7ff8db38eb80 1 -- 172.21.15.19:0/1512742290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 msgr2=0x55a1e10f3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.168+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/1512742290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10f3c20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00a55d0 tx=0x7ff8c00a9c50 comp rx=0 tx=0).stop 2024-05-09T01:30:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.168+0000 7ff8db38eb80 1 -- 172.21.15.19:0/1512742290 shutdown_connections 2024-05-09T01:30:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.168+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/1512742290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10f3c20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.168+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/1512742290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 0x55a1e10f9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.168+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/1512742290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0f724a0 0x55a1e10f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.168+0000 7ff8db38eb80 1 -- 172.21.15.19:0/1512742290 >> 172.21.15.19:0/1512742290 conn(0x55a1e0f75f70 msgr2=0x55a1e10ea220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.168+0000 7ff8db38eb80 1 -- 172.21.15.19:0/1512742290 shutdown_connections 2024-05-09T01:30:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.168+0000 7ff8db38eb80 1 -- 172.21.15.19:0/1512742290 wait complete. 2024-05-09T01:30:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8db38eb80 1 -- start start 2024-05-09T01:30:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8db38eb80 1 Processor -- start 2024-05-09T01:30:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0f724a0 0x55a1e10fd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 0x55a1e10fda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a1e10f4570 con 0x55a1e10f3810 2024-05-09T01:30:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a1e0facb20 con 0x55a1e0f724a0 2024-05-09T01:30:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a1e1100430 con 0x55a1e0fecd00 2024-05-09T01:30:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8db38eb80 10 client.0 _ll_register_callbacks cb 0x55a1e0e836c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.169+0000 7ff8c6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8c6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10fdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43980/0 (socket says 172.21.15.19:43980) 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/213508330 learned_addr learned my addr 172.21.15.19:0/213508330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8c659c700 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0f724a0 0x55a1e10fd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8bdd9b700 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 0x55a1e10fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/213508330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 msgr2=0x55a1e10fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 0x55a1e10fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/213508330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0f724a0 msgr2=0x55a1e10fd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0f724a0 0x55a1e10fd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8c6d9d700 1 -- 172.21.15.19:0/213508330 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e0e8d490 con 0x55a1e10f3810 2024-05-09T01:30:42.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8bdd9b700 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 0x55a1e10fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10fdfa0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00a8e30 tx=0x7ff8c00b2150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e70abba54859f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8adffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8adffb700 1 -- 172.21.15.19:0/213508330 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c006e060 con 0x55a1e10f3810 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8adffb700 1 -- 172.21.15.19:0/213508330 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff8c0004c40 con 0x55a1e10f3810 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8db38eb80 1 -- 172.21.15.19:0/213508330 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8c00ad900 con 0x55a1e10f3810 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8adffb700 1 -- 172.21.15.19:0/213508330 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8c00a9550 con 0x55a1e10f3810 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8db38eb80 10 client.16668 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.170+0000 7ff8db38eb80 1 -- 172.21.15.19:0/213508330 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1e110aea0 con 0x55a1e10f3810 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.171+0000 7ff8adffb700 1 -- 172.21.15.19:0/213508330 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c00ad900 con 0x55a1e10f3810 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.171+0000 7ff8adffb700 1 -- 172.21.15.19:0/213508330 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff8c008fac0 con 0x55a1e10f3810 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.171+0000 7ff8db38eb80 10 client.16668 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.171+0000 7ff8db38eb80 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.171+0000 7ff8db38eb80 20 client.16668 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.171+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a1e110d370 0x55a1e110f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.171+0000 7ff8db38eb80 4 client.16668 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.171+0000 7ff8db38eb80 1 -- 172.21.15.19:0/213508330 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1e101fa80 con 0x55a1e110d370 2024-05-09T01:30:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.171+0000 7ff8c659c700 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a1e110d370 0x55a1e110f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.172+0000 7ff8c659c700 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a1e110d370 0x55a1e110f790 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b0c5fce469acb00 server_cookie=888eb5f7806097a9 in_seq=0 out_seq=0 2024-05-09T01:30:42.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.172+0000 7ff8adffb700 10 client.16668 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8adffb700 1 -- 172.21.15.19:0/213508330 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a1e101fa80 con 0x55a1e110d370 2024-05-09T01:30:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8adffb700 10 client.16668 handle_command_reply: tid=0 2024-05-09T01:30:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 client.16668 shutdown 2024-05-09T01:30:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 -- 172.21.15.19:0/213508330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 msgr2=0x55a1e10fdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10fdfa0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00a8e30 tx=0x7ff8c00b2150 comp rx=0 tx=0).stop 2024-05-09T01:30:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 -- 172.21.15.19:0/213508330 shutdown_connections 2024-05-09T01:30:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a1e110d370 0x55a1e110f790 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e10f3810 0x55a1e10fdfa0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0f724a0 0x55a1e10fd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8adffb700 0 client.16668 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/213508330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 0x55a1e10fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 -- 172.21.15.19:0/213508330 >> 172.21.15.19:0/213508330 conn(0x55a1e0f75f70 msgr2=0x55a1e10e3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 -- 172.21.15.19:0/213508330 shutdown_connections 2024-05-09T01:30:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 1 -- 172.21.15.19:0/213508330 wait complete. 2024-05-09T01:30:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.173+0000 7ff8db38eb80 20 client.16668 trim_cache size 0 max 16384 2024-05-09T01:30:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8db38eb80 1 Processor -- start 2024-05-09T01:30:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8db38eb80 1 -- start start 2024-05-09T01:30:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e10fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 0x55a1e10e4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e10f3810 0x55a1e10ff290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a800d3d0 con 0x55a1e0f724a0 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a800ef10 con 0x55a1e0fecd00 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a800daa0 con 0x55a1e10f3810 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e10fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e10fc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43992/0 (socket says 172.21.15.19:43992) 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8c659c700 1 -- 172.21.15.19:0/3560751855 learned_addr learned my addr 172.21.15.19:0/3560751855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8bdd9b700 1 --2- 172.21.15.19:0/3560751855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 0x55a1e10e4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/3560751855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e10f3810 0x55a1e10ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8c659c700 1 -- 172.21.15.19:0/3560751855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e10f3810 msgr2=0x55a1e10ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8c659c700 1 --2- 172.21.15.19:0/3560751855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e10f3810 0x55a1e10ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8c659c700 1 -- 172.21.15.19:0/3560751855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 msgr2=0x55a1e10e4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.194+0000 7ff8c659c700 1 --2- 172.21.15.19:0/3560751855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 0x55a1e10e4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.195+0000 7ff8c659c700 1 -- 172.21.15.19:0/3560751855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e10fba90 con 0x55a1e0f724a0 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.195+0000 7ff8c659c700 1 --2- 172.21.15.19:0/3560751855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e10fc4e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009690 tx=0x7ff8b8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978822d1772dac09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.195+0000 7ff8affff700 1 -- 172.21.15.19:0/3560751855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8031070 con 0x55a1e0f724a0 2024-05-09T01:30:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.195+0000 7ff8affff700 1 -- 172.21.15.19:0/3560751855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b801d040 con 0x55a1e0f724a0 2024-05-09T01:30:42.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.195+0000 7ff8affff700 1 -- 172.21.15.19:0/3560751855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b801c040 con 0x55a1e0f724a0 2024-05-09T01:30:42.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 -- 172.21.15.19:0/3560751855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 msgr2=0x55a1e10fc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:42.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/3560751855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e10fc4e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009690 tx=0x7ff8b8013000 comp rx=0 tx=0).stop 2024-05-09T01:30:42.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 -- 172.21.15.19:0/3560751855 shutdown_connections 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/3560751855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e10f3810 0x55a1e10ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/3560751855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e10fc4e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/3560751855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e0fecd00 0x55a1e10e4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 -- 172.21.15.19:0/3560751855 >> 172.21.15.19:0/3560751855 conn(0x55a1e0f75f70 msgr2=0x55a1e10feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 -- 172.21.15.19:0/3560751855 shutdown_connections 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 -- 172.21.15.19:0/3560751855 wait complete. 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 -- start start 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.196+0000 7ff8db38eb80 1 Processor -- start 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e1110460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 0x55a1e11109a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8db38eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e10f3810 0x55a1e1110ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a800e840 con 0x55a1e0f724a0 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a800e170 con 0x55a1e10f3810 2024-05-09T01:30:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8db38eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a800b370 con 0x55a1e0fecd00 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8db38eb80 10 client.0 _ll_register_callbacks cb 0x55a1e0e836c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e1110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e1110460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44002/0 (socket says 172.21.15.19:44002) 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8c659c700 1 -- 172.21.15.19:0/2930456157 learned_addr learned my addr 172.21.15.19:0/2930456157 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8bdd9b700 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 0x55a1e11109a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8c6d9d700 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e10f3810 0x55a1e1110ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8c659c700 1 -- 172.21.15.19:0/2930456157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 msgr2=0x55a1e11109a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8c659c700 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 0x55a1e11109a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8c659c700 1 -- 172.21.15.19:0/2930456157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e10f3810 msgr2=0x55a1e1110ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8c659c700 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e10f3810 0x55a1e1110ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.197+0000 7ff8c659c700 1 -- 172.21.15.19:0/2930456157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e1176910 con 0x55a1e0f724a0 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8c659c700 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e1110460 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8003d90 tx=0x7ff8b80244c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e307d666a3109ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8adffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8adffb700 1 -- 172.21.15.19:0/2930456157 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b801c070 con 0x55a1e0f724a0 2024-05-09T01:30:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8adffb700 1 -- 172.21.15.19:0/2930456157 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff8b802b2f0 con 0x55a1e0f724a0 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8db38eb80 1 -- 172.21.15.19:0/2930456157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b801d040 con 0x55a1e0f724a0 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8adffb700 1 -- 172.21.15.19:0/2930456157 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8b8031070 con 0x55a1e0f724a0 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8db38eb80 10 client.16674 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8db38eb80 1 -- 172.21.15.19:0/2930456157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1e1177b80 con 0x55a1e0f724a0 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8adffb700 1 -- 172.21.15.19:0/2930456157 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b801d040 con 0x55a1e0f724a0 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8adffb700 1 -- 172.21.15.19:0/2930456157 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff8b802bce0 con 0x55a1e0f724a0 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8db38eb80 10 client.16674 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.198+0000 7ff8db38eb80 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.199+0000 7ff8db38eb80 20 client.16674 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.199+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a1e11781e0 0x55a1e117a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.199+0000 7ff8db38eb80 4 client.16674 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.199+0000 7ff8db38eb80 1 -- 172.21.15.19:0/2930456157 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a1e1019b20 con 0x55a1e11781e0 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.199+0000 7ff8bdd9b700 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a1e11781e0 0x55a1e117a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.199+0000 7ff8bdd9b700 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a1e11781e0 0x55a1e117a600 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdb4bfe6547e35a server_cookie=d8e2af3ab1561b in_seq=0 out_seq=0 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.199+0000 7ff8adffb700 10 client.16674 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8adffb700 1 -- 172.21.15.19:0/2930456157 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a1e1019b20 con 0x55a1e11781e0 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8adffb700 10 client.16674 handle_command_reply: tid=0 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 client.16674 shutdown 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 -- 172.21.15.19:0/2930456157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 msgr2=0x55a1e1110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e1110460 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8003d90 tx=0x7ff8b80244c0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 -- 172.21.15.19:0/2930456157 shutdown_connections 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a1e11781e0 0x55a1e117a600 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1e10f3810 0x55a1e1110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1e0f724a0 0x55a1e1110460 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8adffb700 0 client.16674 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 --2- 172.21.15.19:0/2930456157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1e0fecd00 0x55a1e11109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 -- 172.21.15.19:0/2930456157 >> 172.21.15.19:0/2930456157 conn(0x55a1e0f75f70 msgr2=0x55a1e10e4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 -- 172.21.15.19:0/2930456157 shutdown_connections 2024-05-09T01:30:42.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 1 -- 172.21.15.19:0/2930456157 wait complete. 2024-05-09T01:30:42.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.200+0000 7ff8db38eb80 20 client.16674 trim_cache size 0 max 16384 2024-05-09T01:30:42.202 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:30:42.202 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:30:42.202 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:30:42.202 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 -- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff8b40b2880 msgr2=0x7ff8b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff8b40b2880 0x7ff8b40b4d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009550 tx=0x7ff8b80091b0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 -- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8109850 msgr2=0x7ff8c819f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8109850 0x7ff8c819f5e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000d980 tx=0x7ff8c000dc90 comp rx=0 tx=0).stop 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 -- 172.21.15.19:0/2320824082 shutdown_connections 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff8b40b2880 0x7ff8b40b4d90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8c8109850 0x7ff8c819f5e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8c81045d0 0x7ff8c819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 --2- 172.21.15.19:0/2320824082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c8108f00 0x7ff8c819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 -- 172.21.15.19:0/2320824082 >> 172.21.15.19:0/2320824082 conn(0x7ff8c80ffea0 msgr2=0x7ff8c810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 -- 172.21.15.19:0/2320824082 shutdown_connections 2024-05-09T01:30:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:42.201+0000 7ff8cc978700 1 -- 172.21.15.19:0/2320824082 wait complete. 2024-05-09T01:30:42.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:42 smithi019 ceph-mon[30158]: pgmap v533: 97 pgs: 97 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:42.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:42 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:30:42.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:42 smithi196 ceph-mon[35041]: pgmap v533: 97 pgs: 97 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:42.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:42 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:30:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:42 smithi059 ceph-mon[34351]: pgmap v533: 97 pgs: 97 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:42 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:30:42.933 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:30:42.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:30:43.189 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:44 smithi019 ceph-mon[30158]: pgmap v534: 97 pgs: 97 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:44.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:44 smithi196 ceph-mon[35041]: pgmap v534: 97 pgs: 97 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:44 smithi059 ceph-mon[34351]: pgmap v534: 97 pgs: 97 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.966+0000 7fb561869700 1 -- 172.21.15.19:0/604385289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c1034c0 msgr2=0x7fb55c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.966+0000 7fb561869700 1 --2- 172.21.15.19:0/604385289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c1034c0 0x7fb55c1038d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0099c0 tx=0x7fb54c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.966+0000 7fb561869700 1 -- 172.21.15.19:0/604385289 shutdown_connections 2024-05-09T01:30:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.966+0000 7fb561869700 1 --2- 172.21.15.19:0/604385289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb55c108740 0x7fb55c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.966+0000 7fb561869700 1 --2- 172.21.15.19:0/604385289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c1034c0 0x7fb55c1038d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.966+0000 7fb561869700 1 --2- 172.21.15.19:0/604385289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb55c107df0 0x7fb55c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.966+0000 7fb561869700 1 -- 172.21.15.19:0/604385289 >> 172.21.15.19:0/604385289 conn(0x7fb55c0fedf0 msgr2=0x7fb55c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.966+0000 7fb561869700 1 -- 172.21.15.19:0/604385289 shutdown_connections 2024-05-09T01:30:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.966+0000 7fb561869700 1 -- 172.21.15.19:0/604385289 wait complete. 2024-05-09T01:30:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.967+0000 7fb561869700 1 Processor -- start 2024-05-09T01:30:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.967+0000 7fb561869700 1 -- start start 2024-05-09T01:30:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb561869700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb55c1034c0 0x7fb55c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb561869700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb55c107df0 0x7fb55c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb561869700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c108740 0x7fb55c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb561869700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c113370 con 0x7fb55c108740 2024-05-09T01:30:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb561869700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1131f0 con 0x7fb55c107df0 2024-05-09T01:30:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb561869700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1134f0 con 0x7fb55c1034c0 2024-05-09T01:30:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb55b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c108740 0x7fb55c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb55b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c108740 0x7fb55c0ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44028/0 (socket says 172.21.15.19:44028) 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb55affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb55c1034c0 0x7fb55c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb55b7fe700 1 -- 172.21.15.19:0/3105924422 learned_addr learned my addr 172.21.15.19:0/3105924422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb5527fc700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb55c107df0 0x7fb55c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb55b7fe700 1 -- 172.21.15.19:0/3105924422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb55c1034c0 msgr2=0x7fb55c0ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb55b7fe700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb55c1034c0 0x7fb55c0ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb55b7fe700 1 -- 172.21.15.19:0/3105924422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb55c107df0 msgr2=0x7fb55c0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb55b7fe700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb55c107df0 0x7fb55c0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb55b7fe700 1 -- 172.21.15.19:0/3105924422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55c1a91d0 con 0x7fb55c108740 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.968+0000 7fb5527fc700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb55c107df0 0x7fb55c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.969+0000 7fb55b7fe700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c108740 0x7fb55c0ffd50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb554007ee0 tx=0x7fb55400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.969+0000 7fb560867700 1 -- 172.21.15.19:0/3105924422 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55400dbd0 con 0x7fb55c108740 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.969+0000 7fb560867700 1 -- 172.21.15.19:0/3105924422 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb55400dd70 con 0x7fb55c108740 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.969+0000 7fb560867700 1 -- 172.21.15.19:0/3105924422 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554005990 con 0x7fb55c108740 2024-05-09T01:30:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.969+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55c1a94c0 con 0x7fb55c108740 2024-05-09T01:30:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.969+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55c1a99b0 con 0x7fb55c108740 2024-05-09T01:30:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.971+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55c0512c0 con 0x7fb55c108740 2024-05-09T01:30:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.972+0000 7fb560867700 1 -- 172.21.15.19:0/3105924422 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb55401a020 con 0x7fb55c108740 2024-05-09T01:30:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.972+0000 7fb560867700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5480b2880 0x7fb5480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.973+0000 7fb55affd700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5480b2880 0x7fb5480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.973+0000 7fb560867700 1 -- 172.21.15.19:0/3105924422 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb5540e1970 con 0x7fb55c108740 2024-05-09T01:30:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.973+0000 7fb55affd700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5480b2880 0x7fb5480b4d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb54c000c00 tx=0x7fb54c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:44.978+0000 7fb560867700 1 -- 172.21.15.19:0/3105924422 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb554010070 con 0x7fb55c108740 2024-05-09T01:30:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 -- 172.21.15.19:0/3062365926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781070a0 msgr2=0x7f02781094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 --2- 172.21.15.19:0/3062365926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781070a0 0x7f02781094c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0268009900 tx=0x7f026802a410 comp rx=0 tx=0).stop 2024-05-09T01:30:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 -- 172.21.15.19:0/3062365926 shutdown_connections 2024-05-09T01:30:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 --2- 172.21.15.19:0/3062365926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0278109a00 0x7f027810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 --2- 172.21.15.19:0/3062365926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02781070a0 0x7f02781094c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 --2- 172.21.15.19:0/3062365926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278104740 0x7f0278106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 -- 172.21.15.19:0/3062365926 >> 172.21.15.19:0/3062365926 conn(0x7f02780fdd40 msgr2=0x7f02781001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 -- 172.21.15.19:0/3062365926 shutdown_connections 2024-05-09T01:30:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 -- 172.21.15.19:0/3062365926 wait complete. 2024-05-09T01:30:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.050+0000 7f027f6bd640 1 Processor -- start 2024-05-09T01:30:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027f6bd640 1 -- start start 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027f6bd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278104740 0x7f027819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027f6bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02781070a0 0x7f027819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027f6bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278109a00 0x7f02781a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027f6bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f027810f780 con 0x7f0278109a00 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027f6bd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f027810f600 con 0x7f0278104740 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027f6bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f027810f900 con 0x7f02781070a0 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027dc5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278109a00 0x7f02781a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027dc5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278109a00 0x7f02781a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44058/0 (socket says 172.21.15.19:44058) 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027dc5a640 1 -- 172.21.15.19:0/1943653147 learned_addr learned my addr 172.21.15.19:0/1943653147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027cc58640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02781070a0 0x7f027819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027d459640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278104740 0x7f027819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027dc5a640 1 -- 172.21.15.19:0/1943653147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02781070a0 msgr2=0x7f027819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027dc5a640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02781070a0 0x7f027819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027dc5a640 1 -- 172.21.15.19:0/1943653147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278104740 msgr2=0x7f027819a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027dc5a640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278104740 0x7f027819a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027dc5a640 1 -- 172.21.15.19:0/1943653147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02781a1ef0 con 0x7f0278109a00 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.051+0000 7f027dc5a640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278109a00 0x7f02781a1880 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f027000eb40 tx=0x7f027000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.052+0000 7f026e7fc640 1 -- 172.21.15.19:0/1943653147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270004b20 con 0x7f0278109a00 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.052+0000 7f027f6bd640 1 -- 172.21.15.19:0/1943653147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02781a21e0 con 0x7f0278109a00 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.052+0000 7f026e7fc640 1 -- 172.21.15.19:0/1943653147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0270004cc0 con 0x7f0278109a00 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.052+0000 7f026e7fc640 1 -- 172.21.15.19:0/1943653147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270010720 con 0x7f0278109a00 2024-05-09T01:30:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.052+0000 7f027f6bd640 1 -- 172.21.15.19:0/1943653147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02781a2730 con 0x7f0278109a00 2024-05-09T01:30:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f028657a740 1 Processor -- start 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f026e7fc640 1 -- 172.21.15.19:0/1943653147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0270005560 con 0x7f0278109a00 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f028657a740 1 -- start start 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f028657a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d6c1b0 0x55a62a110500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f028657a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a110a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f028657a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a117980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f028657a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a62a0747c0 con 0x55a629d432f0 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f028657a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a62a0100e0 con 0x55a629d4b4d0 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f028657a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a62a070780 con 0x55a629d6c1b0 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f027d459640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f027d459640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44062/0 (socket says 172.21.15.19:44062) 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.055+0000 7f027d459640 1 -- 172.21.15.19:0/2787909847 learned_addr learned my addr 172.21.15.19:0/2787909847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027cc58640 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a117980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027cc58640 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d6c1b0 0x55a62a110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027d459640 1 -- 172.21.15.19:0/2787909847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d6c1b0 msgr2=0x55a62a110500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027d459640 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d6c1b0 0x55a62a110500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027d459640 1 -- 172.21.15.19:0/2787909847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 msgr2=0x55a62a117980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027d459640 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a117980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027d459640 1 -- 172.21.15.19:0/2787909847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62a0397c0 con 0x55a629d432f0 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027cc58640 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a117980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f026e7fc640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02640b27e0 0x7f02640b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027cc58640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02640b27e0 0x7f02640b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f026e7fc640 1 -- 172.21.15.19:0/1943653147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f02700e2a10 con 0x7f0278109a00 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f027d459640 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a110a80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0274009c20 tx=0x7f02740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630c33e8e018a88c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f02637fe640 1 -- 172.21.15.19:0/2787909847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274012450 con 0x55a629d432f0 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f02637fe640 1 -- 172.21.15.19:0/2787909847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0274012aa0 con 0x55a629d432f0 2024-05-09T01:30:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.056+0000 7f02637fe640 1 -- 172.21.15.19:0/2787909847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027400fa70 con 0x55a629d432f0 2024-05-09T01:30:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f028657a740 1 -- 172.21.15.19:0/2787909847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 msgr2=0x55a62a110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f027cc58640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02640b27e0 0x7f02640b4cf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0268009230 tx=0x7f026803f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f028657a740 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a110a80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0274009c20 tx=0x7f02740117b0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f028657a740 1 -- 172.21.15.19:0/2787909847 shutdown_connections 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f028657a740 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a117980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f028657a740 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a110a80 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f028657a740 1 --2- 172.21.15.19:0/2787909847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d6c1b0 0x55a62a110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f028657a740 1 -- 172.21.15.19:0/2787909847 >> 172.21.15.19:0/2787909847 conn(0x55a629d60300 msgr2=0x55a62a10be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f028657a740 1 -- 172.21.15.19:0/2787909847 shutdown_connections 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.057+0000 7f028657a740 1 -- 172.21.15.19:0/2787909847 wait complete. 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f028657a740 1 -- start start 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f028657a740 1 Processor -- start 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f028657a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a11a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f028657a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a11a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f028657a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d6c1b0 0x55a62a121670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f028657a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a62a1183d0 con 0x55a629d432f0 2024-05-09T01:30:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f028657a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a629ffb9c0 con 0x55a629d4b4d0 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f028657a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a62a118550 con 0x55a629d6c1b0 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f028657a740 10 client.0 _ll_register_callbacks cb 0x55a62a028020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f027d459640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a11a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f027d459640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a11a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44076/0 (socket says 172.21.15.19:44076) 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.058+0000 7f027d459640 1 -- 172.21.15.19:0/1844819365 learned_addr learned my addr 172.21.15.19:0/1844819365 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f027cc58640 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a11a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f027d459640 1 -- 172.21.15.19:0/1844819365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d6c1b0 msgr2=0x55a62a121670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f027d459640 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d6c1b0 0x55a62a121670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f027d459640 1 -- 172.21.15.19:0/1844819365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 msgr2=0x55a62a11a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f027d459640 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a11a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f027d459640 1 -- 172.21.15.19:0/1844819365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62a0221f0 con 0x55a629d432f0 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f027cc58640 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a11a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f027d459640 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a11a1d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f027401c040 tx=0x7f0274012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f0a6cf26ed7433 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f0260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f0260ff9640 1 -- 172.21.15.19:0/1844819365 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274028020 con 0x55a629d432f0 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f0260ff9640 1 -- 172.21.15.19:0/1844819365 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0274012650 con 0x55a629d432f0 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f028657a740 1 -- 172.21.15.19:0/1844819365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0274012aa0 con 0x55a629d432f0 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f0260ff9640 1 -- 172.21.15.19:0/1844819365 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0274033050 con 0x55a629d432f0 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f028657a740 10 client.16692 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.059+0000 7f028657a740 1 -- 172.21.15.19:0/1844819365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a629dd7790 con 0x55a629d432f0 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.060+0000 7f0260ff9640 1 -- 172.21.15.19:0/1844819365 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0274012aa0 con 0x55a629d432f0 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.060+0000 7f0260ff9640 1 -- 172.21.15.19:0/1844819365 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f027400fb20 con 0x55a629d432f0 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.060+0000 7f028657a740 10 client.16692 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.060+0000 7f028657a740 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.060+0000 7f028657a740 20 client.16692 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.060+0000 7f028657a740 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a629d2ad80 0x55a62a1278a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.060+0000 7f028657a740 4 client.16692 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.060+0000 7f028657a740 1 -- 172.21.15.19:0/1844819365 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a62a0823d0 con 0x55a629d2ad80 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.060+0000 7f027cc58640 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a629d2ad80 0x55a62a1278a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.061+0000 7f027cc58640 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a629d2ad80 0x55a62a1278a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba5f304e95639a5 server_cookie=8c3e71fb7e630099 in_seq=0 out_seq=0 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.061+0000 7f0260ff9640 10 client.16692 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.061+0000 7f0260ff9640 1 -- 172.21.15.19:0/1844819365 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a62a0823d0 con 0x55a629d2ad80 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.061+0000 7f0260ff9640 10 client.16692 handle_command_reply: tid=0 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.061+0000 7f028657a740 1 client.16692 shutdown 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 -- 172.21.15.19:0/1844819365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 msgr2=0x55a62a11a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a11a1d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f027401c040 tx=0x7f0274012f60 comp rx=0 tx=0).stop 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 -- 172.21.15.19:0/1844819365 shutdown_connections 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a629d2ad80 0x55a62a1278a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d6c1b0 0x55a62a121670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a11a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f0260ff9640 0 client.16692 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 --2- 172.21.15.19:0/1844819365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a11a1d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 -- 172.21.15.19:0/1844819365 >> 172.21.15.19:0/1844819365 conn(0x55a629d60300 msgr2=0x55a62a10a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 -- 172.21.15.19:0/1844819365 shutdown_connections 2024-05-09T01:30:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 1 -- 172.21.15.19:0/1844819365 wait complete. 2024-05-09T01:30:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.062+0000 7f028657a740 20 client.16692 trim_cache size 0 max 16384 2024-05-09T01:30:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f028657a740 1 Processor -- start 2024-05-09T01:30:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f028657a740 1 -- start start 2024-05-09T01:30:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f028657a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a113c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f028657a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d4b4d0 0x55a62a119da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f028657a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d6c1b0 0x55a62a11a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f028657a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02781ab4c0 con 0x55a629d432f0 2024-05-09T01:30:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f028657a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02781ad000 con 0x55a629d6c1b0 2024-05-09T01:30:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f028657a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02781abb90 con 0x55a629d4b4d0 2024-05-09T01:30:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f027d459640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a113c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f027d459640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a113c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44080/0 (socket says 172.21.15.19:44080) 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f027d459640 1 -- 172.21.15.19:0/796293518 learned_addr learned my addr 172.21.15.19:0/796293518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f027dc5a640 1 --2- 172.21.15.19:0/796293518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d6c1b0 0x55a62a11a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f027d459640 1 -- 172.21.15.19:0/796293518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d4b4d0 msgr2=0x55a62a119da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f027d459640 1 --2- 172.21.15.19:0/796293518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d4b4d0 0x55a62a119da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f027d459640 1 -- 172.21.15.19:0/796293518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d6c1b0 msgr2=0x55a62a11a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.077+0000 7f027d459640 1 --2- 172.21.15.19:0/796293518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d6c1b0 0x55a62a11a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.078+0000 7f027d459640 1 -- 172.21.15.19:0/796293518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a629e4e5f0 con 0x55a629d432f0 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.078+0000 7f027d459640 1 --2- 172.21.15.19:0/796293518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a113c80 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f02740099e0 tx=0x7f0274009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7ebe48cc051c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.078+0000 7f0262ffd640 1 -- 172.21.15.19:0/796293518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274028070 con 0x55a629d432f0 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.078+0000 7f0262ffd640 1 -- 172.21.15.19:0/796293518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0274004490 con 0x55a629d432f0 2024-05-09T01:30:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.078+0000 7f0262ffd640 1 -- 172.21.15.19:0/796293518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027400fa50 con 0x55a629d432f0 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 -- 172.21.15.19:0/796293518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 msgr2=0x55a62a113c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 --2- 172.21.15.19:0/796293518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a113c80 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f02740099e0 tx=0x7f0274009690 comp rx=0 tx=0).stop 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 -- 172.21.15.19:0/796293518 shutdown_connections 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 --2- 172.21.15.19:0/796293518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d6c1b0 0x55a62a11a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 --2- 172.21.15.19:0/796293518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d4b4d0 0x55a62a119da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 --2- 172.21.15.19:0/796293518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d432f0 0x55a62a113c80 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 -- 172.21.15.19:0/796293518 >> 172.21.15.19:0/796293518 conn(0x55a629d60300 msgr2=0x55a62a113570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 -- 172.21.15.19:0/796293518 shutdown_connections 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 -- 172.21.15.19:0/796293518 wait complete. 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 -- start start 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.079+0000 7f028657a740 1 Processor -- start 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f028657a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d432f0 0x55a62a11de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f028657a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a11e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f028657a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d6c1b0 0x55a62a11f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f028657a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02781ac930 con 0x55a629d6c1b0 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f028657a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02781ac260 con 0x55a629d4b4d0 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f028657a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02781a94a0 con 0x55a629d432f0 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f028657a740 10 client.0 _ll_register_callbacks cb 0x7f0274009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027dc5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d6c1b0 0x55a62a11f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027dc5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d6c1b0 0x55a62a11f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44096/0 (socket says 172.21.15.19:44096) 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027dc5a640 1 -- 172.21.15.19:0/1447733134 learned_addr learned my addr 172.21.15.19:0/1447733134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027cc58640 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a11e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027d459640 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d432f0 0x55a62a11de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027dc5a640 1 -- 172.21.15.19:0/1447733134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d432f0 msgr2=0x55a62a11de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027dc5a640 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d432f0 0x55a62a11de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027dc5a640 1 -- 172.21.15.19:0/1447733134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 msgr2=0x55a62a11e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027dc5a640 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a11e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027dc5a640 1 -- 172.21.15.19:0/1447733134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62a038ed0 con 0x55a629d6c1b0 2024-05-09T01:30:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f027dc5a640 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d6c1b0 0x55a62a11f250 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f02700a9d10 tx=0x7f02700aa630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d329a21057fab35c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f0260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f0260ff9640 1 -- 172.21.15.19:0/1447733134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700c3070 con 0x55a629d6c1b0 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f0260ff9640 1 -- 172.21.15.19:0/1447733134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f02700a9480 con 0x55a629d6c1b0 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.080+0000 7f028657a740 1 -- 172.21.15.19:0/1447733134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0274004490 con 0x55a629d6c1b0 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f0260ff9640 1 -- 172.21.15.19:0/1447733134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f02700aa710 con 0x55a629d6c1b0 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f028657a740 10 client.16698 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f028657a740 1 -- 172.21.15.19:0/1447733134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a62a1209a0 con 0x55a629d6c1b0 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f0260ff9640 1 -- 172.21.15.19:0/1447733134 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0274004490 con 0x55a629d6c1b0 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f0260ff9640 1 -- 172.21.15.19:0/1447733134 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f02700a8ce0 con 0x55a629d6c1b0 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f028657a740 10 client.16698 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f028657a740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f028657a740 20 client.16698 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f028657a740 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a629d001c0 0x55a62a191d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f028657a740 4 client.16698 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:30:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f028657a740 1 -- 172.21.15.19:0/1447733134 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a62a07c240 con 0x55a629d001c0 2024-05-09T01:30:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.081+0000 7f027d459640 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a629d001c0 0x55a62a191d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.082+0000 7f0260ff9640 10 client.16698 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.082+0000 7f027d459640 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a629d001c0 0x55a62a191d50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b419c9dd41c37875 server_cookie=73b9d2020a2f7987 in_seq=0 out_seq=0 2024-05-09T01:30:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.097+0000 7f0260ff9640 1 -- 172.21.15.19:0/1447733134 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a62a07c240 con 0x55a629d001c0 2024-05-09T01:30:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.097+0000 7f0260ff9640 10 client.16698 handle_command_reply: tid=0 2024-05-09T01:30:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.097+0000 7f028657a740 1 client.16698 shutdown 2024-05-09T01:30:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 -- 172.21.15.19:0/1447733134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d6c1b0 msgr2=0x55a62a11f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d6c1b0 0x55a62a11f250 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f02700a9d10 tx=0x7f02700aa630 comp rx=0 tx=0).stop 2024-05-09T01:30:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 -- 172.21.15.19:0/1447733134 shutdown_connections 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a629d001c0 0x55a62a191d50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a629d6c1b0 0x55a62a11f250 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f0260ff9640 0 client.16698 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a629d4b4d0 0x55a62a11e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 --2- 172.21.15.19:0/1447733134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a629d432f0 0x55a62a11de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 -- 172.21.15.19:0/1447733134 >> 172.21.15.19:0/1447733134 conn(0x55a629d60300 msgr2=0x55a62a119d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 -- 172.21.15.19:0/1447733134 shutdown_connections 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 1 -- 172.21.15.19:0/1447733134 wait complete. 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f028657a740 20 client.16698 trim_cache size 0 max 16384 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:30:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:30:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f027f6bd640 1 -- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02640b27e0 msgr2=0x7f02640b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f027f6bd640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02640b27e0 0x7f02640b4cf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0268009230 tx=0x7f026803f040 comp rx=0 tx=0).stop 2024-05-09T01:30:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f027f6bd640 1 -- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278109a00 msgr2=0x7f02781a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.098+0000 7f027f6bd640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278109a00 0x7f02781a1880 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f027000eb40 tx=0x7f027000c2d0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.099+0000 7f027f6bd640 1 -- 172.21.15.19:0/1943653147 shutdown_connections 2024-05-09T01:30:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.099+0000 7f027f6bd640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f02640b27e0 0x7f02640b4cf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.099+0000 7f027f6bd640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0278109a00 0x7f02781a1880 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.099+0000 7f027f6bd640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02781070a0 0x7f027819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.099+0000 7f027f6bd640 1 --2- 172.21.15.19:0/1943653147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278104740 0x7f027819a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.099+0000 7f027f6bd640 1 -- 172.21.15.19:0/1943653147 >> 172.21.15.19:0/1943653147 conn(0x7f02780fdd40 msgr2=0x7f0278105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.099+0000 7f027f6bd640 1 -- 172.21.15.19:0/1943653147 shutdown_connections 2024-05-09T01:30:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:45.099+0000 7f027f6bd640 1 -- 172.21.15.19:0/1943653147 wait complete. 2024-05-09T01:30:45.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.248+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb55c02e5f0 con 0x7fb55c108740 2024-05-09T01:30:45.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.249+0000 7fb560867700 1 -- 172.21.15.19:0/3105924422 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fb5540a9670 con 0x7fb55c108740 2024-05-09T01:30:45.246 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:30:45.247 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.251+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5480b2880 msgr2=0x7fb5480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.251+0000 7fb561869700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5480b2880 0x7fb5480b4d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb54c000c00 tx=0x7fb54c01a040 comp rx=0 tx=0).stop 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.251+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c108740 msgr2=0x7fb55c0ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.251+0000 7fb561869700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c108740 0x7fb55c0ffd50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb554007ee0 tx=0x7fb55400d3b0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.252+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 shutdown_connections 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.252+0000 7fb561869700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5480b2880 0x7fb5480b4d90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.252+0000 7fb561869700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb55c108740 0x7fb55c0ffd50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.252+0000 7fb561869700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb55c1034c0 0x7fb55c0ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.252+0000 7fb561869700 1 --2- 172.21.15.19:0/3105924422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb55c107df0 0x7fb55c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.252+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 >> 172.21.15.19:0/3105924422 conn(0x7fb55c0fedf0 msgr2=0x7fb55c075df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:45.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.252+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 shutdown_connections 2024-05-09T01:30:45.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:45.252+0000 7fb561869700 1 -- 172.21.15.19:0/3105924422 wait complete. 2024-05-09T01:30:45.250 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:30:45.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:45 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3105924422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:30:45.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:45 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3105924422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:30:45.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:45 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3105924422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:30:45.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:30:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:46.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:46 smithi019 ceph-mon[30158]: pgmap v535: 97 pgs: 97 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:46.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:46 smithi196 ceph-mon[35041]: pgmap v535: 97 pgs: 97 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:46 smithi059 ceph-mon[34351]: pgmap v535: 97 pgs: 97 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.050+0000 7fead8fd3700 1 -- 172.21.15.19:0/1209301633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 msgr2=0x7fead41049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.050+0000 7fead8fd3700 1 --2- 172.21.15.19:0/1209301633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 0x7fead41049e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7feac400b0d0 tx=0x7feac400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.051+0000 7fead8fd3700 1 -- 172.21.15.19:0/1209301633 shutdown_connections 2024-05-09T01:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.051+0000 7fead8fd3700 1 --2- 172.21.15.19:0/1209301633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead4109850 0x7fead41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.051+0000 7fead8fd3700 1 --2- 172.21.15.19:0/1209301633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 0x7fead41049e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.051+0000 7fead8fd3700 1 --2- 172.21.15.19:0/1209301633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead4108f00 0x7fead4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.051+0000 7fead8fd3700 1 -- 172.21.15.19:0/1209301633 >> 172.21.15.19:0/1209301633 conn(0x7fead40ffea0 msgr2=0x7fead41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.051+0000 7fead8fd3700 1 -- 172.21.15.19:0/1209301633 shutdown_connections 2024-05-09T01:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.051+0000 7fead8fd3700 1 -- 172.21.15.19:0/1209301633 wait complete. 2024-05-09T01:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.051+0000 7fead8fd3700 1 Processor -- start 2024-05-09T01:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead8fd3700 1 -- start start 2024-05-09T01:30:48.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead8fd3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 0x7fead4076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead8fd3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead4108f00 0x7fead4077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead8fd3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead4109850 0x7fead4077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead8fd3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fead4114480 con 0x7fead41045d0 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead8fd3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fead4114300 con 0x7fead4109850 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead8fd3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fead4114600 con 0x7fead4108f00 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 0x7fead4076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 0x7fead4076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44118/0 (socket says 172.21.15.19:44118) 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead259c700 1 -- 172.21.15.19:0/549580196 learned_addr learned my addr 172.21.15.19:0/549580196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead1d9b700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead4108f00 0x7fead4077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead2d9d700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead4109850 0x7fead4077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead259c700 1 -- 172.21.15.19:0/549580196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead4108f00 msgr2=0x7fead4077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead259c700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead4108f00 0x7fead4077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead259c700 1 -- 172.21.15.19:0/549580196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead4109850 msgr2=0x7fead4077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.052+0000 7fead259c700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead4109850 0x7fead4077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.053+0000 7fead259c700 1 -- 172.21.15.19:0/549580196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead41a91e0 con 0x7fead41045d0 2024-05-09T01:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.053+0000 7fead2d9d700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead4109850 0x7fead4077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.053+0000 7fead259c700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 0x7fead4076bd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7feac400d010 tx=0x7feac4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.053+0000 7feacb7fe700 1 -- 172.21.15.19:0/549580196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac401f070 con 0x7fead41045d0 2024-05-09T01:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.053+0000 7fead8fd3700 1 -- 172.21.15.19:0/549580196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead41a9470 con 0x7fead41045d0 2024-05-09T01:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.053+0000 7feacb7fe700 1 -- 172.21.15.19:0/549580196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac4003c00 con 0x7fead41045d0 2024-05-09T01:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.053+0000 7feacb7fe700 1 -- 172.21.15.19:0/549580196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac401b6a0 con 0x7fead41045d0 2024-05-09T01:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.053+0000 7fead8fd3700 1 -- 172.21.15.19:0/549580196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead41a9920 con 0x7fead41045d0 2024-05-09T01:30:48.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.056+0000 7feacb7fe700 1 -- 172.21.15.19:0/549580196 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7feac4022070 con 0x7fead41045d0 2024-05-09T01:30:48.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.057+0000 7feacb7fe700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feac00b2880 0x7feac00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.057+0000 7fead1d9b700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feac00b2880 0x7feac00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.057+0000 7feacb7fe700 1 -- 172.21.15.19:0/549580196 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7feac400f020 con 0x7fead41045d0 2024-05-09T01:30:48.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.057+0000 7feae79e9b80 1 Processor -- start 2024-05-09T01:30:48.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.057+0000 7fead1d9b700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feac00b2880 0x7feac00b4d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fead4103520 tx=0x7feabc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.057+0000 7feae79e9b80 1 -- start start 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb075804f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 0x55bb07580a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 0x55bb0757aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7feae79e9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb0749cff0 con 0x55bb074e7160 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7feae79e9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bb0743b7f0 con 0x55bb0757a6a0 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7feae79e9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb07498fb0 con 0x55bb073fd4a0 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb075804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb075804f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44130/0 (socket says 172.21.15.19:44130) 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead259c700 1 -- 172.21.15.19:0/579090749 learned_addr learned my addr 172.21.15.19:0/579090749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead1d9b700 1 --2- 172.21.15.19:0/579090749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 0x55bb07580a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead2d9d700 1 --2- 172.21.15.19:0/579090749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 0x55bb0757aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead259c700 1 -- 172.21.15.19:0/579090749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 msgr2=0x55bb07580a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead259c700 1 --2- 172.21.15.19:0/579090749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 0x55bb07580a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead259c700 1 -- 172.21.15.19:0/579090749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 msgr2=0x55bb0757aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:48.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead259c700 1 --2- 172.21.15.19:0/579090749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 0x55bb0757aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead259c700 1 -- 172.21.15.19:0/579090749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb07433370 con 0x55bb074e7160 2024-05-09T01:30:48.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7fead259c700 1 --2- 172.21.15.19:0/579090749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb075804f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7feac40aaa80 tx=0x7feac40aa510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c8de29dfc115c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:48.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.058+0000 7feac8ff9700 1 -- 172.21.15.19:0/579090749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac40aa600 con 0x55bb074e7160 2024-05-09T01:30:48.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.059+0000 7feac8ff9700 1 -- 172.21.15.19:0/579090749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac40ab1f0 con 0x55bb074e7160 2024-05-09T01:30:48.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.059+0000 7feac8ff9700 1 -- 172.21.15.19:0/579090749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4022c10 con 0x55bb074e7160 2024-05-09T01:30:48.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.059+0000 7feae79e9b80 1 -- 172.21.15.19:0/579090749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 msgr2=0x55bb075804f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:48.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.059+0000 7feae79e9b80 1 --2- 172.21.15.19:0/579090749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb075804f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7feac40aaa80 tx=0x7feac40aa510 comp rx=0 tx=0).stop 2024-05-09T01:30:48.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.059+0000 7feae79e9b80 1 -- 172.21.15.19:0/579090749 shutdown_connections 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.059+0000 7feae79e9b80 1 --2- 172.21.15.19:0/579090749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 0x55bb0757aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.059+0000 7feae79e9b80 1 --2- 172.21.15.19:0/579090749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb075804f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.059+0000 7feae79e9b80 1 --2- 172.21.15.19:0/579090749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 0x55bb07580a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.059+0000 7feae79e9b80 1 -- 172.21.15.19:0/579090749 >> 172.21.15.19:0/579090749 conn(0x55bb0745d6a0 msgr2=0x55bb07575400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 -- 172.21.15.19:0/579090749 shutdown_connections 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 -- 172.21.15.19:0/579090749 wait complete. 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 -- start start 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 Processor -- start 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb073fd4a0 0x55bb07589c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb074e7160 0x55bb0758a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb0757a6a0 0x55bb07584090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb07318ba0 con 0x55bb0757a6a0 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bb07457150 con 0x55bb073fd4a0 2024-05-09T01:30:48.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb0757b360 con 0x55bb074e7160 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.060+0000 7feae79e9b80 10 client.0 _ll_register_callbacks cb 0x55bb0730e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb0757a6a0 0x55bb07584090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb0757a6a0 0x55bb07584090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44138/0 (socket says 172.21.15.19:44138) 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead2d9d700 1 -- 172.21.15.19:0/906785485 learned_addr learned my addr 172.21.15.19:0/906785485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead259c700 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb073fd4a0 0x55bb07589c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead1d9b700 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb074e7160 0x55bb0758a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead2d9d700 1 -- 172.21.15.19:0/906785485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb074e7160 msgr2=0x55bb0758a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead2d9d700 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb074e7160 0x55bb0758a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead2d9d700 1 -- 172.21.15.19:0/906785485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb073fd4a0 msgr2=0x55bb07589c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead2d9d700 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb073fd4a0 0x55bb07589c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead2d9d700 1 -- 172.21.15.19:0/906785485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb07318490 con 0x55bb0757a6a0 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead1d9b700 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb074e7160 0x55bb0758a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7fead2d9d700 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb0757a6a0 0x55bb07584090 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7feacc00dfa0 tx=0x7feacc0116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc240f91569eaa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7feab9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7feab9ffb700 1 -- 172.21.15.19:0/906785485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc004830 con 0x55bb0757a6a0 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7feab9ffb700 1 -- 172.21.15.19:0/906785485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7feacc0049d0 con 0x55bb0757a6a0 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.061+0000 7feae79e9b80 1 -- 172.21.15.19:0/906785485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac40ab1f0 con 0x55bb0757a6a0 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.062+0000 7feab9ffb700 1 -- 172.21.15.19:0/906785485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feacc004c10 con 0x55bb0757a6a0 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.062+0000 7feae79e9b80 10 client.16710 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.062+0000 7feae79e9b80 1 -- 172.21.15.19:0/906785485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb07591910 con 0x55bb0757a6a0 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.062+0000 7feab9ffb700 1 -- 172.21.15.19:0/906785485 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac40ab1f0 con 0x55bb0757a6a0 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.062+0000 7feab9ffb700 1 -- 172.21.15.19:0/906785485 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feacc01b6f0 con 0x55bb0757a6a0 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.062+0000 7feae79e9b80 10 client.16710 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.062+0000 7feae79e9b80 10 client.16710 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.063+0000 7feae79e9b80 20 client.16710 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.063+0000 7feae79e9b80 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55bb07593f50 0x55bb07596390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.063+0000 7feae79e9b80 4 client.16710 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.063+0000 7feae79e9b80 1 -- 172.21.15.19:0/906785485 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb074aa9d0 con 0x55bb07593f50 2024-05-09T01:30:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.063+0000 7fead259c700 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55bb07593f50 0x55bb07596390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.063+0000 7fead259c700 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55bb07593f50 0x55bb07596390 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b623b10ce9f3411 server_cookie=84be95c5e5bdaf4b in_seq=0 out_seq=0 2024-05-09T01:30:48.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.063+0000 7feab9ffb700 10 client.16710 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:30:48.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.064+0000 7feab9ffb700 1 -- 172.21.15.19:0/906785485 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb074aa9d0 con 0x55bb07593f50 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.064+0000 7feab9ffb700 10 client.16710 handle_command_reply: tid=0 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.064+0000 7feae79e9b80 1 client.16710 shutdown 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.064+0000 7feae79e9b80 1 -- 172.21.15.19:0/906785485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb0757a6a0 msgr2=0x55bb07584090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.064+0000 7feae79e9b80 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb0757a6a0 0x55bb07584090 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7feacc00dfa0 tx=0x7feacc0116c0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feae79e9b80 1 -- 172.21.15.19:0/906785485 shutdown_connections 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feae79e9b80 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55bb07593f50 0x55bb07596390 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feab9ffb700 0 client.16710 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feae79e9b80 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb0757a6a0 0x55bb07584090 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feae79e9b80 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb073fd4a0 0x55bb07589c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feae79e9b80 1 --2- 172.21.15.19:0/906785485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb074e7160 0x55bb0758a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feae79e9b80 1 -- 172.21.15.19:0/906785485 >> 172.21.15.19:0/906785485 conn(0x55bb0745d6a0 msgr2=0x55bb075751c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feae79e9b80 1 -- 172.21.15.19:0/906785485 shutdown_connections 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feae79e9b80 1 -- 172.21.15.19:0/906785485 wait complete. 2024-05-09T01:30:48.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.065+0000 7feae79e9b80 20 client.16710 trim_cache size 0 max 16384 2024-05-09T01:30:48.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7feae79e9b80 1 Processor -- start 2024-05-09T01:30:48.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7feae79e9b80 1 -- start start 2024-05-09T01:30:48.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb073fd4a0 0x55bb0757e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb074e7160 0x55bb0757ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb0757a6a0 0x55bb0759bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7feae79e9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fead41b0ed0 con 0x55bb073fd4a0 2024-05-09T01:30:48.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7feae79e9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fead41b2a10 con 0x55bb074e7160 2024-05-09T01:30:48.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7feae79e9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fead41b15a0 con 0x55bb0757a6a0 2024-05-09T01:30:48.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7fead259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb073fd4a0 0x55bb0757e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7fead259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb073fd4a0 0x55bb0757e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44148/0 (socket says 172.21.15.19:44148) 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7fead259c700 1 -- 172.21.15.19:0/2193810319 learned_addr learned my addr 172.21.15.19:0/2193810319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7fead1d9b700 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb074e7160 0x55bb0757ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.080+0000 7fead259c700 1 -- 172.21.15.19:0/2193810319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb0757a6a0 msgr2=0x55bb0759bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7fead2d9d700 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb0757a6a0 0x55bb0759bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7fead259c700 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb0757a6a0 0x55bb0759bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7fead259c700 1 -- 172.21.15.19:0/2193810319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb074e7160 msgr2=0x55bb0757ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7fead259c700 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb074e7160 0x55bb0757ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7fead259c700 1 -- 172.21.15.19:0/2193810319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb07576d00 con 0x55bb073fd4a0 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7fead1d9b700 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb074e7160 0x55bb0757ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7fead2d9d700 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb0757a6a0 0x55bb0759bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7fead259c700 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb073fd4a0 0x55bb0757e620 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7feac40a6b70 tx=0x7feac400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f3dc58d03fa02e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7feabbfff700 1 -- 172.21.15.19:0/2193810319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4070070 con 0x55bb073fd4a0 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7feabbfff700 1 -- 172.21.15.19:0/2193810319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac40ab1f0 con 0x55bb073fd4a0 2024-05-09T01:30:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.081+0000 7feabbfff700 1 -- 172.21.15.19:0/2193810319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4003e60 con 0x55bb073fd4a0 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 -- 172.21.15.19:0/2193810319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb073fd4a0 msgr2=0x55bb0757e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb073fd4a0 0x55bb0757e620 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7feac40a6b70 tx=0x7feac400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 -- 172.21.15.19:0/2193810319 shutdown_connections 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb0757a6a0 0x55bb0759bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb073fd4a0 0x55bb0757e620 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 --2- 172.21.15.19:0/2193810319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb074e7160 0x55bb0757ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 -- 172.21.15.19:0/2193810319 >> 172.21.15.19:0/2193810319 conn(0x55bb0745d6a0 msgr2=0x55bb075829a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 -- 172.21.15.19:0/2193810319 shutdown_connections 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 -- 172.21.15.19:0/2193810319 wait complete. 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 -- start start 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.082+0000 7feae79e9b80 1 Processor -- start 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 0x55bb0758fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb0758afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7feae79e9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 0x55bb0758b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7feae79e9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fead41b2340 con 0x55bb074e7160 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7feae79e9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fead41b1c70 con 0x55bb0757a6a0 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7feae79e9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fead41aee70 con 0x55bb073fd4a0 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7feae79e9b80 10 client.0 _ll_register_callbacks cb 0x55bb0730e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb0758afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb0758afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44152/0 (socket says 172.21.15.19:44152) 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead1d9b700 1 -- 172.21.15.19:0/1408679829 learned_addr learned my addr 172.21.15.19:0/1408679829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead259c700 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 0x55bb0758fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead2d9d700 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 0x55bb0758b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead1d9b700 1 -- 172.21.15.19:0/1408679829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 msgr2=0x55bb0758fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead1d9b700 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 0x55bb0758fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead1d9b700 1 -- 172.21.15.19:0/1408679829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 msgr2=0x55bb0758b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead1d9b700 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 0x55bb0758b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead1d9b700 1 -- 172.21.15.19:0/1408679829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb0758bc50 con 0x55bb074e7160 2024-05-09T01:30:48.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead259c700 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 0x55bb0758fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7fead1d9b700 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb0758afe0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feabc005950 tx=0x7feabc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431a4ce0daf265d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.083+0000 7feab9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feab9ffb700 1 -- 172.21.15.19:0/1408679829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc011910 con 0x55bb074e7160 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feab9ffb700 1 -- 172.21.15.19:0/1408679829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7feabc011ab0 con 0x55bb074e7160 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feae79e9b80 1 -- 172.21.15.19:0/1408679829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac40ab1f0 con 0x55bb074e7160 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feab9ffb700 1 -- 172.21.15.19:0/1408679829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feabc011cb0 con 0x55bb074e7160 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feae79e9b80 10 client.16716 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feae79e9b80 1 -- 172.21.15.19:0/1408679829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb075fe550 con 0x55bb074e7160 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feab9ffb700 1 -- 172.21.15.19:0/1408679829 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac40ab1f0 con 0x55bb074e7160 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feab9ffb700 1 -- 172.21.15.19:0/1408679829 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feabc027070 con 0x55bb074e7160 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feae79e9b80 10 client.16716 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feae79e9b80 10 client.16716 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feae79e9b80 20 client.16716 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feae79e9b80 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55bb07593f50 0x55bb07600d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:48.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feae79e9b80 4 client.16716 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.084+0000 7feae79e9b80 1 -- 172.21.15.19:0/1408679829 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb074a4a70 con 0x55bb07593f50 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.085+0000 7fead259c700 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55bb07593f50 0x55bb07600d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.085+0000 7feab9ffb700 10 client.16716 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.085+0000 7fead259c700 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55bb07593f50 0x55bb07600d00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=73a2984220eb87ef server_cookie=bc0c8de4483fbfa2 in_seq=0 out_seq=0 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.085+0000 7feab9ffb700 1 -- 172.21.15.19:0/1408679829 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb074a4a70 con 0x55bb07593f50 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.085+0000 7feab9ffb700 10 client.16716 handle_command_reply: tid=0 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.085+0000 7feae79e9b80 1 client.16716 shutdown 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.085+0000 7feae79e9b80 1 -- 172.21.15.19:0/1408679829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 msgr2=0x55bb0758afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.085+0000 7feae79e9b80 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb0758afe0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feabc005950 tx=0x7feabc0047c0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feae79e9b80 1 -- 172.21.15.19:0/1408679829 shutdown_connections 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feae79e9b80 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55bb07593f50 0x55bb07600d00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feae79e9b80 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bb0757a6a0 0x55bb0758b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feae79e9b80 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb073fd4a0 0x55bb0758fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feae79e9b80 1 --2- 172.21.15.19:0/1408679829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb074e7160 0x55bb0758afe0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feae79e9b80 1 -- 172.21.15.19:0/1408679829 >> 172.21.15.19:0/1408679829 conn(0x55bb0745d6a0 msgr2=0x55bb0757de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:48.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feab9ffb700 0 client.16716 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:30:48.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feae79e9b80 1 -- 172.21.15.19:0/1408679829 shutdown_connections 2024-05-09T01:30:48.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feae79e9b80 1 -- 172.21.15.19:0/1408679829 wait complete. 2024-05-09T01:30:48.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7feae79e9b80 20 client.16716 trim_cache size 0 max 16384 2024-05-09T01:30:48.088 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 -- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feac00b2880 msgr2=0x7feac00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feac00b2880 0x7feac00b4d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fead4103520 tx=0x7feabc00b410 comp rx=0 tx=0).stop 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 -- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 msgr2=0x7fead4076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 0x7fead4076bd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7feac400d010 tx=0x7feac4007a20 comp rx=0 tx=0).stop 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 -- 172.21.15.19:0/549580196 shutdown_connections 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead4109850 0x7fead4077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7feac00b2880 0x7feac00b4d90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead41045d0 0x7fead4076bd0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 --2- 172.21.15.19:0/549580196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead4108f00 0x7fead4077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.086+0000 7fead8fd3700 1 -- 172.21.15.19:0/549580196 >> 172.21.15.19:0/549580196 conn(0x7fead40ffea0 msgr2=0x7fead410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.087+0000 7fead8fd3700 1 -- 172.21.15.19:0/549580196 shutdown_connections 2024-05-09T01:30:48.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:48.087+0000 7fead8fd3700 1 -- 172.21.15.19:0/549580196 wait complete. 2024-05-09T01:30:48.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:48 smithi019 ceph-mon[30158]: pgmap v536: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:30:48.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:48 smithi196 ceph-mon[35041]: pgmap v536: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:30:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:48 smithi059 ceph-mon[34351]: pgmap v536: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:30:48.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:30:49.164 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:49.333 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:127/600: open fd = 3 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:128/600: open fd = 3 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:129/600: open fd = 3 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.334 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:130/600: open fd = 3 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:131/600: open fd = 3 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:132/600: open fd = 3 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:133/600: open fd = 3 2024-05-09T01:30:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:134/600: open fd = 3 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:135/600: open fd = 3 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:136/600: open fd = 3 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.336 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:137/600: open fd = 3 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:138/600: open fd = 3 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:139/600: open fd = 3 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:140/600: open fd = 3 2024-05-09T01:30:49.337 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:141/600: open fd = 3 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:142/600: open fd = 3 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:143/600: open fd = 3 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.338 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:144/600: open fd = 3 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:145/600: open fd = 3 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:146/600: open fd = 3 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:147/600: open fd = 3 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.339 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:148/600: open fd = 3 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:149/600: open fd = 3 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:150/600: open fd = 3 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:151/600: open fd = 3 2024-05-09T01:30:49.340 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:152/600: open fd = 3 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:153/600: open fd = 3 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:154/600: open fd = 3 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:155/600: open fd = 3 2024-05-09T01:30:49.341 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:156/600: open fd = 3 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:157/600: open fd = 3 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:158/600: open fd = 3 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.342 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:159/600: open fd = 3 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:160/600: open fd = 3 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:161/600: open fd = 3 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:162/600: open fd = 3 2024-05-09T01:30:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:163/600: open fd = 3 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:164/600: open fd = 3 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:165/600: open fd = 3 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:166/600: open fd = 3 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:167/600: open fd = 3 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:168/600: open fd = 3 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:169/600: open fd = 3 2024-05-09T01:30:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:170/600: open fd = 3 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:171/600: open fd = 3 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:172/600: open fd = 3 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:173/600: open fd = 3 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:174/600: open fd = 3 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:175/600: open fd = 3 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:176/600: open fd = 3 2024-05-09T01:30:49.347 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:177/600: open fd = 3 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:178/600: open fd = 3 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:179/600: open fd = 3 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.348 INFO:tasks.workunit.client.0.smithi019.stdout:180/600: open fd = 3 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:181/600: open fd = 3 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:182/600: open fd = 3 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:183/600: open fd = 3 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.349 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:184/600: open fd = 3 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:185/600: open fd = 3 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:186/600: open fd = 3 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.350 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.351 INFO:tasks.workunit.client.0.smithi019.stdout:187/600: open fd = 3 2024-05-09T01:30:49.351 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.351 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.351 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:30:49.351 INFO:tasks.workunit.client.0.smithi019.stdout:188/600: open fd = 3 2024-05-09T01:30:49.351 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:49.351 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:30:50.555 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:50 smithi019 ceph-mon[30158]: pgmap v537: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:50.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:50 smithi196 ceph-mon[35041]: pgmap v537: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.797+0000 7fd757abc700 1 -- 172.21.15.19:0/345934174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750108740 msgr2=0x7fd7501100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.797+0000 7fd757abc700 1 --2- 172.21.15.19:0/345934174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750108740 0x7fd7501100a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd74800b0d0 tx=0x7fd74800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.797+0000 7fd757abc700 1 -- 172.21.15.19:0/345934174 shutdown_connections 2024-05-09T01:30:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.797+0000 7fd757abc700 1 --2- 172.21.15.19:0/345934174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750108740 0x7fd7501100a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.797+0000 7fd757abc700 1 --2- 172.21.15.19:0/345934174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7501034c0 0x7fd7501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.797+0000 7fd757abc700 1 --2- 172.21.15.19:0/345934174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd750107df0 0x7fd750108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.797+0000 7fd757abc700 1 -- 172.21.15.19:0/345934174 >> 172.21.15.19:0/345934174 conn(0x7fd7500fedf0 msgr2=0x7fd750101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.797+0000 7fd757abc700 1 -- 172.21.15.19:0/345934174 shutdown_connections 2024-05-09T01:30:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.797+0000 7fd757abc700 1 -- 172.21.15.19:0/345934174 wait complete. 2024-05-09T01:30:50.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.798+0000 7fd757abc700 1 Processor -- start 2024-05-09T01:30:50.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.798+0000 7fd757abc700 1 -- start start 2024-05-09T01:30:50.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd757abc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7501034c0 0x7fd75019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd757abc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750107df0 0x7fd75019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd757abc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd750108740 0x7fd750199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd757abc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd750113370 con 0x7fd750107df0 2024-05-09T01:30:50.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd757abc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501131f0 con 0x7fd7501034c0 2024-05-09T01:30:50.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd757abc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501134f0 con 0x7fd750108740 2024-05-09T01:30:50.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755057700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750107df0 0x7fd75019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755057700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750107df0 0x7fd75019fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56238/0 (socket says 172.21.15.19:56238) 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755057700 1 -- 172.21.15.19:0/3204758668 learned_addr learned my addr 172.21.15.19:0/3204758668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd756059700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd750108740 0x7fd750199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755858700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7501034c0 0x7fd75019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755057700 1 -- 172.21.15.19:0/3204758668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd750108740 msgr2=0x7fd750199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755057700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd750108740 0x7fd750199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755057700 1 -- 172.21.15.19:0/3204758668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7501034c0 msgr2=0x7fd75019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755057700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7501034c0 0x7fd75019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755057700 1 -- 172.21.15.19:0/3204758668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75019a160 con 0x7fd750107df0 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.799+0000 7fd755057700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750107df0 0x7fd75019fdd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd74000d950 tx=0x7fd74000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.800+0000 7fd746ffd700 1 -- 172.21.15.19:0/3204758668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740016070 con 0x7fd750107df0 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.800+0000 7fd746ffd700 1 -- 172.21.15.19:0/3204758668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74000de30 con 0x7fd750107df0 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.800+0000 7fd757abc700 1 -- 172.21.15.19:0/3204758668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75019a450 con 0x7fd750107df0 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.800+0000 7fd746ffd700 1 -- 172.21.15.19:0/3204758668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740005350 con 0x7fd750107df0 2024-05-09T01:30:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.800+0000 7fd757abc700 1 -- 172.21.15.19:0/3204758668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7501a6720 con 0x7fd750107df0 2024-05-09T01:30:50.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.802+0000 7fd746ffd700 1 -- 172.21.15.19:0/3204758668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd740005510 con 0x7fd750107df0 2024-05-09T01:30:50.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.803+0000 7fd746ffd700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd73c0b2900 0x7fd73c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.803+0000 7fd755858700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd73c0b2900 0x7fd73c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.803+0000 7fd746ffd700 1 -- 172.21.15.19:0/3204758668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd7400e1a50 con 0x7fd750107df0 2024-05-09T01:30:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd7664d2b80 1 Processor -- start 2024-05-09T01:30:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd755858700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd73c0b2900 0x7fd73c0b4e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd74c005950 tx=0x7fd74c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd7664d2b80 1 -- start start 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68a66d00 0x558d68b6a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d689ec4a0 0x558d68b6aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b79970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558d68a8c0a0 con 0x558d68a66d00 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558d68a319a0 con 0x558d689ec4a0 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d68a88060 con 0x558d68b6b010 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd755858700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68a66d00 0x558d68b6a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd755858700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68a66d00 0x558d68b6a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56248/0 (socket says 172.21.15.19:56248) 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd755858700 1 -- 172.21.15.19:0/2495243946 learned_addr learned my addr 172.21.15.19:0/2495243946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.804+0000 7fd755057700 1 --2- 172.21.15.19:0/2495243946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d689ec4a0 0x558d68b6aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd756059700 1 --2- 172.21.15.19:0/2495243946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b79970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd755858700 1 -- 172.21.15.19:0/2495243946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 msgr2=0x558d68b79970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd755858700 1 --2- 172.21.15.19:0/2495243946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b79970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd755858700 1 -- 172.21.15.19:0/2495243946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d689ec4a0 msgr2=0x558d68b6aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:50.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd755858700 1 --2- 172.21.15.19:0/2495243946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d689ec4a0 0x558d68b6aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd755858700 1 -- 172.21.15.19:0/2495243946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d68a22500 con 0x558d68a66d00 2024-05-09T01:30:50.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd755858700 1 --2- 172.21.15.19:0/2495243946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68a66d00 0x558d68b6a590 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00fd70 tx=0x7fd74c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b66b070184da1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:50.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd73bfff700 1 -- 172.21.15.19:0/2495243946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c022070 con 0x558d68a66d00 2024-05-09T01:30:50.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd73bfff700 1 -- 172.21.15.19:0/2495243946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c011bb0 con 0x558d68a66d00 2024-05-09T01:30:50.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.805+0000 7fd73bfff700 1 -- 172.21.15.19:0/2495243946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c01d610 con 0x558d68a66d00 2024-05-09T01:30:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.806+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2495243946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68a66d00 msgr2=0x558d68b6a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.806+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2495243946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68a66d00 0x558d68b6a590 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00fd70 tx=0x7fd74c012000 comp rx=0 tx=0).stop 2024-05-09T01:30:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.806+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2495243946 shutdown_connections 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.806+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2495243946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b79970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.806+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2495243946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68a66d00 0x558d68b6a590 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.806+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2495243946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d689ec4a0 0x558d68b6aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.806+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2495243946 >> 172.21.15.19:0/2495243946 conn(0x558d689eff70 msgr2=0x558d68b66b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.806+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2495243946 shutdown_connections 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.806+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2495243946 wait complete. 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.807+0000 7fd7664d2b80 1 -- start start 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.807+0000 7fd7664d2b80 1 Processor -- start 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.807+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b76e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.807+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b773d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.807+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b77910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.807+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558d68b7a1a0 con 0x558d689ec4a0 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.807+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558d68a26b20 con 0x558d68a66d00 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.807+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d68b7a320 con 0x558d68b6b010 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.807+0000 7fd755858700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b76e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755858700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b76e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56254/0 (socket says 172.21.15.19:56254) 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755858700 1 -- 172.21.15.19:0/3089061296 learned_addr learned my addr 172.21.15.19:0/3089061296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd7664d2b80 10 client.0 _ll_register_callbacks cb 0x558d688fd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755057700 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b773d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd756059700 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b77910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755858700 1 -- 172.21.15.19:0/3089061296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 msgr2=0x558d68b77910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755858700 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b77910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755858700 1 -- 172.21.15.19:0/3089061296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 msgr2=0x558d68b773d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755858700 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b773d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755858700 1 -- 172.21.15.19:0/3089061296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d68907490 con 0x558d689ec4a0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755057700 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b773d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd756059700 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b77910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd755858700 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b76e90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00ff10 tx=0x7fd74c01e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f30bd977548132 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd7397fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd7397fa700 1 -- 172.21.15.19:0/3089061296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c036060 con 0x558d689ec4a0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd7397fa700 1 -- 172.21.15.19:0/3089061296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd74c01e8b0 con 0x558d689ec4a0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.808+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3089061296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd74c011bb0 con 0x558d689ec4a0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7397fa700 1 -- 172.21.15.19:0/3089061296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd74c022050 con 0x558d689ec4a0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7664d2b80 10 client.16728 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3089061296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d68b84df0 con 0x558d689ec4a0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7397fa700 1 -- 172.21.15.19:0/3089061296 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c011bb0 con 0x558d689ec4a0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7397fa700 1 -- 172.21.15.19:0/3089061296 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd74c03c400 con 0x558d689ec4a0 2024-05-09T01:30:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7664d2b80 10 client.16728 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7664d2b80 10 client.16728 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:30:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7664d2b80 20 client.16728 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x558d68b87370 0x558d68b89790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7664d2b80 4 client.16728 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3089061296 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d68a99a80 con 0x558d68b87370 2024-05-09T01:30:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.809+0000 7fd755057700 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x558d68b87370 0x558d68b89790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.810+0000 7fd7397fa700 10 client.16728 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.810+0000 7fd755057700 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x558d68b87370 0x558d68b89790 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d0a50084bef99bf4 server_cookie=4757b91a831560f0 in_seq=0 out_seq=0 2024-05-09T01:30:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7397fa700 1 -- 172.21.15.19:0/3089061296 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d68a99a80 con 0x558d68b87370 2024-05-09T01:30:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7397fa700 10 client.16728 handle_command_reply: tid=0 2024-05-09T01:30:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 client.16728 shutdown 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3089061296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 msgr2=0x558d68b76e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b76e90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00ff10 tx=0x7fd74c01e450 comp rx=0 tx=0).stop 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3089061296 shutdown_connections 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x558d68b87370 0x558d68b89790 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b77910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b76e90 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3089061296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b773d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7397fa700 0 client.16728 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3089061296 >> 172.21.15.19:0/3089061296 conn(0x558d689eff70 msgr2=0x558d68b5dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3089061296 shutdown_connections 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3089061296 wait complete. 2024-05-09T01:30:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.811+0000 7fd7664d2b80 20 client.16728 trim_cache size 0 max 16384 2024-05-09T01:30:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.826+0000 7fd7664d2b80 1 Processor -- start 2024-05-09T01:30:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.826+0000 7fd7664d2b80 1 -- start start 2024-05-09T01:30:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d689ec4a0 0x558d68b835c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b77750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68b6b010 0x558d68b5e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501acb00 con 0x558d68b6b010 2024-05-09T01:30:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501ae640 con 0x558d68a66d00 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501ad1d0 con 0x558d689ec4a0 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd756059700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68b6b010 0x558d68b5e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd756059700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68b6b010 0x558d68b5e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56262/0 (socket says 172.21.15.19:56262) 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd756059700 1 -- 172.21.15.19:0/3912426634 learned_addr learned my addr 172.21.15.19:0/3912426634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd755858700 1 --2- 172.21.15.19:0/3912426634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d689ec4a0 0x558d68b835c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd755057700 1 --2- 172.21.15.19:0/3912426634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b77750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd756059700 1 -- 172.21.15.19:0/3912426634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d689ec4a0 msgr2=0x558d68b835c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd756059700 1 --2- 172.21.15.19:0/3912426634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d689ec4a0 0x558d68b835c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd756059700 1 -- 172.21.15.19:0/3912426634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 msgr2=0x558d68b77750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd756059700 1 --2- 172.21.15.19:0/3912426634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b77750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd756059700 1 -- 172.21.15.19:0/3912426634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d68b609d0 con 0x558d68b6b010 2024-05-09T01:30:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.827+0000 7fd756059700 1 --2- 172.21.15.19:0/3912426634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68b6b010 0x558d68b5e660 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd748009df0 tx=0x7fd748009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fa3fbf92f6bb42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd73b7fe700 1 -- 172.21.15.19:0/3912426634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748004a50 con 0x558d68b6b010 2024-05-09T01:30:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd73b7fe700 1 -- 172.21.15.19:0/3912426634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd748004bf0 con 0x558d68b6b010 2024-05-09T01:30:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd73b7fe700 1 -- 172.21.15.19:0/3912426634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748005370 con 0x558d68b6b010 2024-05-09T01:30:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3912426634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68b6b010 msgr2=0x558d68b5e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3912426634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68b6b010 0x558d68b5e660 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd748009df0 tx=0x7fd748009e20 comp rx=0 tx=0).stop 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3912426634 shutdown_connections 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3912426634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d68b6b010 0x558d68b5e660 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3912426634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d689ec4a0 0x558d68b835c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/3912426634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b77750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.828+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3912426634 >> 172.21.15.19:0/3912426634 conn(0x558d689eff70 msgr2=0x558d68b78d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.829+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3912426634 shutdown_connections 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.829+0000 7fd7664d2b80 1 -- 172.21.15.19:0/3912426634 wait complete. 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.829+0000 7fd7664d2b80 1 -- start start 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.829+0000 7fd7664d2b80 1 Processor -- start 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.829+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b8a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.829+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b8aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd7664d2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b8af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501adf70 con 0x558d689ec4a0 2024-05-09T01:30:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501ad8a0 con 0x558d68a66d00 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd7664d2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501aaaa0 con 0x558d68b6b010 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755858700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b8a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755858700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b8a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56274/0 (socket says 172.21.15.19:56274) 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755858700 1 -- 172.21.15.19:0/2177375576 learned_addr learned my addr 172.21.15.19:0/2177375576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd7664d2b80 10 client.0 _ll_register_callbacks cb 0x558d688fd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd756059700 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b8af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755057700 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b8aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755858700 1 -- 172.21.15.19:0/2177375576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 msgr2=0x558d68b8af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755858700 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b8af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755858700 1 -- 172.21.15.19:0/2177375576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 msgr2=0x558d68b8aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755858700 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b8aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755858700 1 -- 172.21.15.19:0/2177375576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d68b79460 con 0x558d689ec4a0 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd755858700 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b8a4c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00ff10 tx=0x7fd74c00fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1438ee457a62c6fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd7397fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd7397fa700 1 -- 172.21.15.19:0/2177375576 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c015070 con 0x558d689ec4a0 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd7397fa700 1 -- 172.21.15.19:0/2177375576 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd74c01eb40 con 0x558d689ec4a0 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.830+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2177375576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd748004bf0 con 0x558d689ec4a0 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7397fa700 1 -- 172.21.15.19:0/2177375576 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd74c036040 con 0x558d689ec4a0 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7664d2b80 10 client.16734 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2177375576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d68bf19e0 con 0x558d689ec4a0 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7397fa700 1 -- 172.21.15.19:0/2177375576 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd748004bf0 con 0x558d689ec4a0 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7397fa700 1 -- 172.21.15.19:0/2177375576 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd74c004710 con 0x558d689ec4a0 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7664d2b80 10 client.16734 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7664d2b80 10 client.16734 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7664d2b80 20 client.16734 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x558d68bf2060 0x558d68bf4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7664d2b80 4 client.16734 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2177375576 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d68a93b20 con 0x558d68bf2060 2024-05-09T01:30:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.831+0000 7fd755057700 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x558d68bf2060 0x558d68bf4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.832+0000 7fd7397fa700 10 client.16734 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.832+0000 7fd755057700 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x558d68bf2060 0x558d68bf4480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a07cbbf0383fe90 server_cookie=6302b9bc83d6d3cb in_seq=0 out_seq=0 2024-05-09T01:30:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.832+0000 7fd7397fa700 1 -- 172.21.15.19:0/2177375576 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d68a93b20 con 0x558d68bf2060 2024-05-09T01:30:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.832+0000 7fd7397fa700 10 client.16734 handle_command_reply: tid=0 2024-05-09T01:30:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.832+0000 7fd7664d2b80 1 client.16734 shutdown 2024-05-09T01:30:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2177375576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 msgr2=0x558d68b8a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b8a4c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00ff10 tx=0x7fd74c00fe50 comp rx=0 tx=0).stop 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2177375576 shutdown_connections 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x558d68bf2060 0x558d68bf4480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d68b6b010 0x558d68b8af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7397fa700 0 client.16734 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d689ec4a0 0x558d68b8a4c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 --2- 172.21.15.19:0/2177375576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d68a66d00 0x558d68b8aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2177375576 >> 172.21.15.19:0/2177375576 conn(0x558d689eff70 msgr2=0x558d68b78c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2177375576 shutdown_connections 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 1 -- 172.21.15.19:0/2177375576 wait complete. 2024-05-09T01:30:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd7664d2b80 20 client.16734 trim_cache size 0 max 16384 2024-05-09T01:30:50.835 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd757abc700 1 -- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd73c0b2900 msgr2=0x7fd73c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd757abc700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd73c0b2900 0x7fd73c0b4e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd74c005950 tx=0x7fd74c0058e0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd757abc700 1 -- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750107df0 msgr2=0x7fd75019fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.833+0000 7fd757abc700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750107df0 0x7fd75019fdd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd74000d950 tx=0x7fd74000dd10 comp rx=0 tx=0).stop 2024-05-09T01:30:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.834+0000 7fd757abc700 1 -- 172.21.15.19:0/3204758668 shutdown_connections 2024-05-09T01:30:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.834+0000 7fd757abc700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd750108740 0x7fd750199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.834+0000 7fd757abc700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7501034c0 0x7fd75019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.834+0000 7fd757abc700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd73c0b2900 0x7fd73c0b4e10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.834+0000 7fd757abc700 1 --2- 172.21.15.19:0/3204758668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd750107df0 0x7fd75019fdd0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.834+0000 7fd757abc700 1 -- 172.21.15.19:0/3204758668 >> 172.21.15.19:0/3204758668 conn(0x7fd7500fedf0 msgr2=0x7fd750107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.834+0000 7fd757abc700 1 -- 172.21.15.19:0/3204758668 shutdown_connections 2024-05-09T01:30:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:50.834+0000 7fd757abc700 1 -- 172.21.15.19:0/3204758668 wait complete. 2024-05-09T01:30:50.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:50 smithi059 ceph-mon[34351]: pgmap v537: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:51.597 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:30:51.853 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:52.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:52 smithi019 ceph-mon[30158]: pgmap v538: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:30:52.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:52 smithi196 ceph-mon[35041]: pgmap v538: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:30:52.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:52 smithi059 ceph-mon[34351]: pgmap v538: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:30:53.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:30:53.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.469 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:30:53.469 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:30:53.469 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:30:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.595+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/1105530532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105480 msgr2=0x7f4eb4105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.595+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/1105530532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105480 0x7f4eb4105890 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400b0a0 tx=0x7f4ea400b3b0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.595+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/1105530532 shutdown_connections 2024-05-09T01:30:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.595+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/1105530532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb410e8f0 0x7f4eb4110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.595+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/1105530532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105480 0x7f4eb4105890 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.595+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/1105530532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eb4105dd0 0x7f4eb410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.595+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/1105530532 >> 172.21.15.19:0/1105530532 conn(0x7f4eb40fef30 msgr2=0x7f4eb4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.596+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/1105530532 shutdown_connections 2024-05-09T01:30:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.596+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/1105530532 wait complete. 2024-05-09T01:30:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.596+0000 7f4eb9ce5700 1 Processor -- start 2024-05-09T01:30:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.596+0000 7f4eb9ce5700 1 -- start start 2024-05-09T01:30:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb9ce5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb4105480 0x7f4eb419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb9ce5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105dd0 0x7f4eb419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb9ce5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eb410e8f0 0x7f4eb419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb9ce5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4114440 con 0x7f4eb4105dd0 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb9ce5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41142c0 con 0x7f4eb410e8f0 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb9ce5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41145c0 con 0x7f4eb4105480 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105dd0 0x7f4eb419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105dd0 0x7f4eb419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56294/0 (socket says 172.21.15.19:56294) 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/3256976679 learned_addr learned my addr 172.21.15.19:0/3256976679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb4105480 0x7f4eb419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb3fff700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eb410e8f0 0x7f4eb419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/3256976679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb4105480 msgr2=0x7f4eb419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb4105480 0x7f4eb419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/3256976679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eb410e8f0 msgr2=0x7f4eb419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eb410e8f0 0x7f4eb419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/3256976679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb419fc10 con 0x7f4eb4105dd0 2024-05-09T01:30:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb4105480 0x7f4eb419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.597+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105dd0 0x7f4eb419b390 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c9b0 tx=0x7f4ea800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.598+0000 7f4eb0ff9700 1 -- 172.21.15.19:0/3256976679 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8007b00 con 0x7f4eb4105dd0 2024-05-09T01:30:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.598+0000 7f4eb0ff9700 1 -- 172.21.15.19:0/3256976679 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea8007ca0 con 0x7f4eb4105dd0 2024-05-09T01:30:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.598+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/3256976679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb419ff00 con 0x7f4eb4105dd0 2024-05-09T01:30:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.598+0000 7f4eb0ff9700 1 -- 172.21.15.19:0/3256976679 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8018660 con 0x7f4eb4105dd0 2024-05-09T01:30:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.598+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/3256976679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb41a0440 con 0x7f4eb4105dd0 2024-05-09T01:30:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.600+0000 7f4eb0ff9700 1 -- 172.21.15.19:0/3256976679 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4ea8004830 con 0x7f4eb4105dd0 2024-05-09T01:30:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.601+0000 7f4eb0ff9700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.601+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.601+0000 7f4eb0ff9700 1 -- 172.21.15.19:0/3256976679 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4ea80e2230 con 0x7f4eb4105dd0 2024-05-09T01:30:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4ec86fbb80 1 Processor -- start 2024-05-09T01:30:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009900 tx=0x7f4ea400b700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4ec86fbb80 1 -- start start 2024-05-09T01:30:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06616ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 0x557f06617050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 0x557f0661b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557f065350a0 con 0x557f0650fd00 2024-05-09T01:30:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557f064da9a0 con 0x557f064954a0 2024-05-09T01:30:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f06531060 con 0x557f0661b1b0 2024-05-09T01:30:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4eb37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06616ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4eb37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06616ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56302/0 (socket says 172.21.15.19:56302) 2024-05-09T01:30:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4eb2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 0x557f06617050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4eb37fe700 1 -- 172.21.15.19:0/2650628006 learned_addr learned my addr 172.21.15.19:0/2650628006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.602+0000 7f4eb3fff700 1 --2- 172.21.15.19:0/2650628006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 0x557f0661b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.603+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/2650628006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 msgr2=0x557f0661b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.603+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/2650628006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 0x557f0661b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.603+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/2650628006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 msgr2=0x557f06616ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.603+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/2650628006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06616ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.603+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/2650628006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f064cb500 con 0x557f064954a0 2024-05-09T01:30:53.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.603+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/2650628006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 0x557f06617050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80dfd10 tx=0x7f4ea800fc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad18a165a3ece89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:53.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.603+0000 7f4e99ffb700 1 -- 172.21.15.19:0/2650628006 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80a9340 con 0x557f064954a0 2024-05-09T01:30:53.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.603+0000 7f4e99ffb700 1 -- 172.21.15.19:0/2650628006 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea80a9f30 con 0x557f064954a0 2024-05-09T01:30:53.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.603+0000 7f4e99ffb700 1 -- 172.21.15.19:0/2650628006 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80a9a90 con 0x557f064954a0 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/2650628006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 msgr2=0x557f06617050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/2650628006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 0x557f06617050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80dfd10 tx=0x7f4ea800fc60 comp rx=0 tx=0).stop 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/2650628006 shutdown_connections 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/2650628006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 0x557f0661b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/2650628006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06616ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/2650628006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 0x557f06617050 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/2650628006 >> 172.21.15.19:0/2650628006 conn(0x557f06498f70 msgr2=0x557f0660d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/2650628006 shutdown_connections 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/2650628006 wait complete. 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.604+0000 7f4ec86fbb80 1 -- start start 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4ec86fbb80 1 Processor -- start 2024-05-09T01:30:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 0x557f06620480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0650fd00 0x557f066209c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0661b1b0 0x557f06620f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557f0661bed0 con 0x557f0661b1b0 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557f064cfb20 con 0x557f064954a0 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f06623460 con 0x557f0650fd00 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4ec86fbb80 10 client.0 _ll_register_callbacks cb 0x557f063a66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0661b1b0 0x557f06620f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0661b1b0 0x557f06620f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56318/0 (socket says 172.21.15.19:56318) 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb3fff700 1 -- 172.21.15.19:0/933244948 learned_addr learned my addr 172.21.15.19:0/933244948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 0x557f06620480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0650fd00 0x557f066209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb3fff700 1 -- 172.21.15.19:0/933244948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0650fd00 msgr2=0x557f066209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb3fff700 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0650fd00 0x557f066209c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb3fff700 1 -- 172.21.15.19:0/933244948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 msgr2=0x557f06620480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb3fff700 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 0x557f06620480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.605+0000 7f4eb3fff700 1 -- 172.21.15.19:0/933244948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f063b0490 con 0x557f0661b1b0 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4eb3fff700 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0661b1b0 0x557f06620f00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4eac003f50 tx=0x7f4eac0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddb56165e74c648 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4e92ffd700 1 -- 172.21.15.19:0/933244948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac01b640 con 0x557f0661b1b0 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4e92ffd700 1 -- 172.21.15.19:0/933244948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4eac01b7e0 con 0x557f0661b1b0 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/933244948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea80a9f30 con 0x557f0661b1b0 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4e92ffd700 1 -- 172.21.15.19:0/933244948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4eac021070 con 0x557f0661b1b0 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4ec86fbb80 10 client.16746 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/933244948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f0662de70 con 0x557f0661b1b0 2024-05-09T01:30:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4e92ffd700 1 -- 172.21.15.19:0/933244948 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea80a9f30 con 0x557f0661b1b0 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4e92ffd700 1 -- 172.21.15.19:0/933244948 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4eac019020 con 0x557f0661b1b0 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4ec86fbb80 10 client.16746 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4ec86fbb80 10 client.16746 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.606+0000 7f4ec86fbb80 20 client.16746 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.607+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557f066304b0 0x557f06632930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.607+0000 7f4ec86fbb80 4 client.16746 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.607+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/933244948 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f06542a80 con 0x557f066304b0 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.607+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557f066304b0 0x557f06632930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.607+0000 7f4e92ffd700 10 client.16746 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.607+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557f066304b0 0x557f06632930 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=178c63dea5b05545 server_cookie=28b9a424c4a8cd1f in_seq=0 out_seq=0 2024-05-09T01:30:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4e92ffd700 1 -- 172.21.15.19:0/933244948 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f06542a80 con 0x557f066304b0 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4e92ffd700 10 client.16746 handle_command_reply: tid=0 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 client.16746 shutdown 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/933244948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0661b1b0 msgr2=0x557f06620f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0661b1b0 0x557f06620f00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4eac003f50 tx=0x7f4eac0098a0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/933244948 shutdown_connections 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0661b1b0 0x557f06620f00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557f066304b0 0x557f06632930 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f064954a0 0x557f06620480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/933244948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0650fd00 0x557f066209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/933244948 >> 172.21.15.19:0/933244948 conn(0x557f06498f70 msgr2=0x557f066113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4e92ffd700 0 client.16746 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/933244948 shutdown_connections 2024-05-09T01:30:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.608+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/933244948 wait complete. 2024-05-09T01:30:53.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.609+0000 7f4ec86fbb80 20 client.16746 trim_cache size 0 max 16384 2024-05-09T01:30:53.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4ec86fbb80 1 Processor -- start 2024-05-09T01:30:53.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4ec86fbb80 1 -- start start 2024-05-09T01:30:53.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f064954a0 0x557f06607a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0650fd00 0x557f06620d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 0x557f06611b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41aca80 con 0x557f064954a0 2024-05-09T01:30:53.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41ae5c0 con 0x557f0650fd00 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41ad150 con 0x557f0661b1b0 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f064954a0 0x557f06607a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f064954a0 0x557f06607a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56328/0 (socket says 172.21.15.19:56328) 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb37fe700 1 -- 172.21.15.19:0/3107733433 learned_addr learned my addr 172.21.15.19:0/3107733433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb3fff700 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 0x557f06611b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0650fd00 0x557f06620d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb37fe700 1 -- 172.21.15.19:0/3107733433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 msgr2=0x557f06611b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 0x557f06611b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb37fe700 1 -- 172.21.15.19:0/3107733433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0650fd00 msgr2=0x557f06620d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0650fd00 0x557f06620d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb37fe700 1 -- 172.21.15.19:0/3107733433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0661e9f0 con 0x557f064954a0 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.624+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0650fd00 0x557f06620d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.625+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f064954a0 0x557f06607a20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400bf70 tx=0x7f4ea401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411f52475a2966eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.625+0000 7f4e997fa700 1 -- 172.21.15.19:0/3107733433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4021070 con 0x557f064954a0 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.625+0000 7f4e997fa700 1 -- 172.21.15.19:0/3107733433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea4007d60 con 0x557f064954a0 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.625+0000 7f4e997fa700 1 -- 172.21.15.19:0/3107733433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4032440 con 0x557f064954a0 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3107733433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f064954a0 msgr2=0x557f06607a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f064954a0 0x557f06607a20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400bf70 tx=0x7f4ea401d000 comp rx=0 tx=0).stop 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3107733433 shutdown_connections 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f0661b1b0 0x557f06611b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f064954a0 0x557f06607a20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3107733433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0650fd00 0x557f06620d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3107733433 >> 172.21.15.19:0/3107733433 conn(0x557f06498f70 msgr2=0x557f066113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3107733433 shutdown_connections 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3107733433 wait complete. 2024-05-09T01:30:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 -- start start 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.626+0000 7f4ec86fbb80 1 Processor -- start 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f064954a0 0x557f066288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06623c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4ec86fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0661b1b0 0x557f06624170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41adef0 con 0x557f0650fd00 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41ad820 con 0x557f0661b1b0 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4ec86fbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41aaa20 con 0x557f064954a0 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4ec86fbb80 10 client.0 _ll_register_callbacks cb 0x557f063a66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06623c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06623c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56338/0 (socket says 172.21.15.19:56338) 2024-05-09T01:30:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/3444911406 learned_addr learned my addr 172.21.15.19:0/3444911406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f064954a0 0x557f066288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb3fff700 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0661b1b0 0x557f06624170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/3444911406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f064954a0 msgr2=0x557f066288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f064954a0 0x557f066288d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/3444911406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0661b1b0 msgr2=0x557f06624170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0661b1b0 0x557f06624170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb2ffd700 1 -- 172.21.15.19:0/3444911406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f066246b0 con 0x557f0650fd00 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.627+0000 7f4eb3fff700 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0661b1b0 0x557f06624170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4eb2ffd700 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06623c30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8000c00 tx=0x7f4ea80a8fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec12cf25dd741746 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4e92ffd700 1 -- 172.21.15.19:0/3444911406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80c6060 con 0x557f0650fd00 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4e92ffd700 1 -- 172.21.15.19:0/3444911406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4ea80a5e20 con 0x557f0650fd00 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3444911406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea4007d60 con 0x557f0650fd00 2024-05-09T01:30:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4e92ffd700 1 -- 172.21.15.19:0/3444911406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ea80a9340 con 0x557f0650fd00 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4ec86fbb80 10 client.16752 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3444911406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f0669a910 con 0x557f0650fd00 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4e92ffd700 1 -- 172.21.15.19:0/3444911406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea4007d60 con 0x557f0650fd00 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4e92ffd700 1 -- 172.21.15.19:0/3444911406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4ea80b2200 con 0x557f0650fd00 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4ec86fbb80 10 client.16752 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4ec86fbb80 10 client.16752 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4ec86fbb80 20 client.16752 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557f0669abf0 0x557f0669d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4ec86fbb80 4 client.16752 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.628+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3444911406 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f0653cb20 con 0x557f0669abf0 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.629+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557f0669abf0 0x557f0669d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.629+0000 7f4e92ffd700 10 client.16752 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.629+0000 7f4eb37fe700 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557f0669abf0 0x557f0669d010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c47ac7cf9fc04a82 server_cookie=197c1a31df47843f in_seq=0 out_seq=0 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.629+0000 7f4e92ffd700 1 -- 172.21.15.19:0/3444911406 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f0653cb20 con 0x557f0669abf0 2024-05-09T01:30:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.629+0000 7f4e92ffd700 10 client.16752 handle_command_reply: tid=0 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.629+0000 7f4ec86fbb80 1 client.16752 shutdown 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3444911406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 msgr2=0x557f06623c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06623c30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8000c00 tx=0x7f4ea80a8fa0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3444911406 shutdown_connections 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557f0669abf0 0x557f0669d010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f0661b1b0 0x557f06624170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f064954a0 0x557f066288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 --2- 172.21.15.19:0/3444911406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f0650fd00 0x557f06623c30 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4e92ffd700 0 client.16752 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3444911406 >> 172.21.15.19:0/3444911406 conn(0x557f06498f70 msgr2=0x557f06607780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3444911406 shutdown_connections 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 1 -- 172.21.15.19:0/3444911406 wait complete. 2024-05-09T01:30:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4ec86fbb80 20 client.16752 trim_cache size 0 max 16384 2024-05-09T01:30:53.633 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4e9c0b2880 msgr2=0x7f4e9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009900 tx=0x7f4ea400b700 comp rx=0 tx=0).stop 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105dd0 msgr2=0x7f4eb419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.630+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105dd0 0x7f4eb419b390 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c9b0 tx=0x7f4ea800ccc0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.631+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/3256976679 shutdown_connections 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.631+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eb410e8f0 0x7f4eb419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.631+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb4105480 0x7f4eb419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.631+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.631+0000 7f4eb9ce5700 1 --2- 172.21.15.19:0/3256976679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4105dd0 0x7f4eb419b390 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.631+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/3256976679 >> 172.21.15.19:0/3256976679 conn(0x7f4eb40fef30 msgr2=0x7f4eb40ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.631+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/3256976679 shutdown_connections 2024-05-09T01:30:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:53.631+0000 7f4eb9ce5700 1 -- 172.21.15.19:0/3256976679 wait complete. 2024-05-09T01:30:53.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:30:53.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:30:53.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:30:53.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:30:53.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:30:53.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:30:53.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:30:53.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:30:53.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:30:54.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:54 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-09T01:30:54.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:54 smithi019 ceph-mon[30158]: Unable to set osd_memory_target on smithi019 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-09T01:30:54.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:54 smithi019 ceph-mon[30158]: pgmap v539: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:54.783 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:54 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-09T01:30:54.783 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:54 smithi196 ceph-mon[35041]: Unable to set osd_memory_target on smithi019 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-09T01:30:54.783 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:54 smithi196 ceph-mon[35041]: pgmap v539: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:54.785 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:30:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:54 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-09T01:30:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:54 smithi059 ceph-mon[34351]: Unable to set osd_memory_target on smithi019 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-09T01:30:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:54 smithi059 ceph-mon[34351]: pgmap v539: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:30:55.051 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:56.576 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:56 smithi019 ceph-mon[30158]: pgmap v540: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:56.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:56 smithi196 ceph-mon[35041]: pgmap v540: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:56.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.696+0000 7f8a27c60700 1 -- 172.21.15.19:0/749932999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 msgr2=0x7f8a20109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.696+0000 7f8a27c60700 1 --2- 172.21.15.19:0/749932999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 0x7f8a20109310 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8a100099c0 tx=0x7f8a10009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.696+0000 7f8a27c60700 1 -- 172.21.15.19:0/749932999 shutdown_connections 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.696+0000 7f8a27c60700 1 --2- 172.21.15.19:0/749932999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20109850 0x7f8a201111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.696+0000 7f8a27c60700 1 --2- 172.21.15.19:0/749932999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a201045d0 0x7f8a201049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.696+0000 7f8a27c60700 1 --2- 172.21.15.19:0/749932999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 0x7f8a20109310 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.696+0000 7f8a27c60700 1 -- 172.21.15.19:0/749932999 >> 172.21.15.19:0/749932999 conn(0x7f8a200ffea0 msgr2=0x7f8a201022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.697+0000 7f8a27c60700 1 -- 172.21.15.19:0/749932999 shutdown_connections 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.697+0000 7f8a27c60700 1 -- 172.21.15.19:0/749932999 wait complete. 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.697+0000 7f8a27c60700 1 Processor -- start 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.697+0000 7f8a27c60700 1 -- start start 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a27c60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a201045d0 0x7f8a2019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a27c60700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 0x7f8a2019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a27c60700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20109850 0x7f8a2019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a27c60700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20114460 con 0x7f8a20108f00 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a27c60700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201142e0 con 0x7f8a201045d0 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a27c60700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201145e0 con 0x7f8a20109850 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a251fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 0x7f8a2019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a251fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 0x7f8a2019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56362/0 (socket says 172.21.15.19:56362) 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a251fb700 1 -- 172.21.15.19:0/1194121005 learned_addr learned my addr 172.21.15.19:0/1194121005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a259fc700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a201045d0 0x7f8a2019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a261fd700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20109850 0x7f8a2019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a251fb700 1 -- 172.21.15.19:0/1194121005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20109850 msgr2=0x7f8a2019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a251fb700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20109850 0x7f8a2019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a251fb700 1 -- 172.21.15.19:0/1194121005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a201045d0 msgr2=0x7f8a2019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a251fb700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a201045d0 0x7f8a2019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a251fb700 1 -- 172.21.15.19:0/1194121005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2019fcb0 con 0x7f8a20108f00 2024-05-09T01:30:56.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a261fd700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20109850 0x7f8a2019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:56.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a251fb700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 0x7f8a2019b430 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009f50 tx=0x7f8a100037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:56.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.698+0000 7f8a16ffd700 1 -- 172.21.15.19:0/1194121005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10027070 con 0x7f8a20108f00 2024-05-09T01:30:56.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.699+0000 7f8a27c60700 1 -- 172.21.15.19:0/1194121005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2019ff40 con 0x7f8a20108f00 2024-05-09T01:30:56.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.699+0000 7f8a27c60700 1 -- 172.21.15.19:0/1194121005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a201a0470 con 0x7f8a20108f00 2024-05-09T01:30:56.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.699+0000 7f8a16ffd700 1 -- 172.21.15.19:0/1194121005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a10004140 con 0x7f8a20108f00 2024-05-09T01:30:56.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.699+0000 7f8a16ffd700 1 -- 172.21.15.19:0/1194121005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000f460 con 0x7f8a20108f00 2024-05-09T01:30:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.701+0000 7f8a16ffd700 1 -- 172.21.15.19:0/1194121005 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8a1002a070 con 0x7f8a20108f00 2024-05-09T01:30:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.702+0000 7f8a16ffd700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.702+0000 7f8a16ffd700 1 -- 172.21.15.19:0/1194121005 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8a100e32d0 con 0x7f8a20108f00 2024-05-09T01:30:56.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.702+0000 7f8a259fc700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.702+0000 7f8a36676b80 1 Processor -- start 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.702+0000 7f8a36676b80 1 -- start start 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a259fc700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005ea0 tx=0x7f8a1c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 0x55b25f3c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3c6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3c0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a36676b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b25f2e30a0 con 0x55b25f3c0400 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a36676b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b25f2889a0 con 0x55b25f2434a0 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a36676b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b25f2df060 con 0x55b25f2bdd00 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a261fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a261fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3c0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56376/0 (socket says 172.21.15.19:56376) 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a261fd700 1 -- 172.21.15.19:0/4075630241 learned_addr learned my addr 172.21.15.19:0/4075630241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a251fb700 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a259fc700 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 0x55b25f3c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a261fd700 1 -- 172.21.15.19:0/4075630241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 msgr2=0x55b25f3c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a261fd700 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 0x55b25f3c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a261fd700 1 -- 172.21.15.19:0/4075630241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 msgr2=0x55b25f3c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a261fd700 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a261fd700 1 -- 172.21.15.19:0/4075630241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b25f279500 con 0x55b25f3c0400 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.703+0000 7f8a259fc700 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 0x55b25f3c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.704+0000 7f8a261fd700 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3c0810 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8a18008470 tx=0x7f8a18008780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6445ee4dc52a3a00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.704+0000 7f8a0bfff700 1 -- 172.21.15.19:0/4075630241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18018070 con 0x55b25f3c0400 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.704+0000 7f8a0bfff700 1 -- 172.21.15.19:0/4075630241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a180119c0 con 0x55b25f3c0400 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.704+0000 7f8a0bfff700 1 -- 172.21.15.19:0/4075630241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18013a90 con 0x55b25f3c0400 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 -- 172.21.15.19:0/4075630241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 msgr2=0x55b25f3c0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3c0810 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8a18008470 tx=0x7f8a18008780 comp rx=0 tx=0).stop 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 -- 172.21.15.19:0/4075630241 shutdown_connections 2024-05-09T01:30:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3c0810 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 0x55b25f3c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 --2- 172.21.15.19:0/4075630241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 -- 172.21.15.19:0/4075630241 >> 172.21.15.19:0/4075630241 conn(0x55b25f246f70 msgr2=0x55b25f3bb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 -- 172.21.15.19:0/4075630241 shutdown_connections 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 -- 172.21.15.19:0/4075630241 wait complete. 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 -- start start 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.705+0000 7f8a36676b80 1 Processor -- start 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 0x55b25f3ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a36676b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b25f3c1160 con 0x55b25f3c0400 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a36676b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b25f27db20 con 0x55b25f2434a0 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a36676b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b25f3cd020 con 0x55b25f2bdd00 2024-05-09T01:30:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a36676b80 10 client.0 _ll_register_callbacks cb 0x55b25f1546c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a261fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a261fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56390/0 (socket says 172.21.15.19:56390) 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a261fd700 1 -- 172.21.15.19:0/2286095318 learned_addr learned my addr 172.21.15.19:0/2286095318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a251fb700 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 0x55b25f3ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a259fc700 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a261fd700 1 -- 172.21.15.19:0/2286095318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 msgr2=0x55b25f3ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a261fd700 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 0x55b25f3ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a261fd700 1 -- 172.21.15.19:0/2286095318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 msgr2=0x55b25f3ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a261fd700 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.706+0000 7f8a261fd700 1 -- 172.21.15.19:0/2286095318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25f15e490 con 0x55b25f3c0400 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.707+0000 7f8a261fd700 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3caac0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8a18007c80 tx=0x7f8a18013480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c465ac684ca06801 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.707+0000 7f8a097fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.707+0000 7f8a097fa700 1 -- 172.21.15.19:0/2286095318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18021070 con 0x55b25f3c0400 2024-05-09T01:30:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.707+0000 7f8a097fa700 1 -- 172.21.15.19:0/2286095318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8a180135f0 con 0x55b25f3c0400 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.707+0000 7f8a36676b80 1 -- 172.21.15.19:0/2286095318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a180119c0 con 0x55b25f3c0400 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.707+0000 7f8a097fa700 1 -- 172.21.15.19:0/2286095318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a1802d050 con 0x55b25f3c0400 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.707+0000 7f8a36676b80 10 client.16764 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.707+0000 7f8a36676b80 1 -- 172.21.15.19:0/2286095318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b25f3d79c0 con 0x55b25f3c0400 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.707+0000 7f8a097fa700 1 -- 172.21.15.19:0/2286095318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a180119c0 con 0x55b25f3c0400 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a097fa700 1 -- 172.21.15.19:0/2286095318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8a1801d930 con 0x55b25f3c0400 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a36676b80 10 client.16764 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a36676b80 10 client.16764 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a36676b80 20 client.16764 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55b25f3d9e90 0x55b25f3dc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a36676b80 4 client.16764 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a36676b80 1 -- 172.21.15.19:0/2286095318 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b25f2f0a80 con 0x55b25f3d9e90 2024-05-09T01:30:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a259fc700 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55b25f3d9e90 0x55b25f3dc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a097fa700 10 client.16764 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.708+0000 7f8a259fc700 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55b25f3d9e90 0x55b25f3dc2b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7678a3997a9898a3 server_cookie=f9b4e2d7be63350 in_seq=0 out_seq=0 2024-05-09T01:30:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a097fa700 1 -- 172.21.15.19:0/2286095318 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b25f2f0a80 con 0x55b25f3d9e90 2024-05-09T01:30:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a097fa700 10 client.16764 handle_command_reply: tid=0 2024-05-09T01:30:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a36676b80 1 client.16764 shutdown 2024-05-09T01:30:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a36676b80 1 -- 172.21.15.19:0/2286095318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 msgr2=0x55b25f3caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3caac0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8a18007c80 tx=0x7f8a18013480 comp rx=0 tx=0).stop 2024-05-09T01:30:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a36676b80 1 -- 172.21.15.19:0/2286095318 shutdown_connections 2024-05-09T01:30:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55b25f3d9e90 0x55b25f3dc2b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f3c0400 0x55b25f3caac0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a097fa700 0 client.16764 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2286095318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2bdd00 0x55b25f3ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.709+0000 7f8a36676b80 1 -- 172.21.15.19:0/2286095318 >> 172.21.15.19:0/2286095318 conn(0x55b25f246f70 msgr2=0x55b25f3b4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.710+0000 7f8a36676b80 1 -- 172.21.15.19:0/2286095318 shutdown_connections 2024-05-09T01:30:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.710+0000 7f8a36676b80 1 -- 172.21.15.19:0/2286095318 wait complete. 2024-05-09T01:30:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.710+0000 7f8a36676b80 20 client.16764 trim_cache size 0 max 16384 2024-05-09T01:30:56.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.725+0000 7f8a36676b80 1 Processor -- start 2024-05-09T01:30:56.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.725+0000 7f8a36676b80 1 -- start start 2024-05-09T01:30:56.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f3c0400 0x55b25f3ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a36676b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400d420 con 0x55b25f2bdd00 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a36676b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400ef60 con 0x55b25f2434a0 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a36676b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400daf0 con 0x55b25f3c0400 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a251fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a251fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56402/0 (socket says 172.21.15.19:56402) 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a251fb700 1 -- 172.21.15.19:0/126982812 learned_addr learned my addr 172.21.15.19:0/126982812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a259fc700 1 --2- 172.21.15.19:0/126982812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a251fb700 1 -- 172.21.15.19:0/126982812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f3c0400 msgr2=0x55b25f3ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a251fb700 1 --2- 172.21.15.19:0/126982812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f3c0400 0x55b25f3ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a251fb700 1 -- 172.21.15.19:0/126982812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 msgr2=0x55b25f3c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a251fb700 1 --2- 172.21.15.19:0/126982812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a251fb700 1 -- 172.21.15.19:0/126982812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b25f3bbf10 con 0x55b25f2bdd00 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a251fb700 1 --2- 172.21.15.19:0/126982812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3cc4b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8a10022a80 tx=0x7f8a10009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb81b01430b5b1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a0b7fe700 1 -- 172.21.15.19:0/126982812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a100aa470 con 0x55b25f2bdd00 2024-05-09T01:30:56.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a0b7fe700 1 -- 172.21.15.19:0/126982812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a100ab060 con 0x55b25f2bdd00 2024-05-09T01:30:56.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.726+0000 7f8a0b7fe700 1 -- 172.21.15.19:0/126982812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a100aabc0 con 0x55b25f2bdd00 2024-05-09T01:30:56.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.727+0000 7f8a36676b80 1 -- 172.21.15.19:0/126982812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 msgr2=0x55b25f3cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:56.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.727+0000 7f8a36676b80 1 --2- 172.21.15.19:0/126982812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3cc4b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8a10022a80 tx=0x7f8a10009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.727+0000 7f8a36676b80 1 -- 172.21.15.19:0/126982812 shutdown_connections 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.727+0000 7f8a36676b80 1 --2- 172.21.15.19:0/126982812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f3c0400 0x55b25f3ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.727+0000 7f8a36676b80 1 --2- 172.21.15.19:0/126982812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f2434a0 0x55b25f3c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.727+0000 7f8a36676b80 1 --2- 172.21.15.19:0/126982812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3cc4b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.727+0000 7f8a36676b80 1 -- 172.21.15.19:0/126982812 >> 172.21.15.19:0/126982812 conn(0x55b25f246f70 msgr2=0x55b25f3b4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.727+0000 7f8a36676b80 1 -- 172.21.15.19:0/126982812 shutdown_connections 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.727+0000 7f8a36676b80 1 -- 172.21.15.19:0/126982812 wait complete. 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.728+0000 7f8a36676b80 1 -- start start 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.728+0000 7f8a36676b80 1 Processor -- start 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.728+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2434a0 0x55b25f3d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.728+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.728+0000 7f8a36676b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f3c0400 0x55b25f3d4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.728+0000 7f8a36676b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400e890 con 0x55b25f2bdd00 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.728+0000 7f8a36676b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400e1c0 con 0x55b25f3c0400 2024-05-09T01:30:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.728+0000 7f8a36676b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400b3c0 con 0x55b25f2434a0 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a36676b80 10 client.0 _ll_register_callbacks cb 0x55b25f1546c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a251fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a251fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3d2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56410/0 (socket says 172.21.15.19:56410) 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a251fb700 1 -- 172.21.15.19:0/2235228547 learned_addr learned my addr 172.21.15.19:0/2235228547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a261fd700 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f3c0400 0x55b25f3d4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a259fc700 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2434a0 0x55b25f3d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a251fb700 1 -- 172.21.15.19:0/2235228547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2434a0 msgr2=0x55b25f3d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a251fb700 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2434a0 0x55b25f3d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a251fb700 1 -- 172.21.15.19:0/2235228547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f3c0400 msgr2=0x55b25f3d4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a251fb700 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f3c0400 0x55b25f3d4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a251fb700 1 -- 172.21.15.19:0/2235228547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25f3d68d0 con 0x55b25f2bdd00 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a251fb700 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3d2620 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a10022b20 tx=0x7f8a100b3550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43d76cfed9442bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a097fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a097fa700 1 -- 172.21.15.19:0/2235228547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a100aed10 con 0x55b25f2bdd00 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a097fa700 1 -- 172.21.15.19:0/2235228547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8a100cbb80 con 0x55b25f2bdd00 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.729+0000 7f8a36676b80 1 -- 172.21.15.19:0/2235228547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a100ab060 con 0x55b25f2bdd00 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a097fa700 1 -- 172.21.15.19:0/2235228547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a100b3080 con 0x55b25f2bdd00 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a36676b80 10 client.16770 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a36676b80 1 -- 172.21.15.19:0/2235228547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b25f443a80 con 0x55b25f2bdd00 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a097fa700 1 -- 172.21.15.19:0/2235228547 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a100ab060 con 0x55b25f2bdd00 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a097fa700 1 -- 172.21.15.19:0/2235228547 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8a100c9030 con 0x55b25f2bdd00 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a36676b80 10 client.16770 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a36676b80 10 client.16770 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a36676b80 20 client.16770 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55b25f4440d0 0x55b25f4464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a36676b80 4 client.16770 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a36676b80 1 -- 172.21.15.19:0/2235228547 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b25f2eab20 con 0x55b25f4440d0 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.730+0000 7f8a259fc700 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55b25f4440d0 0x55b25f4464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a097fa700 10 client.16770 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:56.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a259fc700 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55b25f4440d0 0x55b25f4464f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=892b65cbdba52937 server_cookie=81d90cb91a42a936 in_seq=0 out_seq=0 2024-05-09T01:30:56.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a097fa700 1 -- 172.21.15.19:0/2235228547 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b25f2eab20 con 0x55b25f4440d0 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a097fa700 10 client.16770 handle_command_reply: tid=0 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 client.16770 shutdown 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 -- 172.21.15.19:0/2235228547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 msgr2=0x55b25f3d2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3d2620 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a10022b20 tx=0x7f8a100b3550 comp rx=0 tx=0).stop 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 -- 172.21.15.19:0/2235228547 shutdown_connections 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55b25f4440d0 0x55b25f4464f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b25f3c0400 0x55b25f3d4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b25f2434a0 0x55b25f3d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a097fa700 0 client.16770 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 --2- 172.21.15.19:0/2235228547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b25f2bdd00 0x55b25f3d2620 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 -- 172.21.15.19:0/2235228547 >> 172.21.15.19:0/2235228547 conn(0x55b25f246f70 msgr2=0x55b25f3c3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 -- 172.21.15.19:0/2235228547 shutdown_connections 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 1 -- 172.21.15.19:0/2235228547 wait complete. 2024-05-09T01:30:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.731+0000 7f8a36676b80 20 client.16770 trim_cache size 0 max 16384 2024-05-09T01:30:56.733 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 -- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8a0c0b2900 msgr2=0x7f8a0c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005ea0 tx=0x7f8a1c005e30 comp rx=0 tx=0).stop 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 -- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 msgr2=0x7f8a2019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 0x7f8a2019b430 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009f50 tx=0x7f8a100037c0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 -- 172.21.15.19:0/1194121005 shutdown_connections 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20109850 0x7f8a2019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a201045d0 0x7f8a2019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 --2- 172.21.15.19:0/1194121005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a20108f00 0x7f8a2019b430 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 -- 172.21.15.19:0/1194121005 >> 172.21.15.19:0/1194121005 conn(0x7f8a200ffea0 msgr2=0x7f8a2010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 -- 172.21.15.19:0/1194121005 shutdown_connections 2024-05-09T01:30:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:56.732+0000 7f8a27c60700 1 -- 172.21.15.19:0/1194121005 wait complete. 2024-05-09T01:30:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:56 smithi059 ceph-mon[34351]: pgmap v540: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:30:57.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:30:57.743 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:30:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.203+0000 7f65cc9c0640 1 -- 172.21.15.19:0/885198475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c81070d0 msgr2=0x7f65c81094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.203+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/885198475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c81070d0 0x7f65c81094f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009900 tx=0x7f65bc02a410 comp rx=0 tx=0).stop 2024-05-09T01:30:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.203+0000 7f65cc9c0640 1 -- 172.21.15.19:0/885198475 shutdown_connections 2024-05-09T01:30:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.203+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/885198475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c8109a30 0x7f65c810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.203+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/885198475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c81070d0 0x7f65c81094f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.203+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/885198475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c8104770 0x7f65c8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.203+0000 7f65cc9c0640 1 -- 172.21.15.19:0/885198475 >> 172.21.15.19:0/885198475 conn(0x7f65c80fdd90 msgr2=0x7f65c8100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.203+0000 7f65cc9c0640 1 -- 172.21.15.19:0/885198475 shutdown_connections 2024-05-09T01:30:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.203+0000 7f65cc9c0640 1 -- 172.21.15.19:0/885198475 wait complete. 2024-05-09T01:30:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65cc9c0640 1 Processor -- start 2024-05-09T01:30:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65cc9c0640 1 -- start start 2024-05-09T01:30:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65cc9c0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c8104770 0x7f65c819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65cc9c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c81070d0 0x7f65c819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65cc9c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c8109a30 0x7f65c81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65cc9c0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810f7b0 con 0x7f65c8109a30 2024-05-09T01:30:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65cc9c0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810f630 con 0x7f65c8104770 2024-05-09T01:30:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65cc9c0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810f930 con 0x7f65c81070d0 2024-05-09T01:30:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c8109a30 0x7f65c81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c8109a30 0x7f65c81a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56432/0 (socket says 172.21.15.19:56432) 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c6d9d640 1 -- 172.21.15.19:0/2857125115 learned_addr learned my addr 172.21.15.19:0/2857125115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c659c640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c8104770 0x7f65c819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c81070d0 0x7f65c819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c6d9d640 1 -- 172.21.15.19:0/2857125115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c81070d0 msgr2=0x7f65c819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c6d9d640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c81070d0 0x7f65c819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c6d9d640 1 -- 172.21.15.19:0/2857125115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c8104770 msgr2=0x7f65c819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c6d9d640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c8104770 0x7f65c819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.204+0000 7f65c6d9d640 1 -- 172.21.15.19:0/2857125115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c81a1ec0 con 0x7f65c8109a30 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.205+0000 7f65c6d9d640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c8109a30 0x7f65c81a1850 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f65c000eb40 tx=0x7f65c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.205+0000 7f65b77fe640 1 -- 172.21.15.19:0/2857125115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0004b20 con 0x7f65c8109a30 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.205+0000 7f65b77fe640 1 -- 172.21.15.19:0/2857125115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c0004cc0 con 0x7f65c8109a30 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.205+0000 7f65cc9c0640 1 -- 172.21.15.19:0/2857125115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c81a21b0 con 0x7f65c8109a30 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.205+0000 7f65b77fe640 1 -- 172.21.15.19:0/2857125115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0010720 con 0x7f65c8109a30 2024-05-09T01:30:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.205+0000 7f65cc9c0640 1 -- 172.21.15.19:0/2857125115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c81a2750 con 0x7f65c8109a30 2024-05-09T01:30:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65d3870740 1 Processor -- start 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65b77fe640 1 -- 172.21.15.19:0/2857125115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f65c00108c0 con 0x7f65c8109a30 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65d3870740 1 -- start start 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65d3870740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b461b0 0x55e898eee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65d3870740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eeee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65d3870740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 0x55e898ef5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65d3870740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e898e4e7c0 con 0x55e898b1d2f0 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65d3870740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e898dea0e0 con 0x55e898b461b0 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65d3870740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e898e4a780 con 0x55e898b254d0 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.208+0000 7f65c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eeee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eeee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56438/0 (socket says 172.21.15.19:56438) 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c659c640 1 -- 172.21.15.19:0/4094368237 learned_addr learned my addr 172.21.15.19:0/4094368237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/4094368237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b461b0 0x55e898eee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/4094368237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 0x55e898ef5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c659c640 1 -- 172.21.15.19:0/4094368237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 msgr2=0x55e898ef5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c659c640 1 --2- 172.21.15.19:0/4094368237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 0x55e898ef5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c659c640 1 -- 172.21.15.19:0/4094368237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b461b0 msgr2=0x55e898eee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c659c640 1 --2- 172.21.15.19:0/4094368237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b461b0 0x55e898eee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c659c640 1 -- 172.21.15.19:0/4094368237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e898e137c0 con 0x55e898b1d2f0 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65b77fe640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65b00b2860 0x7f65b00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65b00b2860 0x7f65b00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65b77fe640 1 -- 172.21.15.19:0/2857125115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f65c00e2890 con 0x7f65c8109a30 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65c659c640 1 --2- 172.21.15.19:0/4094368237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eeee90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f65b800bd10 tx=0x7f65b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8fc1c73db77711 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65b4ff9640 1 -- 172.21.15.19:0/4094368237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8005220 con 0x55e898b1d2f0 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65b4ff9640 1 -- 172.21.15.19:0/4094368237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b8004c50 con 0x55e898b1d2f0 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.209+0000 7f65b4ff9640 1 -- 172.21.15.19:0/4094368237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b80117c0 con 0x55e898b1d2f0 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.210+0000 7f65d3870740 1 -- 172.21.15.19:0/4094368237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 msgr2=0x55e898eeee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.210+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65b00b2860 0x7f65b00b4d70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009230 tx=0x7f65bc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.210+0000 7f65d3870740 1 --2- 172.21.15.19:0/4094368237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eeee90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f65b800bd10 tx=0x7f65b80107a0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.210+0000 7f65d3870740 1 -- 172.21.15.19:0/4094368237 shutdown_connections 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.210+0000 7f65d3870740 1 --2- 172.21.15.19:0/4094368237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 0x55e898ef5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.210+0000 7f65d3870740 1 --2- 172.21.15.19:0/4094368237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eeee90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.210+0000 7f65d3870740 1 --2- 172.21.15.19:0/4094368237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b461b0 0x55e898eee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.210+0000 7f65d3870740 1 -- 172.21.15.19:0/4094368237 >> 172.21.15.19:0/4094368237 conn(0x55e898b3a300 msgr2=0x55e898ee5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 -- 172.21.15.19:0/4094368237 shutdown_connections 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 -- 172.21.15.19:0/4094368237 wait complete. 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 -- start start 2024-05-09T01:30:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 Processor -- start 2024-05-09T01:30:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b1d2f0 0x55e898ef85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 0x55e898ef8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b461b0 0x55e898effa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e898ef67e0 con 0x55e898b461b0 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e898dd59c0 con 0x55e898b1d2f0 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e898ef6960 con 0x55e898b254d0 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65d3870740 10 client.0 _ll_register_callbacks cb 0x55e898e02020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b461b0 0x55e898effa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b461b0 0x55e898effa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56454/0 (socket says 172.21.15.19:56454) 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.211+0000 7f65c6d9d640 1 -- 172.21.15.19:0/1367258176 learned_addr learned my addr 172.21.15.19:0/1367258176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65c659c640 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b1d2f0 0x55e898ef85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 0x55e898ef8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65c6d9d640 1 -- 172.21.15.19:0/1367258176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 msgr2=0x55e898ef8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65c6d9d640 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 0x55e898ef8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65c6d9d640 1 -- 172.21.15.19:0/1367258176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b1d2f0 msgr2=0x55e898ef85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65c6d9d640 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b1d2f0 0x55e898ef85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65c6d9d640 1 -- 172.21.15.19:0/1367258176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e898dfc1f0 con 0x55e898b461b0 2024-05-09T01:30:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65c659c640 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b1d2f0 0x55e898ef85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65c6d9d640 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b461b0 0x55e898effa80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f65c00a5f10 tx=0x7f65c00a9a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d893b7f2349dc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65adffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65adffb640 1 -- 172.21.15.19:0/1367258176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00ae240 con 0x55e898b461b0 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65adffb640 1 -- 172.21.15.19:0/1367258176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f65c00b2860 con 0x55e898b461b0 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65d3870740 1 -- 172.21.15.19:0/1367258176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b8004c50 con 0x55e898b461b0 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65adffb640 1 -- 172.21.15.19:0/1367258176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65c00b2cf0 con 0x55e898b461b0 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65d3870740 10 client.16782 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65d3870740 1 -- 172.21.15.19:0/1367258176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e898bb1790 con 0x55e898b461b0 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.212+0000 7f65adffb640 1 -- 172.21.15.19:0/1367258176 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b8004c50 con 0x55e898b461b0 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.213+0000 7f65adffb640 1 -- 172.21.15.19:0/1367258176 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f65c00a5800 con 0x55e898b461b0 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.213+0000 7f65d3870740 10 client.16782 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.213+0000 7f65d3870740 10 client.16782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.213+0000 7f65d3870740 20 client.16782 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.213+0000 7f65d3870740 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e898b04d80 0x55e898f05fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.213+0000 7f65d3870740 4 client.16782 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.213+0000 7f65d3870740 1 -- 172.21.15.19:0/1367258176 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e898e5c3d0 con 0x55e898b04d80 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.213+0000 7f65c659c640 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e898b04d80 0x55e898f05fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.214+0000 7f65c659c640 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e898b04d80 0x55e898f05fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741791401e257bd3 server_cookie=da46f93dea12364f in_seq=0 out_seq=0 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.214+0000 7f65adffb640 10 client.16782 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.214+0000 7f65adffb640 1 -- 172.21.15.19:0/1367258176 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e898e5c3d0 con 0x55e898b04d80 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.214+0000 7f65adffb640 10 client.16782 handle_command_reply: tid=0 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 client.16782 shutdown 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 -- 172.21.15.19:0/1367258176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b461b0 msgr2=0x55e898effa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b461b0 0x55e898effa80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f65c00a5f10 tx=0x7f65c00a9a60 comp rx=0 tx=0).stop 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 -- 172.21.15.19:0/1367258176 shutdown_connections 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e898b04d80 0x55e898f05fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b461b0 0x55e898effa80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b254d0 0x55e898ef8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 --2- 172.21.15.19:0/1367258176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b1d2f0 0x55e898ef85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65adffb640 0 client.16782 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 -- 172.21.15.19:0/1367258176 >> 172.21.15.19:0/1367258176 conn(0x55e898b3a300 msgr2=0x55e898ee8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 -- 172.21.15.19:0/1367258176 shutdown_connections 2024-05-09T01:30:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 1 -- 172.21.15.19:0/1367258176 wait complete. 2024-05-09T01:30:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.215+0000 7f65d3870740 20 client.16782 trim_cache size 0 max 16384 2024-05-09T01:30:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65d3870740 1 Processor -- start 2024-05-09T01:30:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65d3870740 1 -- start start 2024-05-09T01:30:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65d3870740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65d3870740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 0x55e898eeedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65d3870740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 0x55e898ef2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65d3870740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81a91a0 con 0x55e898b1d2f0 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65d3870740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81aace0 con 0x55e898b254d0 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65d3870740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81a9870 con 0x55e898b461b0 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eee890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56464/0 (socket says 172.21.15.19:56464) 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.230+0000 7f65c659c640 1 -- 172.21.15.19:0/94962223 learned_addr learned my addr 172.21.15.19:0/94962223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 0x55e898eeedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65c6d9d640 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 0x55e898ef2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65c659c640 1 -- 172.21.15.19:0/94962223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 msgr2=0x55e898ef2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65c659c640 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 0x55e898ef2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65c659c640 1 -- 172.21.15.19:0/94962223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 msgr2=0x55e898eeedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65c659c640 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 0x55e898eeedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65c659c640 1 -- 172.21.15.19:0/94962223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e898c285f0 con 0x55e898b1d2f0 2024-05-09T01:30:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65c6d9d640 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 0x55e898ef2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:30:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65c659c640 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eee890 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006010 tx=0x7f65b800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b167c42b6163fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65affff640 1 -- 172.21.15.19:0/94962223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b802a370 con 0x55e898b1d2f0 2024-05-09T01:30:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65affff640 1 -- 172.21.15.19:0/94962223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b802a9c0 con 0x55e898b1d2f0 2024-05-09T01:30:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.231+0000 7f65affff640 1 -- 172.21.15.19:0/94962223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8011d20 con 0x55e898b1d2f0 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 -- 172.21.15.19:0/94962223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 msgr2=0x55e898eee890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eee890 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006010 tx=0x7f65b800b410 comp rx=0 tx=0).stop 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 -- 172.21.15.19:0/94962223 shutdown_connections 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 0x55e898ef2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 0x55e898eeedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 --2- 172.21.15.19:0/94962223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898eee890 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 -- 172.21.15.19:0/94962223 >> 172.21.15.19:0/94962223 conn(0x55e898b3a300 msgr2=0x55e898ef6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 -- 172.21.15.19:0/94962223 shutdown_connections 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 -- 172.21.15.19:0/94962223 wait complete. 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.232+0000 7f65d3870740 1 -- start start 2024-05-09T01:30:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65d3870740 1 Processor -- start 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65d3870740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898efc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65d3870740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 0x55e898efcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65d3870740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 0x55e898efdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65d3870740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81aa610 con 0x55e898b1d2f0 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65d3870740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81a9f40 con 0x55e898b254d0 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65d3870740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81a7180 con 0x55e898b461b0 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65d3870740 10 client.0 _ll_register_callbacks cb 0x7f65c000ceb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898efc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898efc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56474/0 (socket says 172.21.15.19:56474) 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c659c640 1 -- 172.21.15.19:0/1133435613 learned_addr learned my addr 172.21.15.19:0/1133435613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 0x55e898efcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c6d9d640 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 0x55e898efdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c659c640 1 -- 172.21.15.19:0/1133435613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 msgr2=0x55e898efdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c659c640 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 0x55e898efdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c659c640 1 -- 172.21.15.19:0/1133435613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 msgr2=0x55e898efcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c659c640 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 0x55e898efcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.233+0000 7f65c659c640 1 -- 172.21.15.19:0/1133435613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e898e12ed0 con 0x55e898b1d2f0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65c659c640 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898efc6c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f65b8000c00 tx=0x7f65b8010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f61a58569ee811d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65adffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65adffb640 1 -- 172.21.15.19:0/1133435613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8038070 con 0x55e898b1d2f0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65adffb640 1 -- 172.21.15.19:0/1133435613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f65b8011aa0 con 0x55e898b1d2f0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65d3870740 1 -- 172.21.15.19:0/1133435613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b802a9c0 con 0x55e898b1d2f0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65adffb640 1 -- 172.21.15.19:0/1133435613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65b8011cc0 con 0x55e898b1d2f0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65d3870740 10 client.16788 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65d3870740 1 -- 172.21.15.19:0/1133435613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e898eff120 con 0x55e898b1d2f0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65adffb640 1 -- 172.21.15.19:0/1133435613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b802a9c0 con 0x55e898b1d2f0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65adffb640 1 -- 172.21.15.19:0/1133435613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f65b803b030 con 0x55e898b1d2f0 2024-05-09T01:30:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65d3870740 10 client.16788 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65d3870740 10 client.16788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65d3870740 20 client.16788 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65d3870740 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e898ada1c0 0x55e898f706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65d3870740 4 client.16788 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:30:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.234+0000 7f65d3870740 1 -- 172.21.15.19:0/1133435613 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e898e56240 con 0x55e898ada1c0 2024-05-09T01:30:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.235+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e898ada1c0 0x55e898f706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.235+0000 7f65adffb640 10 client.16788 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.235+0000 7f65c5d9b640 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e898ada1c0 0x55e898f706b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25bb24aeabde8ff1 server_cookie=ca56f5867cca2c6b in_seq=0 out_seq=0 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65adffb640 1 -- 172.21.15.19:0/1133435613 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e898e56240 con 0x55e898ada1c0 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65adffb640 10 client.16788 handle_command_reply: tid=0 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 client.16788 shutdown 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 -- 172.21.15.19:0/1133435613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 msgr2=0x55e898efc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898efc6c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f65b8000c00 tx=0x7f65b8010d60 comp rx=0 tx=0).stop 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 -- 172.21.15.19:0/1133435613 shutdown_connections 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e898ada1c0 0x55e898f706b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e898b461b0 0x55e898efdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65adffb640 0 client.16788 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e898b254d0 0x55e898efcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 --2- 172.21.15.19:0/1133435613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e898b1d2f0 0x55e898efc6c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 -- 172.21.15.19:0/1133435613 >> 172.21.15.19:0/1133435613 conn(0x55e898b3a300 msgr2=0x55e898ef6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 -- 172.21.15.19:0/1133435613 shutdown_connections 2024-05-09T01:30:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 1 -- 172.21.15.19:0/1133435613 wait complete. 2024-05-09T01:30:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.253+0000 7f65d3870740 20 client.16788 trim_cache size 0 max 16384 2024-05-09T01:30:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:30:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:30:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 -- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65b00b2860 msgr2=0x7f65b00b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65b00b2860 0x7f65b00b4d70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009230 tx=0x7f65bc000bc0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 -- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c8109a30 msgr2=0x7f65c81a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c8109a30 0x7f65c81a1850 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f65c000eb40 tx=0x7f65c000c2d0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 -- 172.21.15.19:0/2857125115 shutdown_connections 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f65b00b2860 0x7f65b00b4d70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65c8109a30 0x7f65c81a1850 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c81070d0 0x7f65c819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 --2- 172.21.15.19:0/2857125115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c8104770 0x7f65c819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 -- 172.21.15.19:0/2857125115 >> 172.21.15.19:0/2857125115 conn(0x7f65c80fdd90 msgr2=0x7f65c8105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 -- 172.21.15.19:0/2857125115 shutdown_connections 2024-05-09T01:30:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:30:58.254+0000 7f65cc9c0640 1 -- 172.21.15.19:0/2857125115 wait complete. 2024-05-09T01:30:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:30:58 smithi196 ceph-mon[35041]: pgmap v541: 97 pgs: 97 active+clean; 3.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:30:58 smithi019 ceph-mon[30158]: pgmap v541: 97 pgs: 97 active+clean; 3.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:30:58 smithi059 ceph-mon[34351]: pgmap v541: 97 pgs: 97 active+clean; 3.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:30:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.329+0000 7f5d21bfe700 1 -- 172.21.15.19:0/109069806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d1c107e10 msgr2=0x7f5d1c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.329+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/109069806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d1c107e10 0x7f5d1c108220 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5d100099c0 tx=0x7f5d10009cd0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.329+0000 7f5d21bfe700 1 -- 172.21.15.19:0/109069806 shutdown_connections 2024-05-09T01:30:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.329+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/109069806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c108760 0x7f5d1c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.329+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/109069806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d1c1034e0 0x7f5d1c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.329+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/109069806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d1c107e10 0x7f5d1c108220 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.329+0000 7f5d21bfe700 1 -- 172.21.15.19:0/109069806 >> 172.21.15.19:0/109069806 conn(0x7f5d1c0fef30 msgr2=0x7f5d1c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.329+0000 7f5d21bfe700 1 -- 172.21.15.19:0/109069806 shutdown_connections 2024-05-09T01:30:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.329+0000 7f5d21bfe700 1 -- 172.21.15.19:0/109069806 wait complete. 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.330+0000 7f5d21bfe700 1 Processor -- start 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d21bfe700 1 -- start start 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d21bfe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c1034e0 0x7f5d1c199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d21bfe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d1c107e10 0x7f5d1c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d21bfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d1c108760 0x7f5d1c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d21bfe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c113370 con 0x7f5d1c108760 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d21bfe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1131f0 con 0x7f5d1c1034e0 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d21bfe700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1134f0 con 0x7f5d1c107e10 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c1034e0 0x7f5d1c199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d1bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d1c108760 0x7f5d1c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c1034e0 0x7f5d1c199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47084/0 (socket says 172.21.15.19:47084) 2024-05-09T01:30:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.331+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/2370675649 learned_addr learned my addr 172.21.15.19:0/2370675649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.332+0000 7f5d1affd700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d1c107e10 0x7f5d1c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.332+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/2370675649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d1c107e10 msgr2=0x7f5d1c19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.332+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d1c107e10 0x7f5d1c19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.332+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/2370675649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d1c108760 msgr2=0x7f5d1c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.332+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d1c108760 0x7f5d1c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.332+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/2370675649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1c19ecf0 con 0x7f5d1c1034e0 2024-05-09T01:30:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.332+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c1034e0 0x7f5d1c199e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00d950 tx=0x7f5d0c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.332+0000 7f5d18ff9700 1 -- 172.21.15.19:0/2370675649 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c016070 con 0x7f5d1c1034e0 2024-05-09T01:30:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.333+0000 7f5d21bfe700 1 -- 172.21.15.19:0/2370675649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1c19efe0 con 0x7f5d1c1034e0 2024-05-09T01:30:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.333+0000 7f5d18ff9700 1 -- 172.21.15.19:0/2370675649 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0c00fd40 con 0x7f5d1c1034e0 2024-05-09T01:30:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.333+0000 7f5d18ff9700 1 -- 172.21.15.19:0/2370675649 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c00e3b0 con 0x7f5d1c1034e0 2024-05-09T01:30:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.333+0000 7f5d21bfe700 1 -- 172.21.15.19:0/2370675649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d1c1a6900 con 0x7f5d1c1034e0 2024-05-09T01:30:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.336+0000 7f5d18ff9700 1 -- 172.21.15.19:0/2370675649 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5d0c019070 con 0x7f5d1c1034e0 2024-05-09T01:30:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.336+0000 7f5d18ff9700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d040b2880 0x7f5d040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d30614b80 1 Processor -- start 2024-05-09T01:30:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d30614b80 1 -- start start 2024-05-09T01:30:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d1affd700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d040b2880 0x7f5d040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d18ff9700 1 -- 172.21.15.19:0/2370675649 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5d0c0e1fc0 con 0x7f5d1c1034e0 2024-05-09T01:30:59.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74bfbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74bfc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c08bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d30614b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f74b1cff0 con 0x559f74a7d4a0 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d30614b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559f74abb7f0 con 0x559f74b67160 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d30614b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f74b18fb0 con 0x559f74c067b0 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74bfc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74bfc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56492/0 (socket says 172.21.15.19:56492) 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d1affd700 1 -- 172.21.15.19:0/62937830 learned_addr learned my addr 172.21.15.19:0/62937830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d1affd700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d040b2880 0x7f5d040b4d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5d10006010 tx=0x7f5d10000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d1bfff700 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c08bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.337+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74bfbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d1affd700 1 -- 172.21.15.19:0/62937830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 msgr2=0x559f74c08bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d1affd700 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c08bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d1affd700 1 -- 172.21.15.19:0/62937830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 msgr2=0x559f74bfbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d1affd700 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74bfbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d1affd700 1 -- 172.21.15.19:0/62937830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f74ab3370 con 0x559f74a7d4a0 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d1bfff700 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c08bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:30:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d1affd700 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74bfc040 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5d10015000 tx=0x7f5d1001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b558769b3046a2a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:59.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d01ffb700 1 -- 172.21.15.19:0/62937830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10005460 con 0x559f74a7d4a0 2024-05-09T01:30:59.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d01ffb700 1 -- 172.21.15.19:0/62937830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d10005600 con 0x559f74a7d4a0 2024-05-09T01:30:59.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.338+0000 7f5d01ffb700 1 -- 172.21.15.19:0/62937830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000fae0 con 0x559f74a7d4a0 2024-05-09T01:30:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 -- 172.21.15.19:0/62937830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 msgr2=0x559f74bfc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74bfc040 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5d10015000 tx=0x7f5d1001e000 comp rx=0 tx=0).stop 2024-05-09T01:30:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 -- 172.21.15.19:0/62937830 shutdown_connections 2024-05-09T01:30:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c08bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74bfbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 --2- 172.21.15.19:0/62937830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74bfc040 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 -- 172.21.15.19:0/62937830 >> 172.21.15.19:0/62937830 conn(0x559f74add6a0 msgr2=0x559f74bf7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 -- 172.21.15.19:0/62937830 shutdown_connections 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 -- 172.21.15.19:0/62937830 wait complete. 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.339+0000 7f5d30614b80 1 -- start start 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d30614b80 1 Processor -- start 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c03760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74c03ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d30614b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559f74998ba0 con 0x559f74a7d4a0 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d30614b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559f74ad7150 con 0x559f74b67160 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d30614b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f74c09200 con 0x559f74c067b0 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d30614b80 10 client.0 _ll_register_callbacks cb 0x559f7498e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c03760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c03760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56500/0 (socket says 172.21.15.19:56500) 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/1582548819 learned_addr learned my addr 172.21.15.19:0/1582548819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d1affd700 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74c03ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.340+0000 7f5d1bfff700 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/1582548819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 msgr2=0x559f74c041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/1582548819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 msgr2=0x559f74c03ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74c03ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/1582548819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f74998490 con 0x559f74a7d4a0 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c03760 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0a9360 tx=0x7f5d0c0a9e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d829e092ea0991b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5cfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5cfaffd700 1 -- 172.21.15.19:0/1582548819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c020040 con 0x559f74a7d4a0 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5cfaffd700 1 -- 172.21.15.19:0/1582548819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5d0c0106a0 con 0x559f74a7d4a0 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5d30614b80 1 -- 172.21.15.19:0/1582548819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d10005600 con 0x559f74a7d4a0 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5cfaffd700 1 -- 172.21.15.19:0/1582548819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d0c0b24c0 con 0x559f74a7d4a0 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5d30614b80 10 client.16800 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5d30614b80 1 -- 172.21.15.19:0/1582548819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f74c13d20 con 0x559f74a7d4a0 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.341+0000 7f5cfaffd700 1 -- 172.21.15.19:0/1582548819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d10005600 con 0x559f74a7d4a0 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5cfaffd700 1 -- 172.21.15.19:0/1582548819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d0c0b2030 con 0x559f74a7d4a0 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5d30614b80 10 client.16800 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5d30614b80 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5d30614b80 20 client.16800 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5d30614b80 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559f74c161f0 0x559f74c18650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5d30614b80 4 client.16800 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:30:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5d30614b80 1 -- 172.21.15.19:0/1582548819 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f74b2a9d0 con 0x559f74c161f0 2024-05-09T01:30:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5d1affd700 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559f74c161f0 0x559f74c18650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5d1affd700 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559f74c161f0 0x559f74c18650 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a4c9ad7cd8e044 server_cookie=204beef7313c1cd3 in_seq=0 out_seq=0 2024-05-09T01:30:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.342+0000 7f5cfaffd700 10 client.16800 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.343+0000 7f5cfaffd700 1 -- 172.21.15.19:0/1582548819 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559f74b2a9d0 con 0x559f74c161f0 2024-05-09T01:30:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.343+0000 7f5cfaffd700 10 client.16800 handle_command_reply: tid=0 2024-05-09T01:30:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.343+0000 7f5d30614b80 1 client.16800 shutdown 2024-05-09T01:30:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.343+0000 7f5d30614b80 1 -- 172.21.15.19:0/1582548819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 msgr2=0x559f74c03760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.343+0000 7f5d30614b80 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c03760 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0a9360 tx=0x7f5d0c0a9e30 comp rx=0 tx=0).stop 2024-05-09T01:30:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5d30614b80 1 -- 172.21.15.19:0/1582548819 shutdown_connections 2024-05-09T01:30:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5d30614b80 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559f74c161f0 0x559f74c18650 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5d30614b80 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5d30614b80 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c03760 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5cfaffd700 0 client.16800 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:59.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5d30614b80 1 --2- 172.21.15.19:0/1582548819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74c03ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5d30614b80 1 -- 172.21.15.19:0/1582548819 >> 172.21.15.19:0/1582548819 conn(0x559f74add6a0 msgr2=0x559f74bfbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5d30614b80 1 -- 172.21.15.19:0/1582548819 shutdown_connections 2024-05-09T01:30:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5d30614b80 1 -- 172.21.15.19:0/1582548819 wait complete. 2024-05-09T01:30:59.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.344+0000 7f5d30614b80 20 client.16800 trim_cache size 0 max 16384 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d30614b80 1 Processor -- start 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d30614b80 1 -- start start 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74a7d4a0 0x559f74c05c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74b67160 0x559f74bfedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74c067b0 0x559f74c04020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d30614b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1ac980 con 0x559f74b67160 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d30614b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1ae4c0 con 0x559f74c067b0 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d30614b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1ad050 con 0x559f74a7d4a0 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74b67160 0x559f74bfedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74b67160 0x559f74bfedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56504/0 (socket says 172.21.15.19:56504) 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.359+0000 7f5d1affd700 1 -- 172.21.15.19:0/2105880603 learned_addr learned my addr 172.21.15.19:0/2105880603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/2105880603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74a7d4a0 0x559f74c05c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d1affd700 1 -- 172.21.15.19:0/2105880603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74a7d4a0 msgr2=0x559f74c05c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d1bfff700 1 --2- 172.21.15.19:0/2105880603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74c067b0 0x559f74c04020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d1affd700 1 --2- 172.21.15.19:0/2105880603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74a7d4a0 0x559f74c05c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d1affd700 1 -- 172.21.15.19:0/2105880603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74c067b0 msgr2=0x559f74c04020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d1affd700 1 --2- 172.21.15.19:0/2105880603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74c067b0 0x559f74c04020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d1affd700 1 -- 172.21.15.19:0/2105880603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f74beef60 con 0x559f74b67160 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d1affd700 1 --2- 172.21.15.19:0/2105880603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74b67160 0x559f74bfedb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000b700 tx=0x7f5d1000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c765c269b5e98c04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d017fa700 1 -- 172.21.15.19:0/2105880603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000f790 con 0x559f74b67160 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d017fa700 1 -- 172.21.15.19:0/2105880603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1003c070 con 0x559f74b67160 2024-05-09T01:30:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.360+0000 7f5d017fa700 1 -- 172.21.15.19:0/2105880603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10037260 con 0x559f74b67160 2024-05-09T01:30:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 -- 172.21.15.19:0/2105880603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74b67160 msgr2=0x559f74bfedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 --2- 172.21.15.19:0/2105880603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74b67160 0x559f74bfedb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000b700 tx=0x7f5d1000f720 comp rx=0 tx=0).stop 2024-05-09T01:30:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 -- 172.21.15.19:0/2105880603 shutdown_connections 2024-05-09T01:30:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 --2- 172.21.15.19:0/2105880603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74c067b0 0x559f74c04020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 --2- 172.21.15.19:0/2105880603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74a7d4a0 0x559f74c05c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 --2- 172.21.15.19:0/2105880603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74b67160 0x559f74bfedb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 -- 172.21.15.19:0/2105880603 >> 172.21.15.19:0/2105880603 conn(0x559f74add6a0 msgr2=0x559f74bfe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 -- 172.21.15.19:0/2105880603 shutdown_connections 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 -- 172.21.15.19:0/2105880603 wait complete. 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.361+0000 7f5d30614b80 1 -- start start 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d30614b80 1 Processor -- start 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c06bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74c07100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d30614b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c07640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d30614b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1addf0 con 0x559f74a7d4a0 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d30614b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1ad720 con 0x559f74b67160 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d30614b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1aa940 con 0x559f74c067b0 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d30614b80 10 client.0 _ll_register_callbacks cb 0x559f7498e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c06bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c06bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56516/0 (socket says 172.21.15.19:56516) 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/429509666 learned_addr learned my addr 172.21.15.19:0/429509666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:30:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1bfff700 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c07640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1affd700 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74c07100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/429509666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 msgr2=0x559f74c07640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c07640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/429509666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 msgr2=0x559f74c07100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74c07100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.362+0000 7f5d1b7fe700 1 -- 172.21.15.19:0/429509666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f74c0d4c0 con 0x559f74a7d4a0 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d1b7fe700 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c06bc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0a95f0 tx=0x7f5d0c0108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdda35042a3c608 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5cfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5cfaffd700 1 -- 172.21.15.19:0/429509666 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c020050 con 0x559f74a7d4a0 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5cfaffd700 1 -- 172.21.15.19:0/429509666 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5d0c0a8990 con 0x559f74a7d4a0 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d30614b80 1 -- 172.21.15.19:0/429509666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1003c070 con 0x559f74a7d4a0 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5cfaffd700 1 -- 172.21.15.19:0/429509666 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d0c0c9070 con 0x559f74a7d4a0 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d30614b80 10 client.16806 fetch_fsmap learned FSMap version 28 2024-05-09T01:30:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d30614b80 1 -- 172.21.15.19:0/429509666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f74c0e720 con 0x559f74a7d4a0 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5cfaffd700 1 -- 172.21.15.19:0/429509666 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1003c070 con 0x559f74a7d4a0 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5cfaffd700 1 -- 172.21.15.19:0/429509666 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d0c0a4fd0 con 0x559f74a7d4a0 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d30614b80 10 client.16806 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d30614b80 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d30614b80 20 client.16806 populate_metadata read hostname 'smithi019' 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d30614b80 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559f74c0ec00 0x559f74c829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d30614b80 4 client.16806 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.363+0000 7f5d30614b80 1 -- 172.21.15.19:0/429509666 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f74b24a70 con 0x559f74c0ec00 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.364+0000 7f5d1affd700 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559f74c0ec00 0x559f74c829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.364+0000 7f5d1affd700 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559f74c0ec00 0x559f74c829d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be02765f2f4be633 server_cookie=909b7b99dd33e864 in_seq=0 out_seq=0 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.364+0000 7f5cfaffd700 10 client.16806 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.364+0000 7f5cfaffd700 1 -- 172.21.15.19:0/429509666 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f74b24a70 con 0x559f74c0ec00 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.364+0000 7f5cfaffd700 10 client.16806 handle_command_reply: tid=0 2024-05-09T01:30:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.364+0000 7f5d30614b80 1 client.16806 shutdown 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.364+0000 7f5d30614b80 1 -- 172.21.15.19:0/429509666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 msgr2=0x559f74c06bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.364+0000 7f5d30614b80 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c06bc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0a95f0 tx=0x7f5d0c0108d0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d30614b80 1 -- 172.21.15.19:0/429509666 shutdown_connections 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d30614b80 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559f74c0ec00 0x559f74c829d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d30614b80 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f74c067b0 0x559f74c07640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d30614b80 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559f74a7d4a0 0x559f74c06bc0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d30614b80 1 --2- 172.21.15.19:0/429509666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f74b67160 0x559f74c07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5cfaffd700 0 client.16806 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d30614b80 1 -- 172.21.15.19:0/429509666 >> 172.21.15.19:0/429509666 conn(0x559f74add6a0 msgr2=0x559f74bfd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d30614b80 1 -- 172.21.15.19:0/429509666 shutdown_connections 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d30614b80 1 -- 172.21.15.19:0/429509666 wait complete. 2024-05-09T01:30:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d30614b80 20 client.16806 trim_cache size 0 max 16384 2024-05-09T01:30:59.367 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:30:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d21bfe700 1 -- 172.21.15.19:0/2370675649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d040b2880 msgr2=0x7f5d040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d040b2880 0x7f5d040b4d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5d10006010 tx=0x7f5d10000bc0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d21bfe700 1 -- 172.21.15.19:0/2370675649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c1034e0 msgr2=0x7f5d1c199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:30:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.365+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c1034e0 0x7f5d1c199e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00d950 tx=0x7f5d0c00dc60 comp rx=0 tx=0).stop 2024-05-09T01:30:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.366+0000 7f5d21bfe700 1 -- 172.21.15.19:0/2370675649 shutdown_connections 2024-05-09T01:30:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.366+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5d040b2880 0x7f5d040b4d90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.366+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d1c108760 0x7f5d1c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.366+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c1034e0 0x7f5d1c199e20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.366+0000 7f5d21bfe700 1 --2- 172.21.15.19:0/2370675649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d1c107e10 0x7f5d1c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:30:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.366+0000 7f5d21bfe700 1 -- 172.21.15.19:0/2370675649 >> 172.21.15.19:0/2370675649 conn(0x7f5d1c0fef30 msgr2=0x7f5d1c1096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:30:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.366+0000 7f5d21bfe700 1 -- 172.21.15.19:0/2370675649 shutdown_connections 2024-05-09T01:30:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:30:59.366+0000 7f5d21bfe700 1 -- 172.21.15.19:0/2370675649 wait complete. 2024-05-09T01:31:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:00 smithi196 ceph-mon[35041]: pgmap v542: 97 pgs: 97 active+clean; 3.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:00 smithi059 ceph-mon[34351]: pgmap v542: 97 pgs: 97 active+clean; 3.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:00.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:00 smithi019 ceph-mon[30158]: pgmap v542: 97 pgs: 97 active+clean; 3.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:01.143 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:31:01.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:31:01.412 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:02.638 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:02 smithi019 ceph-mon[30158]: pgmap v543: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:02.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:02 smithi196 ceph-mon[35041]: pgmap v543: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:02 smithi059 ceph-mon[34351]: pgmap v543: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:03.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.183+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2797470111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 msgr2=0x7fb668102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.183+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2797470111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 0x7fb668102c80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009990 tx=0x7fb65c009ca0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.183+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2797470111 shutdown_connections 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.183+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2797470111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6681031c0 0x7fb668111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.183+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2797470111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6681092d0 0x7fb6681096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.183+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2797470111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 0x7fb668102c80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.183+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2797470111 >> 172.21.15.19:0/2797470111 conn(0x7fb6680fe4e0 msgr2=0x7fb668100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.183+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2797470111 shutdown_connections 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.183+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2797470111 wait complete. 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.184+0000 7fb66c8ce700 1 Processor -- start 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.184+0000 7fb66c8ce700 1 -- start start 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.184+0000 7fb66c8ce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 0x7fb66819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.184+0000 7fb66c8ce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6681031c0 0x7fb66819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.184+0000 7fb66c8ce700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6681092d0 0x7fb66819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.184+0000 7fb66c8ce700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb668114590 con 0x7fb668102870 2024-05-09T01:31:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.184+0000 7fb66c8ce700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb668114410 con 0x7fb6681031c0 2024-05-09T01:31:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.184+0000 7fb66c8ce700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb668114710 con 0x7fb6681092d0 2024-05-09T01:31:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb66659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 0x7fb66819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb66659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 0x7fb66819aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55654/0 (socket says 172.21.15.19:55654) 2024-05-09T01:31:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb66659c700 1 -- 172.21.15.19:0/2617891088 learned_addr learned my addr 172.21.15.19:0/2617891088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb665d9b700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6681031c0 0x7fb66819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb666d9d700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6681092d0 0x7fb66819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb66659c700 1 -- 172.21.15.19:0/2617891088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6681092d0 msgr2=0x7fb66819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:03.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb66659c700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6681092d0 0x7fb66819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb66659c700 1 -- 172.21.15.19:0/2617891088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6681031c0 msgr2=0x7fb66819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:03.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb66659c700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6681031c0 0x7fb66819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb66659c700 1 -- 172.21.15.19:0/2617891088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66819fd70 con 0x7fb668102870 2024-05-09T01:31:03.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb66659c700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 0x7fb66819aea0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb65800d980 tx=0x7fb65800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:03.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb6577fe700 1 -- 172.21.15.19:0/2617891088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658017070 con 0x7fb668102870 2024-05-09T01:31:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb6577fe700 1 -- 172.21.15.19:0/2617891088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb65800fd40 con 0x7fb668102870 2024-05-09T01:31:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.185+0000 7fb6577fe700 1 -- 172.21.15.19:0/2617891088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65800e3b0 con 0x7fb668102870 2024-05-09T01:31:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.186+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2617891088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6681a0060 con 0x7fb668102870 2024-05-09T01:31:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.186+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2617891088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6681a7980 con 0x7fb668102870 2024-05-09T01:31:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.189+0000 7fb6577fe700 1 -- 172.21.15.19:0/2617891088 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb658010610 con 0x7fb668102870 2024-05-09T01:31:03.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.189+0000 7fb67b2e4b80 1 Processor -- start 2024-05-09T01:31:03.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb67b2e4b80 1 -- start start 2024-05-09T01:31:03.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb6577fe700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb6500b2880 0x7fb6500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae689b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb666d9d700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb6500b2880 0x7fb6500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae68a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae68e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598ae5a8120 con 0x5598ae68e260 2024-05-09T01:31:03.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5598ae54da20 con 0x5598ae508520 2024-05-09T01:31:03.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598ae5a40e0 con 0x5598ae582d80 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb6577fe700 1 -- 172.21.15.19:0/2617891088 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb6580e1ca0 con 0x7fb668102870 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb666d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae68e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb665d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae68a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb666d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae68e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55656/0 (socket says 172.21.15.19:55656) 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb666d9d700 1 -- 172.21.15.19:0/2866908089 learned_addr learned my addr 172.21.15.19:0/2866908089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb66659c700 1 --2- 172.21.15.19:0/2866908089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae689b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.190+0000 7fb666d9d700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb6500b2880 0x7fb6500b4d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb660009d40 tx=0x7fb660004320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb665d9b700 1 -- 172.21.15.19:0/2866908089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 msgr2=0x5598ae689b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb665d9b700 1 --2- 172.21.15.19:0/2866908089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae689b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb665d9b700 1 -- 172.21.15.19:0/2866908089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 msgr2=0x5598ae68e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb665d9b700 1 --2- 172.21.15.19:0/2866908089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae68e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb665d9b700 1 -- 172.21.15.19:0/2866908089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598ae53e580 con 0x5598ae508520 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb666d9d700 1 --2- 172.21.15.19:0/2866908089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae68e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:31:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb665d9b700 1 --2- 172.21.15.19:0/2866908089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae68a0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0098e0 tx=0x7fb65c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e6a508912278d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:03.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb654ff9700 1 -- 172.21.15.19:0/2866908089 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c003a60 con 0x5598ae508520 2024-05-09T01:31:03.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb654ff9700 1 -- 172.21.15.19:0/2866908089 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb65c005520 con 0x5598ae508520 2024-05-09T01:31:03.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.191+0000 7fb654ff9700 1 -- 172.21.15.19:0/2866908089 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00ba10 con 0x5598ae508520 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.192+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2866908089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 msgr2=0x5598ae68a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.192+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2866908089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae68a0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0098e0 tx=0x7fb65c009f90 comp rx=0 tx=0).stop 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.192+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2866908089 shutdown_connections 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.192+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2866908089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae68e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.192+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2866908089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae689b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.192+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2866908089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae68a0b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.192+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2866908089 >> 172.21.15.19:0/2866908089 conn(0x5598ae50bff0 msgr2=0x5598ae6802a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.192+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2866908089 shutdown_connections 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.192+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2866908089 wait complete. 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb67b2e4b80 1 -- start start 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb67b2e4b80 1 Processor -- start 2024-05-09T01:31:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae693460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae6939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae693ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598ae68eec0 con 0x5598ae68e260 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5598ae542ba0 con 0x5598ae508520 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598ae68f040 con 0x5598ae582d80 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb67b2e4b80 10 client.0 _ll_register_callbacks cb 0x5598ae419740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb66659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae693460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb666d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae693ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb66659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae693460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:50082/0 (socket says 172.21.15.19:50082) 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb66659c700 1 -- 172.21.15.19:0/2057959710 learned_addr learned my addr 172.21.15.19:0/2057959710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.193+0000 7fb665d9b700 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae6939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb66659c700 1 -- 172.21.15.19:0/2057959710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 msgr2=0x5598ae6939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb66659c700 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae6939a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb66659c700 1 -- 172.21.15.19:0/2057959710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 msgr2=0x5598ae693ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb66659c700 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae693ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb66659c700 1 -- 172.21.15.19:0/2057959710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598ae423510 con 0x5598ae508520 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb666d9d700 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae693ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb66659c700 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae693460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6580dfaf0 tx=0x7fb6580a97e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7463bef42f8e0fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb64dffb700 1 -- 172.21.15.19:0/2057959710 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658020050 con 0x5598ae508520 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2057959710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb65c005520 con 0x5598ae508520 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb64dffb700 1 -- 172.21.15.19:0/2057959710 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb6580ad380 con 0x5598ae508520 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb64dffb700 1 -- 172.21.15.19:0/2057959710 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6580a99a0 con 0x5598ae508520 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb67b2e4b80 10 client.26315 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.194+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2057959710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598ae6a0cf0 con 0x5598ae508520 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb64dffb700 1 -- 172.21.15.19:0/2057959710 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb65c005520 con 0x5598ae508520 2024-05-09T01:31:03.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb64dffb700 1 -- 172.21.15.19:0/2057959710 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb6580a4c10 con 0x5598ae508520 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb67b2e4b80 10 client.26315 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb67b2e4b80 10 client.26315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb67b2e4b80 20 client.26315 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5598ae6a3330 0x5598ae6a5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb67b2e4b80 4 client.26315 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2057959710 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598ae5b5b00 con 0x5598ae6a3330 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb665d9b700 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5598ae6a3330 0x5598ae6a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb665d9b700 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5598ae6a3330 0x5598ae6a5750 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ad4c583e2b6c0c server_cookie=a5b438c41f276bd1 in_seq=0 out_seq=0 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.195+0000 7fb64dffb700 10 client.26315 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.196+0000 7fb64dffb700 1 -- 172.21.15.19:0/2057959710 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598ae5b5b00 con 0x5598ae6a3330 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.196+0000 7fb64dffb700 10 client.26315 handle_command_reply: tid=0 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.196+0000 7fb67b2e4b80 1 client.26315 shutdown 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.196+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2057959710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 msgr2=0x5598ae693460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:03.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.196+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae693460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6580dfaf0 tx=0x7fb6580a97e0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.196+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2057959710 shutdown_connections 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.196+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5598ae6a3330 0x5598ae6a5750 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.197+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae693ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.197+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae693460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.197+0000 7fb64dffb700 0 client.26315 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.197+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/2057959710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae6939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.197+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2057959710 >> 172.21.15.19:0/2057959710 conn(0x5598ae50bff0 msgr2=0x5598ae67a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.197+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2057959710 shutdown_connections 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.197+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/2057959710 wait complete. 2024-05-09T01:31:03.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.197+0000 7fb67b2e4b80 20 client.26315 trim_cache size 0 max 16384 2024-05-09T01:31:03.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.217+0000 7fb67b2e4b80 1 Processor -- start 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.217+0000 7fb67b2e4b80 1 -- start start 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae508520 0x5598ae67aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae582d80 0x5598ae693d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae69f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681aca40 con 0x5598ae68e260 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681ae580 con 0x5598ae582d80 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681ad110 con 0x5598ae508520 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb666d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae69f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb666d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae69f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55666/0 (socket says 172.21.15.19:55666) 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb666d9d700 1 -- 172.21.15.19:0/344612357 learned_addr learned my addr 172.21.15.19:0/344612357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb665d9b700 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae582d80 0x5598ae693d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb66659c700 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae508520 0x5598ae67aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb666d9d700 1 -- 172.21.15.19:0/344612357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae508520 msgr2=0x5598ae67aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb666d9d700 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae508520 0x5598ae67aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb666d9d700 1 -- 172.21.15.19:0/344612357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae582d80 msgr2=0x5598ae693d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb666d9d700 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae582d80 0x5598ae693d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb666d9d700 1 -- 172.21.15.19:0/344612357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598ae695cb0 con 0x5598ae68e260 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb66659c700 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae508520 0x5598ae67aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb666d9d700 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae69f240 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb660008ae0 tx=0x7fb66001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a1e10e54f78fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb64ffff700 1 -- 172.21.15.19:0/344612357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660004db0 con 0x5598ae68e260 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb64ffff700 1 -- 172.21.15.19:0/344612357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb660008ea0 con 0x5598ae68e260 2024-05-09T01:31:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.218+0000 7fb64ffff700 1 -- 172.21.15.19:0/344612357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660025b40 con 0x5598ae68e260 2024-05-09T01:31:03.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.219+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/344612357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 msgr2=0x5598ae69f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:03.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.219+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae69f240 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb660008ae0 tx=0x7fb66001a000 comp rx=0 tx=0).stop 2024-05-09T01:31:03.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.219+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/344612357 shutdown_connections 2024-05-09T01:31:03.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.219+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae69f240 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.219+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae508520 0x5598ae67aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.219+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/344612357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae582d80 0x5598ae693d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.219+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/344612357 >> 172.21.15.19:0/344612357 conn(0x5598ae50bff0 msgr2=0x5598ae694ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:03.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.219+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/344612357 shutdown_connections 2024-05-09T01:31:03.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.219+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/344612357 wait complete. 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb67b2e4b80 1 -- start start 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb67b2e4b80 1 Processor -- start 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae69b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae698590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb67b2e4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae698ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681adeb0 con 0x5598ae68e260 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681ad7e0 con 0x5598ae508520 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb67b2e4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681aa9e0 con 0x5598ae582d80 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb67b2e4b80 10 client.0 _ll_register_callbacks cb 0x5598ae681290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb666d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae698ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb666d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae698ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55674/0 (socket says 172.21.15.19:55674) 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb666d9d700 1 -- 172.21.15.19:0/1142813854 learned_addr learned my addr 172.21.15.19:0/1142813854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb66659c700 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae69b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.220+0000 7fb665d9b700 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae698590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb666d9d700 1 -- 172.21.15.19:0/1142813854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 msgr2=0x5598ae698590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb666d9d700 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae698590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb666d9d700 1 -- 172.21.15.19:0/1142813854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 msgr2=0x5598ae69b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb666d9d700 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae69b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb666d9d700 1 -- 172.21.15.19:0/1142813854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598ae69bf10 con 0x5598ae68e260 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb665d9b700 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae698590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb666d9d700 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae698ad0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb660008ab0 tx=0x7fb66001af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6747702b6936627 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb64dffb700 1 -- 172.21.15.19:0/1142813854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660025450 con 0x5598ae68e260 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb64dffb700 1 -- 172.21.15.19:0/1142813854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb6600255f0 con 0x5598ae68e260 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/1142813854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb660008ea0 con 0x5598ae68e260 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb64dffb700 1 -- 172.21.15.19:0/1142813854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6600257f0 con 0x5598ae68e260 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb67b2e4b80 10 client.16824 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/1142813854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598ae70e420 con 0x5598ae68e260 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.221+0000 7fb64dffb700 1 -- 172.21.15.19:0/1142813854 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb660008ea0 con 0x5598ae68e260 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb64dffb700 1 -- 172.21.15.19:0/1142813854 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb660018040 con 0x5598ae68e260 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb67b2e4b80 10 client.16824 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb67b2e4b80 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb67b2e4b80 20 client.16824 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5598ae70ea50 0x5598ae710e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb67b2e4b80 4 client.16824 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/1142813854 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5598ae5afba0 con 0x5598ae70ea50 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb66659c700 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5598ae70ea50 0x5598ae710e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb64dffb700 10 client.16824 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.222+0000 7fb66659c700 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5598ae70ea50 0x5598ae710e70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d194574d7f68e4 server_cookie=a017738646e7a823 in_seq=0 out_seq=0 2024-05-09T01:31:03.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb64dffb700 1 -- 172.21.15.19:0/1142813854 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5598ae5afba0 con 0x5598ae70ea50 2024-05-09T01:31:03.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb64dffb700 10 client.16824 handle_command_reply: tid=0 2024-05-09T01:31:03.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 client.16824 shutdown 2024-05-09T01:31:03.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/1142813854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 msgr2=0x5598ae698ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae698ad0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb660008ab0 tx=0x7fb66001af30 comp rx=0 tx=0).stop 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/1142813854 shutdown_connections 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5598ae70ea50 0x5598ae710e70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598ae68e260 0x5598ae698ad0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598ae508520 0x5598ae69b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 --2- 172.21.15.19:0/1142813854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ae582d80 0x5598ae698590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/1142813854 >> 172.21.15.19:0/1142813854 conn(0x5598ae50bff0 msgr2=0x5598ae67a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb64dffb700 0 client.16824 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/1142813854 shutdown_connections 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 1 -- 172.21.15.19:0/1142813854 wait complete. 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb67b2e4b80 20 client.16824 trim_cache size 0 max 16384 2024-05-09T01:31:03.225 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:31:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:31:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "7de93181-f716-46bd-ba40-4c00d9e7bf8d", 2024-05-09T01:31:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:31:03.226 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb6500b2880 msgr2=0x7fb6500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb6500b2880 0x7fb6500b4d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb660009d40 tx=0x7fb660004320 comp rx=0 tx=0).stop 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 msgr2=0x7fb66819aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 0x7fb66819aea0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb65800d980 tx=0x7fb65800dc90 comp rx=0 tx=0).stop 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2617891088 shutdown_connections 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6681092d0 0x7fb66819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb6500b2880 0x7fb6500b4d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb668102870 0x7fb66819aea0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 --2- 172.21.15.19:0/2617891088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6681031c0 0x7fb66819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2617891088 >> 172.21.15.19:0/2617891088 conn(0x7fb6680fe4e0 msgr2=0x7fb66810a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2617891088 shutdown_connections 2024-05-09T01:31:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:03.225+0000 7fb66c8ce700 1 -- 172.21.15.19:0/2617891088 wait complete. 2024-05-09T01:31:04.077 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:31:04.332 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:04 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:31:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:04 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:31:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:04 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:31:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:04 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:31:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:04 smithi196 ceph-mon[35041]: pgmap v544: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:31:04.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:04 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:31:04.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:04 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:31:04.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:04 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:31:04.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:04 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:31:04.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:04 smithi019 ceph-mon[30158]: pgmap v544: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:31:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:04 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:31:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:04 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:31:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:04 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:31:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:04 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:31:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:04 smithi059 ceph-mon[34351]: pgmap v544: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:31:05.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.990+0000 7f75d8d38700 1 -- 172.21.15.19:0/2551731374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4102b20 msgr2=0x7f75d4102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:05.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.990+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2551731374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4102b20 0x7f75d4102f30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f75c40099c0 tx=0x7f75c4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:31:05.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.991+0000 7f75d8d38700 1 -- 172.21.15.19:0/2551731374 shutdown_connections 2024-05-09T01:31:05.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.991+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2551731374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75d4109890 0x7f75d41111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:05.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.991+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2551731374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4102b20 0x7f75d4102f30 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:05.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.991+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2551731374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d4108f40 0x7f75d4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:05.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.991+0000 7f75d8d38700 1 -- 172.21.15.19:0/2551731374 >> 172.21.15.19:0/2551731374 conn(0x7f75d40fe4e0 msgr2=0x7f75d4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:05.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.991+0000 7f75d8d38700 1 -- 172.21.15.19:0/2551731374 shutdown_connections 2024-05-09T01:31:05.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.991+0000 7f75d8d38700 1 -- 172.21.15.19:0/2551731374 wait complete. 2024-05-09T01:31:05.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d8d38700 1 Processor -- start 2024-05-09T01:31:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d8d38700 1 -- start start 2024-05-09T01:31:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d8d38700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d4102b20 0x7f75d419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d8d38700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4108f40 0x7f75d419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d8d38700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75d4109890 0x7f75d419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d8d38700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4114530 con 0x7f75d4108f40 2024-05-09T01:31:05.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d8d38700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41143b0 con 0x7f75d4102b20 2024-05-09T01:31:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d8d38700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41146b0 con 0x7f75d4109890 2024-05-09T01:31:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4108f40 0x7f75d419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4108f40 0x7f75d419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55690/0 (socket says 172.21.15.19:55690) 2024-05-09T01:31:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d1d9b700 1 -- 172.21.15.19:0/2065025121 learned_addr learned my addr 172.21.15.19:0/2065025121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:05.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d259c700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d4102b20 0x7f75d419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d2d9d700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75d4109890 0x7f75d419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d1d9b700 1 -- 172.21.15.19:0/2065025121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75d4109890 msgr2=0x7f75d419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75d4109890 0x7f75d419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d1d9b700 1 -- 172.21.15.19:0/2065025121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d4102b20 msgr2=0x7f75d419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.992+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d4102b20 0x7f75d419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.993+0000 7f75d1d9b700 1 -- 172.21.15.19:0/2065025121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d419fd30 con 0x7f75d4108f40 2024-05-09T01:31:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.993+0000 7f75d259c700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d4102b20 0x7f75d419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.993+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4108f40 0x7f75d419b400 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f75c800e970 tx=0x7f75c800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.993+0000 7f75c37fe700 1 -- 172.21.15.19:0/2065025121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c800cbd0 con 0x7f75d4108f40 2024-05-09T01:31:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.993+0000 7f75d8d38700 1 -- 172.21.15.19:0/2065025121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d41a0020 con 0x7f75d4108f40 2024-05-09T01:31:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.993+0000 7f75c37fe700 1 -- 172.21.15.19:0/2065025121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c800cd70 con 0x7f75d4108f40 2024-05-09T01:31:05.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.993+0000 7f75c37fe700 1 -- 172.21.15.19:0/2065025121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8010430 con 0x7f75d4108f40 2024-05-09T01:31:05.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.993+0000 7f75d8d38700 1 -- 172.21.15.19:0/2065025121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41a7940 con 0x7f75d4108f40 2024-05-09T01:31:06.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.996+0000 7f75c37fe700 1 -- 172.21.15.19:0/2065025121 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f75c80105d0 con 0x7f75d4108f40 2024-05-09T01:31:06.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.996+0000 7f75c37fe700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75bc0b2880 0x7f75bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.997+0000 7f75d259c700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75bc0b2880 0x7f75bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.997+0000 7f75c37fe700 1 -- 172.21.15.19:0/2065025121 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f75c80e2560 con 0x7f75d4108f40 2024-05-09T01:31:06.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.997+0000 7f75e774eb80 1 Processor -- start 2024-05-09T01:31:06.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.997+0000 7f75d259c700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75bc0b2880 0x7f75bc0b4d90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f75c4000c00 tx=0x7f75c401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:06.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.997+0000 7f75e774eb80 1 -- start start 2024-05-09T01:31:06.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.997+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073d0d160 0x557073da64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.997+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073da6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073da0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75e774eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557073cc2ff0 con 0x557073c234a0 2024-05-09T01:31:06.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75e774eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557073c617f0 con 0x557073d0d160 2024-05-09T01:31:06.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75e774eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557073cbefb0 con 0x557073da06a0 2024-05-09T01:31:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073da6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073da6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55702/0 (socket says 172.21.15.19:55702) 2024-05-09T01:31:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d1d9b700 1 -- 172.21.15.19:0/3240317136 learned_addr learned my addr 172.21.15.19:0/3240317136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d259c700 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073d0d160 0x557073da64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d2d9d700 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073da0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d1d9b700 1 -- 172.21.15.19:0/3240317136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 msgr2=0x557073da0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073da0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d1d9b700 1 -- 172.21.15.19:0/3240317136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073d0d160 msgr2=0x557073da64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073d0d160 0x557073da64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d1d9b700 1 -- 172.21.15.19:0/3240317136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557073c59370 con 0x557073c234a0 2024-05-09T01:31:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d2d9d700 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073da0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:06.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073da6a30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f75c80a5be0 tx=0x7f75c80a92d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45043a299cfd59e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:06.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75c0ff9700 1 -- 172.21.15.19:0/3240317136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80c3070 con 0x557073c234a0 2024-05-09T01:31:06.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.998+0000 7f75c0ff9700 1 -- 172.21.15.19:0/3240317136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c80b2310 con 0x557073c234a0 2024-05-09T01:31:06.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.999+0000 7f75c0ff9700 1 -- 172.21.15.19:0/3240317136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80b29c0 con 0x557073c234a0 2024-05-09T01:31:06.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.999+0000 7f75e774eb80 1 -- 172.21.15.19:0/3240317136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 msgr2=0x557073da6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:06.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.999+0000 7f75e774eb80 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073da6a30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f75c80a5be0 tx=0x7f75c80a92d0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.999+0000 7f75e774eb80 1 -- 172.21.15.19:0/3240317136 shutdown_connections 2024-05-09T01:31:06.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.999+0000 7f75e774eb80 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073da0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.999+0000 7f75e774eb80 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073d0d160 0x557073da64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.999+0000 7f75e774eb80 1 --2- 172.21.15.19:0/3240317136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073da6a30 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:05.999+0000 7f75e774eb80 1 -- 172.21.15.19:0/3240317136 >> 172.21.15.19:0/3240317136 conn(0x557073c836a0 msgr2=0x557073d9b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 -- 172.21.15.19:0/3240317136 shutdown_connections 2024-05-09T01:31:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 -- 172.21.15.19:0/3240317136 wait complete. 2024-05-09T01:31:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 -- start start 2024-05-09T01:31:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 Processor -- start 2024-05-09T01:31:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 0x557073daa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073daad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557073b3eba0 con 0x557073d0d160 2024-05-09T01:31:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557073c7d150 con 0x557073c234a0 2024-05-09T01:31:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.000+0000 7f75e774eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557073dad1d0 con 0x557073da06a0 2024-05-09T01:31:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75e774eb80 10 client.0 _ll_register_callbacks cb 0x557073b346c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:06.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55716/0 (socket says 172.21.15.19:55716) 2024-05-09T01:31:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d1d9b700 1 -- 172.21.15.19:0/2110746206 learned_addr learned my addr 172.21.15.19:0/2110746206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d259c700 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 0x557073daa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d2d9d700 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073daad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d1d9b700 1 -- 172.21.15.19:0/2110746206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 msgr2=0x557073daad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073daad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d1d9b700 1 -- 172.21.15.19:0/2110746206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 msgr2=0x557073daa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 0x557073daa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d1d9b700 1 -- 172.21.15.19:0/2110746206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557073b3e490 con 0x557073d0d160 2024-05-09T01:31:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d2d9d700 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073daad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daa830 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c5b0 tx=0x7f75c80aa0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f81bd831fb5fcc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75b9ffb700 1 -- 172.21.15.19:0/2110746206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c806f020 con 0x557073d0d160 2024-05-09T01:31:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75b9ffb700 1 -- 172.21.15.19:0/2110746206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f75c80b2740 con 0x557073d0d160 2024-05-09T01:31:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.001+0000 7f75e774eb80 1 -- 172.21.15.19:0/2110746206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c80b2310 con 0x557073d0d160 2024-05-09T01:31:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75b9ffb700 1 -- 172.21.15.19:0/2110746206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75c80c3070 con 0x557073d0d160 2024-05-09T01:31:06.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75e774eb80 10 client.16836 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75e774eb80 1 -- 172.21.15.19:0/2110746206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557073db7cc0 con 0x557073d0d160 2024-05-09T01:31:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75b9ffb700 1 -- 172.21.15.19:0/2110746206 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c80b2310 con 0x557073d0d160 2024-05-09T01:31:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75b9ffb700 1 -- 172.21.15.19:0/2110746206 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f75c80e2110 con 0x557073d0d160 2024-05-09T01:31:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75e774eb80 10 client.16836 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75e774eb80 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:06.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75e774eb80 20 client.16836 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75e774eb80 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557073dba190 0x557073dbc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75e774eb80 4 client.16836 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75e774eb80 1 -- 172.21.15.19:0/2110746206 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557073cd09d0 con 0x557073dba190 2024-05-09T01:31:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.002+0000 7f75d259c700 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557073dba190 0x557073dbc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.003+0000 7f75d259c700 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557073dba190 0x557073dbc5b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12bd6a925874ac6 server_cookie=461ef8561e091962 in_seq=0 out_seq=0 2024-05-09T01:31:06.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.003+0000 7f75b9ffb700 10 client.16836 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:06.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.003+0000 7f75b9ffb700 1 -- 172.21.15.19:0/2110746206 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557073cd09d0 con 0x557073dba190 2024-05-09T01:31:06.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.003+0000 7f75b9ffb700 10 client.16836 handle_command_reply: tid=0 2024-05-09T01:31:06.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 client.16836 shutdown 2024-05-09T01:31:06.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 -- 172.21.15.19:0/2110746206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 msgr2=0x557073daa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:06.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daa830 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c5b0 tx=0x7f75c80aa0a0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 -- 172.21.15.19:0/2110746206 shutdown_connections 2024-05-09T01:31:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557073dba190 0x557073dbc5b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75b9ffb700 0 client.16836 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073daad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 0x557073daa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 --2- 172.21.15.19:0/2110746206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daa830 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 -- 172.21.15.19:0/2110746206 >> 172.21.15.19:0/2110746206 conn(0x557073c836a0 msgr2=0x557073d9b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:06.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 -- 172.21.15.19:0/2110746206 shutdown_connections 2024-05-09T01:31:06.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 1 -- 172.21.15.19:0/2110746206 wait complete. 2024-05-09T01:31:06.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.004+0000 7f75e774eb80 20 client.16836 trim_cache size 0 max 16384 2024-05-09T01:31:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.024+0000 7f75e774eb80 1 Processor -- start 2024-05-09T01:31:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.024+0000 7f75e774eb80 1 -- start start 2024-05-09T01:31:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 0x557073da00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073db5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75e774eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41aede0 con 0x557073d0d160 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75e774eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b0920 con 0x557073c234a0 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75e774eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41af4b0 con 0x557073da06a0 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55718/0 (socket says 172.21.15.19:55718) 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d1d9b700 1 -- 172.21.15.19:0/1026710797 learned_addr learned my addr 172.21.15.19:0/1026710797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 0x557073da00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d2d9d700 1 --2- 172.21.15.19:0/1026710797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073db5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d1d9b700 1 -- 172.21.15.19:0/1026710797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 msgr2=0x557073db5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/1026710797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073db5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d1d9b700 1 -- 172.21.15.19:0/1026710797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 msgr2=0x557073da00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/1026710797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 0x557073da00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d1d9b700 1 -- 172.21.15.19:0/1026710797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557073dacb90 con 0x557073d0d160 2024-05-09T01:31:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d259c700 1 --2- 172.21.15.19:0/1026710797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 0x557073da00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:31:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/1026710797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daabb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f75c80a9af0 tx=0x7f75c80051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63b64d231c5f687 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.025+0000 7f75bbfff700 1 -- 172.21.15.19:0/1026710797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c806f070 con 0x557073d0d160 2024-05-09T01:31:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75bbfff700 1 -- 172.21.15.19:0/1026710797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c80b2310 con 0x557073d0d160 2024-05-09T01:31:06.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75bbfff700 1 -- 172.21.15.19:0/1026710797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80a9b60 con 0x557073d0d160 2024-05-09T01:31:06.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75e774eb80 1 -- 172.21.15.19:0/1026710797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 msgr2=0x557073daabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:06.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1026710797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daabb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f75c80a9af0 tx=0x7f75c80051c0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75e774eb80 1 -- 172.21.15.19:0/1026710797 shutdown_connections 2024-05-09T01:31:06.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1026710797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073da06a0 0x557073db5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1026710797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073c234a0 0x557073da00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1026710797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073d0d160 0x557073daabb0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75e774eb80 1 -- 172.21.15.19:0/1026710797 >> 172.21.15.19:0/1026710797 conn(0x557073c836a0 msgr2=0x557073da9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:06.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75e774eb80 1 -- 172.21.15.19:0/1026710797 shutdown_connections 2024-05-09T01:31:06.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.026+0000 7f75e774eb80 1 -- 172.21.15.19:0/1026710797 wait complete. 2024-05-09T01:31:06.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75e774eb80 1 -- start start 2024-05-09T01:31:06.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75e774eb80 1 Processor -- start 2024-05-09T01:31:06.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073e1c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073d0d160 0x557073e1cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75e774eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073da06a0 0x557073e27710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75e774eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b0250 con 0x557073c234a0 2024-05-09T01:31:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75e774eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41afb80 con 0x557073da06a0 2024-05-09T01:31:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75e774eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41acd80 con 0x557073d0d160 2024-05-09T01:31:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75e774eb80 10 client.0 _ll_register_callbacks cb 0x557073b346c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75d259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073e1c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75d259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073e1c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55730/0 (socket says 172.21.15.19:55730) 2024-05-09T01:31:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.027+0000 7f75d259c700 1 -- 172.21.15.19:0/1600809838 learned_addr learned my addr 172.21.15.19:0/1600809838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75d2d9d700 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073da06a0 0x557073e27710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073d0d160 0x557073e1cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75d259c700 1 -- 172.21.15.19:0/1600809838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073d0d160 msgr2=0x557073e1cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75d259c700 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073d0d160 0x557073e1cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75d259c700 1 -- 172.21.15.19:0/1600809838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073da06a0 msgr2=0x557073e27710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75d259c700 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073da06a0 0x557073e27710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75d259c700 1 -- 172.21.15.19:0/1600809838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557073e27c50 con 0x557073c234a0 2024-05-09T01:31:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75d259c700 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073e1c890 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009750 tx=0x7f75c4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4e3da36229d1a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75b9ffb700 1 -- 172.21.15.19:0/1600809838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4031380 con 0x557073c234a0 2024-05-09T01:31:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75e774eb80 1 -- 172.21.15.19:0/1600809838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c80b2310 con 0x557073c234a0 2024-05-09T01:31:06.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75b9ffb700 1 -- 172.21.15.19:0/1600809838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f75c4031520 con 0x557073c234a0 2024-05-09T01:31:06.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75b9ffb700 1 -- 172.21.15.19:0/1600809838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75c4040070 con 0x557073c234a0 2024-05-09T01:31:06.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.028+0000 7f75e774eb80 10 client.16842 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:06.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75e774eb80 1 -- 172.21.15.19:0/1600809838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557073e28e10 con 0x557073c234a0 2024-05-09T01:31:06.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75b9ffb700 1 -- 172.21.15.19:0/1600809838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c80b2310 con 0x557073c234a0 2024-05-09T01:31:06.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75b9ffb700 1 -- 172.21.15.19:0/1600809838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f75c4022070 con 0x557073c234a0 2024-05-09T01:31:06.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75e774eb80 10 client.16842 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:06.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75e774eb80 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:06.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75e774eb80 20 client.16842 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:06.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557073dba190 0x557073e2b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:06.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75e774eb80 4 client.16842 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:31:06.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75e774eb80 1 -- 172.21.15.19:0/1600809838 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557073ccaa70 con 0x557073dba190 2024-05-09T01:31:06.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.029+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557073dba190 0x557073e2b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:06.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75b9ffb700 10 client.16842 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:06.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75d1d9b700 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557073dba190 0x557073e2b4b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feb7799b844790dc server_cookie=faff7bca4a363ba2 in_seq=0 out_seq=0 2024-05-09T01:31:06.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75b9ffb700 1 -- 172.21.15.19:0/1600809838 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557073ccaa70 con 0x557073dba190 2024-05-09T01:31:06.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75b9ffb700 10 client.16842 handle_command_reply: tid=0 2024-05-09T01:31:06.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 client.16842 shutdown 2024-05-09T01:31:06.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 -- 172.21.15.19:0/1600809838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 msgr2=0x557073e1c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:06.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073e1c890 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009750 tx=0x7f75c4009310 comp rx=0 tx=0).stop 2024-05-09T01:31:06.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 -- 172.21.15.19:0/1600809838 shutdown_connections 2024-05-09T01:31:06.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557073dba190 0x557073e2b4b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557073da06a0 0x557073e27710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557073c234a0 0x557073e1c890 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 --2- 172.21.15.19:0/1600809838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557073d0d160 0x557073e1cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75b9ffb700 0 client.16842 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:06.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 -- 172.21.15.19:0/1600809838 >> 172.21.15.19:0/1600809838 conn(0x557073c836a0 msgr2=0x557073da3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:06.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 -- 172.21.15.19:0/1600809838 shutdown_connections 2024-05-09T01:31:06.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 1 -- 172.21.15.19:0/1600809838 wait complete. 2024-05-09T01:31:06.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.030+0000 7f75e774eb80 20 client.16842 trim_cache size 0 max 16384 2024-05-09T01:31:06.045 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:31:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T01:31:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T01:31:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "7de93181-f716-46bd-ba40-4c00d9e7bf8d": { 2024-05-09T01:31:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T01:31:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "7de93181-f716-46bd-ba40-4c00d9e7bf8d", 2024-05-09T01:31:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T01:31:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:31:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:31:06.047 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:31:06.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 -- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75bc0b2880 msgr2=0x7f75bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:06.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75bc0b2880 0x7f75bc0b4d90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f75c4000c00 tx=0x7f75c401a040 comp rx=0 tx=0).stop 2024-05-09T01:31:06.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 -- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4108f40 msgr2=0x7f75d419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:06.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4108f40 0x7f75d419b400 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f75c800e970 tx=0x7f75c800ed30 comp rx=0 tx=0).stop 2024-05-09T01:31:06.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 -- 172.21.15.19:0/2065025121 shutdown_connections 2024-05-09T01:31:06.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75d4109890 0x7f75d419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d4102b20 0x7f75d419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f75bc0b2880 0x7f75bc0b4d90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 --2- 172.21.15.19:0/2065025121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d4108f40 0x7f75d419b400 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:06.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 -- 172.21.15.19:0/2065025121 >> 172.21.15.19:0/2065025121 conn(0x7f75d40fe4e0 msgr2=0x7f75d4108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:06.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 -- 172.21.15.19:0/2065025121 shutdown_connections 2024-05-09T01:31:06.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:06.031+0000 7f75d8d38700 1 -- 172.21.15.19:0/2065025121 wait complete. 2024-05-09T01:31:06.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:06 smithi196 ceph-mon[35041]: pgmap v545: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:31:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:06 smithi059 ceph-mon[34351]: pgmap v545: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:31:06.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:06 smithi019 ceph-mon[30158]: pgmap v545: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:31:06.957 INFO:tasks.cephfs.filesystem:scrub status for tag:7de93181-f716-46bd-ba40-4c00d9e7bf8d - {'path': '/', 'tag': '7de93181-f716-46bd-ba40-4c00d9e7bf8d', 'options': 'recursive,force'} 2024-05-09T01:31:07.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:07 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:31:07.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:07 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:31:07.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:07 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:31:08.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:08 smithi019 ceph-mon[30158]: pgmap v546: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:31:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:08 smithi196 ceph-mon[35041]: pgmap v546: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:31:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:08 smithi059 ceph-mon[34351]: pgmap v546: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:31:10.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:10 smithi196 ceph-mon[35041]: pgmap v547: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:31:10.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:10 smithi059 ceph-mon[34351]: pgmap v547: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:31:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:10 smithi019 ceph-mon[30158]: pgmap v547: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.368+0000 7f25190ee640 1 -- 172.21.15.19:0/2184976395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 msgr2=0x7f2514104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.368+0000 7f25190ee640 1 --2- 172.21.15.19:0/2184976395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 0x7f2514104970 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2504009900 tx=0x7f250402a410 comp rx=0 tx=0).stop 2024-05-09T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.368+0000 7f25190ee640 1 -- 172.21.15.19:0/2184976395 shutdown_connections 2024-05-09T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.368+0000 7f25190ee640 1 --2- 172.21.15.19:0/2184976395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514109bb0 0x7f251410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.368+0000 7f25190ee640 1 --2- 172.21.15.19:0/2184976395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2514104eb0 0x7f25141052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.368+0000 7f25190ee640 1 --2- 172.21.15.19:0/2184976395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 0x7f2514104970 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.368+0000 7f25190ee640 1 -- 172.21.15.19:0/2184976395 >> 172.21.15.19:0/2184976395 conn(0x7f25140fee40 msgr2=0x7f2514101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.369+0000 7f25190ee640 1 -- 172.21.15.19:0/2184976395 shutdown_connections 2024-05-09T01:31:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.369+0000 7f25190ee640 1 -- 172.21.15.19:0/2184976395 wait complete. 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.369+0000 7f25190ee640 1 Processor -- start 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f25190ee640 1 -- start start 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f25190ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 0x7f251419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f25190ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2514104eb0 0x7f251419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f25190ee640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514109bb0 0x7f25141a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f25190ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f251410f7c0 con 0x7f2514103c70 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f25190ee640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f251410f640 con 0x7f2514109bb0 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f25190ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f251410f940 con 0x7f2514104eb0 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f2512d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 0x7f251419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f2512d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 0x7f251419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51852/0 (socket says 172.21.15.19:51852) 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f2512d9d640 1 -- 172.21.15.19:0/856599566 learned_addr learned my addr 172.21.15.19:0/856599566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f251259c640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2514104eb0 0x7f251419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f251359e640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514109bb0 0x7f25141a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f2512d9d640 1 -- 172.21.15.19:0/856599566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2514104eb0 msgr2=0x7f251419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f2512d9d640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2514104eb0 0x7f251419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f2512d9d640 1 -- 172.21.15.19:0/856599566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514109bb0 msgr2=0x7f25141a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f2512d9d640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514109bb0 0x7f25141a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.370+0000 7f2512d9d640 1 -- 172.21.15.19:0/856599566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25141a1f00 con 0x7f2514103c70 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.371+0000 7f2512d9d640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 0x7f251419a3c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2504031440 tx=0x7f2504003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.371+0000 7f24fbfff640 1 -- 172.21.15.19:0/856599566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250403f070 con 0x7f2514103c70 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.371+0000 7f25190ee640 1 -- 172.21.15.19:0/856599566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25141a2190 con 0x7f2514103c70 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.371+0000 7f24fbfff640 1 -- 172.21.15.19:0/856599566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2504031c50 con 0x7f2514103c70 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.371+0000 7f24fbfff640 1 -- 172.21.15.19:0/856599566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504035030 con 0x7f2514103c70 2024-05-09T01:31:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.371+0000 7f25190ee640 1 -- 172.21.15.19:0/856599566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25141a2680 con 0x7f2514103c70 2024-05-09T01:31:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251ffa7740 1 Processor -- start 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251ffa7740 1 -- start start 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f24fbfff640 1 -- 172.21.15.19:0/856599566 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2504042070 con 0x7f2514103c70 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55350500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f832f0 0x55ad55350a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f8b4d0 0x55ad55357980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251ffa7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad552b47c0 con 0x55ad54fac1b0 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251ffa7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad552500e0 con 0x55ad54f8b4d0 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251ffa7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ad552b0780 con 0x55ad54f832f0 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55350500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55350500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51868/0 (socket says 172.21.15.19:51868) 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251359e640 1 -- 172.21.15.19:0/106402842 learned_addr learned my addr 172.21.15.19:0/106402842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f2512d9d640 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f832f0 0x55ad55350a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.374+0000 7f251259c640 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f8b4d0 0x55ad55357980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f251359e640 1 -- 172.21.15.19:0/106402842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f832f0 msgr2=0x55ad55350a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f251359e640 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f832f0 0x55ad55350a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f251359e640 1 -- 172.21.15.19:0/106402842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f8b4d0 msgr2=0x55ad55357980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f251359e640 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f8b4d0 0x55ad55357980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f251359e640 1 -- 172.21.15.19:0/106402842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad552797c0 con 0x55ad54fac1b0 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f251259c640 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f8b4d0 0x55ad55357980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f251359e640 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55350500 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f250c009fd0 tx=0x7f250c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c66e9895f5c2a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f24fbfff640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24fc0b27e0 0x7f24fc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f24f97fa640 1 -- 172.21.15.19:0/106402842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c004d20 con 0x55ad54fac1b0 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f24f97fa640 1 -- 172.21.15.19:0/106402842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250c004050 con 0x55ad54fac1b0 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f24f97fa640 1 -- 172.21.15.19:0/106402842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c005280 con 0x55ad54fac1b0 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f2512d9d640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24fc0b27e0 0x7f24fc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.375+0000 7f24fbfff640 1 -- 172.21.15.19:0/856599566 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f25041029b0 con 0x7f2514103c70 2024-05-09T01:31:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f2512d9d640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24fc0b27e0 0x7f24fc0b4cf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55ad553518e0 tx=0x7f25041084f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f251ffa7740 1 -- 172.21.15.19:0/106402842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 msgr2=0x55ad55350500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f251ffa7740 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55350500 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f250c009fd0 tx=0x7f250c0118d0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f251ffa7740 1 -- 172.21.15.19:0/106402842 shutdown_connections 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f251ffa7740 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f8b4d0 0x55ad55357980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f251ffa7740 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f832f0 0x55ad55350a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f251ffa7740 1 --2- 172.21.15.19:0/106402842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55350500 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f251ffa7740 1 -- 172.21.15.19:0/106402842 >> 172.21.15.19:0/106402842 conn(0x55ad54fa0300 msgr2=0x55ad5534be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f251ffa7740 1 -- 172.21.15.19:0/106402842 shutdown_connections 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.376+0000 7f251ffa7740 1 -- 172.21.15.19:0/106402842 wait complete. 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251ffa7740 1 -- start start 2024-05-09T01:31:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251ffa7740 1 Processor -- start 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 0x55ad5535a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 0x55ad5535a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55361760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251ffa7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad553583d0 con 0x55ad54fac1b0 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251ffa7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad5523b9c0 con 0x55ad54f832f0 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251ffa7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ad55358550 con 0x55ad54f8b4d0 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251ffa7740 10 client.0 _ll_register_callbacks cb 0x55ad55268020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55361760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55361760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51870/0 (socket says 172.21.15.19:51870) 2024-05-09T01:31:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251359e640 1 -- 172.21.15.19:0/2053578259 learned_addr learned my addr 172.21.15.19:0/2053578259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251259c640 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 0x55ad5535a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.377+0000 7f251359e640 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 0x55ad5535a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f251359e640 1 -- 172.21.15.19:0/2053578259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 msgr2=0x55ad5535a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f251359e640 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 0x55ad5535a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f251359e640 1 -- 172.21.15.19:0/2053578259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 msgr2=0x55ad5535a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f251359e640 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 0x55ad5535a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f251359e640 1 -- 172.21.15.19:0/2053578259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad552621f0 con 0x55ad54fac1b0 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f251359e640 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55361760 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f250c005b40 tx=0x7f250c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fa74dd623588fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f24f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f24f6ff5640 1 -- 172.21.15.19:0/2053578259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c02d060 con 0x55ad54fac1b0 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f24f6ff5640 1 -- 172.21.15.19:0/2053578259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f250c00f4e0 con 0x55ad54fac1b0 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f251ffa7740 1 -- 172.21.15.19:0/2053578259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f250c004050 con 0x55ad54fac1b0 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f24f6ff5640 1 -- 172.21.15.19:0/2053578259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f250c00f6e0 con 0x55ad54fac1b0 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f251ffa7740 10 client.16854 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f251ffa7740 1 -- 172.21.15.19:0/2053578259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad55017790 con 0x55ad54fac1b0 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.378+0000 7f24f6ff5640 1 -- 172.21.15.19:0/2053578259 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250c004050 con 0x55ad54fac1b0 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.379+0000 7f24f6ff5640 1 -- 172.21.15.19:0/2053578259 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f250c019730 con 0x55ad54fac1b0 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.379+0000 7f251ffa7740 10 client.16854 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.379+0000 7f251ffa7740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.379+0000 7f251ffa7740 20 client.16854 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.379+0000 7f251ffa7740 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ad54f6ad80 0x55ad553679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.379+0000 7f251ffa7740 4 client.16854 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.379+0000 7f251ffa7740 1 -- 172.21.15.19:0/2053578259 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad552c23d0 con 0x55ad54f6ad80 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.379+0000 7f251259c640 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ad54f6ad80 0x55ad553679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.380+0000 7f24f6ff5640 10 client.16854 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.380+0000 7f251259c640 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ad54f6ad80 0x55ad553679a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e79c6bee7e75fe27 server_cookie=69177c5de56fc137 in_seq=0 out_seq=0 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.380+0000 7f24f6ff5640 1 -- 172.21.15.19:0/2053578259 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ad552c23d0 con 0x55ad54f6ad80 2024-05-09T01:31:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.380+0000 7f24f6ff5640 10 client.16854 handle_command_reply: tid=0 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.380+0000 7f251ffa7740 1 client.16854 shutdown 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.380+0000 7f251ffa7740 1 -- 172.21.15.19:0/2053578259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 msgr2=0x55ad55361760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.380+0000 7f251ffa7740 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55361760 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f250c005b40 tx=0x7f250c005a50 comp rx=0 tx=0).stop 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f251ffa7740 1 -- 172.21.15.19:0/2053578259 shutdown_connections 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f251ffa7740 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ad54f6ad80 0x55ad553679a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f251ffa7740 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54fac1b0 0x55ad55361760 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f251ffa7740 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 0x55ad5535a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f24f6ff5640 0 client.16854 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f251ffa7740 1 --2- 172.21.15.19:0/2053578259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 0x55ad5535a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f251ffa7740 1 -- 172.21.15.19:0/2053578259 >> 172.21.15.19:0/2053578259 conn(0x55ad54fa0300 msgr2=0x55ad5534a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f251ffa7740 1 -- 172.21.15.19:0/2053578259 shutdown_connections 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f251ffa7740 1 -- 172.21.15.19:0/2053578259 wait complete. 2024-05-09T01:31:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.381+0000 7f251ffa7740 20 client.16854 trim_cache size 0 max 16384 2024-05-09T01:31:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.396+0000 7f251ffa7740 1 Processor -- start 2024-05-09T01:31:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.396+0000 7f251ffa7740 1 -- start start 2024-05-09T01:31:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.396+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f6ad80 0x55ad5534bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.396+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 0x55ad55361680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.396+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f8b4d0 0x55ad55361bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.396+0000 7f251ffa7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25141ab560 con 0x55ad54f8b4d0 2024-05-09T01:31:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.396+0000 7f251ffa7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25141ad0a0 con 0x55ad54f832f0 2024-05-09T01:31:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.396+0000 7f251ffa7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f25141abc30 con 0x55ad54f6ad80 2024-05-09T01:31:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f8b4d0 0x55ad55361bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f8b4d0 0x55ad55361bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51886/0 (socket says 172.21.15.19:51886) 2024-05-09T01:31:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 -- 172.21.15.19:0/1048739052 learned_addr learned my addr 172.21.15.19:0/1048739052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251259c640 1 --2- 172.21.15.19:0/1048739052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 0x55ad55361680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 --2- 172.21.15.19:0/1048739052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f6ad80 0x55ad5534bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 -- 172.21.15.19:0/1048739052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f6ad80 msgr2=0x55ad5534bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 --2- 172.21.15.19:0/1048739052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f6ad80 0x55ad5534bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 -- 172.21.15.19:0/1048739052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 msgr2=0x55ad55361680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 --2- 172.21.15.19:0/1048739052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 0x55ad55361680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 -- 172.21.15.19:0/1048739052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad5508e5f0 con 0x55ad54f8b4d0 2024-05-09T01:31:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f251359e640 1 --2- 172.21.15.19:0/1048739052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f8b4d0 0x55ad55361bc0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f250c010010 tx=0x7f250c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47fd05b9493d349 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f24f8ff9640 1 -- 172.21.15.19:0/1048739052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c02d040 con 0x55ad54f8b4d0 2024-05-09T01:31:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f24f8ff9640 1 -- 172.21.15.19:0/1048739052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250c00fd30 con 0x55ad54f8b4d0 2024-05-09T01:31:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.397+0000 7f24f8ff9640 1 -- 172.21.15.19:0/1048739052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c019690 con 0x55ad54f8b4d0 2024-05-09T01:31:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 -- 172.21.15.19:0/1048739052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f8b4d0 msgr2=0x55ad55361bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 --2- 172.21.15.19:0/1048739052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f8b4d0 0x55ad55361bc0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f250c010010 tx=0x7f250c005cf0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 -- 172.21.15.19:0/1048739052 shutdown_connections 2024-05-09T01:31:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 --2- 172.21.15.19:0/1048739052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f8b4d0 0x55ad55361bc0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 --2- 172.21.15.19:0/1048739052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f832f0 0x55ad55361680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 --2- 172.21.15.19:0/1048739052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f6ad80 0x55ad5534bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 -- 172.21.15.19:0/1048739052 >> 172.21.15.19:0/1048739052 conn(0x55ad54fa0300 msgr2=0x55ad5534b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 -- 172.21.15.19:0/1048739052 shutdown_connections 2024-05-09T01:31:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 -- 172.21.15.19:0/1048739052 wait complete. 2024-05-09T01:31:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.398+0000 7f251ffa7740 1 -- start start 2024-05-09T01:31:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251ffa7740 1 Processor -- start 2024-05-09T01:31:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f6ad80 0x55ad5535ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f832f0 0x55ad5535f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251ffa7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 0x55ad553cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251ffa7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25141ac9d0 con 0x55ad54f832f0 2024-05-09T01:31:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251ffa7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25141ac300 con 0x55ad54f6ad80 2024-05-09T01:31:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251ffa7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f25141a9540 con 0x55ad54f8b4d0 2024-05-09T01:31:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251ffa7740 10 client.0 _ll_register_callbacks cb 0x7f25041083c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f832f0 0x55ad5535f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f832f0 0x55ad5535f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51894/0 (socket says 172.21.15.19:51894) 2024-05-09T01:31:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251259c640 1 -- 172.21.15.19:0/3380833321 learned_addr learned my addr 172.21.15.19:0/3380833321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251359e640 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 0x55ad553cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251359e640 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f6ad80 0x55ad5535ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251259c640 1 -- 172.21.15.19:0/3380833321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 msgr2=0x55ad553cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251259c640 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 0x55ad553cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251259c640 1 -- 172.21.15.19:0/3380833321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f6ad80 msgr2=0x55ad5535ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251259c640 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f6ad80 0x55ad5535ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251259c640 1 -- 172.21.15.19:0/3380833321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad55278ed0 con 0x55ad54f832f0 2024-05-09T01:31:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251359e640 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f6ad80 0x55ad5535ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f251259c640 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f832f0 0x55ad5535f360 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x55ad5534cc20 tx=0x7f250800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c3bd32126603fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.399+0000 7f24f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f24f6ff5640 1 -- 172.21.15.19:0/3380833321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250801d610 con 0x55ad54f832f0 2024-05-09T01:31:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f24f6ff5640 1 -- 172.21.15.19:0/3380833321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f250801d7b0 con 0x55ad54f832f0 2024-05-09T01:31:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251ffa7740 1 -- 172.21.15.19:0/3380833321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ad552bc240 con 0x55ad54f832f0 2024-05-09T01:31:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f24f6ff5640 1 -- 172.21.15.19:0/3380833321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f250802d050 con 0x55ad54f832f0 2024-05-09T01:31:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251ffa7740 10 client.16860 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251ffa7740 1 -- 172.21.15.19:0/3380833321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad553cf310 con 0x55ad54f832f0 2024-05-09T01:31:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f24f6ff5640 1 -- 172.21.15.19:0/3380833321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ad552bc240 con 0x55ad54f832f0 2024-05-09T01:31:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f24f6ff5640 1 -- 172.21.15.19:0/3380833321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2508010430 con 0x55ad54f832f0 2024-05-09T01:31:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251ffa7740 10 client.16860 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251ffa7740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251ffa7740 20 client.16860 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251ffa7740 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ad54fac1b0 0x55ad553d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251ffa7740 4 client.16860 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:31:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251ffa7740 1 -- 172.21.15.19:0/3380833321 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad55237350 con 0x55ad54fac1b0 2024-05-09T01:31:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.400+0000 7f251359e640 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ad54fac1b0 0x55ad553d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.401+0000 7f251359e640 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ad54fac1b0 0x55ad553d1c80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70ebdabb55544ce server_cookie=44dd2e9096c4b481 in_seq=0 out_seq=0 2024-05-09T01:31:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.401+0000 7f24f6ff5640 10 client.16860 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.417+0000 7f24f6ff5640 1 -- 172.21.15.19:0/3380833321 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ad55237350 con 0x55ad54fac1b0 2024-05-09T01:31:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.417+0000 7f24f6ff5640 10 client.16860 handle_command_reply: tid=0 2024-05-09T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.417+0000 7f251ffa7740 1 client.16860 shutdown 2024-05-09T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 -- 172.21.15.19:0/3380833321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f832f0 msgr2=0x55ad5535f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f832f0 0x55ad5535f360 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x55ad5534cc20 tx=0x7f250800bf90 comp rx=0 tx=0).stop 2024-05-09T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 -- 172.21.15.19:0/3380833321 shutdown_connections 2024-05-09T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ad54fac1b0 0x55ad553d1c80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad54f8b4d0 0x55ad553cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad54f832f0 0x55ad5535f360 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f24f6ff5640 0 client.16860 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 --2- 172.21.15.19:0/3380833321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad54f6ad80 0x55ad5535ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 -- 172.21.15.19:0/3380833321 >> 172.21.15.19:0/3380833321 conn(0x55ad54fa0300 msgr2=0x55ad553588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 -- 172.21.15.19:0/3380833321 shutdown_connections 2024-05-09T01:31:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 1 -- 172.21.15.19:0/3380833321 wait complete. 2024-05-09T01:31:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f251ffa7740 20 client.16860 trim_cache size 0 max 16384 2024-05-09T01:31:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:31:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:31:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:31:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 -- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24fc0b27e0 msgr2=0x7f24fc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24fc0b27e0 0x7f24fc0b4cf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55ad553518e0 tx=0x7f25041084f0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 -- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 msgr2=0x7f251419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 0x7f251419a3c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2504031440 tx=0x7f2504003c10 comp rx=0 tx=0).stop 2024-05-09T01:31:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 -- 172.21.15.19:0/856599566 shutdown_connections 2024-05-09T01:31:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24fc0b27e0 0x7f24fc0b4cf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514109bb0 0x7f25141a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2514104eb0 0x7f251419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 --2- 172.21.15.19:0/856599566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2514103c70 0x7f251419a3c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.418+0000 7f25190ee640 1 -- 172.21.15.19:0/856599566 >> 172.21.15.19:0/856599566 conn(0x7f25140fee40 msgr2=0x7f2514105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.419+0000 7f25190ee640 1 -- 172.21.15.19:0/856599566 shutdown_connections 2024-05-09T01:31:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:11.419+0000 7f25190ee640 1 -- 172.21.15.19:0/856599566 wait complete. 2024-05-09T01:31:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:12 smithi196 ceph-mon[35041]: pgmap v548: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:12.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:12 smithi019 ceph-mon[30158]: pgmap v548: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:12 smithi059 ceph-mon[34351]: pgmap v548: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:14.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:14 smithi196 ceph-mon[35041]: pgmap v549: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:31:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:14 smithi059 ceph-mon[34351]: pgmap v549: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:31:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:14 smithi019 ceph-mon[30158]: pgmap v549: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:31:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:16 smithi196 ceph-mon[35041]: pgmap v550: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:31:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:16 smithi059 ceph-mon[34351]: pgmap v550: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:31:16.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:16 smithi019 ceph-mon[30158]: pgmap v550: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:31:18.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:18 smithi196 ceph-mon[35041]: pgmap v551: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:18 smithi059 ceph-mon[34351]: pgmap v551: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:18 smithi019 ceph-mon[30158]: pgmap v551: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:20.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:20 smithi059 ceph-mon[34351]: pgmap v552: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:31:20.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:20 smithi019 ceph-mon[30158]: pgmap v552: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:31:20.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:20 smithi196 ceph-mon[35041]: pgmap v552: 97 pgs: 97 active+clean; 3.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:31:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:22 smithi059 ceph-mon[34351]: pgmap v553: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:31:22.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:22 smithi019 ceph-mon[30158]: pgmap v553: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:31:22.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:22 smithi196 ceph-mon[35041]: pgmap v553: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:31:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.530+0000 7f503d2f0640 1 -- 172.21.15.19:0/377028809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 msgr2=0x7f503810a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.530+0000 7f503d2f0640 1 --2- 172.21.15.19:0/377028809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 0x7f503810a640 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f503000adb0 tx=0x7f503002b7a0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.530+0000 7f503d2f0640 1 -- 172.21.15.19:0/377028809 shutdown_connections 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.530+0000 7f503d2f0640 1 --2- 172.21.15.19:0/377028809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 0x7f503810a640 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.530+0000 7f503d2f0640 1 --2- 172.21.15.19:0/377028809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5038103520 0x7f5038103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.530+0000 7f503d2f0640 1 --2- 172.21.15.19:0/377028809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50381022e0 0x7f5038102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.530+0000 7f503d2f0640 1 -- 172.21.15.19:0/377028809 >> 172.21.15.19:0/377028809 conn(0x7f50380fd4d0 msgr2=0x7f50380ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.530+0000 7f503d2f0640 1 -- 172.21.15.19:0/377028809 shutdown_connections 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.530+0000 7f503d2f0640 1 -- 172.21.15.19:0/377028809 wait complete. 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f503d2f0640 1 Processor -- start 2024-05-09T01:31:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f503d2f0640 1 -- start start 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f503d2f0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50381022e0 0x7f503819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f503d2f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5038103520 0x7f503819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f503d2f0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 0x7f50381a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f503d2f0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f503810d570 con 0x7f5038108220 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f503d2f0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f503810d3f0 con 0x7f50381022e0 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f503d2f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f503810d6f0 con 0x7f5038103520 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f50377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 0x7f50381a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f50377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 0x7f50381a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41992/0 (socket says 172.21.15.19:41992) 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f50377fe640 1 -- 172.21.15.19:0/2459301328 learned_addr learned my addr 172.21.15.19:0/2459301328 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f5036ffd640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50381022e0 0x7f503819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.531+0000 7f50367fc640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5038103520 0x7f503819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f50377fe640 1 -- 172.21.15.19:0/2459301328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5038103520 msgr2=0x7f503819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f50377fe640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5038103520 0x7f503819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f50377fe640 1 -- 172.21.15.19:0/2459301328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50381022e0 msgr2=0x7f503819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f50377fe640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50381022e0 0x7f503819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f50377fe640 1 -- 172.21.15.19:0/2459301328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50381a1e20 con 0x7f5038108220 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f50377fe640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 0x7f50381a17b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5030031040 tx=0x7f5030031410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f501ffff640 1 -- 172.21.15.19:0/2459301328 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030040070 con 0x7f5038108220 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f501ffff640 1 -- 172.21.15.19:0/2459301328 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50300309c0 con 0x7f5038108220 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f503d2f0640 1 -- 172.21.15.19:0/2459301328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50381a20b0 con 0x7f5038108220 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f501ffff640 1 -- 172.21.15.19:0/2459301328 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50300047c0 con 0x7f5038108220 2024-05-09T01:31:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.532+0000 7f503d2f0640 1 -- 172.21.15.19:0/2459301328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381a25a0 con 0x7f5038108220 2024-05-09T01:31:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f501ffff640 1 -- 172.21.15.19:0/2459301328 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5030030510 con 0x7f5038108220 2024-05-09T01:31:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f504419e740 1 Processor -- start 2024-05-09T01:31:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f504419e740 1 -- start start 2024-05-09T01:31:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f504419e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e529910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f504419e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e529e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f504419e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e530d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f504419e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556d1e4897c0 con 0x556d1e1811b0 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f504419e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556d1e4250e0 con 0x556d1e1582f0 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f504419e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d1e485780 con 0x556d1e1604d0 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f50367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e529910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f50367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e529910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42006/0 (socket says 172.21.15.19:42006) 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.535+0000 7f50367fc640 1 -- 172.21.15.19:0/1650645495 learned_addr learned my addr 172.21.15.19:0/1650645495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f501ffff640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50200b27e0 0x7f50200b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f5036ffd640 1 --2- 172.21.15.19:0/1650645495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e529e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f50367fc640 1 --2- 172.21.15.19:0/1650645495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e530d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f50377fe640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50200b27e0 0x7f50200b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f50367fc640 1 -- 172.21.15.19:0/1650645495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 msgr2=0x556d1e530d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f50367fc640 1 --2- 172.21.15.19:0/1650645495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e530d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f50367fc640 1 -- 172.21.15.19:0/1650645495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 msgr2=0x556d1e529e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f50367fc640 1 --2- 172.21.15.19:0/1650645495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e529e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f50367fc640 1 -- 172.21.15.19:0/1650645495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1e44e7c0 con 0x556d1e1811b0 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f501ffff640 1 -- 172.21.15.19:0/2459301328 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5030103670 con 0x7f5038108220 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f50367fc640 1 --2- 172.21.15.19:0/1650645495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e529910 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f502c00bfc0 tx=0x7f502c00b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9abb835caf3097 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f501d7fa640 1 -- 172.21.15.19:0/1650645495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c004030 con 0x556d1e1811b0 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f501d7fa640 1 -- 172.21.15.19:0/1650645495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c0041d0 con 0x556d1e1811b0 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f501d7fa640 1 -- 172.21.15.19:0/1650645495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c00f6e0 con 0x556d1e1811b0 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.536+0000 7f50377fe640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50200b27e0 0x7f50200b4cf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f50300c6cf0 tx=0x7f50300975f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.537+0000 7f504419e740 1 -- 172.21.15.19:0/1650645495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 msgr2=0x556d1e529910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.537+0000 7f504419e740 1 --2- 172.21.15.19:0/1650645495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e529910 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f502c00bfc0 tx=0x7f502c00b7a0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.537+0000 7f504419e740 1 -- 172.21.15.19:0/1650645495 shutdown_connections 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.537+0000 7f504419e740 1 --2- 172.21.15.19:0/1650645495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e530d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.537+0000 7f504419e740 1 --2- 172.21.15.19:0/1650645495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e529e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.537+0000 7f504419e740 1 --2- 172.21.15.19:0/1650645495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e529910 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.537+0000 7f504419e740 1 -- 172.21.15.19:0/1650645495 >> 172.21.15.19:0/1650645495 conn(0x556d1e175300 msgr2=0x556d1e520e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.537+0000 7f504419e740 1 -- 172.21.15.19:0/1650645495 shutdown_connections 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.537+0000 7f504419e740 1 -- 172.21.15.19:0/1650645495 wait complete. 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f504419e740 1 -- start start 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f504419e740 1 Processor -- start 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f504419e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e533680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f504419e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1604d0 0x556d1e533bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f504419e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1811b0 0x556d1e53ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f504419e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556d1e5317e0 con 0x556d1e1604d0 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f504419e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556d1e4109c0 con 0x556d1e1582f0 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f504419e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d1e531960 con 0x556d1e1811b0 2024-05-09T01:31:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f504419e740 10 client.0 _ll_register_callbacks cb 0x556d1e43d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f5036ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1604d0 0x556d1e533bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f5036ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1604d0 0x556d1e533bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42020/0 (socket says 172.21.15.19:42020) 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f5036ffd640 1 -- 172.21.15.19:0/3319200508 learned_addr learned my addr 172.21.15.19:0/3319200508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f50367fc640 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1811b0 0x556d1e53ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f50367fc640 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e533680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f5036ffd640 1 -- 172.21.15.19:0/3319200508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1811b0 msgr2=0x556d1e53ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f5036ffd640 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1811b0 0x556d1e53ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f5036ffd640 1 -- 172.21.15.19:0/3319200508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 msgr2=0x556d1e533680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f5036ffd640 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e533680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.538+0000 7f5036ffd640 1 -- 172.21.15.19:0/3319200508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1e4371f0 con 0x556d1e1604d0 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f5036ffd640 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1604d0 0x556d1e533bc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f502800ba40 tx=0x7f502800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bc0d5b213479b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f501aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f501aff5640 1 -- 172.21.15.19:0/3319200508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028003e80 con 0x556d1e1604d0 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f501aff5640 1 -- 172.21.15.19:0/3319200508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5028011440 con 0x556d1e1604d0 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f504419e740 1 -- 172.21.15.19:0/3319200508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502c0041d0 con 0x556d1e1604d0 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f501aff5640 1 -- 172.21.15.19:0/3319200508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5028011640 con 0x556d1e1604d0 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f504419e740 10 client.16872 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f504419e740 1 -- 172.21.15.19:0/3319200508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d1e1ec790 con 0x556d1e1604d0 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f501aff5640 1 -- 172.21.15.19:0/3319200508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c0041d0 con 0x556d1e1604d0 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f501aff5640 1 -- 172.21.15.19:0/3319200508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50280259c0 con 0x556d1e1604d0 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f504419e740 10 client.16872 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f504419e740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.539+0000 7f504419e740 20 client.16872 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.540+0000 7f504419e740 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556d1e13fd80 0x556d1e541060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.540+0000 7f504419e740 4 client.16872 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.540+0000 7f504419e740 1 -- 172.21.15.19:0/3319200508 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d1e4973d0 con 0x556d1e13fd80 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.540+0000 7f50367fc640 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556d1e13fd80 0x556d1e541060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.540+0000 7f50367fc640 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556d1e13fd80 0x556d1e541060 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f99c654b2f1ea2c server_cookie=69891c502b20d39f in_seq=0 out_seq=0 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.540+0000 7f501aff5640 10 client.16872 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f501aff5640 1 -- 172.21.15.19:0/3319200508 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d1e4973d0 con 0x556d1e13fd80 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f501aff5640 10 client.16872 handle_command_reply: tid=0 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 client.16872 shutdown 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 -- 172.21.15.19:0/3319200508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1604d0 msgr2=0x556d1e533bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1604d0 0x556d1e533bc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f502800ba40 tx=0x7f502800bf00 comp rx=0 tx=0).stop 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 -- 172.21.15.19:0/3319200508 shutdown_connections 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556d1e13fd80 0x556d1e541060 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1811b0 0x556d1e53ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1604d0 0x556d1e533bc0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f501aff5640 0 client.16872 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 --2- 172.21.15.19:0/3319200508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e533680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 -- 172.21.15.19:0/3319200508 >> 172.21.15.19:0/3319200508 conn(0x556d1e175300 msgr2=0x556d1e51f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.541+0000 7f504419e740 1 -- 172.21.15.19:0/3319200508 shutdown_connections 2024-05-09T01:31:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.542+0000 7f504419e740 1 -- 172.21.15.19:0/3319200508 wait complete. 2024-05-09T01:31:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.542+0000 7f504419e740 20 client.16872 trim_cache size 0 max 16384 2024-05-09T01:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f504419e740 1 Processor -- start 2024-05-09T01:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f504419e740 1 -- start start 2024-05-09T01:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f504419e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1582f0 0x556d1e52d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f504419e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e532370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f504419e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1811b0 0x556d1e53aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f504419e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50381a9170 con 0x556d1e1582f0 2024-05-09T01:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f504419e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50381aacb0 con 0x556d1e1811b0 2024-05-09T01:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f504419e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50381a9840 con 0x556d1e1604d0 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f50367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1582f0 0x556d1e52d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f50367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1582f0 0x556d1e52d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42024/0 (socket says 172.21.15.19:42024) 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.557+0000 7f50367fc640 1 -- 172.21.15.19:0/402809551 learned_addr learned my addr 172.21.15.19:0/402809551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f5036ffd640 1 --2- 172.21.15.19:0/402809551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e532370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f50367fc640 1 -- 172.21.15.19:0/402809551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 msgr2=0x556d1e532370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f50367fc640 1 --2- 172.21.15.19:0/402809551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e532370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f50367fc640 1 -- 172.21.15.19:0/402809551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1811b0 msgr2=0x556d1e53aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f50367fc640 1 --2- 172.21.15.19:0/402809551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1811b0 0x556d1e53aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f50367fc640 1 -- 172.21.15.19:0/402809551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1e2635f0 con 0x556d1e1582f0 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f50367fc640 1 --2- 172.21.15.19:0/402809551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1582f0 0x556d1e52d230 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f502c012e30 tx=0x7f502c00b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f614e467d559d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f501cff9640 1 -- 172.21.15.19:0/402809551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c027040 con 0x556d1e1582f0 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f501cff9640 1 -- 172.21.15.19:0/402809551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c005500 con 0x556d1e1582f0 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.558+0000 7f501cff9640 1 -- 172.21.15.19:0/402809551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c010460 con 0x556d1e1582f0 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 -- 172.21.15.19:0/402809551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1582f0 msgr2=0x556d1e52d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 --2- 172.21.15.19:0/402809551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1582f0 0x556d1e52d230 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f502c012e30 tx=0x7f502c00b590 comp rx=0 tx=0).stop 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 -- 172.21.15.19:0/402809551 shutdown_connections 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 --2- 172.21.15.19:0/402809551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1811b0 0x556d1e53aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 --2- 172.21.15.19:0/402809551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e532370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 --2- 172.21.15.19:0/402809551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1582f0 0x556d1e52d230 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 -- 172.21.15.19:0/402809551 >> 172.21.15.19:0/402809551 conn(0x556d1e175300 msgr2=0x556d1e52cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 -- 172.21.15.19:0/402809551 shutdown_connections 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 -- 172.21.15.19:0/402809551 wait complete. 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 -- start start 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 Processor -- start 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e5381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e5386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.559+0000 7f504419e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e5a7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f504419e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50381aa5e0 con 0x556d1e1811b0 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f504419e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50381a9f10 con 0x556d1e1582f0 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f504419e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50381a7150 con 0x556d1e1604d0 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f504419e740 10 client.0 _ll_register_callbacks cb 0x7f50280042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e5a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e5a7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42028/0 (socket says 172.21.15.19:42028) 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 -- 172.21.15.19:0/4269449096 learned_addr learned my addr 172.21.15.19:0/4269449096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e5381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f5036ffd640 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e5386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 -- 172.21.15.19:0/4269449096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 msgr2=0x556d1e5386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e5386f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 -- 172.21.15.19:0/4269449096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 msgr2=0x556d1e5381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e5381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 -- 172.21.15.19:0/4269449096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1e44ded0 con 0x556d1e1811b0 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f5036ffd640 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e5386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f50367fc640 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e5a7400 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f502c005940 tx=0x7f502c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad24f3d3f5871fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f501aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f501aff5640 1 -- 172.21.15.19:0/4269449096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c004c30 con 0x556d1e1811b0 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f501aff5640 1 -- 172.21.15.19:0/4269449096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f502c004dd0 con 0x556d1e1811b0 2024-05-09T01:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f504419e740 1 -- 172.21.15.19:0/4269449096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502c005500 con 0x556d1e1811b0 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f501aff5640 1 -- 172.21.15.19:0/4269449096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f502c004fd0 con 0x556d1e1811b0 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f504419e740 10 client.16878 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.560+0000 7f504419e740 1 -- 172.21.15.19:0/4269449096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d1e5a8810 con 0x556d1e1811b0 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.561+0000 7f501aff5640 1 -- 172.21.15.19:0/4269449096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c005500 con 0x556d1e1811b0 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.561+0000 7f501aff5640 1 -- 172.21.15.19:0/4269449096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f502c010c10 con 0x556d1e1811b0 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.561+0000 7f504419e740 10 client.16878 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.561+0000 7f504419e740 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.561+0000 7f504419e740 20 client.16878 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.561+0000 7f504419e740 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556d1e1151c0 0x556d1e5aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.561+0000 7f504419e740 4 client.16878 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.561+0000 7f504419e740 1 -- 172.21.15.19:0/4269449096 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d1e491240 con 0x556d1e1151c0 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.561+0000 7f5036ffd640 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556d1e1151c0 0x556d1e5aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.562+0000 7f501aff5640 10 client.16878 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.562+0000 7f5036ffd640 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556d1e1151c0 0x556d1e5aafd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b3e2019aa7cd9a server_cookie=52f835b3ff3bfdbc in_seq=0 out_seq=0 2024-05-09T01:31:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.579+0000 7f501aff5640 1 -- 172.21.15.19:0/4269449096 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556d1e491240 con 0x556d1e1151c0 2024-05-09T01:31:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.579+0000 7f501aff5640 10 client.16878 handle_command_reply: tid=0 2024-05-09T01:31:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.579+0000 7f504419e740 1 client.16878 shutdown 2024-05-09T01:31:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 -- 172.21.15.19:0/4269449096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 msgr2=0x556d1e5a7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e5a7400 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f502c005940 tx=0x7f502c00bf00 comp rx=0 tx=0).stop 2024-05-09T01:31:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 -- 172.21.15.19:0/4269449096 shutdown_connections 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556d1e1151c0 0x556d1e5aafd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d1e1811b0 0x556d1e5a7400 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f501aff5640 0 client.16878 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d1e1604d0 0x556d1e5386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 --2- 172.21.15.19:0/4269449096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d1e1582f0 0x556d1e5381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 -- 172.21.15.19:0/4269449096 >> 172.21.15.19:0/4269449096 conn(0x556d1e175300 msgr2=0x556d1e5207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 -- 172.21.15.19:0/4269449096 shutdown_connections 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 1 -- 172.21.15.19:0/4269449096 wait complete. 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f504419e740 20 client.16878 trim_cache size 0 max 16384 2024-05-09T01:31:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:31:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:31:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:31:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f503d2f0640 1 -- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50200b27e0 msgr2=0x7f50200b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f503d2f0640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50200b27e0 0x7f50200b4cf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f50300c6cf0 tx=0x7f50300975f0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f503d2f0640 1 -- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 msgr2=0x7f50381a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.580+0000 7f503d2f0640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 0x7f50381a17b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5030031040 tx=0x7f5030031410 comp rx=0 tx=0).stop 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.581+0000 7f503d2f0640 1 -- 172.21.15.19:0/2459301328 shutdown_connections 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.581+0000 7f503d2f0640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f50200b27e0 0x7f50200b4cf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.581+0000 7f503d2f0640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5038108220 0x7f50381a17b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.581+0000 7f503d2f0640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5038103520 0x7f503819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.581+0000 7f503d2f0640 1 --2- 172.21.15.19:0/2459301328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50381022e0 0x7f503819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.581+0000 7f503d2f0640 1 -- 172.21.15.19:0/2459301328 >> 172.21.15.19:0/2459301328 conn(0x7f50380fd4d0 msgr2=0x7f5038109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.581+0000 7f503d2f0640 1 -- 172.21.15.19:0/2459301328 shutdown_connections 2024-05-09T01:31:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:24.581+0000 7f503d2f0640 1 -- 172.21.15.19:0/2459301328 wait complete. 2024-05-09T01:31:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:24 smithi059 ceph-mon[34351]: pgmap v554: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:31:24.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:24 smithi019 ceph-mon[30158]: pgmap v554: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:31:24.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:24 smithi196 ceph-mon[35041]: pgmap v554: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:31:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:26 smithi059 ceph-mon[34351]: pgmap v555: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:31:26.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:26 smithi019 ceph-mon[30158]: pgmap v555: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:31:26.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:26 smithi196 ceph-mon[35041]: pgmap v555: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:31:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:28 smithi059 ceph-mon[34351]: pgmap v556: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:28.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:28 smithi019 ceph-mon[30158]: pgmap v556: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:28.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:28 smithi196 ceph-mon[35041]: pgmap v556: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:31:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:30 smithi059 ceph-mon[34351]: pgmap v557: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:30.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:30 smithi019 ceph-mon[30158]: pgmap v557: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:30 smithi196 ceph-mon[35041]: pgmap v557: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:32.746 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:32 smithi196 ceph-mon[35041]: pgmap v558: 97 pgs: 97 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:31:32.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:32 smithi059 ceph-mon[34351]: pgmap v558: 97 pgs: 97 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:31:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:32 smithi019 ceph-mon[30158]: pgmap v558: 97 pgs: 97 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:31:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:34 smithi059 ceph-mon[34351]: pgmap v559: 97 pgs: 97 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:31:34.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:34 smithi019 ceph-mon[30158]: pgmap v559: 97 pgs: 97 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:31:34.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:34 smithi196 ceph-mon[35041]: pgmap v559: 97 pgs: 97 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:31:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:36 smithi059 ceph-mon[34351]: pgmap v560: 97 pgs: 97 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:36 smithi019 ceph-mon[30158]: pgmap v560: 97 pgs: 97 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:36.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:36 smithi196 ceph-mon[35041]: pgmap v560: 97 pgs: 97 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:36.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:31:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.685+0000 7fee84eb1640 1 -- 172.21.15.19:0/3538503689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee80103ca0 msgr2=0x7fee80104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.685+0000 7fee84eb1640 1 --2- 172.21.15.19:0/3538503689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee80103ca0 0x7fee80104170 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fee68009900 tx=0x7fee6802a410 comp rx=0 tx=0).stop 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.685+0000 7fee84eb1640 1 -- 172.21.15.19:0/3538503689 shutdown_connections 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.685+0000 7fee84eb1640 1 --2- 172.21.15.19:0/3538503689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee80104800 0x7fee8010af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.685+0000 7fee84eb1640 1 --2- 172.21.15.19:0/3538503689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee80103ca0 0x7fee80104170 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.685+0000 7fee84eb1640 1 --2- 172.21.15.19:0/3538503689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee80103140 0x7fee80103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.685+0000 7fee84eb1640 1 -- 172.21.15.19:0/3538503689 >> 172.21.15.19:0/3538503689 conn(0x7fee800fd4c0 msgr2=0x7fee800ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.685+0000 7fee84eb1640 1 -- 172.21.15.19:0/3538503689 shutdown_connections 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.685+0000 7fee84eb1640 1 -- 172.21.15.19:0/3538503689 wait complete. 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee84eb1640 1 Processor -- start 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee84eb1640 1 -- start start 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee84eb1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee80103140 0x7fee8019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee84eb1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee80103ca0 0x7fee8019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee84eb1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8019f620 0x7fee801a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee84eb1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010f9d0 con 0x7fee8019f620 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee84eb1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010f850 con 0x7fee80103140 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee84eb1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010fb50 con 0x7fee80103ca0 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee7ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8019f620 0x7fee801a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee7ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8019f620 0x7fee801a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48394/0 (socket says 172.21.15.19:48394) 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee7ed9d640 1 -- 172.21.15.19:0/2131887267 learned_addr learned my addr 172.21.15.19:0/2131887267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee7e59c640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee80103140 0x7fee8019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.686+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee80103ca0 0x7fee8019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee7ed9d640 1 -- 172.21.15.19:0/2131887267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee80103ca0 msgr2=0x7fee8019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee7ed9d640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee80103ca0 0x7fee8019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee7ed9d640 1 -- 172.21.15.19:0/2131887267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee80103140 msgr2=0x7fee8019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee7ed9d640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee80103140 0x7fee8019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee7ed9d640 1 -- 172.21.15.19:0/2131887267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee801a20b0 con 0x7fee8019f620 2024-05-09T01:31:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee80103ca0 0x7fee8019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee7ed9d640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8019f620 0x7fee801a1a40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fee7800eb10 tx=0x7fee7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee777fe640 1 -- 172.21.15.19:0/2131887267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78004b20 con 0x7fee8019f620 2024-05-09T01:31:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee777fe640 1 -- 172.21.15.19:0/2131887267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78004cc0 con 0x7fee8019f620 2024-05-09T01:31:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee84eb1640 1 -- 172.21.15.19:0/2131887267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee801a23a0 con 0x7fee8019f620 2024-05-09T01:31:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee777fe640 1 -- 172.21.15.19:0/2131887267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78010670 con 0x7fee8019f620 2024-05-09T01:31:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.687+0000 7fee84eb1640 1 -- 172.21.15.19:0/2131887267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee801a28f0 con 0x7fee8019f620 2024-05-09T01:31:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee8bd66740 1 Processor -- start 2024-05-09T01:31:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee777fe640 1 -- 172.21.15.19:0/2131887267 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fee78005020 con 0x7fee8019f620 2024-05-09T01:31:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee8bd66740 1 -- start start 2024-05-09T01:31:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x556122713500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561223462f0 0x556122713a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612234e4d0 0x55612271a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee8bd66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561226777c0 con 0x55612234e4d0 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee8bd66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5561226130e0 con 0x55612236f1b0 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee8bd66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556122673780 con 0x5561223462f0 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.690+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612234e4d0 0x55612271a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612234e4d0 0x55612271a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48410/0 (socket says 172.21.15.19:48410) 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7dd9b640 1 -- 172.21.15.19:0/617313879 learned_addr learned my addr 172.21.15.19:0/617313879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/617313879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x556122713500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7e59c640 1 --2- 172.21.15.19:0/617313879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561223462f0 0x556122713a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee777fe640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee6c0b2860 0x7fee6c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7dd9b640 1 -- 172.21.15.19:0/617313879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561223462f0 msgr2=0x556122713a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/617313879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561223462f0 0x556122713a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7dd9b640 1 -- 172.21.15.19:0/617313879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 msgr2=0x556122713500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/617313879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x556122713500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7dd9b640 1 -- 172.21.15.19:0/617313879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612263c7c0 con 0x55612234e4d0 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee777fe640 1 -- 172.21.15.19:0/2131887267 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fee780e27e0 con 0x7fee8019f620 2024-05-09T01:31:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7ed9d640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee6c0b2860 0x7fee6c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/617313879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612234e4d0 0x55612271a980 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fee68005940 tx=0x7fee68005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b176ae250783102 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee74ff9640 1 -- 172.21.15.19:0/617313879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68035040 con 0x55612234e4d0 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.691+0000 7fee74ff9640 1 -- 172.21.15.19:0/617313879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee68005020 con 0x55612234e4d0 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.692+0000 7fee74ff9640 1 -- 172.21.15.19:0/617313879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee680044e0 con 0x55612234e4d0 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.692+0000 7fee8bd66740 1 -- 172.21.15.19:0/617313879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612234e4d0 msgr2=0x55612271a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.692+0000 7fee8bd66740 1 --2- 172.21.15.19:0/617313879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612234e4d0 0x55612271a980 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fee68005940 tx=0x7fee68005a40 comp rx=0 tx=0).stop 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.692+0000 7fee7ed9d640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee6c0b2860 0x7fee6c0b4d70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fee780049d0 tx=0x7fee7800cf70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.692+0000 7fee8bd66740 1 -- 172.21.15.19:0/617313879 shutdown_connections 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.692+0000 7fee8bd66740 1 --2- 172.21.15.19:0/617313879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612234e4d0 0x55612271a980 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.692+0000 7fee8bd66740 1 --2- 172.21.15.19:0/617313879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561223462f0 0x556122713a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.692+0000 7fee8bd66740 1 --2- 172.21.15.19:0/617313879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x556122713500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.692+0000 7fee8bd66740 1 -- 172.21.15.19:0/617313879 >> 172.21.15.19:0/617313879 conn(0x556122363300 msgr2=0x55612270ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 -- 172.21.15.19:0/617313879 shutdown_connections 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 -- 172.21.15.19:0/617313879 wait complete. 2024-05-09T01:31:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 -- start start 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 Processor -- start 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x55612271d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x55612271d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x556122724710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55612271b410 con 0x5561223462f0 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5561225fe9c0 con 0x55612236f1b0 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55612271b590 con 0x55612234e4d0 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.693+0000 7fee8bd66740 10 client.0 _ll_register_callbacks cb 0x55612262b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x55612271d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x55612271d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48418/0 (socket says 172.21.15.19:48418) 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7dd9b640 1 -- 172.21.15.19:0/2417212354 learned_addr learned my addr 172.21.15.19:0/2417212354 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7e59c640 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x55612271d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7dd9b640 1 -- 172.21.15.19:0/2417212354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 msgr2=0x55612271d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x55612271d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7dd9b640 1 -- 172.21.15.19:0/2417212354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 msgr2=0x556122724710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x556122724710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7dd9b640 1 -- 172.21.15.19:0/2417212354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561226251f0 con 0x5561223462f0 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x55612271d270 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fee68033000 tx=0x7fee680092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533b5207ea212a49 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee65ffb640 1 -- 172.21.15.19:0/2417212354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6804e070 con 0x5561223462f0 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee65ffb640 1 -- 172.21.15.19:0/2417212354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fee68005490 con 0x5561223462f0 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.694+0000 7fee8bd66740 1 -- 172.21.15.19:0/2417212354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee68005020 con 0x5561223462f0 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee65ffb640 1 -- 172.21.15.19:0/2417212354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee68035030 con 0x5561223462f0 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee8bd66740 10 client.16890 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee8bd66740 1 -- 172.21.15.19:0/2417212354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5561223da790 con 0x5561223462f0 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee65ffb640 1 -- 172.21.15.19:0/2417212354 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee68005020 con 0x5561223462f0 2024-05-09T01:31:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee65ffb640 1 -- 172.21.15.19:0/2417212354 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fee6803a040 con 0x5561223462f0 2024-05-09T01:31:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee8bd66740 10 client.16890 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee8bd66740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee8bd66740 20 client.16890 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee8bd66740 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55612232dd80 0x55612272a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee8bd66740 4 client.16890 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.695+0000 7fee8bd66740 1 -- 172.21.15.19:0/2417212354 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561226853d0 con 0x55612232dd80 2024-05-09T01:31:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.696+0000 7fee7e59c640 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55612232dd80 0x55612272a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.696+0000 7fee7e59c640 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55612232dd80 0x55612272a7d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa02759706e6153b server_cookie=fc47066e44d63c5a in_seq=0 out_seq=0 2024-05-09T01:31:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.696+0000 7fee65ffb640 10 client.16890 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee65ffb640 1 -- 172.21.15.19:0/2417212354 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5561226853d0 con 0x55612232dd80 2024-05-09T01:31:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee65ffb640 10 client.16890 handle_command_reply: tid=0 2024-05-09T01:31:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 client.16890 shutdown 2024-05-09T01:31:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 -- 172.21.15.19:0/2417212354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 msgr2=0x55612271d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x55612271d270 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fee68033000 tx=0x7fee680092c0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 -- 172.21.15.19:0/2417212354 shutdown_connections 2024-05-09T01:31:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55612232dd80 0x55612272a7d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x556122724710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x55612271d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 --2- 172.21.15.19:0/2417212354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x55612271d270 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee65ffb640 0 client.16890 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 -- 172.21.15.19:0/2417212354 >> 172.21.15.19:0/2417212354 conn(0x556122363300 msgr2=0x55612270d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 -- 172.21.15.19:0/2417212354 shutdown_connections 2024-05-09T01:31:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.697+0000 7fee8bd66740 1 -- 172.21.15.19:0/2417212354 wait complete. 2024-05-09T01:31:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.698+0000 7fee8bd66740 20 client.16890 trim_cache size 0 max 16384 2024-05-09T01:31:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee8bd66740 1 Processor -- start 2024-05-09T01:31:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee8bd66740 1 -- start start 2024-05-09T01:31:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561223462f0 0x55612271d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x5561227246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612236f1b0 0x556122724be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee8bd66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee801ab400 con 0x55612236f1b0 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee8bd66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee801acf40 con 0x5561223462f0 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee8bd66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee801abad0 con 0x55612234e4d0 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612236f1b0 0x556122724be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612236f1b0 0x556122724be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48426/0 (socket says 172.21.15.19:48426) 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 -- 172.21.15.19:0/4263514829 learned_addr learned my addr 172.21.15.19:0/4263514829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/4263514829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561223462f0 0x55612271d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7e59c640 1 --2- 172.21.15.19:0/4263514829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x5561227246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 -- 172.21.15.19:0/4263514829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 msgr2=0x5561227246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/4263514829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x5561227246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 -- 172.21.15.19:0/4263514829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561223462f0 msgr2=0x55612271d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/4263514829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561223462f0 0x55612271d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 -- 172.21.15.19:0/4263514829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612272ab40 con 0x55612236f1b0 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.713+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/4263514829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612236f1b0 0x556122724be0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fee68005f70 tx=0x7fee680314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b853a20f941d913e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee67fff640 1 -- 172.21.15.19:0/4263514829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6804e070 con 0x55612236f1b0 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee67fff640 1 -- 172.21.15.19:0/4263514829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee680035f0 con 0x55612236f1b0 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee67fff640 1 -- 172.21.15.19:0/4263514829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6803d480 con 0x55612236f1b0 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee8bd66740 1 -- 172.21.15.19:0/4263514829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612236f1b0 msgr2=0x556122724be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4263514829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612236f1b0 0x556122724be0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fee68005f70 tx=0x7fee680314a0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee8bd66740 1 -- 172.21.15.19:0/4263514829 shutdown_connections 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4263514829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55612236f1b0 0x556122724be0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4263514829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x5561227246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4263514829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561223462f0 0x55612271d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.714+0000 7fee8bd66740 1 -- 172.21.15.19:0/4263514829 >> 172.21.15.19:0/4263514829 conn(0x556122363300 msgr2=0x55612271ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 -- 172.21.15.19:0/4263514829 shutdown_connections 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 -- 172.21.15.19:0/4263514829 wait complete. 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 -- start start 2024-05-09T01:31:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 Processor -- start 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x556122721eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x5561227223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x5561227232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee801ac870 con 0x5561223462f0 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee801ac1a0 con 0x55612236f1b0 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee801a93e0 con 0x55612234e4d0 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee8bd66740 10 client.0 _ll_register_callbacks cb 0x7fee68009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x556122721eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x556122721eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48436/0 (socket says 172.21.15.19:48436) 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.715+0000 7fee7dd9b640 1 -- 172.21.15.19:0/4257564861 learned_addr learned my addr 172.21.15.19:0/4257564861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee7dd9b640 1 -- 172.21.15.19:0/4257564861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 msgr2=0x5561227223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee7e59c640 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x5561227223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x5561227223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee7dd9b640 1 -- 172.21.15.19:0/4257564861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 msgr2=0x5561227232d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x5561227232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee7dd9b640 1 -- 172.21.15.19:0/4257564861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561224515f0 con 0x5561223462f0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee7dd9b640 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x556122721eb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fee680047e0 tx=0x7fee68009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594103c011b7b7a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee65ffb640 1 -- 172.21.15.19:0/4257564861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6804e040 con 0x5561223462f0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee65ffb640 1 -- 172.21.15.19:0/4257564861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fee68004810 con 0x5561223462f0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee8bd66740 1 -- 172.21.15.19:0/4257564861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee680035f0 con 0x5561223462f0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee65ffb640 1 -- 172.21.15.19:0/4257564861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee68004a10 con 0x5561223462f0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee8bd66740 10 client.16896 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee8bd66740 1 -- 172.21.15.19:0/4257564861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55612263bed0 con 0x5561223462f0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.716+0000 7fee65ffb640 1 -- 172.21.15.19:0/4257564861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee680035f0 con 0x5561223462f0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee65ffb640 1 -- 172.21.15.19:0/4257564861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fee6804a450 con 0x5561223462f0 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee8bd66740 10 client.16896 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee8bd66740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee8bd66740 20 client.16896 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5561223031c0 0x556122794960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee8bd66740 4 client.16896 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:31:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee8bd66740 1 -- 172.21.15.19:0/4257564861 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55612267f240 con 0x5561223031c0 2024-05-09T01:31:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee7e59c640 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5561223031c0 0x556122794960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee65ffb640 10 client.16896 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.717+0000 7fee7e59c640 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5561223031c0 0x556122794960 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e2f0fa16ebf87a server_cookie=ac15af5d5704473c in_seq=0 out_seq=0 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee65ffb640 1 -- 172.21.15.19:0/4257564861 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55612267f240 con 0x5561223031c0 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee65ffb640 10 client.16896 handle_command_reply: tid=0 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee8bd66740 1 client.16896 shutdown 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee8bd66740 1 -- 172.21.15.19:0/4257564861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 msgr2=0x556122721eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x556122721eb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fee680047e0 tx=0x7fee68009230 comp rx=0 tx=0).stop 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee8bd66740 1 -- 172.21.15.19:0/4257564861 shutdown_connections 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5561223031c0 0x556122794960 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55612236f1b0 0x5561227232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55612234e4d0 0x5561227223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee8bd66740 1 --2- 172.21.15.19:0/4257564861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561223462f0 0x556122721eb0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee65ffb640 0 client.16896 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.735+0000 7fee8bd66740 1 -- 172.21.15.19:0/4257564861 >> 172.21.15.19:0/4257564861 conn(0x556122363300 msgr2=0x55612271c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee8bd66740 1 -- 172.21.15.19:0/4257564861 shutdown_connections 2024-05-09T01:31:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee8bd66740 1 -- 172.21.15.19:0/4257564861 wait complete. 2024-05-09T01:31:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee8bd66740 20 client.16896 trim_cache size 0 max 16384 2024-05-09T01:31:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:31:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:31:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:31:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 -- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee6c0b2860 msgr2=0x7fee6c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee6c0b2860 0x7fee6c0b4d70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fee780049d0 tx=0x7fee7800cf70 comp rx=0 tx=0).stop 2024-05-09T01:31:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 -- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8019f620 msgr2=0x7fee801a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8019f620 0x7fee801a1a40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fee7800eb10 tx=0x7fee7800c2d0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 -- 172.21.15.19:0/2131887267 shutdown_connections 2024-05-09T01:31:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fee6c0b2860 0x7fee6c0b4d70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee8019f620 0x7fee801a1a40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee80103ca0 0x7fee8019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 --2- 172.21.15.19:0/2131887267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee80103140 0x7fee8019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 -- 172.21.15.19:0/2131887267 >> 172.21.15.19:0/2131887267 conn(0x7fee800fd4c0 msgr2=0x7fee800ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 -- 172.21.15.19:0/2131887267 shutdown_connections 2024-05-09T01:31:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:37.736+0000 7fee84eb1640 1 -- 172.21.15.19:0/2131887267 wait complete. 2024-05-09T01:31:38.790 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:38 smithi019 ceph-mon[30158]: pgmap v561: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:38 smithi059 ceph-mon[34351]: pgmap v561: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:38.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:38 smithi196 ceph-mon[35041]: pgmap v561: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:39.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.082+0000 7fdb08e77700 1 -- 172.21.15.19:0/1533230634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04108740 msgr2=0x7fdb041100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:39.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.082+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1533230634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04108740 0x7fdb041100a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00b0d0 tx=0x7fdafc00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.082+0000 7fdb08e77700 1 -- 172.21.15.19:0/1533230634 shutdown_connections 2024-05-09T01:31:39.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.082+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1533230634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04108740 0x7fdb041100a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.082+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1533230634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb041034c0 0x7fdb041038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.082+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1533230634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb04107df0 0x7fdb04108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.082+0000 7fdb08e77700 1 -- 172.21.15.19:0/1533230634 >> 172.21.15.19:0/1533230634 conn(0x7fdb040fedc0 msgr2=0x7fdb041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.082+0000 7fdb08e77700 1 -- 172.21.15.19:0/1533230634 shutdown_connections 2024-05-09T01:31:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.082+0000 7fdb08e77700 1 -- 172.21.15.19:0/1533230634 wait complete. 2024-05-09T01:31:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.083+0000 7fdb08e77700 1 Processor -- start 2024-05-09T01:31:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.083+0000 7fdb08e77700 1 -- start start 2024-05-09T01:31:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.083+0000 7fdb08e77700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb041034c0 0x7fdb04199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.083+0000 7fdb08e77700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04107df0 0x7fdb0419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.083+0000 7fdb08e77700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb04108740 0x7fdb0419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.083+0000 7fdb08e77700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04113370 con 0x7fdb04107df0 2024-05-09T01:31:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb08e77700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041131f0 con 0x7fdb041034c0 2024-05-09T01:31:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb08e77700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041134f0 con 0x7fdb04108740 2024-05-09T01:31:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb01d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04107df0 0x7fdb0419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb01d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04107df0 0x7fdb0419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48460/0 (socket says 172.21.15.19:48460) 2024-05-09T01:31:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb01d9b700 1 -- 172.21.15.19:0/1437167223 learned_addr learned my addr 172.21.15.19:0/1437167223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb041034c0 0x7fdb04199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb02d9d700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb04108740 0x7fdb0419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb01d9b700 1 -- 172.21.15.19:0/1437167223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb04108740 msgr2=0x7fdb0419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb04108740 0x7fdb0419e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb01d9b700 1 -- 172.21.15.19:0/1437167223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb041034c0 msgr2=0x7fdb04199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb041034c0 0x7fdb04199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb01d9b700 1 -- 172.21.15.19:0/1437167223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0419ec10 con 0x7fdb04107df0 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04107df0 0x7fdb0419a2e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800d950 tx=0x7fdaf800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdaf37fe700 1 -- 172.21.15.19:0/1437167223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8016070 con 0x7fdb04107df0 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdaf37fe700 1 -- 172.21.15.19:0/1437167223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf8004830 con 0x7fdb04107df0 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdb08e77700 1 -- 172.21.15.19:0/1437167223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0419ef00 con 0x7fdb04107df0 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.084+0000 7fdaf37fe700 1 -- 172.21.15.19:0/1437167223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8005440 con 0x7fdb04107df0 2024-05-09T01:31:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.085+0000 7fdb08e77700 1 -- 172.21.15.19:0/1437167223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb041a6820 con 0x7fdb04107df0 2024-05-09T01:31:39.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.088+0000 7fdaf37fe700 1 -- 172.21.15.19:0/1437167223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdaf8004220 con 0x7fdb04107df0 2024-05-09T01:31:39.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.088+0000 7fdaf37fe700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdaec0b2880 0x7fdaec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.088+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdaec0b2880 0x7fdaec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.088+0000 7fdaf37fe700 1 -- 172.21.15.19:0/1437167223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdaf80e1f90 con 0x7fdb04107df0 2024-05-09T01:31:39.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.089+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdaec0b2880 0x7fdaec0b4d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009c60 tx=0x7fdaf4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:39.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.089+0000 7fdb1788db80 1 Processor -- start 2024-05-09T01:31:39.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.089+0000 7fdb1788db80 1 -- start start 2024-05-09T01:31:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a09e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f194a0 0x55e85a09ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 0x55e85a098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb1788db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e859fb90a0 con 0x55e859f93d00 2024-05-09T01:31:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb1788db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e859f5e9a0 con 0x55e859f194a0 2024-05-09T01:31:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb1788db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e859fb5060 con 0x55e85a098610 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb0259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a09e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb0259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a09e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48472/0 (socket says 172.21.15.19:48472) 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb0259c700 1 -- 172.21.15.19:0/1189151655 learned_addr learned my addr 172.21.15.19:0/1189151655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb02d9d700 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 0x55e85a098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f194a0 0x55e85a09ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb0259c700 1 -- 172.21.15.19:0/1189151655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 msgr2=0x55e85a098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 0x55e85a098a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb0259c700 1 -- 172.21.15.19:0/1189151655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f194a0 msgr2=0x55e85a09ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f194a0 0x55e85a09ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb0259c700 1 -- 172.21.15.19:0/1189151655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e859f4f500 con 0x55e859f93d00 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb02d9d700 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 0x55e85a098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:31:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.090+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f194a0 0x55e85a09ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.091+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a09e4b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400e4e0 tx=0x7fdaf4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca428838831cf33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.091+0000 7fdaf0ff9700 1 -- 172.21.15.19:0/1189151655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf400ee00 con 0x55e859f93d00 2024-05-09T01:31:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.091+0000 7fdaf0ff9700 1 -- 172.21.15.19:0/1189151655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf401c8c0 con 0x55e859f93d00 2024-05-09T01:31:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.091+0000 7fdaf0ff9700 1 -- 172.21.15.19:0/1189151655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf401da20 con 0x55e859f93d00 2024-05-09T01:31:39.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 -- 172.21.15.19:0/1189151655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 msgr2=0x55e85a09e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:39.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a09e4b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400e4e0 tx=0x7fdaf4014000 comp rx=0 tx=0).stop 2024-05-09T01:31:39.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 -- 172.21.15.19:0/1189151655 shutdown_connections 2024-05-09T01:31:39.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 0x55e85a098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a09e4b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1189151655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f194a0 0x55e85a09ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 -- 172.21.15.19:0/1189151655 >> 172.21.15.19:0/1189151655 conn(0x55e859f1cf70 msgr2=0x55e85a091220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 -- 172.21.15.19:0/1189151655 shutdown_connections 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 -- 172.21.15.19:0/1189151655 wait complete. 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.092+0000 7fdb1788db80 1 -- start start 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb1788db80 1 Processor -- start 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a098610 0x55e85a0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb1788db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e85a099360 con 0x55e85a098610 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb1788db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e859f53b20 con 0x55e859f93d00 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb1788db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e85a0a5270 con 0x55e859f194a0 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb1788db80 10 client.0 _ll_register_callbacks cb 0x55e859e2a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb02d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a098610 0x55e85a0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb02d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a098610 0x55e85a0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48488/0 (socket says 172.21.15.19:48488) 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb02d9d700 1 -- 172.21.15.19:0/1214813694 learned_addr learned my addr 172.21.15.19:0/1214813694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb02d9d700 1 -- 172.21.15.19:0/1214813694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 msgr2=0x55e85a0a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb02d9d700 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb02d9d700 1 -- 172.21.15.19:0/1214813694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 msgr2=0x55e85a0a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb02d9d700 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb02d9d700 1 -- 172.21.15.19:0/1214813694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e859e34490 con 0x55e85a098610 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.093+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.094+0000 7fdb02d9d700 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a098610 0x55e85a0b04b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00baa0 tx=0x7fdafc00bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52be824d67fd751 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.094+0000 7fdae9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.094+0000 7fdae9ffb700 1 -- 172.21.15.19:0/1214813694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0079d0 con 0x55e85a098610 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.094+0000 7fdae9ffb700 1 -- 172.21.15.19:0/1214813694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdafc007b70 con 0x55e85a098610 2024-05-09T01:31:39.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.094+0000 7fdb1788db80 1 -- 172.21.15.19:0/1214813694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf401c8c0 con 0x55e85a098610 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.094+0000 7fdae9ffb700 1 -- 172.21.15.19:0/1214813694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdafc00be50 con 0x55e85a098610 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.094+0000 7fdb1788db80 10 client.16908 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.094+0000 7fdb1788db80 1 -- 172.21.15.19:0/1214813694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e85a0b1ed0 con 0x55e85a098610 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.094+0000 7fdae9ffb700 1 -- 172.21.15.19:0/1214813694 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdafc007d70 con 0x55e85a098610 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdae9ffb700 1 -- 172.21.15.19:0/1214813694 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdafc011070 con 0x55e85a098610 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdb1788db80 10 client.16908 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdb1788db80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdb1788db80 20 client.16908 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e85a0b4510 0x55e85a0b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdb1788db80 4 client.16908 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdb1788db80 1 -- 172.21.15.19:0/1214813694 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdaf401c8c0 con 0x55e85a0b4510 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e85a0b4510 0x55e85a0b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdb0259c700 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e85a0b4510 0x55e85a0b6970 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c56c1f0f666f34 server_cookie=b6b99e2ccd9b88a1 in_seq=0 out_seq=0 2024-05-09T01:31:39.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.095+0000 7fdae9ffb700 10 client.16908 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.096+0000 7fdae9ffb700 1 -- 172.21.15.19:0/1214813694 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fdaf401c8c0 con 0x55e85a0b4510 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.096+0000 7fdae9ffb700 10 client.16908 handle_command_reply: tid=0 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.096+0000 7fdb1788db80 1 client.16908 shutdown 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.096+0000 7fdb1788db80 1 -- 172.21.15.19:0/1214813694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a098610 msgr2=0x55e85a0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.096+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a098610 0x55e85a0b04b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00baa0 tx=0x7fdafc00bb80 comp rx=0 tx=0).stop 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.096+0000 7fdb1788db80 1 -- 172.21.15.19:0/1214813694 shutdown_connections 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.096+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e85a0b4510 0x55e85a0b6970 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.097+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e85a098610 0x55e85a0b04b0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.097+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.097+0000 7fdb1788db80 1 --2- 172.21.15.19:0/1214813694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.097+0000 7fdb1788db80 1 -- 172.21.15.19:0/1214813694 >> 172.21.15.19:0/1214813694 conn(0x55e859f1cf70 msgr2=0x55e85a0a3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.097+0000 7fdae9ffb700 0 client.16908 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.097+0000 7fdb1788db80 1 -- 172.21.15.19:0/1214813694 shutdown_connections 2024-05-09T01:31:39.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.097+0000 7fdb1788db80 1 -- 172.21.15.19:0/1214813694 wait complete. 2024-05-09T01:31:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.097+0000 7fdb1788db80 20 client.16908 trim_cache size 0 max 16384 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.117+0000 7fdb1788db80 1 Processor -- start 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.117+0000 7fdb1788db80 1 -- start start 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a0bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e85a098610 0x55e85a0bc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb1788db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041acb80 con 0x55e859f93d00 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb1788db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041ae6c0 con 0x55e85a098610 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb1788db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041ad250 con 0x55e859f194a0 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb01d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a0bc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb01d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a0bc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48504/0 (socket says 172.21.15.19:48504) 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb01d9b700 1 -- 172.21.15.19:0/2555688207 learned_addr learned my addr 172.21.15.19:0/2555688207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb02d9d700 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e85a098610 0x55e85a0bc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb0259c700 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb01d9b700 1 -- 172.21.15.19:0/2555688207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 msgr2=0x55e85a0a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb01d9b700 1 -- 172.21.15.19:0/2555688207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e85a098610 msgr2=0x55e85a0bc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e85a098610 0x55e85a0bc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb01d9b700 1 -- 172.21.15.19:0/2555688207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85a0933e0 con 0x55e859f93d00 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb0259c700 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.118+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a0bc170 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80a9520 tx=0x7fdaf80df940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76cdd687ac7e72b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.119+0000 7fdaebfff700 1 -- 172.21.15.19:0/2555688207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80a9c90 con 0x55e859f93d00 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.119+0000 7fdaebfff700 1 -- 172.21.15.19:0/2555688207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf80a9590 con 0x55e859f93d00 2024-05-09T01:31:39.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.119+0000 7fdaebfff700 1 -- 172.21.15.19:0/2555688207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80a9a50 con 0x55e859f93d00 2024-05-09T01:31:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 -- 172.21.15.19:0/2555688207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 msgr2=0x55e85a0bc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a0bc170 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80a9520 tx=0x7fdaf80df940 comp rx=0 tx=0).stop 2024-05-09T01:31:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 -- 172.21.15.19:0/2555688207 shutdown_connections 2024-05-09T01:31:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e85a098610 0x55e85a0bc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e859f194a0 0x55e85a0a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 --2- 172.21.15.19:0/2555688207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f93d00 0x55e85a0bc170 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 -- 172.21.15.19:0/2555688207 >> 172.21.15.19:0/2555688207 conn(0x55e859f1cf70 msgr2=0x55e85a0a53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 -- 172.21.15.19:0/2555688207 shutdown_connections 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 -- 172.21.15.19:0/2555688207 wait complete. 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 -- start start 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.120+0000 7fdb1788db80 1 Processor -- start 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f194a0 0x55e85a0ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb1788db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 0x55e85a0a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb1788db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041adff0 con 0x55e859f194a0 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb1788db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041ad920 con 0x55e859f93d00 2024-05-09T01:31:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb1788db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041aab20 con 0x55e85a098610 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb1788db80 10 client.0 _ll_register_callbacks cb 0x55e859e2a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb0259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f194a0 0x55e85a0ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb0259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f194a0 0x55e85a0ab810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48514/0 (socket says 172.21.15.19:48514) 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb0259c700 1 -- 172.21.15.19:0/3749940125 learned_addr learned my addr 172.21.15.19:0/3749940125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb02d9d700 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 0x55e85a0a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb01d9b700 1 -- 172.21.15.19:0/3749940125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 msgr2=0x55e85a0a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 0x55e85a0a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb01d9b700 1 -- 172.21.15.19:0/3749940125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f194a0 msgr2=0x55e85a0ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f194a0 0x55e85a0ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb01d9b700 1 -- 172.21.15.19:0/3749940125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85a0a96b0 con 0x55e859f93d00 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.121+0000 7fdb0259c700 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f194a0 0x55e85a0ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb01d9b700 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a8be0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800b540 tx=0x7fdaf80b2540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b589647bc84b3b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdae9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdae9ffb700 1 -- 172.21.15.19:0/3749940125 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80a97f0 con 0x55e859f93d00 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdae9ffb700 1 -- 172.21.15.19:0/3749940125 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdaf80a9990 con 0x55e859f93d00 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb1788db80 1 -- 172.21.15.19:0/3749940125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf80a9590 con 0x55e859f93d00 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdae9ffb700 1 -- 172.21.15.19:0/3749940125 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdaf80b1f60 con 0x55e859f93d00 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb1788db80 10 client.26399 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb1788db80 1 -- 172.21.15.19:0/3749940125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e85a11e9d0 con 0x55e859f93d00 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdae9ffb700 1 -- 172.21.15.19:0/3749940125 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf80a9590 con 0x55e859f93d00 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdae9ffb700 1 -- 172.21.15.19:0/3749940125 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdaf80a4f00 con 0x55e859f93d00 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb1788db80 10 client.26399 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb1788db80 10 client.26399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb1788db80 20 client.26399 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb1788db80 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e85a11f130 0x55e85a121550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb1788db80 4 client.26399 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:31:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.122+0000 7fdb1788db80 1 -- 172.21.15.19:0/3749940125 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e859fc6a80 con 0x55e85a11f130 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.123+0000 7fdb0259c700 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e85a11f130 0x55e85a121550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.123+0000 7fdb0259c700 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e85a11f130 0x55e85a121550 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659a4d4354254ed4 server_cookie=27ea49eb6d8d8cd1 in_seq=0 out_seq=0 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.123+0000 7fdae9ffb700 10 client.26399 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.123+0000 7fdae9ffb700 1 -- 172.21.15.19:0/3749940125 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e859fc6a80 con 0x55e85a11f130 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.123+0000 7fdae9ffb700 10 client.26399 handle_command_reply: tid=0 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.123+0000 7fdb1788db80 1 client.26399 shutdown 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.123+0000 7fdb1788db80 1 -- 172.21.15.19:0/3749940125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 msgr2=0x55e85a0a8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.123+0000 7fdb1788db80 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a8be0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800b540 tx=0x7fdaf80b2540 comp rx=0 tx=0).stop 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb1788db80 1 -- 172.21.15.19:0/3749940125 shutdown_connections 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb1788db80 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e85a11f130 0x55e85a121550 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb1788db80 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e85a098610 0x55e85a0a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb1788db80 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e859f194a0 0x55e85a0ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdae9ffb700 0 client.26399 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb1788db80 1 --2- 172.21.15.19:0/3749940125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e859f93d00 0x55e85a0a8be0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb1788db80 1 -- 172.21.15.19:0/3749940125 >> 172.21.15.19:0/3749940125 conn(0x55e859f1cf70 msgr2=0x55e85a090b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb1788db80 1 -- 172.21.15.19:0/3749940125 shutdown_connections 2024-05-09T01:31:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb1788db80 1 -- 172.21.15.19:0/3749940125 wait complete. 2024-05-09T01:31:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb1788db80 20 client.26399 trim_cache size 0 max 16384 2024-05-09T01:31:39.126 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:31:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:31:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:31:39.126 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:31:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 -- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdaec0b2880 msgr2=0x7fdaec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdaec0b2880 0x7fdaec0b4d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009c60 tx=0x7fdaf4009260 comp rx=0 tx=0).stop 2024-05-09T01:31:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 -- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04107df0 msgr2=0x7fdb0419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04107df0 0x7fdb0419a2e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800d950 tx=0x7fdaf800dc60 comp rx=0 tx=0).stop 2024-05-09T01:31:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 -- 172.21.15.19:0/1437167223 shutdown_connections 2024-05-09T01:31:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb04108740 0x7fdb0419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb041034c0 0x7fdb04199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdaec0b2880 0x7fdaec0b4d90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 --2- 172.21.15.19:0/1437167223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb04107df0 0x7fdb0419a2e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.124+0000 7fdb08e77700 1 -- 172.21.15.19:0/1437167223 >> 172.21.15.19:0/1437167223 conn(0x7fdb040fedc0 msgr2=0x7fdb04107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.125+0000 7fdb08e77700 1 -- 172.21.15.19:0/1437167223 shutdown_connections 2024-05-09T01:31:39.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:39.125+0000 7fdb08e77700 1 -- 172.21.15.19:0/1437167223 wait complete. 2024-05-09T01:31:39.883 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:31:39.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:31:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:40.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:40 smithi019 ceph-mon[30158]: pgmap v562: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:31:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:40 smithi059 ceph-mon[34351]: pgmap v562: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:31:40.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:40 smithi196 ceph-mon[35041]: pgmap v562: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:31:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.835+0000 7f6ad62a9700 1 -- 172.21.15.19:0/1314216947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 msgr2=0x7f6ad01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.835+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/1314216947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 0x7f6ad01100a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b0d0 tx=0x7f6ac800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.835+0000 7f6ad62a9700 1 -- 172.21.15.19:0/1314216947 shutdown_connections 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.835+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/1314216947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 0x7f6ad01100a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.835+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/1314216947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad01034c0 0x7f6ad01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.835+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/1314216947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad0107df0 0x7f6ad0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.835+0000 7f6ad62a9700 1 -- 172.21.15.19:0/1314216947 >> 172.21.15.19:0/1314216947 conn(0x7f6ad00fedf0 msgr2=0x7f6ad0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.835+0000 7f6ad62a9700 1 -- 172.21.15.19:0/1314216947 shutdown_connections 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.835+0000 7f6ad62a9700 1 -- 172.21.15.19:0/1314216947 wait complete. 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.836+0000 7f6ad62a9700 1 Processor -- start 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.836+0000 7f6ad62a9700 1 -- start start 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad62a9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad01034c0 0x7f6ad019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:41.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad62a9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad0107df0 0x7f6ad019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:41.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad62a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 0x7f6ad01a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:41.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad62a9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0113370 con 0x7f6ad0108740 2024-05-09T01:31:41.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad62a9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01131f0 con 0x7f6ad0107df0 2024-05-09T01:31:41.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad62a9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01134f0 con 0x7f6ad01034c0 2024-05-09T01:31:41.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad4846700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 0x7f6ad01a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad4846700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 0x7f6ad01a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39636/0 (socket says 172.21.15.19:39636) 2024-05-09T01:31:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad4846700 1 -- 172.21.15.19:0/36167481 learned_addr learned my addr 172.21.15.19:0/36167481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6acffff700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad01034c0 0x7f6ad019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6acf7fe700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad0107df0 0x7f6ad019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad4846700 1 -- 172.21.15.19:0/36167481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad01034c0 msgr2=0x7f6ad019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad4846700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad01034c0 0x7f6ad019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad4846700 1 -- 172.21.15.19:0/36167481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad0107df0 msgr2=0x7f6ad019d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad4846700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad0107df0 0x7f6ad019d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad4846700 1 -- 172.21.15.19:0/36167481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad01a1dc0 con 0x7f6ad0108740 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.837+0000 7f6ad4846700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 0x7f6ad01a17d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80095c0 tx=0x7f6ac8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.838+0000 7f6acd7fa700 1 -- 172.21.15.19:0/36167481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac801f070 con 0x7f6ad0108740 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.838+0000 7f6acd7fa700 1 -- 172.21.15.19:0/36167481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac8007e20 con 0x7f6ad0108740 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.838+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad01a2050 con 0x7f6ad0108740 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.838+0000 7f6acd7fa700 1 -- 172.21.15.19:0/36167481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac801b4a0 con 0x7f6ad0108740 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.838+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad01a2580 con 0x7f6ad0108740 2024-05-09T01:31:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.839+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad00512c0 con 0x7f6ad0108740 2024-05-09T01:31:41.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.841+0000 7f6acd7fa700 1 -- 172.21.15.19:0/36167481 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6ac8022070 con 0x7f6ad0108740 2024-05-09T01:31:41.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.842+0000 7f6acd7fa700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6ab80b2900 0x7f6ab80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:41.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.842+0000 7f6acffff700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6ab80b2900 0x7f6ab80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:41.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.842+0000 7f6acd7fa700 1 -- 172.21.15.19:0/36167481 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6ac800f020 con 0x7f6ad0108740 2024-05-09T01:31:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.843+0000 7f6acffff700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6ab80b2900 0x7f6ab80b4e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000ac50 tx=0x7f6ac000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:41.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:41.846+0000 7f6acd7fa700 1 -- 172.21.15.19:0/36167481 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ac8029020 con 0x7f6ad0108740 2024-05-09T01:31:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.110+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ad019e300 con 0x7f6ad0108740 2024-05-09T01:31:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.111+0000 7f6acd7fa700 1 -- 172.21.15.19:0/36167481 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f6ad019e300 con 0x7f6ad0108740 2024-05-09T01:31:42.108 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:31:42.108 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:31:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6ab80b2900 msgr2=0x7f6ab80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6ab80b2900 0x7f6ab80b4e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000ac50 tx=0x7f6ac000a380 comp rx=0 tx=0).stop 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 msgr2=0x7f6ad01a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 0x7f6ad01a17d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80095c0 tx=0x7f6ac8007a20 comp rx=0 tx=0).stop 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 shutdown_connections 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f6ab80b2900 0x7f6ab80b4e10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad0108740 0x7f6ad01a17d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad01034c0 0x7f6ad019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 --2- 172.21.15.19:0/36167481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad0107df0 0x7f6ad019d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.113+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 >> 172.21.15.19:0/36167481 conn(0x7f6ad00fedf0 msgr2=0x7f6ad0109740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.114+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 shutdown_connections 2024-05-09T01:31:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:42.114+0000 7f6ad62a9700 1 -- 172.21.15.19:0/36167481 wait complete. 2024-05-09T01:31:42.112 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:31:42.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:42 smithi019 ceph-mon[30158]: pgmap v563: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:42.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:42 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/36167481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:31:42.795 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:31:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:42 smithi059 ceph-mon[34351]: pgmap v563: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:42 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/36167481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:31:42.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:42 smithi196 ceph-mon[35041]: pgmap v563: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:42.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:42 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/36167481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:31:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:44.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:44 smithi019 ceph-mon[30158]: pgmap v564: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:31:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:44 smithi059 ceph-mon[34351]: pgmap v564: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:31:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.896+0000 7f32d928e700 1 -- 172.21.15.19:0/2040253843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4108740 msgr2=0x7f32d41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.896+0000 7f32d928e700 1 --2- 172.21.15.19:0/2040253843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4108740 0x7f32d41100a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00b0d0 tx=0x7f32cc00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.896+0000 7f32d928e700 1 -- 172.21.15.19:0/2040253843 shutdown_connections 2024-05-09T01:31:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.896+0000 7f32d928e700 1 --2- 172.21.15.19:0/2040253843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4108740 0x7f32d41100a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.896+0000 7f32d928e700 1 --2- 172.21.15.19:0/2040253843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32d41034c0 0x7f32d41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.896+0000 7f32d928e700 1 --2- 172.21.15.19:0/2040253843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32d4107df0 0x7f32d4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.896+0000 7f32d928e700 1 -- 172.21.15.19:0/2040253843 >> 172.21.15.19:0/2040253843 conn(0x7f32d40fedf0 msgr2=0x7f32d4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.896+0000 7f32d928e700 1 -- 172.21.15.19:0/2040253843 shutdown_connections 2024-05-09T01:31:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.896+0000 7f32d928e700 1 -- 172.21.15.19:0/2040253843 wait complete. 2024-05-09T01:31:44.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d928e700 1 Processor -- start 2024-05-09T01:31:44.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d928e700 1 -- start start 2024-05-09T01:31:44.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d928e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32d41034c0 0x7f32d419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d928e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4107df0 0x7f32d419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d928e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32d4108740 0x7f32d419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d928e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41144c0 con 0x7f32d4107df0 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d928e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4114340 con 0x7f32d4108740 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d928e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4114640 con 0x7f32d41034c0 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4107df0 0x7f32d419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.897+0000 7f32d27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4107df0 0x7f32d419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39648/0 (socket says 172.21.15.19:39648) 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d27fc700 1 -- 172.21.15.19:0/1763503334 learned_addr learned my addr 172.21.15.19:0/1763503334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32d41034c0 0x7f32d419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d37fe700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32d4108740 0x7f32d419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d27fc700 1 -- 172.21.15.19:0/1763503334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32d41034c0 msgr2=0x7f32d419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d27fc700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32d41034c0 0x7f32d419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d27fc700 1 -- 172.21.15.19:0/1763503334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32d4108740 msgr2=0x7f32d419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d27fc700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32d4108740 0x7f32d419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d27fc700 1 -- 172.21.15.19:0/1763503334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d419fd60 con 0x7f32d4107df0 2024-05-09T01:31:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d27fc700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4107df0 0x7f32d419b430 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f32c800d980 tx=0x7f32c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32bbfff700 1 -- 172.21.15.19:0/1763503334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8017070 con 0x7f32d4107df0 2024-05-09T01:31:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d928e700 1 -- 172.21.15.19:0/1763503334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d41a0050 con 0x7f32d4107df0 2024-05-09T01:31:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32bbfff700 1 -- 172.21.15.19:0/1763503334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c800fd40 con 0x7f32d4107df0 2024-05-09T01:31:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32bbfff700 1 -- 172.21.15.19:0/1763503334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c800e3b0 con 0x7f32d4107df0 2024-05-09T01:31:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.898+0000 7f32d928e700 1 -- 172.21.15.19:0/1763503334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d41a7970 con 0x7f32d4107df0 2024-05-09T01:31:44.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.901+0000 7f32bbfff700 1 -- 172.21.15.19:0/1763503334 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f32c800e550 con 0x7f32d4107df0 2024-05-09T01:31:44.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.902+0000 7f32bbfff700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32bc0b2880 0x7f32bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.902+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32bc0b2880 0x7f32bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.902+0000 7f32bbfff700 1 -- 172.21.15.19:0/1763503334 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f32c80e1cb0 con 0x7f32d4107df0 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.902+0000 7f32e7ca4b80 1 Processor -- start 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.902+0000 7f32e7ca4b80 1 -- start start 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.902+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32bc0b2880 0x7f32bc0b4d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f32c4007be0 tx=0x7f32c40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.902+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 0x55c0e0188940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.902+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 0x55c0e0188e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e0182ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c0e00a0ff0 con 0x55c0e0182ab0 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c0e003f7f0 con 0x55c0e00eb160 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0e009cfb0 con 0x55c0e00014a0 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e0182ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e0182ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39650/0 (socket says 172.21.15.19:39650) 2024-05-09T01:31:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d37fe700 1 -- 172.21.15.19:0/2237324389 learned_addr learned my addr 172.21.15.19:0/2237324389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 0x55c0e0188940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d27fc700 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 0x55c0e0188e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d37fe700 1 -- 172.21.15.19:0/2237324389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 msgr2=0x55c0e0188e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d37fe700 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 0x55c0e0188e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d37fe700 1 -- 172.21.15.19:0/2237324389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 msgr2=0x55c0e0188940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d37fe700 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 0x55c0e0188940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d37fe700 1 -- 172.21.15.19:0/2237324389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e0037370 con 0x55c0e0182ab0 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d27fc700 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 0x55c0e0188e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32d37fe700 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e0182ec0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009cb0 tx=0x7f32cc009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4095cb31d24859a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.903+0000 7f32b97fa700 1 -- 172.21.15.19:0/2237324389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc009450 con 0x55c0e0182ab0 2024-05-09T01:31:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32b97fa700 1 -- 172.21.15.19:0/2237324389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32cc006000 con 0x55c0e0182ab0 2024-05-09T01:31:44.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32b97fa700 1 -- 172.21.15.19:0/2237324389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0075c0 con 0x55c0e0182ab0 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2237324389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 msgr2=0x55c0e0182ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e0182ec0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009cb0 tx=0x7f32cc009ce0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2237324389 shutdown_connections 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e0182ec0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 0x55c0e0188940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/2237324389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 0x55c0e0188e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2237324389 >> 172.21.15.19:0/2237324389 conn(0x55c0e00616a0 msgr2=0x55c0e0179400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.904+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2237324389 shutdown_connections 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2237324389 wait complete. 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 1 -- start start 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 1 Processor -- start 2024-05-09T01:31:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00014a0 0x55c0e018c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 0x55c0e018cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e0182ab0 0x55c0e018d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c0dff1cba0 con 0x55c0e00014a0 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c0e005b150 con 0x55c0e00eb160 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0183820 con 0x55c0e0182ab0 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.905+0000 7f32e7ca4b80 10 client.0 _ll_register_callbacks cb 0x55c0dff126c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00014a0 0x55c0e018c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00014a0 0x55c0e018c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39664/0 (socket says 172.21.15.19:39664) 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d2ffd700 1 -- 172.21.15.19:0/1107916018 learned_addr learned my addr 172.21.15.19:0/1107916018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d27fc700 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 0x55c0e018cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d37fe700 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e0182ab0 0x55c0e018d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d2ffd700 1 -- 172.21.15.19:0/1107916018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e0182ab0 msgr2=0x55c0e018d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e0182ab0 0x55c0e018d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d2ffd700 1 -- 172.21.15.19:0/1107916018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 msgr2=0x55c0e018cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 0x55c0e018cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d2ffd700 1 -- 172.21.15.19:0/1107916018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0dff1c490 con 0x55c0e00014a0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00014a0 0x55c0e018c670 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f32c40042b0 tx=0x7f32c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c782d2719fe866 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107916018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4024070 con 0x55c0e00014a0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107916018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f32c4003b10 con 0x55c0e00014a0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.906+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107916018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32cc006000 con 0x55c0e00014a0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107916018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32c4030050 con 0x55c0e00014a0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32e7ca4b80 10 client.16932 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107916018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c0e0199ee0 con 0x55c0e00014a0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107916018 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32cc006000 con 0x55c0e00014a0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107916018 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f32c4020410 con 0x55c0e00014a0 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32e7ca4b80 10 client.16932 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32e7ca4b80 10 client.16932 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:31:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32e7ca4b80 20 client.16932 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55c0e019c4c0 0x55c0e019e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32e7ca4b80 4 client.16932 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107916018 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0e00ae9d0 con 0x55c0e019c4c0 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.907+0000 7f32d27fc700 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55c0e019c4c0 0x55c0e019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.908+0000 7f32d27fc700 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55c0e019c4c0 0x55c0e019e960 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a1e9238d1aae2ed server_cookie=85474261fd93aa95 in_seq=0 out_seq=0 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.908+0000 7f32b6ff5700 10 client.16932 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107916018 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0e00ae9d0 con 0x55c0e019c4c0 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32b6ff5700 10 client.16932 handle_command_reply: tid=0 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 client.16932 shutdown 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107916018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00014a0 msgr2=0x55c0e018c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00014a0 0x55c0e018c670 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f32c40042b0 tx=0x7f32c40047c0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107916018 shutdown_connections 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55c0e019c4c0 0x55c0e019e960 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e0182ab0 0x55c0e018d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00014a0 0x55c0e018c670 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107916018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00eb160 0x55c0e018cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107916018 >> 172.21.15.19:0/1107916018 conn(0x55c0e00616a0 msgr2=0x55c0e01791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:44.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32b6ff5700 0 client.16932 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:31:44.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107916018 shutdown_connections 2024-05-09T01:31:44.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107916018 wait complete. 2024-05-09T01:31:44.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.909+0000 7f32e7ca4b80 20 client.16932 trim_cache size 0 max 16384 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.924+0000 7f32e7ca4b80 1 Processor -- start 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.924+0000 7f32e7ca4b80 1 -- start start 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.924+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 0x55c0e0187df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.924+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00eb160 0x55c0e0186c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.924+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e0182ab0 0x55c0e018cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.924+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41ac9b0 con 0x55c0e00eb160 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.924+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41ae4f0 con 0x55c0e0182ab0 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.924+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41ad080 con 0x55c0e00014a0 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00eb160 0x55c0e0186c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00eb160 0x55c0e0186c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39672/0 (socket says 172.21.15.19:39672) 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d27fc700 1 -- 172.21.15.19:0/2651535844 learned_addr learned my addr 172.21.15.19:0/2651535844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/2651535844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 0x55c0e0187df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d27fc700 1 -- 172.21.15.19:0/2651535844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 msgr2=0x55c0e0187df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d27fc700 1 --2- 172.21.15.19:0/2651535844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 0x55c0e0187df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d27fc700 1 -- 172.21.15.19:0/2651535844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e0182ab0 msgr2=0x55c0e018cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d27fc700 1 --2- 172.21.15.19:0/2651535844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e0182ab0 0x55c0e018cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d27fc700 1 -- 172.21.15.19:0/2651535844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e0172f60 con 0x55c0e00eb160 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32d27fc700 1 --2- 172.21.15.19:0/2651535844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00eb160 0x55c0e0186c40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f32c80a9180 tx=0x7f32c8091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a906dae05dfee0f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32b8ff9700 1 -- 172.21.15.19:0/2651535844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8020040 con 0x55c0e00eb160 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32b8ff9700 1 -- 172.21.15.19:0/2651535844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c80b1c20 con 0x55c0e00eb160 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.925+0000 7f32b8ff9700 1 -- 172.21.15.19:0/2651535844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c80afe60 con 0x55c0e00eb160 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2651535844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00eb160 msgr2=0x55c0e0186c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/2651535844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00eb160 0x55c0e0186c40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f32c80a9180 tx=0x7f32c8091040 comp rx=0 tx=0).stop 2024-05-09T01:31:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2651535844 shutdown_connections 2024-05-09T01:31:44.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/2651535844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e0182ab0 0x55c0e018cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/2651535844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00014a0 0x55c0e0187df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/2651535844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e00eb160 0x55c0e0186c40 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2651535844 >> 172.21.15.19:0/2651535844 conn(0x55c0e00616a0 msgr2=0x55c0e01876e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:44.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2651535844 shutdown_connections 2024-05-09T01:31:44.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/2651535844 wait complete. 2024-05-09T01:31:44.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 -- start start 2024-05-09T01:31:44.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.926+0000 7f32e7ca4b80 1 Processor -- start 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00014a0 0x55c0e01a3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00eb160 0x55c0e01a3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32e7ca4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e01a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41ade20 con 0x55c0e0182ab0 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41ad750 con 0x55c0e00014a0 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32e7ca4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41aa950 con 0x55c0e00eb160 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32e7ca4b80 10 client.0 _ll_register_callbacks cb 0x55c0dff126c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e01a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e01a3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39674/0 (socket says 172.21.15.19:39674) 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d37fe700 1 -- 172.21.15.19:0/1107491527 learned_addr learned my addr 172.21.15.19:0/1107491527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d27fc700 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00eb160 0x55c0e01a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00014a0 0x55c0e01a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d37fe700 1 -- 172.21.15.19:0/1107491527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00eb160 msgr2=0x55c0e01a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:44.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d37fe700 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00eb160 0x55c0e01a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d37fe700 1 -- 172.21.15.19:0/1107491527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00014a0 msgr2=0x55c0e01a3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d37fe700 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00014a0 0x55c0e01a3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d37fe700 1 -- 172.21.15.19:0/1107491527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e0178f90 con 0x55c0e0182ab0 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d27fc700 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00eb160 0x55c0e01a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00014a0 0x55c0e01a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32d37fe700 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e01a3b90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f32cc000f80 tx=0x7f32cc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7769d5152ca35b73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.927+0000 7f32b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107491527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0052c0 con 0x55c0e0182ab0 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107491527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f32cc005460 con 0x55c0e0182ab0 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107491527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32c80b1c20 con 0x55c0e0182ab0 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107491527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32cc005660 con 0x55c0e0182ab0 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32e7ca4b80 10 client.16938 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107491527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c0e01944a0 con 0x55c0e0182ab0 2024-05-09T01:31:44.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107491527 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c80b1c20 con 0x55c0e0182ab0 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107491527 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f32cc011070 con 0x55c0e0182ab0 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32e7ca4b80 10 client.16938 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32e7ca4b80 10 client.16938 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32e7ca4b80 20 client.16938 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55c0e0194730 0x55c0e0194b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32e7ca4b80 4 client.16938 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.928+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107491527 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0e00a8a70 con 0x55c0e0194730 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.929+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55c0e0194730 0x55c0e0194b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.929+0000 7f32b6ff5700 10 client.16938 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.929+0000 7f32d2ffd700 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55c0e0194730 0x55c0e0194b40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=796512424f91137d server_cookie=546abb0b0493995 in_seq=0 out_seq=0 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.929+0000 7f32b6ff5700 1 -- 172.21.15.19:0/1107491527 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0e00a8a70 con 0x55c0e0194730 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.929+0000 7f32b6ff5700 10 client.16938 handle_command_reply: tid=0 2024-05-09T01:31:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.929+0000 7f32e7ca4b80 1 client.16938 shutdown 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.929+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107491527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 msgr2=0x55c0e01a3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.929+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e01a3b90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f32cc000f80 tx=0x7f32cc003ce0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107491527 shutdown_connections 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55c0e0194730 0x55c0e0194b40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0e0182ab0 0x55c0e01a3b90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0e00014a0 0x55c0e01a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32b6ff5700 0 client.16938 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32e7ca4b80 1 --2- 172.21.15.19:0/1107491527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0e00eb160 0x55c0e01a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107491527 >> 172.21.15.19:0/1107491527 conn(0x55c0e00616a0 msgr2=0x55c0e018af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107491527 shutdown_connections 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32e7ca4b80 1 -- 172.21.15.19:0/1107491527 wait complete. 2024-05-09T01:31:44.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32e7ca4b80 20 client.16938 trim_cache size 0 max 16384 2024-05-09T01:31:44.933 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:31:44.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 -- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32bc0b2880 msgr2=0x7f32bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:44.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32bc0b2880 0x7f32bc0b4d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f32c4007be0 tx=0x7f32c40073d0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 -- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4107df0 msgr2=0x7f32d419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4107df0 0x7f32d419b430 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f32c800d980 tx=0x7f32c800dc90 comp rx=0 tx=0).stop 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 -- 172.21.15.19:0/1763503334 shutdown_connections 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32d4108740 0x7f32d419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32d41034c0 0x7f32d419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f32bc0b2880 0x7f32bc0b4d90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 --2- 172.21.15.19:0/1763503334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d4107df0 0x7f32d419b430 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.930+0000 7f32d928e700 1 -- 172.21.15.19:0/1763503334 >> 172.21.15.19:0/1763503334 conn(0x7f32d40fedf0 msgr2=0x7f32d4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.931+0000 7f32d928e700 1 -- 172.21.15.19:0/1763503334 shutdown_connections 2024-05-09T01:31:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:44.931+0000 7f32d928e700 1 -- 172.21.15.19:0/1763503334 wait complete. 2024-05-09T01:31:44.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:44 smithi196 ceph-mon[35041]: pgmap v564: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:31:45.573 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:31:45.825 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:46.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:46 smithi059 ceph-mon[34351]: pgmap v565: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:31:46.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:46 smithi019 ceph-mon[30158]: pgmap v565: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:31:46.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:46 smithi196 ceph-mon[35041]: pgmap v565: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:31:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.611+0000 7f3cdebc8700 1 -- 172.21.15.19:0/4265565334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8108740 msgr2=0x7f3cd81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.611+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/4265565334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8108740 0x7f3cd81100a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00099c0 tx=0x7f3cd0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.611+0000 7f3cdebc8700 1 -- 172.21.15.19:0/4265565334 shutdown_connections 2024-05-09T01:31:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.611+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/4265565334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8108740 0x7f3cd81100a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.611+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/4265565334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd81034c0 0x7f3cd81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.611+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/4265565334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8107df0 0x7f3cd8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.611+0000 7f3cdebc8700 1 -- 172.21.15.19:0/4265565334 >> 172.21.15.19:0/4265565334 conn(0x7f3cd80fedc0 msgr2=0x7f3cd81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.611+0000 7f3cdebc8700 1 -- 172.21.15.19:0/4265565334 shutdown_connections 2024-05-09T01:31:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.611+0000 7f3cdebc8700 1 -- 172.21.15.19:0/4265565334 wait complete. 2024-05-09T01:31:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.612+0000 7f3cdebc8700 1 Processor -- start 2024-05-09T01:31:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.612+0000 7f3cdebc8700 1 -- start start 2024-05-09T01:31:47.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.612+0000 7f3cdebc8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd81034c0 0x7f3cd80ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.612+0000 7f3cdebc8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8107df0 0x7f3cd80ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.612+0000 7f3cdebc8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8108740 0x7f3cd80ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.612+0000 7f3cdebc8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8113370 con 0x7f3cd8107df0 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.612+0000 7f3cdebc8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81131f0 con 0x7f3cd8108740 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.612+0000 7f3cdebc8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81134f0 con 0x7f3cd81034c0 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8107df0 0x7f3cd80ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8107df0 0x7f3cd80ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39684/0 (socket says 172.21.15.19:39684) 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd7fff700 1 -- 172.21.15.19:0/573219734 learned_addr learned my addr 172.21.15.19:0/573219734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cdd165700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8108740 0x7f3cd80ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cdc964700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd81034c0 0x7f3cd80ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd7fff700 1 -- 172.21.15.19:0/573219734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd81034c0 msgr2=0x7f3cd80ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd81034c0 0x7f3cd80ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd7fff700 1 -- 172.21.15.19:0/573219734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8108740 msgr2=0x7f3cd80ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8108740 0x7f3cd80ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd7fff700 1 -- 172.21.15.19:0/573219734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd81a9170 con 0x7f3cd8107df0 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cdc964700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd81034c0 0x7f3cd80ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8107df0 0x7f3cd80ff7e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0007f10 tx=0x7f3cc000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd5ffb700 1 -- 172.21.15.19:0/573219734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc000ddd0 con 0x7f3cd8107df0 2024-05-09T01:31:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cdebc8700 1 -- 172.21.15.19:0/573219734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd81a9460 con 0x7f3cd8107df0 2024-05-09T01:31:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.613+0000 7f3cd5ffb700 1 -- 172.21.15.19:0/573219734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc0018070 con 0x7f3cd8107df0 2024-05-09T01:31:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.614+0000 7f3cd5ffb700 1 -- 172.21.15.19:0/573219734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0012640 con 0x7f3cd8107df0 2024-05-09T01:31:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.614+0000 7f3cdebc8700 1 -- 172.21.15.19:0/573219734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd81a99f0 con 0x7f3cd8107df0 2024-05-09T01:31:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.617+0000 7f3cd5ffb700 1 -- 172.21.15.19:0/573219734 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3cc00139c0 con 0x7f3cd8107df0 2024-05-09T01:31:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.617+0000 7f3cd5ffb700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cc40b2900 0x7f3cc40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3ced5deb80 1 Processor -- start 2024-05-09T01:31:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3cdc964700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cc40b2900 0x7f3cc40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3cd5ffb700 1 -- 172.21.15.19:0/573219734 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3cc00e1f20 con 0x7f3cd8107df0 2024-05-09T01:31:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3ced5deb80 1 -- start start 2024-05-09T01:31:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 0x555cb89be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555cb88e10a0 con 0x555cb88414a0 2024-05-09T01:31:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555cb88869a0 con 0x555cb89be400 2024-05-09T01:31:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555cb88dd060 con 0x555cb88bbd00 2024-05-09T01:31:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3cdc964700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cc40b2900 0x7f3cc40b4e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005950 tx=0x7f3cc80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3cd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3cd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39696/0 (socket says 172.21.15.19:39696) 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3cd7fff700 1 -- 172.21.15.19:0/1735662776 learned_addr learned my addr 172.21.15.19:0/1735662776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3cdd165700 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 0x555cb89be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.618+0000 7f3cdc964700 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cd7fff700 1 -- 172.21.15.19:0/1735662776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 msgr2=0x555cb89c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cd7fff700 1 -- 172.21.15.19:0/1735662776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 msgr2=0x555cb89be810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 0x555cb89be810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cd7fff700 1 -- 172.21.15.19:0/1735662776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb8877500 con 0x555cb88414a0 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cdc964700 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cdd165700 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 0x555cb89be810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:31:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c4780 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00a55a0 tx=0x7f3cc00a5510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db411a51b15deff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cceffd700 1 -- 172.21.15.19:0/1735662776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00a9520 con 0x555cb88414a0 2024-05-09T01:31:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cceffd700 1 -- 172.21.15.19:0/1735662776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc00ad8d0 con 0x555cb88414a0 2024-05-09T01:31:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.619+0000 7f3cceffd700 1 -- 172.21.15.19:0/1735662776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00a4e90 con 0x555cb88414a0 2024-05-09T01:31:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 -- 172.21.15.19:0/1735662776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 msgr2=0x555cb89c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c4780 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00a55a0 tx=0x7f3cc00a5510 comp rx=0 tx=0).stop 2024-05-09T01:31:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 -- 172.21.15.19:0/1735662776 shutdown_connections 2024-05-09T01:31:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 0x555cb89be810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/1735662776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c4780 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 -- 172.21.15.19:0/1735662776 >> 172.21.15.19:0/1735662776 conn(0x555cb8844f70 msgr2=0x555cb89b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 -- 172.21.15.19:0/1735662776 shutdown_connections 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 -- 172.21.15.19:0/1735662776 wait complete. 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.620+0000 7f3ced5deb80 1 -- start start 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3ced5deb80 1 Processor -- start 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 0x555cb89c8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88bbd00 0x555cb89c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb89be400 0x555cb89c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555cb89bf100 con 0x555cb88bbd00 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555cb887bb20 con 0x555cb88414a0 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555cb89d41d0 con 0x555cb89be400 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3ced5deb80 10 client.0 _ll_register_callbacks cb 0x555cb87526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3cd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88bbd00 0x555cb89c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3cd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88bbd00 0x555cb89c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39710/0 (socket says 172.21.15.19:39710) 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3cd7fff700 1 -- 172.21.15.19:0/3371968401 learned_addr learned my addr 172.21.15.19:0/3371968401 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3cdd165700 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb89be400 0x555cb89c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.621+0000 7f3cdc964700 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 0x555cb89c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cd7fff700 1 -- 172.21.15.19:0/3371968401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb89be400 msgr2=0x555cb89c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb89be400 0x555cb89c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cd7fff700 1 -- 172.21.15.19:0/3371968401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 msgr2=0x555cb89c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 0x555cb89c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cd7fff700 1 -- 172.21.15.19:0/3371968401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb875c490 con 0x555cb88bbd00 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cdc964700 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 0x555cb89c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88bbd00 0x555cb89c8560 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00095b0 tx=0x7f3cc00a9d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa38d9344ba78a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cb7fff700 1 -- 172.21.15.19:0/3371968401 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00c2070 con 0x555cb88bbd00 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cb7fff700 1 -- 172.21.15.19:0/3371968401 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3cc00a9990 con 0x555cb88bbd00 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3371968401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc00ad8d0 con 0x555cb88bbd00 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cb7fff700 1 -- 172.21.15.19:0/3371968401 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3cc00a8780 con 0x555cb88bbd00 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3ced5deb80 10 client.16950 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3371968401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cb89d5b00 con 0x555cb88bbd00 2024-05-09T01:31:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.622+0000 7f3cb7fff700 1 -- 172.21.15.19:0/3371968401 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc00ad8d0 con 0x555cb88bbd00 2024-05-09T01:31:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.623+0000 7f3cb7fff700 1 -- 172.21.15.19:0/3371968401 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3cc00a81f0 con 0x555cb88bbd00 2024-05-09T01:31:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.623+0000 7f3ced5deb80 10 client.16950 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.623+0000 7f3ced5deb80 10 client.16950 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:31:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.623+0000 7f3ced5deb80 20 client.16950 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.623+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x555cb89d7fd0 0x555cb89da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.623+0000 7f3ced5deb80 4 client.16950 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.623+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3371968401 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cb88eea80 con 0x555cb89d7fd0 2024-05-09T01:31:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.623+0000 7f3cdc964700 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x555cb89d7fd0 0x555cb89da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.624+0000 7f3cdc964700 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x555cb89d7fd0 0x555cb89da3f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf3e9a81512bbf4b server_cookie=7474e5cc7b63d250 in_seq=0 out_seq=0 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.624+0000 7f3cb7fff700 10 client.16950 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.624+0000 7f3cb7fff700 1 -- 172.21.15.19:0/3371968401 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555cb88eea80 con 0x555cb89d7fd0 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.624+0000 7f3cb7fff700 10 client.16950 handle_command_reply: tid=0 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.624+0000 7f3ced5deb80 1 client.16950 shutdown 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.624+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3371968401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88bbd00 msgr2=0x555cb89c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.624+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88bbd00 0x555cb89c8560 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00095b0 tx=0x7f3cc00a9d90 comp rx=0 tx=0).stop 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3371968401 shutdown_connections 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x555cb89d7fd0 0x555cb89da3f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb89be400 0x555cb89c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 0x555cb89c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3371968401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88bbd00 0x555cb89c8560 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3371968401 >> 172.21.15.19:0/3371968401 conn(0x555cb8844f70 msgr2=0x555cb89b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3cb7fff700 0 client.16950 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3371968401 shutdown_connections 2024-05-09T01:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3371968401 wait complete. 2024-05-09T01:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.625+0000 7f3ced5deb80 20 client.16950 trim_cache size 0 max 16384 2024-05-09T01:31:47.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.640+0000 7f3ced5deb80 1 Processor -- start 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.640+0000 7f3ced5deb80 1 -- start start 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.640+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.640+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 0x555cb89b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81b0f90 con 0x555cb88414a0 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81b2ad0 con 0x555cb89be400 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81b1660 con 0x555cb88bbd00 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdc964700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdc964700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39722/0 (socket says 172.21.15.19:39722) 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdc964700 1 -- 172.21.15.19:0/3725875902 learned_addr learned my addr 172.21.15.19:0/3725875902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdd165700 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 0x555cb89b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdc964700 1 -- 172.21.15.19:0/3725875902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 msgr2=0x555cb89d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdc964700 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89d4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdc964700 1 -- 172.21.15.19:0/3725875902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 msgr2=0x555cb89b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdc964700 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 0x555cb89b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdc964700 1 -- 172.21.15.19:0/3725875902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb89c7df0 con 0x555cb88414a0 2024-05-09T01:31:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cdc964700 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c3890 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800f9d0 tx=0x7f3cc8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7f56e71c48f9df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cce7fc700 1 -- 172.21.15.19:0/3725875902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8003680 con 0x555cb88414a0 2024-05-09T01:31:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cce7fc700 1 -- 172.21.15.19:0/3725875902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc802e070 con 0x555cb88414a0 2024-05-09T01:31:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.641+0000 7f3cce7fc700 1 -- 172.21.15.19:0/3725875902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8028ad0 con 0x555cb88414a0 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.642+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3725875902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 msgr2=0x555cb89c3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.642+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c3890 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800f9d0 tx=0x7f3cc8015000 comp rx=0 tx=0).stop 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.642+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3725875902 shutdown_connections 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.642+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb89be400 0x555cb89b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.642+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb88414a0 0x555cb89c3890 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.642+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/3725875902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.642+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3725875902 >> 172.21.15.19:0/3725875902 conn(0x555cb8844f70 msgr2=0x555cb89c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.642+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3725875902 shutdown_connections 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.642+0000 7f3ced5deb80 1 -- 172.21.15.19:0/3725875902 wait complete. 2024-05-09T01:31:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3ced5deb80 1 -- start start 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3ced5deb80 1 Processor -- start 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 0x555cb89d9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3ced5deb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb89be400 0x555cb89daa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81b2400 con 0x555cb89be400 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81b1d30 con 0x555cb88414a0 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3ced5deb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81aef30 con 0x555cb88bbd00 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3ced5deb80 10 client.0 _ll_register_callbacks cb 0x555cb87526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.643+0000 7f3cdd165700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb89be400 0x555cb89daa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cdd165700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb89be400 0x555cb89daa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39734/0 (socket says 172.21.15.19:39734) 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cdd165700 1 -- 172.21.15.19:0/2410773904 learned_addr learned my addr 172.21.15.19:0/2410773904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cdc964700 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 0x555cb89d9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cd7fff700 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cdd165700 1 -- 172.21.15.19:0/2410773904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 msgr2=0x555cb89da520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cdd165700 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89da520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cdd165700 1 -- 172.21.15.19:0/2410773904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 msgr2=0x555cb89d9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cdd165700 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 0x555cb89d9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cdd165700 1 -- 172.21.15.19:0/2410773904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb8a40510 con 0x555cb89be400 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cdd165700 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb89be400 0x555cb89daa60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00099c0 tx=0x7f3cd0003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427a4ac3cb9cf1cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cb7fff700 1 -- 172.21.15.19:0/2410773904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00055c0 con 0x555cb89be400 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3ced5deb80 1 -- 172.21.15.19:0/2410773904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc802e070 con 0x555cb89be400 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cb7fff700 1 -- 172.21.15.19:0/2410773904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3cd000b790 con 0x555cb89be400 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3cb7fff700 1 -- 172.21.15.19:0/2410773904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3cd0033070 con 0x555cb89be400 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3ced5deb80 10 client.16956 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.644+0000 7f3ced5deb80 1 -- 172.21.15.19:0/2410773904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cb8a417c0 con 0x555cb89be400 2024-05-09T01:31:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.645+0000 7f3cb7fff700 1 -- 172.21.15.19:0/2410773904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc802e070 con 0x555cb89be400 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.645+0000 7f3cb7fff700 1 -- 172.21.15.19:0/2410773904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3cd000faa0 con 0x555cb89be400 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.645+0000 7f3ced5deb80 10 client.16956 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.645+0000 7f3ced5deb80 10 client.16956 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.645+0000 7f3ced5deb80 20 client.16956 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.645+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x555cb8a41eb0 0x555cb8a442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.645+0000 7f3ced5deb80 4 client.16956 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.645+0000 7f3ced5deb80 1 -- 172.21.15.19:0/2410773904 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cb89baf00 con 0x555cb8a41eb0 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.645+0000 7f3cdc964700 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x555cb8a41eb0 0x555cb8a442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3cb7fff700 10 client.16956 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3cdc964700 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x555cb8a41eb0 0x555cb8a442d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53007ec83f069b32 server_cookie=ea4e01e2a5873572 in_seq=0 out_seq=0 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3cb7fff700 1 -- 172.21.15.19:0/2410773904 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cb89baf00 con 0x555cb8a41eb0 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3cb7fff700 10 client.16956 handle_command_reply: tid=0 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 client.16956 shutdown 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 -- 172.21.15.19:0/2410773904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb89be400 msgr2=0x555cb89daa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb89be400 0x555cb89daa60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00099c0 tx=0x7f3cd0003f40 comp rx=0 tx=0).stop 2024-05-09T01:31:47.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 -- 172.21.15.19:0/2410773904 shutdown_connections 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x555cb8a41eb0 0x555cb8a442d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cb89be400 0x555cb89daa60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cb88414a0 0x555cb89d9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3cb7fff700 0 client.16956 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 --2- 172.21.15.19:0/2410773904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555cb88bbd00 0x555cb89da520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 -- 172.21.15.19:0/2410773904 >> 172.21.15.19:0/2410773904 conn(0x555cb8844f70 msgr2=0x555cb89c6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 -- 172.21.15.19:0/2410773904 shutdown_connections 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.646+0000 7f3ced5deb80 1 -- 172.21.15.19:0/2410773904 wait complete. 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3ced5deb80 20 client.16956 trim_cache size 0 max 16384 2024-05-09T01:31:47.649 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:31:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 -- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cc40b2900 msgr2=0x7f3cc40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cc40b2900 0x7f3cc40b4e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005950 tx=0x7f3cc80058e0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 -- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8107df0 msgr2=0x7f3cd80ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8107df0 0x7f3cd80ff7e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0007f10 tx=0x7f3cc000d3b0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 -- 172.21.15.19:0/573219734 shutdown_connections 2024-05-09T01:31:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8108740 0x7f3cd80ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd81034c0 0x7f3cd80ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3cc40b2900 0x7f3cc40b4e10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 --2- 172.21.15.19:0/573219734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd8107df0 0x7f3cd80ff7e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:47.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 -- 172.21.15.19:0/573219734 >> 172.21.15.19:0/573219734 conn(0x7f3cd80fedc0 msgr2=0x7f3cd806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:47.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 -- 172.21.15.19:0/573219734 shutdown_connections 2024-05-09T01:31:47.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:47.647+0000 7f3cdebc8700 1 -- 172.21.15.19:0/573219734 wait complete. 2024-05-09T01:31:48.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:31:48.704 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:48.826 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:48 smithi019 ceph-mon[30158]: pgmap v566: 97 pgs: 97 active+clean; 3.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:48 smithi059 ceph-mon[34351]: pgmap v566: 97 pgs: 97 active+clean; 3.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:48 smithi196 ceph-mon[35041]: pgmap v566: 97 pgs: 97 active+clean; 3.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:50.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.529+0000 7f56be446700 1 -- 172.21.15.19:0/1832280042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8107df0 msgr2=0x7f56b8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.529+0000 7f56be446700 1 --2- 172.21.15.19:0/1832280042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8107df0 0x7f56b8108200 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0099c0 tx=0x7f56ac009cd0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:50 smithi019 ceph-mon[30158]: pgmap v567: 97 pgs: 97 active+clean; 3.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:50.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.529+0000 7f56be446700 1 -- 172.21.15.19:0/1832280042 shutdown_connections 2024-05-09T01:31:50.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.529+0000 7f56be446700 1 --2- 172.21.15.19:0/1832280042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b8108740 0x7f56b81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.529+0000 7f56be446700 1 --2- 172.21.15.19:0/1832280042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b81034c0 0x7f56b81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.529+0000 7f56be446700 1 --2- 172.21.15.19:0/1832280042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8107df0 0x7f56b8108200 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.529+0000 7f56be446700 1 -- 172.21.15.19:0/1832280042 >> 172.21.15.19:0/1832280042 conn(0x7f56b80fedf0 msgr2=0x7f56b8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.529+0000 7f56be446700 1 -- 172.21.15.19:0/1832280042 shutdown_connections 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.529+0000 7f56be446700 1 -- 172.21.15.19:0/1832280042 wait complete. 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.530+0000 7f56be446700 1 Processor -- start 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.530+0000 7f56be446700 1 -- start start 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.530+0000 7f56be446700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b81034c0 0x7f56b810d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.530+0000 7f56be446700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b8107df0 0x7f56b810da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.530+0000 7f56be446700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8108740 0x7f56b810dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56be446700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56b8116650 con 0x7f56b8108740 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56be446700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56b81164d0 con 0x7f56b8107df0 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56be446700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56b81167d0 con 0x7f56b81034c0 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56bc9e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8108740 0x7f56b810dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56bc9e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8108740 0x7f56b810dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44194/0 (socket says 172.21.15.19:44194) 2024-05-09T01:31:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56bc9e3700 1 -- 172.21.15.19:0/4242844376 learned_addr learned my addr 172.21.15.19:0/4242844376 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56b7fff700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b81034c0 0x7f56b810d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56b77fe700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b8107df0 0x7f56b810da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56bc9e3700 1 -- 172.21.15.19:0/4242844376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b81034c0 msgr2=0x7f56b810d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56bc9e3700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b81034c0 0x7f56b810d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56bc9e3700 1 -- 172.21.15.19:0/4242844376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b8107df0 msgr2=0x7f56b810da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56bc9e3700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b8107df0 0x7f56b810da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56bc9e3700 1 -- 172.21.15.19:0/4242844376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b81ad5a0 con 0x7f56b8108740 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.531+0000 7f56bc9e3700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8108740 0x7f56b810dfc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007ee0 tx=0x7f56b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.532+0000 7f56b57fa700 1 -- 172.21.15.19:0/4242844376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0004bb0 con 0x7f56b8108740 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.532+0000 7f56b57fa700 1 -- 172.21.15.19:0/4242844376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b0004d50 con 0x7f56b8108740 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.532+0000 7f56be446700 1 -- 172.21.15.19:0/4242844376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b81ad890 con 0x7f56b8108740 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.532+0000 7f56b57fa700 1 -- 172.21.15.19:0/4242844376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0005920 con 0x7f56b8108740 2024-05-09T01:31:50.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.532+0000 7f56be446700 1 -- 172.21.15.19:0/4242844376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b81add50 con 0x7f56b8108740 2024-05-09T01:31:50.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.536+0000 7f56b57fa700 1 -- 172.21.15.19:0/4242844376 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f56b001a020 con 0x7f56b8108740 2024-05-09T01:31:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.536+0000 7f56cce5cb80 1 Processor -- start 2024-05-09T01:31:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.536+0000 7f56cce5cb80 1 -- start start 2024-05-09T01:31:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56b57fa700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56a00b2860 0x7f56a00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 0x55ee033c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 0x55ee033c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee033bb400 0x55ee033bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee032de0a0 con 0x55ee033bb400 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ee032839a0 con 0x55ee032b8d00 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ee032da060 con 0x55ee0323e4a0 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56b77fe700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56a00b2860 0x7f56a00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56b57fa700 1 -- 172.21.15.19:0/4242844376 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f56b00e1880 con 0x7f56b8108740 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56bc9e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee033bb400 0x55ee033bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56bc9e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee033bb400 0x55ee033bb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44206/0 (socket says 172.21.15.19:44206) 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56bc9e3700 1 -- 172.21.15.19:0/4087543607 learned_addr learned my addr 172.21.15.19:0/4087543607 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56b77fe700 1 --2- 172.21.15.19:0/4087543607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 0x55ee033c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56bc9e3700 1 -- 172.21.15.19:0/4087543607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 msgr2=0x55ee033c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56bc9e3700 1 --2- 172.21.15.19:0/4087543607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 0x55ee033c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56bc9e3700 1 -- 172.21.15.19:0/4087543607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 msgr2=0x55ee033c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56bc9e3700 1 --2- 172.21.15.19:0/4087543607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 0x55ee033c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56bc9e3700 1 -- 172.21.15.19:0/4087543607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee03274500 con 0x55ee033bb400 2024-05-09T01:31:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.537+0000 7f56b77fe700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56a00b2860 0x7f56a00b4d70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f56ac000c00 tx=0x7f56ac005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.538+0000 7f56bc9e3700 1 --2- 172.21.15.19:0/4087543607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee033bb400 0x55ee033bb810 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f56b00a8b80 tx=0x7f56b00a88a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788de2e87422549c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.538+0000 7f569e7fc700 1 -- 172.21.15.19:0/4087543607 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b00a9580 con 0x55ee033bb400 2024-05-09T01:31:50.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.538+0000 7f569e7fc700 1 -- 172.21.15.19:0/4087543607 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b00a8e80 con 0x55ee033bb400 2024-05-09T01:31:50.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.538+0000 7f569e7fc700 1 -- 172.21.15.19:0/4087543607 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b00a9340 con 0x55ee033bb400 2024-05-09T01:31:50.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 -- 172.21.15.19:0/4087543607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee033bb400 msgr2=0x55ee033bb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/4087543607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee033bb400 0x55ee033bb810 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f56b00a8b80 tx=0x7f56b00a88a0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 -- 172.21.15.19:0/4087543607 shutdown_connections 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/4087543607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee033bb400 0x55ee033bb810 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/4087543607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 0x55ee033c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/4087543607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 0x55ee033c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 -- 172.21.15.19:0/4087543607 >> 172.21.15.19:0/4087543607 conn(0x55ee03241f70 msgr2=0x55ee033b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 -- 172.21.15.19:0/4087543607 shutdown_connections 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 -- 172.21.15.19:0/4087543607 wait complete. 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.539+0000 7f56cce5cb80 1 -- start start 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56cce5cb80 1 Processor -- start 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee0323e4a0 0x55ee033c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee033bb400 0x55ee033c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee033bc160 con 0x55ee032b8d00 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ee03278b20 con 0x55ee0323e4a0 2024-05-09T01:31:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ee033c8020 con 0x55ee033bb400 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56cce5cb80 10 client.0 _ll_register_callbacks cb 0x55ee0314f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56b77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56b77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44216/0 (socket says 172.21.15.19:44216) 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56b77fe700 1 -- 172.21.15.19:0/2517053933 learned_addr learned my addr 172.21.15.19:0/2517053933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56bc9e3700 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee033bb400 0x55ee033c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.540+0000 7f56b7fff700 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee0323e4a0 0x55ee033c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56b77fe700 1 -- 172.21.15.19:0/2517053933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee033bb400 msgr2=0x55ee033c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56b77fe700 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee033bb400 0x55ee033c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56b77fe700 1 -- 172.21.15.19:0/2517053933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee0323e4a0 msgr2=0x55ee033c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56b77fe700 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee0323e4a0 0x55ee033c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56b77fe700 1 -- 172.21.15.19:0/2517053933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee03159490 con 0x55ee032b8d00 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56b7fff700 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee0323e4a0 0x55ee033c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56b77fe700 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c5610 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f56ac015000 tx=0x7f56ac009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ee4e197edd0ac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56977fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56977fe700 1 -- 172.21.15.19:0/2517053933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac035060 con 0x55ee032b8d00 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56977fe700 1 -- 172.21.15.19:0/2517053933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f56ac00b990 con 0x55ee032b8d00 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56cce5cb80 1 -- 172.21.15.19:0/2517053933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b00a8e80 con 0x55ee032b8d00 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56977fe700 1 -- 172.21.15.19:0/2517053933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56ac021040 con 0x55ee032b8d00 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56cce5cb80 10 client.16968 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56cce5cb80 1 -- 172.21.15.19:0/2517053933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee033d2ab0 con 0x55ee032b8d00 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.541+0000 7f56977fe700 1 -- 172.21.15.19:0/2517053933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b00a8e80 con 0x55ee032b8d00 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.542+0000 7f56977fe700 1 -- 172.21.15.19:0/2517053933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f56ac02f030 con 0x55ee032b8d00 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.542+0000 7f56cce5cb80 10 client.16968 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.542+0000 7f56cce5cb80 10 client.16968 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.542+0000 7f56cce5cb80 20 client.16968 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.542+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ee033d50f0 0x55ee033d7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.542+0000 7f56cce5cb80 4 client.16968 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.542+0000 7f56cce5cb80 1 -- 172.21.15.19:0/2517053933 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee032eba80 con 0x55ee033d50f0 2024-05-09T01:31:50.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.542+0000 7f56b7fff700 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ee033d50f0 0x55ee033d7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.543+0000 7f56977fe700 10 client.16968 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:31:50.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.543+0000 7f56b7fff700 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ee033d50f0 0x55ee033d7510 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c1244453639141a server_cookie=f92748441f59c302 in_seq=0 out_seq=0 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.543+0000 7f56977fe700 1 -- 172.21.15.19:0/2517053933 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee032eba80 con 0x55ee033d50f0 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.543+0000 7f56977fe700 10 client.16968 handle_command_reply: tid=0 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.543+0000 7f56cce5cb80 1 client.16968 shutdown 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 -- 172.21.15.19:0/2517053933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 msgr2=0x55ee033c5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c5610 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f56ac015000 tx=0x7f56ac009cd0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 -- 172.21.15.19:0/2517053933 shutdown_connections 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee033bb400 0x55ee033c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ee033d50f0 0x55ee033d7510 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee0323e4a0 0x55ee033c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/2517053933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c5610 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 -- 172.21.15.19:0/2517053933 >> 172.21.15.19:0/2517053933 conn(0x55ee03241f70 msgr2=0x55ee033afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56977fe700 0 client.16968 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:31:50.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 -- 172.21.15.19:0/2517053933 shutdown_connections 2024-05-09T01:31:50.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 1 -- 172.21.15.19:0/2517053933 wait complete. 2024-05-09T01:31:50.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.544+0000 7f56cce5cb80 20 client.16968 trim_cache size 0 max 16384 2024-05-09T01:31:50.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.559+0000 7f56cce5cb80 1 Processor -- start 2024-05-09T01:31:50.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.559+0000 7f56cce5cb80 1 -- start start 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee0323e4a0 0x55ee033c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 0x55ee033dcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee033bb400 0x55ee033dd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56b81b5290 con 0x55ee0323e4a0 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56b81b6dd0 con 0x55ee032b8d00 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56b81b5960 con 0x55ee033bb400 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee0323e4a0 0x55ee033c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee0323e4a0 0x55ee033c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44220/0 (socket says 172.21.15.19:44220) 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b7fff700 1 -- 172.21.15.19:0/1229379892 learned_addr learned my addr 172.21.15.19:0/1229379892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b77fe700 1 --2- 172.21.15.19:0/1229379892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 0x55ee033dcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b7fff700 1 -- 172.21.15.19:0/1229379892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee033bb400 msgr2=0x55ee033dd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b7fff700 1 --2- 172.21.15.19:0/1229379892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee033bb400 0x55ee033dd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b7fff700 1 -- 172.21.15.19:0/1229379892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 msgr2=0x55ee033dcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b7fff700 1 --2- 172.21.15.19:0/1229379892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 0x55ee033dcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b7fff700 1 -- 172.21.15.19:0/1229379892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee033b2510 con 0x55ee0323e4a0 2024-05-09T01:31:50.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b77fe700 1 --2- 172.21.15.19:0/1229379892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 0x55ee033dcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:50.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f56b7fff700 1 --2- 172.21.15.19:0/1229379892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee0323e4a0 0x55ee033c7570 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x55ee033d7c00 tx=0x7f56a8010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11454c1b73fa65d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f569dffb700 1 -- 172.21.15.19:0/1229379892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8010cd0 con 0x55ee0323e4a0 2024-05-09T01:31:50.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f569dffb700 1 -- 172.21.15.19:0/1229379892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a8004500 con 0x55ee0323e4a0 2024-05-09T01:31:50.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.560+0000 7f569dffb700 1 -- 172.21.15.19:0/1229379892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a801d6a0 con 0x55ee0323e4a0 2024-05-09T01:31:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.561+0000 7f56cce5cb80 1 -- 172.21.15.19:0/1229379892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee0323e4a0 msgr2=0x55ee033c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.561+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/1229379892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee0323e4a0 0x55ee033c7570 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x55ee033d7c00 tx=0x7f56a8010710 comp rx=0 tx=0).stop 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.561+0000 7f56cce5cb80 1 -- 172.21.15.19:0/1229379892 shutdown_connections 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.561+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/1229379892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee033bb400 0x55ee033dd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.561+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/1229379892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee0323e4a0 0x55ee033c7570 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.561+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/1229379892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee032b8d00 0x55ee033dcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.561+0000 7f56cce5cb80 1 -- 172.21.15.19:0/1229379892 >> 172.21.15.19:0/1229379892 conn(0x55ee03241f70 msgr2=0x55ee033c1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.561+0000 7f56cce5cb80 1 -- 172.21.15.19:0/1229379892 shutdown_connections 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.561+0000 7f56cce5cb80 1 -- 172.21.15.19:0/1229379892 wait complete. 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56cce5cb80 1 -- start start 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56cce5cb80 1 Processor -- start 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 0x55ee033cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56cce5cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee033bb400 0x55ee033c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56b81b6700 con 0x55ee032b8d00 2024-05-09T01:31:50.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56b81b6030 con 0x55ee033bb400 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56cce5cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56b81b3230 con 0x55ee0323e4a0 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56cce5cb80 10 client.0 _ll_register_callbacks cb 0x55ee0314f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56b77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56b77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44224/0 (socket says 172.21.15.19:44224) 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.562+0000 7f56b77fe700 1 -- 172.21.15.19:0/3900432987 learned_addr learned my addr 172.21.15.19:0/3900432987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56b7fff700 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 0x55ee033cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56b77fe700 1 -- 172.21.15.19:0/3900432987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 msgr2=0x55ee033cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56b77fe700 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 0x55ee033cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56b77fe700 1 -- 172.21.15.19:0/3900432987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee033bb400 msgr2=0x55ee033c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56bc9e3700 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee033bb400 0x55ee033c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56b77fe700 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee033bb400 0x55ee033c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56b77fe700 1 -- 172.21.15.19:0/3900432987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee033c7390 con 0x55ee032b8d00 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56bc9e3700 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee033bb400 0x55ee033c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:50.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56b77fe700 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c65a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00fd10 tx=0x7f56ac00f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960634330a83952c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56977fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56977fe700 1 -- 172.21.15.19:0/3900432987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac035070 con 0x55ee032b8d00 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56977fe700 1 -- 172.21.15.19:0/3900432987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f56ac029910 con 0x55ee032b8d00 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56cce5cb80 1 -- 172.21.15.19:0/3900432987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56a8004500 con 0x55ee032b8d00 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56977fe700 1 -- 172.21.15.19:0/3900432987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56ac04a070 con 0x55ee032b8d00 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56cce5cb80 10 client.16974 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.563+0000 7f56cce5cb80 1 -- 172.21.15.19:0/3900432987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee0343f7f0 con 0x55ee032b8d00 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.564+0000 7f56977fe700 1 -- 172.21.15.19:0/3900432987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a8004500 con 0x55ee032b8d00 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.564+0000 7f56977fe700 1 -- 172.21.15.19:0/3900432987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f56ac02f040 con 0x55ee032b8d00 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.564+0000 7f56cce5cb80 10 client.16974 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.564+0000 7f56cce5cb80 10 client.16974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.564+0000 7f56cce5cb80 20 client.16974 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.564+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ee0343ff00 0x55ee03442320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.564+0000 7f56cce5cb80 4 client.16974 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.564+0000 7f56cce5cb80 1 -- 172.21.15.19:0/3900432987 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee032e5b20 con 0x55ee0343ff00 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.564+0000 7f56b7fff700 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ee0343ff00 0x55ee03442320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56977fe700 10 client.16974 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56b7fff700 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ee0343ff00 0x55ee03442320 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67f11532e98aec9b server_cookie=8006fb8b14bf260c in_seq=0 out_seq=0 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56977fe700 1 -- 172.21.15.19:0/3900432987 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee032e5b20 con 0x55ee0343ff00 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56977fe700 10 client.16974 handle_command_reply: tid=0 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 client.16974 shutdown 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 -- 172.21.15.19:0/3900432987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 msgr2=0x55ee033c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c65a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00fd10 tx=0x7f56ac00f970 comp rx=0 tx=0).stop 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 -- 172.21.15.19:0/3900432987 shutdown_connections 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55ee0343ff00 0x55ee03442320 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee033bb400 0x55ee033c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ee0323e4a0 0x55ee033cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56977fe700 0 client.16974 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:31:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 --2- 172.21.15.19:0/3900432987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee032b8d00 0x55ee033c65a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 -- 172.21.15.19:0/3900432987 >> 172.21.15.19:0/3900432987 conn(0x55ee03241f70 msgr2=0x55ee033afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.565+0000 7f56cce5cb80 1 -- 172.21.15.19:0/3900432987 shutdown_connections 2024-05-09T01:31:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56cce5cb80 1 -- 172.21.15.19:0/3900432987 wait complete. 2024-05-09T01:31:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56cce5cb80 20 client.16974 trim_cache size 0 max 16384 2024-05-09T01:31:50.568 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:31:50.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 -- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56a00b2860 msgr2=0x7f56a00b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56a00b2860 0x7f56a00b4d70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f56ac000c00 tx=0x7f56ac005fb0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 -- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8108740 msgr2=0x7f56b810dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8108740 0x7f56b810dfc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007ee0 tx=0x7f56b000d3b0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 -- 172.21.15.19:0/4242844376 shutdown_connections 2024-05-09T01:31:50.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56a00b2860 0x7f56a00b4d70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b8108740 0x7f56b810dfc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b81034c0 0x7f56b810d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 --2- 172.21.15.19:0/4242844376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b8107df0 0x7f56b810da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.566+0000 7f56be446700 1 -- 172.21.15.19:0/4242844376 >> 172.21.15.19:0/4242844376 conn(0x7f56b80fedf0 msgr2=0x7f56b81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.567+0000 7f56be446700 1 -- 172.21.15.19:0/4242844376 shutdown_connections 2024-05-09T01:31:50.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:50.567+0000 7f56be446700 1 -- 172.21.15.19:0/4242844376 wait complete. 2024-05-09T01:31:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.838+0000 7f4309da8640 1 -- 172.21.15.19:0/570748377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43041099f0 msgr2=0x7f430410be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.838+0000 7f4309da8640 1 --2- 172.21.15.19:0/570748377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43041099f0 0x7f430410be10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009ca0 tx=0x7f42fc02a8c0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.839+0000 7f4309da8640 1 -- 172.21.15.19:0/570748377 shutdown_connections 2024-05-09T01:31:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.839+0000 7f4309da8640 1 --2- 172.21.15.19:0/570748377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43041099f0 0x7f430410be10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.839+0000 7f4309da8640 1 --2- 172.21.15.19:0/570748377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4304107090 0x7f43041094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.839+0000 7f4309da8640 1 --2- 172.21.15.19:0/570748377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4304104730 0x7f4304106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.839+0000 7f4309da8640 1 -- 172.21.15.19:0/570748377 >> 172.21.15.19:0/570748377 conn(0x7f43040fdd30 msgr2=0x7f43041001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.839+0000 7f4309da8640 1 -- 172.21.15.19:0/570748377 shutdown_connections 2024-05-09T01:31:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.839+0000 7f4309da8640 1 -- 172.21.15.19:0/570748377 wait complete. 2024-05-09T01:31:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.839+0000 7f4309da8640 1 Processor -- start 2024-05-09T01:31:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.839+0000 7f4309da8640 1 -- start start 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f4309da8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4304104730 0x7f430419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f4309da8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4304107090 0x7f430419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f4309da8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43041099f0 0x7f43041a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f4309da8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f430410f750 con 0x7f4304107090 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f4309da8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f430410f5d0 con 0x7f4304104730 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f4309da8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f430410f8d0 con 0x7f43041099f0 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f42faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4304107090 0x7f430419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f42faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4304107090 0x7f430419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44248/0 (socket says 172.21.15.19:44248) 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f42faffd640 1 -- 172.21.15.19:0/1159151492 learned_addr learned my addr 172.21.15.19:0/1159151492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f4303fff640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43041099f0 0x7f43041a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f43037fe640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4304104730 0x7f430419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f42faffd640 1 -- 172.21.15.19:0/1159151492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43041099f0 msgr2=0x7f43041a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f42faffd640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43041099f0 0x7f43041a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f42faffd640 1 -- 172.21.15.19:0/1159151492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4304104730 msgr2=0x7f430419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f42faffd640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4304104730 0x7f430419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f42faffd640 1 -- 172.21.15.19:0/1159151492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43041a1e90 con 0x7f4304107090 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f43037fe640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4304104730 0x7f430419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.840+0000 7f42faffd640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4304107090 0x7f430419a890 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00e9f0 tx=0x7f42ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.841+0000 7f43017fa640 1 -- 172.21.15.19:0/1159151492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00cd80 con 0x7f4304107090 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.841+0000 7f4309da8640 1 -- 172.21.15.19:0/1159151492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43041a2180 con 0x7f4304107090 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.841+0000 7f43017fa640 1 -- 172.21.15.19:0/1159151492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ec0108e0 con 0x7f4304107090 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.841+0000 7f43017fa640 1 -- 172.21.15.19:0/1159151492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec018980 con 0x7f4304107090 2024-05-09T01:31:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.841+0000 7f4309da8640 1 -- 172.21.15.19:0/1159151492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43041a2720 con 0x7f4304107090 2024-05-09T01:31:50.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:50 smithi059 ceph-mon[34351]: pgmap v567: 97 pgs: 97 active+clean; 3.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.843+0000 7f43017fa640 1 -- 172.21.15.19:0/1159151492 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f42ec010430 con 0x7f4304107090 2024-05-09T01:31:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.843+0000 7f4310c58740 1 Processor -- start 2024-05-09T01:31:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f4310c58740 1 -- start start 2024-05-09T01:31:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f4310c58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 0x5596cfa29500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f4310c58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa29a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f4310c58740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6644d0 0x5596cfa30980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f4310c58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596cf98d7c0 con 0x5596cf65c2f0 2024-05-09T01:31:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f4310c58740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5596cf9290e0 con 0x5596cf6644d0 2024-05-09T01:31:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f4310c58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596cf989780 con 0x5596cf6851b0 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa29a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa29a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44256/0 (socket says 172.21.15.19:44256) 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43037fe640 1 -- 172.21.15.19:0/1597640465 learned_addr learned my addr 172.21.15.19:0/1597640465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f4303fff640 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 0x5596cfa29500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f42faffd640 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6644d0 0x5596cfa30980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43017fa640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f42f00b27e0 0x7f42f00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43037fe640 1 -- 172.21.15.19:0/1597640465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 msgr2=0x5596cfa29500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43037fe640 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 0x5596cfa29500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43037fe640 1 -- 172.21.15.19:0/1597640465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6644d0 msgr2=0x5596cfa30980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43037fe640 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6644d0 0x5596cfa30980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43037fe640 1 -- 172.21.15.19:0/1597640465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596cf9527c0 con 0x5596cf65c2f0 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f42faffd640 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6644d0 0x5596cfa30980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f4303fff640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f42f00b27e0 0x7f42f00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43037fe640 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa29a80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009cf0 tx=0x7f42f40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511553eada1cdcb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.844+0000 7f43017fa640 1 -- 172.21.15.19:0/1159151492 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f42ec0e2600 con 0x7f4304107090 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.845+0000 7f42f9ffb640 1 -- 172.21.15.19:0/1597640465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4012450 con 0x5596cf65c2f0 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.845+0000 7f42f9ffb640 1 -- 172.21.15.19:0/1597640465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f4012aa0 con 0x5596cf65c2f0 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.845+0000 7f42f9ffb640 1 -- 172.21.15.19:0/1597640465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f400fae0 con 0x5596cf65c2f0 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.845+0000 7f4310c58740 1 -- 172.21.15.19:0/1597640465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 msgr2=0x5596cfa29a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.845+0000 7f4303fff640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f42f00b27e0 0x7f42f00b4cf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f42fc004af0 tx=0x7f42fc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.845+0000 7f4310c58740 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa29a80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009cf0 tx=0x7f42f40117b0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.846+0000 7f4310c58740 1 -- 172.21.15.19:0/1597640465 shutdown_connections 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.846+0000 7f4310c58740 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6644d0 0x5596cfa30980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.846+0000 7f4310c58740 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa29a80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.846+0000 7f4310c58740 1 --2- 172.21.15.19:0/1597640465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 0x5596cfa29500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.846+0000 7f4310c58740 1 -- 172.21.15.19:0/1597640465 >> 172.21.15.19:0/1597640465 conn(0x5596cf679300 msgr2=0x5596cfa24e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.846+0000 7f4310c58740 1 -- 172.21.15.19:0/1597640465 shutdown_connections 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.846+0000 7f4310c58740 1 -- 172.21.15.19:0/1597640465 wait complete. 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.846+0000 7f4310c58740 1 -- start start 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.846+0000 7f4310c58740 1 Processor -- start 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4310c58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf65c2f0 0x5596cfa331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4310c58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfa336f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4310c58740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 0x5596cfa3a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4310c58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596cfa313d0 con 0x5596cf6644d0 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4310c58740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5596cf9149c0 con 0x5596cf6851b0 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4310c58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596cfa31550 con 0x5596cf65c2f0 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4310c58740 10 client.0 _ll_register_callbacks cb 0x5596cf941020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfa336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfa336f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44272/0 (socket says 172.21.15.19:44272) 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42faffd640 1 -- 172.21.15.19:0/2188093532 learned_addr learned my addr 172.21.15.19:0/2188093532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f43037fe640 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf65c2f0 0x5596cfa331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42faffd640 1 -- 172.21.15.19:0/2188093532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf65c2f0 msgr2=0x5596cfa331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4303fff640 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 0x5596cfa3a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42faffd640 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf65c2f0 0x5596cfa331b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42faffd640 1 -- 172.21.15.19:0/2188093532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 msgr2=0x5596cfa3a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42faffd640 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 0x5596cfa3a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42faffd640 1 -- 172.21.15.19:0/2188093532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596cf93b1f0 con 0x5596cf6644d0 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4303fff640 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 0x5596cfa3a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42faffd640 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfa336f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0a9900 tx=0x7f42ec010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e99310d3c69a5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42e2ffd640 1 -- 172.21.15.19:0/2188093532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec091030 con 0x5596cf6644d0 2024-05-09T01:31:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f42e2ffd640 1 -- 172.21.15.19:0/2188093532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f42ec0b25d0 con 0x5596cf6644d0 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.847+0000 7f4310c58740 1 -- 172.21.15.19:0/2188093532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f4012aa0 con 0x5596cf6644d0 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f42e2ffd640 1 -- 172.21.15.19:0/2188093532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42ec0aa300 con 0x5596cf6644d0 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f4310c58740 10 client.16986 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f4310c58740 1 -- 172.21.15.19:0/2188093532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596cf6f0790 con 0x5596cf6644d0 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f42e2ffd640 1 -- 172.21.15.19:0/2188093532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f4012aa0 con 0x5596cf6644d0 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f42e2ffd640 1 -- 172.21.15.19:0/2188093532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f42ec0a5570 con 0x5596cf6644d0 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f4310c58740 10 client.16986 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f4310c58740 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f4310c58740 20 client.16986 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f4310c58740 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5596cf643d80 0x5596cfa40b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f4310c58740 4 client.16986 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f4310c58740 1 -- 172.21.15.19:0/2188093532 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596cf99b3d0 con 0x5596cf643d80 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.848+0000 7f43037fe640 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5596cf643d80 0x5596cfa40b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.849+0000 7f43037fe640 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5596cf643d80 0x5596cfa40b10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77065158a3a0211 server_cookie=257c0dc550e873b1 in_seq=0 out_seq=0 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.849+0000 7f42e2ffd640 10 client.16986 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f42e2ffd640 1 -- 172.21.15.19:0/2188093532 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596cf99b3d0 con 0x5596cf643d80 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f42e2ffd640 10 client.16986 handle_command_reply: tid=0 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 client.16986 shutdown 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 -- 172.21.15.19:0/2188093532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 msgr2=0x5596cfa336f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfa336f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0a9900 tx=0x7f42ec010860 comp rx=0 tx=0).stop 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 -- 172.21.15.19:0/2188093532 shutdown_connections 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5596cf643d80 0x5596cfa40b10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 0x5596cfa3a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfa336f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 --2- 172.21.15.19:0/2188093532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf65c2f0 0x5596cfa331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f42e2ffd640 0 client.16986 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 -- 172.21.15.19:0/2188093532 >> 172.21.15.19:0/2188093532 conn(0x5596cf679300 msgr2=0x5596cfa23700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 -- 172.21.15.19:0/2188093532 shutdown_connections 2024-05-09T01:31:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 1 -- 172.21.15.19:0/2188093532 wait complete. 2024-05-09T01:31:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.850+0000 7f4310c58740 20 client.16986 trim_cache size 0 max 16384 2024-05-09T01:31:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.865+0000 7f4310c58740 1 Processor -- start 2024-05-09T01:31:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.865+0000 7f4310c58740 1 -- start start 2024-05-09T01:31:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.865+0000 7f4310c58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.865+0000 7f4310c58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6644d0 0x5596cfa457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.865+0000 7f4310c58740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 0x5596cfa45d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.865+0000 7f4310c58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43041ab550 con 0x5596cf65c2f0 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.865+0000 7f4310c58740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43041ad090 con 0x5596cf6851b0 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.865+0000 7f4310c58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43041abc20 con 0x5596cf6644d0 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.865+0000 7f43037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f43037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44282/0 (socket says 172.21.15.19:44282) 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f43037fe640 1 -- 172.21.15.19:0/1564479645 learned_addr learned my addr 172.21.15.19:0/1564479645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f4303fff640 1 --2- 172.21.15.19:0/1564479645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 0x5596cfa45d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f42faffd640 1 --2- 172.21.15.19:0/1564479645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6644d0 0x5596cfa457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f43037fe640 1 -- 172.21.15.19:0/1564479645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6644d0 msgr2=0x5596cfa457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f43037fe640 1 --2- 172.21.15.19:0/1564479645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6644d0 0x5596cfa457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f43037fe640 1 -- 172.21.15.19:0/1564479645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 msgr2=0x5596cfa45d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f43037fe640 1 --2- 172.21.15.19:0/1564479645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 0x5596cfa45d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f43037fe640 1 -- 172.21.15.19:0/1564479645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596cf7675f0 con 0x5596cf65c2f0 2024-05-09T01:31:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f43037fe640 1 --2- 172.21.15.19:0/1564479645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa452b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f42f4011cb0 tx=0x7f42f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16aa1bd8196bed56 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f42f97fa640 1 -- 172.21.15.19:0/1564479645 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f400f440 con 0x5596cf65c2f0 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f42f97fa640 1 -- 172.21.15.19:0/1564479645 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f400f5e0 con 0x5596cf65c2f0 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.866+0000 7f42f97fa640 1 -- 172.21.15.19:0/1564479645 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4004660 con 0x5596cf65c2f0 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 -- 172.21.15.19:0/1564479645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 msgr2=0x5596cfa452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 --2- 172.21.15.19:0/1564479645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa452b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f42f4011cb0 tx=0x7f42f4005f90 comp rx=0 tx=0).stop 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 -- 172.21.15.19:0/1564479645 shutdown_connections 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 --2- 172.21.15.19:0/1564479645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf6851b0 0x5596cfa45d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 --2- 172.21.15.19:0/1564479645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6644d0 0x5596cfa457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 --2- 172.21.15.19:0/1564479645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf65c2f0 0x5596cfa452b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 -- 172.21.15.19:0/1564479645 >> 172.21.15.19:0/1564479645 conn(0x5596cf679300 msgr2=0x5596cfa2d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 -- 172.21.15.19:0/1564479645 shutdown_connections 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 -- 172.21.15.19:0/1564479645 wait complete. 2024-05-09T01:31:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.867+0000 7f4310c58740 1 -- start start 2024-05-09T01:31:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4310c58740 1 Processor -- start 2024-05-09T01:31:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4310c58740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf65c2f0 0x5596cfaa4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4310c58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfaa4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4310c58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 0x5596cfaab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4310c58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43041ac9c0 con 0x5596cf6644d0 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4310c58740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43041ac2f0 con 0x5596cf65c2f0 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4310c58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43041a9530 con 0x5596cf6851b0 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4310c58740 10 client.0 _ll_register_callbacks cb 0x7f42ec00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfaa4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfaa4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44298/0 (socket says 172.21.15.19:44298) 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42faffd640 1 -- 172.21.15.19:0/3704088380 learned_addr learned my addr 172.21.15.19:0/3704088380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f43037fe640 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf65c2f0 0x5596cfaa4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4303fff640 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 0x5596cfaab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42faffd640 1 -- 172.21.15.19:0/3704088380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 msgr2=0x5596cfaab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42faffd640 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 0x5596cfaab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42faffd640 1 -- 172.21.15.19:0/3704088380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf65c2f0 msgr2=0x5596cfaa4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42faffd640 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf65c2f0 0x5596cfaa4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42faffd640 1 -- 172.21.15.19:0/3704088380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596cf951ed0 con 0x5596cf6644d0 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f43037fe640 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf65c2f0 0x5596cfaa4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f4303fff640 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 0x5596cfaab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42faffd640 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfaa4590 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0a58e0 tx=0x7f42ec0b27e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c790d256cdf77a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.868+0000 7f42e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f42e2ffd640 1 -- 172.21.15.19:0/3704088380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0c8030 con 0x5596cf6644d0 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f42e2ffd640 1 -- 172.21.15.19:0/3704088380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f42ec0a61f0 con 0x5596cf6644d0 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f4310c58740 1 -- 172.21.15.19:0/3704088380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f400f5e0 con 0x5596cf6644d0 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f42e2ffd640 1 -- 172.21.15.19:0/3704088380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42ec0b8070 con 0x5596cf6644d0 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f4310c58740 10 client.16992 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f4310c58740 1 -- 172.21.15.19:0/3704088380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596cfaacc10 con 0x5596cf6644d0 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f42e2ffd640 1 -- 172.21.15.19:0/3704088380 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f400f5e0 con 0x5596cf6644d0 2024-05-09T01:31:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f42e2ffd640 1 -- 172.21.15.19:0/3704088380 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f42ec018b20 con 0x5596cf6644d0 2024-05-09T01:31:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f4310c58740 10 client.16992 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f4310c58740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f4310c58740 20 client.16992 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f4310c58740 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5596cf6191c0 0x5596cfaaf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f4310c58740 4 client.16992 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:31:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f4310c58740 1 -- 172.21.15.19:0/3704088380 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596cf995240 con 0x5596cf6191c0 2024-05-09T01:31:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.869+0000 7f43037fe640 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5596cf6191c0 0x5596cfaaf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.870+0000 7f42e2ffd640 10 client.16992 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.870+0000 7f43037fe640 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5596cf6191c0 0x5596cfaaf440 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9caab2eca3208e3 server_cookie=54458921f14766de in_seq=0 out_seq=0 2024-05-09T01:31:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f42e2ffd640 1 -- 172.21.15.19:0/3704088380 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5596cf995240 con 0x5596cf6191c0 2024-05-09T01:31:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f42e2ffd640 10 client.16992 handle_command_reply: tid=0 2024-05-09T01:31:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f4310c58740 1 client.16992 shutdown 2024-05-09T01:31:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f4310c58740 1 -- 172.21.15.19:0/3704088380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 msgr2=0x5596cfaa4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f4310c58740 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfaa4590 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0a58e0 tx=0x7f42ec0b27e0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f4310c58740 1 -- 172.21.15.19:0/3704088380 shutdown_connections 2024-05-09T01:31:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f4310c58740 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5596cf6191c0 0x5596cfaaf440 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f4310c58740 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596cf6851b0 0x5596cfaab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f4310c58740 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596cf6644d0 0x5596cfaa4590 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f4310c58740 1 --2- 172.21.15.19:0/3704088380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596cf65c2f0 0x5596cfaa4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.888+0000 7f42e2ffd640 0 client.16992 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4310c58740 1 -- 172.21.15.19:0/3704088380 >> 172.21.15.19:0/3704088380 conn(0x5596cf679300 msgr2=0x5596cfa27c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4310c58740 1 -- 172.21.15.19:0/3704088380 shutdown_connections 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4310c58740 1 -- 172.21.15.19:0/3704088380 wait complete. 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4310c58740 20 client.16992 trim_cache size 0 max 16384 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:31:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:31:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:31:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 -- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f42f00b27e0 msgr2=0x7f42f00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f42f00b27e0 0x7f42f00b4cf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f42fc004af0 tx=0x7f42fc0095c0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 -- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4304107090 msgr2=0x7f430419a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4304107090 0x7f430419a890 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00e9f0 tx=0x7f42ec00eeb0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 -- 172.21.15.19:0/1159151492 shutdown_connections 2024-05-09T01:31:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f42f00b27e0 0x7f42f00b4cf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43041099f0 0x7f43041a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4304107090 0x7f430419a890 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 --2- 172.21.15.19:0/1159151492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4304104730 0x7f430419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 -- 172.21.15.19:0/1159151492 >> 172.21.15.19:0/1159151492 conn(0x7f43040fdd30 msgr2=0x7f43041056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 -- 172.21.15.19:0/1159151492 shutdown_connections 2024-05-09T01:31:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:31:50.889+0000 7f4309da8640 1 -- 172.21.15.19:0/1159151492 wait complete. 2024-05-09T01:31:50.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:50 smithi196 ceph-mon[35041]: pgmap v567: 97 pgs: 97 active+clean; 3.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:31:51.324 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:31:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:51.890 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:189/600: open fd = 3 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:190/600: open fd = 3 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:191/600: open fd = 3 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.891 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:192/600: open fd = 3 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:193/600: open fd = 3 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:194/600: open fd = 3 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:195/600: open fd = 3 2024-05-09T01:31:51.892 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:196/600: open fd = 3 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:197/600: open fd = 3 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:198/600: open fd = 3 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.893 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:199/600: open fd = 3 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:200/600: open fd = 3 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:201/600: open fd = 3 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.894 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:202/600: open fd = 3 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:203/600: open fd = 3 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:204/600: open fd = 3 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:205/600: open fd = 3 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.895 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.896 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.896 INFO:tasks.workunit.client.0.smithi019.stdout:206/600: open fd = 3 2024-05-09T01:31:51.896 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.896 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.896 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.896 INFO:tasks.workunit.client.0.smithi019.stdout:207/600: open fd = 3 2024-05-09T01:31:51.896 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.896 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.896 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:208/600: open fd = 3 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:209/600: open fd = 3 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:210/600: open fd = 3 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:211/600: open fd = 3 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.897 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:212/600: open fd = 3 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:213/600: open fd = 3 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:214/600: open fd = 3 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.898 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:215/600: open fd = 3 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:216/600: open fd = 3 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:217/600: open fd = 3 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:218/600: open fd = 3 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.899 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:219/600: open fd = 3 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:220/600: open fd = 3 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:221/600: open fd = 3 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:222/600: open fd = 3 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.900 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:223/600: open fd = 3 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:224/600: open fd = 3 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:225/600: open fd = 3 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:226/600: open fd = 3 2024-05-09T01:31:51.901 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:227/600: open fd = 3 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:228/600: open fd = 3 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:229/600: open fd = 3 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.902 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:230/600: open fd = 3 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:231/600: open fd = 3 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:232/600: open fd = 3 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:233/600: open fd = 3 2024-05-09T01:31:51.903 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:234/600: open fd = 3 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:235/600: open fd = 3 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:236/600: open fd = 3 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.904 INFO:tasks.workunit.client.0.smithi019.stdout:237/600: open fd = 3 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:238/600: open fd = 3 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:239/600: open fd = 3 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:240/600: open fd = 3 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.905 INFO:tasks.workunit.client.0.smithi019.stdout:241/600: open fd = 3 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:242/600: open fd = 3 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:243/600: open fd = 3 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.906 INFO:tasks.workunit.client.0.smithi019.stdout:244/600: open fd = 3 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:245/600: open fd = 3 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:246/600: open fd = 3 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:247/600: open fd = 3 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.907 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:248/600: open fd = 3 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:249/600: open fd = 3 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:250/600: open fd = 3 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:51.908 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:31:52.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:52 smithi019 ceph-mon[30158]: pgmap v568: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:52.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:52 smithi059 ceph-mon[34351]: pgmap v568: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:52.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:52 smithi196 ceph-mon[35041]: pgmap v568: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.235+0000 7f21eae69700 1 -- 172.21.15.19:0/1312864939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 msgr2=0x7f21e4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.235+0000 7f21eae69700 1 --2- 172.21.15.19:0/1312864939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 0x7f21e4108200 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f21d400b0d0 tx=0x7f21d400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.235+0000 7f21eae69700 1 -- 172.21.15.19:0/1312864939 shutdown_connections 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.235+0000 7f21eae69700 1 --2- 172.21.15.19:0/1312864939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21e4108740 0x7f21e41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.235+0000 7f21eae69700 1 --2- 172.21.15.19:0/1312864939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e41034c0 0x7f21e41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.235+0000 7f21eae69700 1 --2- 172.21.15.19:0/1312864939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 0x7f21e4108200 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.235+0000 7f21eae69700 1 -- 172.21.15.19:0/1312864939 >> 172.21.15.19:0/1312864939 conn(0x7f21e40fedf0 msgr2=0x7f21e4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.235+0000 7f21eae69700 1 -- 172.21.15.19:0/1312864939 shutdown_connections 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.235+0000 7f21eae69700 1 -- 172.21.15.19:0/1312864939 wait complete. 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21eae69700 1 Processor -- start 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21eae69700 1 -- start start 2024-05-09T01:31:53.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21eae69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21e41034c0 0x7f21e40ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21eae69700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 0x7f21e40ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21eae69700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e4108740 0x7f21e40ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21eae69700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21e4113350 con 0x7f21e4107df0 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21eae69700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41131d0 con 0x7f21e41034c0 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21eae69700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41134d0 con 0x7f21e4108740 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 0x7f21e40ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 0x7f21e40ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44328/0 (socket says 172.21.15.19:44328) 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e3fff700 1 -- 172.21.15.19:0/3598333097 learned_addr learned my addr 172.21.15.19:0/3598333097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e8c05700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21e41034c0 0x7f21e40ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e9406700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e4108740 0x7f21e40ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e3fff700 1 -- 172.21.15.19:0/3598333097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e4108740 msgr2=0x7f21e40ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e3fff700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e4108740 0x7f21e40ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e3fff700 1 -- 172.21.15.19:0/3598333097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21e41034c0 msgr2=0x7f21e40ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e3fff700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21e41034c0 0x7f21e40ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.236+0000 7f21e3fff700 1 -- 172.21.15.19:0/3598333097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e41a90e0 con 0x7f21e4107df0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.237+0000 7f21e3fff700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 0x7f21e40ff810 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f21d40095c0 tx=0x7f21d4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.237+0000 7f21e1ffb700 1 -- 172.21.15.19:0/3598333097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4021020 con 0x7f21e4107df0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.237+0000 7f21e1ffb700 1 -- 172.21.15.19:0/3598333097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d4004c50 con 0x7f21e4107df0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.237+0000 7f21eae69700 1 -- 172.21.15.19:0/3598333097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e41a9370 con 0x7f21e4107df0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.237+0000 7f21e1ffb700 1 -- 172.21.15.19:0/3598333097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d40052b0 con 0x7f21e4107df0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.237+0000 7f21eae69700 1 -- 172.21.15.19:0/3598333097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e41a9820 con 0x7f21e4107df0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.240+0000 7f21e1ffb700 1 -- 172.21.15.19:0/3598333097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f21d4024020 con 0x7f21e4107df0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.240+0000 7f21e1ffb700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f21d00b2880 0x7f21d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21e1ffb700 1 -- 172.21.15.19:0/3598333097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f21d400f020 con 0x7f21e4107df0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21e8c05700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f21d00b2880 0x7f21d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21e8c05700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f21d00b2880 0x7f21d00b4d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00ac50 tx=0x7f21cc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21f987fb80 1 Processor -- start 2024-05-09T01:31:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21f987fb80 1 -- start start 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d0a160 0x562b08da34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 0x562b08da3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 0x562b08d9dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21f987fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b08cbfff0 con 0x562b08d0a160 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21f987fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562b08c5e7f0 con 0x562b08d9d6a0 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.241+0000 7f21f987fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b08cbbfb0 con 0x562b08c204a0 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e8c05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d0a160 0x562b08da34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e8c05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d0a160 0x562b08da34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44336/0 (socket says 172.21.15.19:44336) 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e8c05700 1 -- 172.21.15.19:0/3605711226 learned_addr learned my addr 172.21.15.19:0/3605711226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e3fff700 1 --2- 172.21.15.19:0/3605711226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 0x562b08da3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e8c05700 1 -- 172.21.15.19:0/3605711226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 msgr2=0x562b08da3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e8c05700 1 --2- 172.21.15.19:0/3605711226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 0x562b08da3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e8c05700 1 -- 172.21.15.19:0/3605711226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 msgr2=0x562b08d9dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e9406700 1 --2- 172.21.15.19:0/3605711226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 0x562b08d9dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e8c05700 1 --2- 172.21.15.19:0/3605711226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 0x562b08d9dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e8c05700 1 -- 172.21.15.19:0/3605711226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b08c56370 con 0x562b08d0a160 2024-05-09T01:31:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21e8c05700 1 --2- 172.21.15.19:0/3605711226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d0a160 0x562b08da34f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00ff90 tx=0x7f21cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac4491e4e2a3906 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21daffd700 1 -- 172.21.15.19:0/3605711226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc016070 con 0x562b08d0a160 2024-05-09T01:31:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21daffd700 1 -- 172.21.15.19:0/3605711226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21cc014070 con 0x562b08d0a160 2024-05-09T01:31:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.242+0000 7f21daffd700 1 -- 172.21.15.19:0/3605711226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc01d400 con 0x562b08d0a160 2024-05-09T01:31:53.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.243+0000 7f21f987fb80 1 -- 172.21.15.19:0/3605711226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d0a160 msgr2=0x562b08da34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:53.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.243+0000 7f21f987fb80 1 --2- 172.21.15.19:0/3605711226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d0a160 0x562b08da34f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00ff90 tx=0x7f21cc012000 comp rx=0 tx=0).stop 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.243+0000 7f21f987fb80 1 -- 172.21.15.19:0/3605711226 shutdown_connections 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.243+0000 7f21f987fb80 1 --2- 172.21.15.19:0/3605711226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 0x562b08d9dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.243+0000 7f21f987fb80 1 --2- 172.21.15.19:0/3605711226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d0a160 0x562b08da34f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.243+0000 7f21f987fb80 1 --2- 172.21.15.19:0/3605711226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 0x562b08da3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.243+0000 7f21f987fb80 1 -- 172.21.15.19:0/3605711226 >> 172.21.15.19:0/3605711226 conn(0x562b08c806a0 msgr2=0x562b08d98400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.243+0000 7f21f987fb80 1 -- 172.21.15.19:0/3605711226 shutdown_connections 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.243+0000 7f21f987fb80 1 -- 172.21.15.19:0/3605711226 wait complete. 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.244+0000 7f21f987fb80 1 -- start start 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.244+0000 7f21f987fb80 1 Processor -- start 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08c204a0 0x562b08da7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 0x562b08da7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21f987fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b08b3bba0 con 0x562b08d9d6a0 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21f987fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562b08c7a150 con 0x562b08c204a0 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21f987fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b08daa1d0 con 0x562b08d0a160 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21f987fb80 10 client.0 _ll_register_callbacks cb 0x562b08b316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e9406700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e9406700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44348/0 (socket says 172.21.15.19:44348) 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e9406700 1 -- 172.21.15.19:0/2331101572 learned_addr learned my addr 172.21.15.19:0/2331101572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e3fff700 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 0x562b08da7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e8c05700 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08c204a0 0x562b08da7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e9406700 1 -- 172.21.15.19:0/2331101572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 msgr2=0x562b08da7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e9406700 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 0x562b08da7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e9406700 1 -- 172.21.15.19:0/2331101572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08c204a0 msgr2=0x562b08da7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e9406700 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08c204a0 0x562b08da7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e9406700 1 -- 172.21.15.19:0/2331101572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b08b3b490 con 0x562b08d9d6a0 2024-05-09T01:31:53.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e3fff700 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 0x562b08da7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21e9406700 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da7c90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f21dc010470 tx=0x7f21dc010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fd9e513756e61a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21c3fff700 1 -- 172.21.15.19:0/2331101572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc012860 con 0x562b08d9d6a0 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21c3fff700 1 -- 172.21.15.19:0/2331101572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f21dc012a00 con 0x562b08d9d6a0 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.245+0000 7f21f987fb80 1 -- 172.21.15.19:0/2331101572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21cc014070 con 0x562b08d9d6a0 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21c3fff700 1 -- 172.21.15.19:0/2331101572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21dc012c00 con 0x562b08d9d6a0 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21f987fb80 10 client.17004 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21f987fb80 1 -- 172.21.15.19:0/2331101572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b08db4c40 con 0x562b08d9d6a0 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21c3fff700 1 -- 172.21.15.19:0/2331101572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21cc014070 con 0x562b08d9d6a0 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21c3fff700 1 -- 172.21.15.19:0/2331101572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f21dc00f040 con 0x562b08d9d6a0 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21f987fb80 10 client.17004 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21f987fb80 10 client.17004 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:31:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21f987fb80 20 client.17004 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x562b08db7280 0x562b08db96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21f987fb80 4 client.17004 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.246+0000 7f21f987fb80 1 -- 172.21.15.19:0/2331101572 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b08ccd9d0 con 0x562b08db7280 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.247+0000 7f21e8c05700 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x562b08db7280 0x562b08db96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.247+0000 7f21e8c05700 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x562b08db7280 0x562b08db96f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69db17e3368fc571 server_cookie=d2670ef80e76a3ff in_seq=0 out_seq=0 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.247+0000 7f21c3fff700 10 client.17004 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21c3fff700 1 -- 172.21.15.19:0/2331101572 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b08ccd9d0 con 0x562b08db7280 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21c3fff700 10 client.17004 handle_command_reply: tid=0 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 client.17004 shutdown 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 -- 172.21.15.19:0/2331101572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 msgr2=0x562b08da7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da7c90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f21dc010470 tx=0x7f21dc010830 comp rx=0 tx=0).stop 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 -- 172.21.15.19:0/2331101572 shutdown_connections 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x562b08db7280 0x562b08db96f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da7c90 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08c204a0 0x562b08da7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2331101572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 0x562b08da7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 -- 172.21.15.19:0/2331101572 >> 172.21.15.19:0/2331101572 conn(0x562b08c806a0 msgr2=0x562b08d981c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:53.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21c3fff700 0 client.17004 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:31:53.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 -- 172.21.15.19:0/2331101572 shutdown_connections 2024-05-09T01:31:53.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 1 -- 172.21.15.19:0/2331101572 wait complete. 2024-05-09T01:31:53.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.248+0000 7f21f987fb80 20 client.17004 trim_cache size 0 max 16384 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21f987fb80 1 Processor -- start 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21f987fb80 1 -- start start 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 0x562b08db9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d0a160 0x562b08da6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21f987fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41b0d50 con 0x562b08d9d6a0 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21f987fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41b2890 con 0x562b08d0a160 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21f987fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41b1420 con 0x562b08c204a0 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21e9406700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21e9406700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44356/0 (socket says 172.21.15.19:44356) 2024-05-09T01:31:53.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21e9406700 1 -- 172.21.15.19:0/2391421646 learned_addr learned my addr 172.21.15.19:0/2391421646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21e3fff700 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d0a160 0x562b08da6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.270+0000 7f21e8c05700 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 0x562b08db9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21e9406700 1 -- 172.21.15.19:0/2391421646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 msgr2=0x562b08db9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21e9406700 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 0x562b08db9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21e9406700 1 -- 172.21.15.19:0/2391421646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d0a160 msgr2=0x562b08da6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21e9406700 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d0a160 0x562b08da6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21e9406700 1 -- 172.21.15.19:0/2391421646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b08da82d0 con 0x562b08d9d6a0 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21e8c05700 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 0x562b08db9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21e9406700 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da0e00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f21dc010db0 tx=0x7f21dc012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d35874c15b70a4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21da7fc700 1 -- 172.21.15.19:0/2391421646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc029040 con 0x562b08d9d6a0 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21da7fc700 1 -- 172.21.15.19:0/2391421646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc034070 con 0x562b08d9d6a0 2024-05-09T01:31:53.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.271+0000 7f21da7fc700 1 -- 172.21.15.19:0/2391421646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc005570 con 0x562b08d9d6a0 2024-05-09T01:31:53.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 -- 172.21.15.19:0/2391421646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 msgr2=0x562b08da0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:53.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da0e00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f21dc010db0 tx=0x7f21dc012f30 comp rx=0 tx=0).stop 2024-05-09T01:31:53.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 -- 172.21.15.19:0/2391421646 shutdown_connections 2024-05-09T01:31:53.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08d9d6a0 0x562b08da0e00 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08c204a0 0x562b08db9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 --2- 172.21.15.19:0/2391421646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d0a160 0x562b08da6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 -- 172.21.15.19:0/2391421646 >> 172.21.15.19:0/2391421646 conn(0x562b08c806a0 msgr2=0x562b08da5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 -- 172.21.15.19:0/2391421646 shutdown_connections 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 -- 172.21.15.19:0/2391421646 wait complete. 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 -- start start 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.272+0000 7f21f987fb80 1 Processor -- start 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08c204a0 0x562b08dafa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 0x562b08daa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21f987fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 0x562b08daa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21f987fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41b21c0 con 0x562b08c204a0 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21f987fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41b1af0 con 0x562b08d9d6a0 2024-05-09T01:31:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21f987fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41aecf0 con 0x562b08d0a160 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21f987fb80 10 client.0 _ll_register_callbacks cb 0x562b08b316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21e8c05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08c204a0 0x562b08dafa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21e8c05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08c204a0 0x562b08dafa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44370/0 (socket says 172.21.15.19:44370) 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21e8c05700 1 -- 172.21.15.19:0/1710201785 learned_addr learned my addr 172.21.15.19:0/1710201785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21e3fff700 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 0x562b08daa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.273+0000 7f21e9406700 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 0x562b08daa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21e8c05700 1 -- 172.21.15.19:0/1710201785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 msgr2=0x562b08daa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21e8c05700 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 0x562b08daa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21e8c05700 1 -- 172.21.15.19:0/1710201785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 msgr2=0x562b08daa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21e8c05700 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 0x562b08daa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21e8c05700 1 -- 172.21.15.19:0/1710201785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b08dab0e0 con 0x562b08c204a0 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21e9406700 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 0x562b08daa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21e8c05700 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08c204a0 0x562b08dafa60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00ff60 tx=0x7f21cc005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8612b2b5d2dcde08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21c3fff700 1 -- 172.21.15.19:0/1710201785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc016050 con 0x562b08c204a0 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21c3fff700 1 -- 172.21.15.19:0/1710201785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f21cc010840 con 0x562b08c204a0 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21f987fb80 1 -- 172.21.15.19:0/1710201785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21dc034070 con 0x562b08c204a0 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21c3fff700 1 -- 172.21.15.19:0/1710201785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21cc003910 con 0x562b08c204a0 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21f987fb80 10 client.17010 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21f987fb80 1 -- 172.21.15.19:0/1710201785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b08e20ae0 con 0x562b08c204a0 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.274+0000 7f21c3fff700 1 -- 172.21.15.19:0/1710201785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc034070 con 0x562b08c204a0 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.275+0000 7f21c3fff700 1 -- 172.21.15.19:0/1710201785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f21cc031b30 con 0x562b08c204a0 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.275+0000 7f21f987fb80 10 client.17010 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.275+0000 7f21f987fb80 10 client.17010 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:31:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.275+0000 7f21f987fb80 20 client.17010 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.275+0000 7f21f987fb80 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x562b08e20f70 0x562b08e23390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.275+0000 7f21f987fb80 4 client.17010 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.275+0000 7f21f987fb80 1 -- 172.21.15.19:0/1710201785 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b08cc7a70 con 0x562b08e20f70 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.275+0000 7f21e3fff700 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x562b08e20f70 0x562b08e23390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21c3fff700 10 client.17010 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.275+0000 7f21e3fff700 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x562b08e20f70 0x562b08e23390 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30333d762743dad9 server_cookie=8d43e72614eb9f6e in_seq=0 out_seq=0 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21c3fff700 1 -- 172.21.15.19:0/1710201785 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b08cc7a70 con 0x562b08e20f70 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21c3fff700 10 client.17010 handle_command_reply: tid=0 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 client.17010 shutdown 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 -- 172.21.15.19:0/1710201785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08c204a0 msgr2=0x562b08dafa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08c204a0 0x562b08dafa60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00ff60 tx=0x7f21cc005730 comp rx=0 tx=0).stop 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 -- 172.21.15.19:0/1710201785 shutdown_connections 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x562b08e20f70 0x562b08e23390 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b08d9d6a0 0x562b08daa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b08c204a0 0x562b08dafa60 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 --2- 172.21.15.19:0/1710201785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b08d0a160 0x562b08daa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 -- 172.21.15.19:0/1710201785 >> 172.21.15.19:0/1710201785 conn(0x562b08c806a0 msgr2=0x562b08da7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21c3fff700 0 client.17010 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:31:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 -- 172.21.15.19:0/1710201785 shutdown_connections 2024-05-09T01:31:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 1 -- 172.21.15.19:0/1710201785 wait complete. 2024-05-09T01:31:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.276+0000 7f21f987fb80 20 client.17010 trim_cache size 0 max 16384 2024-05-09T01:31:53.278 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:31:53.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 -- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f21d00b2880 msgr2=0x7f21d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:53.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f21d00b2880 0x7f21d00b4d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00ac50 tx=0x7f21cc00a380 comp rx=0 tx=0).stop 2024-05-09T01:31:53.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 -- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 msgr2=0x7f21e40ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:53.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 0x7f21e40ff810 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f21d40095c0 tx=0x7f21d4007a20 comp rx=0 tx=0).stop 2024-05-09T01:31:53.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 -- 172.21.15.19:0/3598333097 shutdown_connections 2024-05-09T01:31:53.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e4108740 0x7f21e40ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21e41034c0 0x7f21e40ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f21d00b2880 0x7f21d00b4d90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 --2- 172.21.15.19:0/3598333097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e4107df0 0x7f21e40ff810 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 -- 172.21.15.19:0/3598333097 >> 172.21.15.19:0/3598333097 conn(0x7f21e40fedf0 msgr2=0x7f21e4075cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 -- 172.21.15.19:0/3598333097 shutdown_connections 2024-05-09T01:31:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:53.277+0000 7f21eae69700 1 -- 172.21.15.19:0/3598333097 wait complete. 2024-05-09T01:31:53.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:53 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:31:53.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:53 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:31:53.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:53 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:31:54.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:31:54.372 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: pgmap v569: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:31:54.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:31:54.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:54 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:31:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: pgmap v569: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-09T01:31:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:31:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:31:54.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:54 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:31:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: pgmap v569: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-09T01:31:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:31:54.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:31:54.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:31:54.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:54 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:31:55.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:55 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-09T01:31:55.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:55 smithi019 ceph-mon[30158]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-09T01:31:55.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:55 smithi019 ceph-mon[30158]: Unable to set osd_memory_target on smithi059 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-09T01:31:55.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:55 smithi019 ceph-mon[30158]: Unable to set osd_memory_target on smithi196 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-09T01:31:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:55 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-09T01:31:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:55 smithi059 ceph-mon[34351]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-09T01:31:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:55 smithi059 ceph-mon[34351]: Unable to set osd_memory_target on smithi059 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-09T01:31:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:55 smithi059 ceph-mon[34351]: Unable to set osd_memory_target on smithi196 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-09T01:31:55.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:55 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-09T01:31:55.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:55 smithi196 ceph-mon[35041]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-09T01:31:55.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:55 smithi196 ceph-mon[35041]: Unable to set osd_memory_target on smithi059 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-09T01:31:55.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:55 smithi196 ceph-mon[35041]: Unable to set osd_memory_target on smithi196 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-09T01:31:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.156+0000 7f4adabec700 1 -- 172.21.15.19:0/3073403565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad41034c0 msgr2=0x7f4ad41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.156+0000 7f4adabec700 1 --2- 172.21.15.19:0/3073403565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad41034c0 0x7f4ad41038d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40099c0 tx=0x7f4ac4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.156+0000 7f4adabec700 1 -- 172.21.15.19:0/3073403565 shutdown_connections 2024-05-09T01:31:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.156+0000 7f4adabec700 1 --2- 172.21.15.19:0/3073403565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad4108740 0x7f4ad41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.156+0000 7f4adabec700 1 --2- 172.21.15.19:0/3073403565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad41034c0 0x7f4ad41038d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.156+0000 7f4adabec700 1 --2- 172.21.15.19:0/3073403565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ad4107df0 0x7f4ad4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.156+0000 7f4adabec700 1 -- 172.21.15.19:0/3073403565 >> 172.21.15.19:0/3073403565 conn(0x7f4ad40fedc0 msgr2=0x7f4ad41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.156+0000 7f4adabec700 1 -- 172.21.15.19:0/3073403565 shutdown_connections 2024-05-09T01:31:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.156+0000 7f4adabec700 1 -- 172.21.15.19:0/3073403565 wait complete. 2024-05-09T01:31:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4adabec700 1 Processor -- start 2024-05-09T01:31:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4adabec700 1 -- start start 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4adabec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad41034c0 0x7f4ad419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4adabec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad4107df0 0x7f4ad419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4adabec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ad4108740 0x7f4ad419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4adabec700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad41133e0 con 0x7f4ad4107df0 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4adabec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4113260 con 0x7f4ad4108740 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4adabec700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4113560 con 0x7f4ad41034c0 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4ad3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad4107df0 0x7f4ad419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4ad3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad4107df0 0x7f4ad419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44382/0 (socket says 172.21.15.19:44382) 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4ad3fff700 1 -- 172.21.15.19:0/3030390465 learned_addr learned my addr 172.21.15.19:0/3030390465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4ad9189700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ad4108740 0x7f4ad419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.157+0000 7f4ad8988700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad41034c0 0x7f4ad419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad3fff700 1 -- 172.21.15.19:0/3030390465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad41034c0 msgr2=0x7f4ad419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad41034c0 0x7f4ad419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad3fff700 1 -- 172.21.15.19:0/3030390465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ad4108740 msgr2=0x7f4ad419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ad4108740 0x7f4ad419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad3fff700 1 -- 172.21.15.19:0/3030390465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad419fd50 con 0x7f4ad4107df0 2024-05-09T01:31:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad9189700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ad4108740 0x7f4ad419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad4107df0 0x7f4ad419b420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00b500 tx=0x7f4abc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad1ffb700 1 -- 172.21.15.19:0/3030390465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc015020 con 0x7f4ad4107df0 2024-05-09T01:31:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad1ffb700 1 -- 172.21.15.19:0/3030390465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc011ce0 con 0x7f4ad4107df0 2024-05-09T01:31:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4adabec700 1 -- 172.21.15.19:0/3030390465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad41a0040 con 0x7f4ad4107df0 2024-05-09T01:31:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4ad1ffb700 1 -- 172.21.15.19:0/3030390465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc010d40 con 0x7f4ad4107df0 2024-05-09T01:31:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.158+0000 7f4adabec700 1 -- 172.21.15.19:0/3030390465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad41a7960 con 0x7f4ad4107df0 2024-05-09T01:31:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.161+0000 7f4ad1ffb700 1 -- 172.21.15.19:0/3030390465 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4abc018020 con 0x7f4ad4107df0 2024-05-09T01:31:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.162+0000 7f4ad1ffb700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ac00b2880 0x7f4ac00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.162+0000 7f4ad8988700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ac00b2880 0x7f4ac00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.162+0000 7f4ad1ffb700 1 -- 172.21.15.19:0/3030390465 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4abc0e1940 con 0x7f4ad4107df0 2024-05-09T01:31:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.162+0000 7f4ae9602b80 1 Processor -- start 2024-05-09T01:31:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.162+0000 7f4ae9602b80 1 -- start start 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.162+0000 7f4ad8988700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ac00b2880 0x7f4ac00b4d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009990 tx=0x7f4ac4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 0x558ed95b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed94344a0 0x558ed95b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 0x558ed95ba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ed94d3ff0 con 0x558ed94344a0 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558ed94727f0 con 0x558ed95ba400 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ed94cffb0 con 0x558ed951e160 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed94344a0 0x558ed95b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed94344a0 0x558ed95b62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44394/0 (socket says 172.21.15.19:44394) 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad3fff700 1 -- 172.21.15.19:0/304650346 learned_addr learned my addr 172.21.15.19:0/304650346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad8988700 1 --2- 172.21.15.19:0/304650346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 0x558ed95b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad9189700 1 --2- 172.21.15.19:0/304650346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 0x558ed95ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad3fff700 1 -- 172.21.15.19:0/304650346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 msgr2=0x558ed95b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/304650346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 0x558ed95b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad3fff700 1 -- 172.21.15.19:0/304650346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 msgr2=0x558ed95ba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/304650346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 0x558ed95ba810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad3fff700 1 -- 172.21.15.19:0/304650346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ed946a370 con 0x558ed94344a0 2024-05-09T01:31:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.163+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/304650346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed94344a0 0x558ed95b62f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0a61b0 tx=0x7f4abc004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fa430458a3b467 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.164+0000 7f4acaffd700 1 -- 172.21.15.19:0/304650346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0a9640 con 0x558ed94344a0 2024-05-09T01:31:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.164+0000 7f4acaffd700 1 -- 172.21.15.19:0/304650346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc0a8f40 con 0x558ed94344a0 2024-05-09T01:31:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.164+0000 7f4acaffd700 1 -- 172.21.15.19:0/304650346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0a81a0 con 0x558ed94344a0 2024-05-09T01:31:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.164+0000 7f4ae9602b80 1 -- 172.21.15.19:0/304650346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed94344a0 msgr2=0x558ed95b62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.164+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/304650346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed94344a0 0x558ed95b62f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0a61b0 tx=0x7f4abc004860 comp rx=0 tx=0).stop 2024-05-09T01:31:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.165+0000 7f4ae9602b80 1 -- 172.21.15.19:0/304650346 shutdown_connections 2024-05-09T01:31:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.165+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/304650346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 0x558ed95ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.165+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/304650346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 0x558ed95b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.165+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/304650346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed94344a0 0x558ed95b62f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.165+0000 7f4ae9602b80 1 -- 172.21.15.19:0/304650346 >> 172.21.15.19:0/304650346 conn(0x558ed94946a0 msgr2=0x558ed95ac400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.165+0000 7f4ae9602b80 1 -- 172.21.15.19:0/304650346 shutdown_connections 2024-05-09T01:31:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.165+0000 7f4ae9602b80 1 -- 172.21.15.19:0/304650346 wait complete. 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.165+0000 7f4ae9602b80 1 -- start start 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.165+0000 7f4ae9602b80 1 Processor -- start 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 0x558ed95bf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed951e160 0x558ed95bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ed934fba0 con 0x558ed95ba400 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558ed948e150 con 0x558ed951e160 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ed95bb1b0 con 0x558ed94344a0 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ae9602b80 10 client.0 _ll_register_callbacks cb 0x558ed93456c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad9189700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad9189700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95c01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44396/0 (socket says 172.21.15.19:44396) 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad9189700 1 -- 172.21.15.19:0/2770479680 learned_addr learned my addr 172.21.15.19:0/2770479680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed951e160 0x558ed95bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad8988700 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 0x558ed95bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad9189700 1 -- 172.21.15.19:0/2770479680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 msgr2=0x558ed95bf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad9189700 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 0x558ed95bf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad9189700 1 -- 172.21.15.19:0/2770479680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed951e160 msgr2=0x558ed95bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad9189700 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed951e160 0x558ed95bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad9189700 1 -- 172.21.15.19:0/2770479680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ed934f490 con 0x558ed95ba400 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad8988700 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 0x558ed95bf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ad9189700 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95c01b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4acc008f50 tx=0x7f4acc008f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a77ed4645166927 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ab3fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.166+0000 7f4ab3fff700 1 -- 172.21.15.19:0/2770479680 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc004bb0 con 0x558ed95ba400 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ab3fff700 1 -- 172.21.15.19:0/2770479680 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4acc004d50 con 0x558ed95ba400 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ae9602b80 1 -- 172.21.15.19:0/2770479680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4abc0a8f40 con 0x558ed95ba400 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ab3fff700 1 -- 172.21.15.19:0/2770479680 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4acc00f9b0 con 0x558ed95ba400 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ae9602b80 10 client.17022 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ae9602b80 1 -- 172.21.15.19:0/2770479680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ed95cd0b0 con 0x558ed95ba400 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ab3fff700 1 -- 172.21.15.19:0/2770479680 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc0a8f40 con 0x558ed95ba400 2024-05-09T01:31:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ab3fff700 1 -- 172.21.15.19:0/2770479680 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4acc020bc0 con 0x558ed95ba400 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ae9602b80 10 client.17022 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ae9602b80 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ae9602b80 20 client.17022 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558ed95cf740 0x558ed95d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ae9602b80 4 client.17022 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.167+0000 7f4ae9602b80 1 -- 172.21.15.19:0/2770479680 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ed94e19d0 con 0x558ed95cf740 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.168+0000 7f4ad8988700 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558ed95cf740 0x558ed95d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.168+0000 7f4ad8988700 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558ed95cf740 0x558ed95d1b60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8762baf6a83b990 server_cookie=2f7e839477e7ba71 in_seq=0 out_seq=0 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.168+0000 7f4ab3fff700 10 client.17022 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ab3fff700 1 -- 172.21.15.19:0/2770479680 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ed94e19d0 con 0x558ed95cf740 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ab3fff700 10 client.17022 handle_command_reply: tid=0 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 client.17022 shutdown 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 -- 172.21.15.19:0/2770479680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 msgr2=0x558ed95c01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95c01b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4acc008f50 tx=0x7f4acc008f80 comp rx=0 tx=0).stop 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 -- 172.21.15.19:0/2770479680 shutdown_connections 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558ed95cf740 0x558ed95d1b60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95c01b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 0x558ed95bf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/2770479680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed951e160 0x558ed95bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ab3fff700 0 client.17022 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 -- 172.21.15.19:0/2770479680 >> 172.21.15.19:0/2770479680 conn(0x558ed94946a0 msgr2=0x558ed95ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 -- 172.21.15.19:0/2770479680 shutdown_connections 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.169+0000 7f4ae9602b80 1 -- 172.21.15.19:0/2770479680 wait complete. 2024-05-09T01:31:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.170+0000 7f4ae9602b80 20 client.17022 trim_cache size 0 max 16384 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ae9602b80 1 Processor -- start 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ae9602b80 1 -- start start 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed94344a0 0x558ed95c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 0x558ed95b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad41acbf0 con 0x558ed95ba400 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad41ae730 con 0x558ed94344a0 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad41ad2c0 con 0x558ed951e160 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ad9189700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ad9189700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95bfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44406/0 (socket says 172.21.15.19:44406) 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ad9189700 1 -- 172.21.15.19:0/1471325518 learned_addr learned my addr 172.21.15.19:0/1471325518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ad8988700 1 --2- 172.21.15.19:0/1471325518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed94344a0 0x558ed95c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/1471325518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 0x558ed95b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ad9189700 1 -- 172.21.15.19:0/1471325518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 msgr2=0x558ed95b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ad9189700 1 --2- 172.21.15.19:0/1471325518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 0x558ed95b9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ad9189700 1 -- 172.21.15.19:0/1471325518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed94344a0 msgr2=0x558ed95c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.185+0000 7f4ad9189700 1 --2- 172.21.15.19:0/1471325518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed94344a0 0x558ed95c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.186+0000 7f4ad9189700 1 -- 172.21.15.19:0/1471325518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ed95a5f60 con 0x558ed95ba400 2024-05-09T01:31:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.186+0000 7f4ad9189700 1 --2- 172.21.15.19:0/1471325518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95bfca0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4acc002230 tx=0x7f4acc007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fba2f5532439e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.186+0000 7f4aca7fc700 1 -- 172.21.15.19:0/1471325518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc00fb50 con 0x558ed95ba400 2024-05-09T01:31:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.186+0000 7f4aca7fc700 1 -- 172.21.15.19:0/1471325518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4acc00fcf0 con 0x558ed95ba400 2024-05-09T01:31:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.186+0000 7f4aca7fc700 1 -- 172.21.15.19:0/1471325518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc003d40 con 0x558ed95ba400 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1471325518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 msgr2=0x558ed95bfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1471325518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95bfca0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4acc002230 tx=0x7f4acc007c70 comp rx=0 tx=0).stop 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1471325518 shutdown_connections 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1471325518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed95ba400 0x558ed95bfca0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1471325518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed94344a0 0x558ed95c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1471325518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed951e160 0x558ed95b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1471325518 >> 172.21.15.19:0/1471325518 conn(0x558ed94946a0 msgr2=0x558ed95b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1471325518 shutdown_connections 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1471325518 wait complete. 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 -- start start 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.187+0000 7f4ae9602b80 1 Processor -- start 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 0x558ed95c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed951e160 0x558ed95c3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ae9602b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 0x558ed9631eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad41ae060 con 0x558ed951e160 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad41ad990 con 0x558ed95ba400 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ae9602b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad41aab90 con 0x558ed94344a0 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ae9602b80 10 client.0 _ll_register_callbacks cb 0x558ed93456c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed951e160 0x558ed95c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed951e160 0x558ed95c3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44414/0 (socket says 172.21.15.19:44414) 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad3fff700 1 -- 172.21.15.19:0/1640642562 learned_addr learned my addr 172.21.15.19:0/1640642562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad8988700 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 0x558ed95c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad9189700 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 0x558ed9631eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad3fff700 1 -- 172.21.15.19:0/1640642562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 msgr2=0x558ed95c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 0x558ed95c2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad3fff700 1 -- 172.21.15.19:0/1640642562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 msgr2=0x558ed9631eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 0x558ed9631eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad3fff700 1 -- 172.21.15.19:0/1640642562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ed96323f0 con 0x558ed951e160 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ad3fff700 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed951e160 0x558ed95c3080 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0a4fc0 tx=0x7f4abc010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd2a1ab07a10283 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:31:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ab3fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.188+0000 7f4ab3fff700 1 -- 172.21.15.19:0/1640642562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc088060 con 0x558ed951e160 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ab3fff700 1 -- 172.21.15.19:0/1640642562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4abc0b1b20 con 0x558ed951e160 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1640642562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4acc00fcf0 con 0x558ed951e160 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ab3fff700 1 -- 172.21.15.19:0/1640642562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4abc0ad2f0 con 0x558ed951e160 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ae9602b80 10 client.17028 fetch_fsmap learned FSMap version 28 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1640642562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ed96396e0 con 0x558ed951e160 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ab3fff700 1 -- 172.21.15.19:0/1640642562 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4acc00fcf0 con 0x558ed951e160 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ab3fff700 1 -- 172.21.15.19:0/1640642562 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4abc0a48b0 con 0x558ed951e160 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ae9602b80 10 client.17028 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ae9602b80 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ae9602b80 20 client.17028 populate_metadata read hostname 'smithi019' 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558ed9639960 0x558ed963bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:31:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ae9602b80 4 client.17028 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1640642562 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ed94dba70 con 0x558ed9639960 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.189+0000 7f4ad8988700 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558ed9639960 0x558ed963bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ab3fff700 10 client.17028 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ad8988700 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558ed9639960 0x558ed963bd80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5804f08bf8b8d08 server_cookie=54e3713decb2d398 in_seq=0 out_seq=0 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ab3fff700 1 -- 172.21.15.19:0/1640642562 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ed94dba70 con 0x558ed9639960 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ab3fff700 10 client.17028 handle_command_reply: tid=0 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 client.17028 shutdown 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1640642562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed951e160 msgr2=0x558ed95c3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed951e160 0x558ed95c3080 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0a4fc0 tx=0x7f4abc010640 comp rx=0 tx=0).stop 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1640642562 shutdown_connections 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558ed9639960 0x558ed963bd80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ed95ba400 0x558ed9631eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ed94344a0 0x558ed95c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 --2- 172.21.15.19:0/1640642562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ed951e160 0x558ed95c3080 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1640642562 >> 172.21.15.19:0/1640642562 conn(0x558ed94946a0 msgr2=0x558ed95b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:56.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ab3fff700 0 client.17028 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:31:56.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1640642562 shutdown_connections 2024-05-09T01:31:56.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.190+0000 7f4ae9602b80 1 -- 172.21.15.19:0/1640642562 wait complete. 2024-05-09T01:31:56.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4ae9602b80 20 client.17028 trim_cache size 0 max 16384 2024-05-09T01:31:56.192 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 -- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ac00b2880 msgr2=0x7f4ac00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ac00b2880 0x7f4ac00b4d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009990 tx=0x7f4ac4000bc0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 -- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad4107df0 msgr2=0x7f4ad419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad4107df0 0x7f4ad419b420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00b500 tx=0x7f4abc00b810 comp rx=0 tx=0).stop 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 -- 172.21.15.19:0/3030390465 shutdown_connections 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ad4108740 0x7f4ad419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad41034c0 0x7f4ad419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4ac00b2880 0x7f4ac00b4d90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 --2- 172.21.15.19:0/3030390465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad4107df0 0x7f4ad419b420 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 -- 172.21.15.19:0/3030390465 >> 172.21.15.19:0/3030390465 conn(0x7f4ad40fedc0 msgr2=0x7f4ad406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 -- 172.21.15.19:0/3030390465 shutdown_connections 2024-05-09T01:31:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:31:56.191+0000 7f4adabec700 1 -- 172.21.15.19:0/3030390465 wait complete. 2024-05-09T01:31:56.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:56 smithi059 ceph-mon[34351]: pgmap v570: 97 pgs: 97 active+clean; 3.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:31:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:56 smithi019 ceph-mon[30158]: pgmap v570: 97 pgs: 97 active+clean; 3.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:31:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:56 smithi196 ceph-mon[35041]: pgmap v570: 97 pgs: 97 active+clean; 3.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:31:57.976 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:31:57.976 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:31:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:31:58.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:31:58 smithi019 ceph-mon[30158]: pgmap v571: 97 pgs: 97 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:31:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:31:58 smithi059 ceph-mon[34351]: pgmap v571: 97 pgs: 97 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:31:58.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:31:58 smithi196 ceph-mon[35041]: pgmap v571: 97 pgs: 97 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-05-09T01:32:00.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.054+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1867287426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f141054a0 msgr2=0x7f2f141058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:00.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.054+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1867287426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f141054a0 0x7f2f141058b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009990 tx=0x7f2f04009ca0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.054+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1867287426 shutdown_connections 2024-05-09T01:32:00.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.054+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1867287426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1410e910 0x7f2f14110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.054+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1867287426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f141054a0 0x7f2f141058b0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.054+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1867287426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f14105df0 0x7f2f1410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.054+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1867287426 >> 172.21.15.19:0/1867287426 conn(0x7f2f140fedc0 msgr2=0x7f2f141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:00.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.054+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1867287426 shutdown_connections 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.054+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1867287426 wait complete. 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f1a8b5700 1 Processor -- start 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f1a8b5700 1 -- start start 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f1a8b5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f141054a0 0x7f2f14103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f1a8b5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f14105df0 0x7f2f141036d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f1a8b5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1410e910 0x7f2f14103c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f1a8b5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f14114460 con 0x7f2f14105df0 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f1a8b5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141142e0 con 0x7f2f1410e910 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f1a8b5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141145e0 con 0x7f2f141054a0 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f0bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f14105df0 0x7f2f141036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f0bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f14105df0 0x7f2f141036d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53544/0 (socket says 172.21.15.19:53544) 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f0bfff700 1 -- 172.21.15.19:0/1904062736 learned_addr learned my addr 172.21.15.19:0/1904062736 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f18e52700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1410e910 0x7f2f14103c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.055+0000 7f2f13fff700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f141054a0 0x7f2f14103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f0bfff700 1 -- 172.21.15.19:0/1904062736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f141054a0 msgr2=0x7f2f14103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f141054a0 0x7f2f14103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f0bfff700 1 -- 172.21.15.19:0/1904062736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1410e910 msgr2=0x7f2f14103c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1410e910 0x7f2f14103c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f0bfff700 1 -- 172.21.15.19:0/1904062736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f141a9140 con 0x7f2f14105df0 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f13fff700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f141054a0 0x7f2f14103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f14105df0 0x7f2f141036d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00e970 tx=0x7f2efc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f11ffb700 1 -- 172.21.15.19:0/1904062736 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00cbd0 con 0x7f2f14105df0 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f11ffb700 1 -- 172.21.15.19:0/1904062736 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2efc00cd70 con 0x7f2f14105df0 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1904062736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f141a9430 con 0x7f2f14105df0 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f11ffb700 1 -- 172.21.15.19:0/1904062736 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc005680 con 0x7f2f14105df0 2024-05-09T01:32:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.056+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1904062736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f141a99c0 con 0x7f2f14105df0 2024-05-09T01:32:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.059+0000 7f2f11ffb700 1 -- 172.21.15.19:0/1904062736 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2efc004750 con 0x7f2f14105df0 2024-05-09T01:32:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.059+0000 7f2f11ffb700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f000b2900 0x7f2f000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f11ffb700 1 -- 172.21.15.19:0/1904062736 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2efc0e2450 con 0x7f2f14105df0 2024-05-09T01:32:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f13fff700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f000b2900 0x7f2f000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f13fff700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f000b2900 0x7f2f000b4e10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400b2e0 tx=0x7f2f04005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:00.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f292cbb80 1 Processor -- start 2024-05-09T01:32:00.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f292cbb80 1 -- start start 2024-05-09T01:32:00.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 0x557acb9e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 0x557acb9e9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb9e3480 0x557acb9e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557acb906120 con 0x557acb9e3480 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557acb8aba20 con 0x557acb8e0d80 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.060+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557acb9020e0 con 0x557acb866520 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f18e52700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb9e3480 0x557acb9e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f18e52700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb9e3480 0x557acb9e3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53558/0 (socket says 172.21.15.19:53558) 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f18e52700 1 -- 172.21.15.19:0/2405091683 learned_addr learned my addr 172.21.15.19:0/2405091683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f13fff700 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 0x557acb9e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 0x557acb9e9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f18e52700 1 -- 172.21.15.19:0/2405091683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 msgr2=0x557acb9e9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f18e52700 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 0x557acb9e9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f18e52700 1 -- 172.21.15.19:0/2405091683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 msgr2=0x557acb9e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f18e52700 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 0x557acb9e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f18e52700 1 -- 172.21.15.19:0/2405091683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557acb89c580 con 0x557acb9e3480 2024-05-09T01:32:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 0x557acb9e9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f18e52700 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb9e3480 0x557acb9e3890 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00f410 tx=0x7f2f0c00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb080f0e300c9953 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f0a7fc700 1 -- 172.21.15.19:0/2405091683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c018030 con 0x557acb9e3480 2024-05-09T01:32:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f0a7fc700 1 -- 172.21.15.19:0/2405091683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f0c004bb0 con 0x557acb9e3480 2024-05-09T01:32:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.061+0000 7f2f0a7fc700 1 -- 172.21.15.19:0/2405091683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c005020 con 0x557acb9e3480 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.062+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2405091683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb9e3480 msgr2=0x557acb9e3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.062+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb9e3480 0x557acb9e3890 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00f410 tx=0x7f2f0c00f720 comp rx=0 tx=0).stop 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.062+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2405091683 shutdown_connections 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.062+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb9e3480 0x557acb9e3890 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.062+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 0x557acb9e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.062+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2405091683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 0x557acb9e9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.062+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2405091683 >> 172.21.15.19:0/2405091683 conn(0x557acb869ff0 msgr2=0x557acb9de2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.062+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2405091683 shutdown_connections 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.062+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2405091683 wait complete. 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f292cbb80 1 -- start start 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f292cbb80 1 Processor -- start 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9ed130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 0x557acb9ed670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb9e3480 0x557acb9edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557acb9e41e0 con 0x557acb866520 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557acb8a0ba0 con 0x557acb8e0d80 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557acb9f00a0 con 0x557acb9e3480 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f292cbb80 10 client.0 _ll_register_callbacks cb 0x557acb777740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f13fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9ed130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f13fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9ed130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53566/0 (socket says 172.21.15.19:53566) 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f13fff700 1 -- 172.21.15.19:0/2893137778 learned_addr learned my addr 172.21.15.19:0/2893137778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 0x557acb9ed670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.063+0000 7f2f18e52700 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb9e3480 0x557acb9edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2f13fff700 1 -- 172.21.15.19:0/2893137778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb9e3480 msgr2=0x557acb9edbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2f13fff700 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb9e3480 0x557acb9edbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2f13fff700 1 -- 172.21.15.19:0/2893137778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 msgr2=0x557acb9ed670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2f13fff700 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 0x557acb9ed670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2f13fff700 1 -- 172.21.15.19:0/2893137778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557acb781510 con 0x557acb866520 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2f13fff700 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9ed130 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2f04015000 tx=0x7f2f04009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b2b0e0d42d0eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2ef37fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2893137778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04033060 con 0x557acb866520 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2893137778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2f0400f460 con 0x557acb866520 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2893137778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f0c004bb0 con 0x557acb866520 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2893137778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f04004e50 con 0x557acb866520 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2f292cbb80 10 client.17040 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.064+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2893137778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557acb9faa80 con 0x557acb866520 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.065+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2893137778 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f0c004bb0 con 0x557acb866520 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.065+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2893137778 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2f040236b0 con 0x557acb866520 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.065+0000 7f2f292cbb80 10 client.17040 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.065+0000 7f2f292cbb80 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.065+0000 7f2f292cbb80 20 client.17040 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:00.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.065+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557acb9fd110 0x557acb9ff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.065+0000 7f2f292cbb80 4 client.17040 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:00.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.065+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2893137778 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557acb913b00 con 0x557acb9fd110 2024-05-09T01:32:00.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.065+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557acb9fd110 0x557acb9ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.066+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557acb9fd110 0x557acb9ff550 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf7aa9391e1028b9 server_cookie=f9c663f63883db19 in_seq=0 out_seq=0 2024-05-09T01:32:00.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.066+0000 7f2ef37fe700 10 client.17040 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:00.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.066+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2893137778 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557acb913b00 con 0x557acb9fd110 2024-05-09T01:32:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.066+0000 7f2ef37fe700 10 client.17040 handle_command_reply: tid=0 2024-05-09T01:32:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 client.17040 shutdown 2024-05-09T01:32:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2893137778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 msgr2=0x557acb9ed130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9ed130 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2f04015000 tx=0x7f2f04009ca0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2893137778 shutdown_connections 2024-05-09T01:32:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557acb9fd110 0x557acb9ff550 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb9e3480 0x557acb9edbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2ef37fe700 0 client.17040 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9ed130 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2893137778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb8e0d80 0x557acb9ed670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2893137778 >> 172.21.15.19:0/2893137778 conn(0x557acb869ff0 msgr2=0x557acb9d8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2893137778 shutdown_connections 2024-05-09T01:32:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2893137778 wait complete. 2024-05-09T01:32:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.067+0000 7f2f292cbb80 20 client.17040 trim_cache size 0 max 16384 2024-05-09T01:32:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f292cbb80 1 Processor -- start 2024-05-09T01:32:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f292cbb80 1 -- start start 2024-05-09T01:32:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb8e0d80 0x557acb9e2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 0x557acb9e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b0e80 con 0x557acb866520 2024-05-09T01:32:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b29c0 con 0x557acb9e3480 2024-05-09T01:32:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b1550 con 0x557acb8e0d80 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f13fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f13fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9d8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53580/0 (socket says 172.21.15.19:53580) 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f13fff700 1 -- 172.21.15.19:0/602384882 learned_addr learned my addr 172.21.15.19:0/602384882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.088+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb8e0d80 0x557acb9e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f13fff700 1 -- 172.21.15.19:0/602384882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb8e0d80 msgr2=0x557acb9e2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f13fff700 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb8e0d80 0x557acb9e2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f18e52700 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 0x557acb9e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f13fff700 1 -- 172.21.15.19:0/602384882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 msgr2=0x557acb9e29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f13fff700 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 0x557acb9e29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f13fff700 1 -- 172.21.15.19:0/602384882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557acb9ef9d0 con 0x557acb866520 2024-05-09T01:32:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f18e52700 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 0x557acb9e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f13fff700 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9d8aa0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400bd70 tx=0x7f2f04009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500c2fa8dfea9a8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f09ffb700 1 -- 172.21.15.19:0/602384882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04023440 con 0x557acb866520 2024-05-09T01:32:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f09ffb700 1 -- 172.21.15.19:0/602384882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f040235e0 con 0x557acb866520 2024-05-09T01:32:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.089+0000 7f2f09ffb700 1 -- 172.21.15.19:0/602384882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400f460 con 0x557acb866520 2024-05-09T01:32:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 -- 172.21.15.19:0/602384882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 msgr2=0x557acb9d8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9d8aa0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400bd70 tx=0x7f2f04009940 comp rx=0 tx=0).stop 2024-05-09T01:32:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 -- 172.21.15.19:0/602384882 shutdown_connections 2024-05-09T01:32:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 0x557acb9e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb866520 0x557acb9d8aa0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/602384882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb8e0d80 0x557acb9e2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 -- 172.21.15.19:0/602384882 >> 172.21.15.19:0/602384882 conn(0x557acb869ff0 msgr2=0x557acb9ec090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 -- 172.21.15.19:0/602384882 shutdown_connections 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 -- 172.21.15.19:0/602384882 wait complete. 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.090+0000 7f2f292cbb80 1 -- start start 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f292cbb80 1 Processor -- start 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 0x557acb9f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb8e0d80 0x557acb9f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f292cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 0x557acb9f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b22f0 con 0x557acb8e0d80 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b1c20 con 0x557acb9e3480 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f292cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141aee20 con 0x557acb866520 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f292cbb80 10 client.0 _ll_register_callbacks cb 0x557acb9e3e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f0bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb8e0d80 0x557acb9f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f0bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb8e0d80 0x557acb9f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53596/0 (socket says 172.21.15.19:53596) 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f0bfff700 1 -- 172.21.15.19:0/2728490857 learned_addr learned my addr 172.21.15.19:0/2728490857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f13fff700 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 0x557acb9f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f18e52700 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 0x557acb9f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f0bfff700 1 -- 172.21.15.19:0/2728490857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 msgr2=0x557acb9f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 0x557acb9f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f0bfff700 1 -- 172.21.15.19:0/2728490857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 msgr2=0x557acb9f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 0x557acb9f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.091+0000 7f2f0bfff700 1 -- 172.21.15.19:0/2728490857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557acb9f2c20 con 0x557acb8e0d80 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f0bfff700 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb8e0d80 0x557acb9f21a0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0a6cc0 tx=0x7f2efc091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa79e9bfaa01746 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2ef37fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2728490857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc0a5040 con 0x557acb8e0d80 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2728490857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2efc08ae00 con 0x557acb8e0d80 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2728490857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f040235e0 con 0x557acb8e0d80 2024-05-09T01:32:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2728490857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2efc0aa150 con 0x557acb8e0d80 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f292cbb80 10 client.17046 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2728490857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557acba68530 con 0x557acb8e0d80 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2728490857 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f040235e0 con 0x557acb8e0d80 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2728490857 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2efc0a9cb0 con 0x557acb8e0d80 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f292cbb80 10 client.17046 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f292cbb80 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f292cbb80 20 client.17046 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557acb9fd110 0x557acba6aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f292cbb80 4 client.17046 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.092+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2728490857 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557acb90dba0 con 0x557acb9fd110 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.093+0000 7f2f13fff700 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557acb9fd110 0x557acba6aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.093+0000 7f2f13fff700 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557acb9fd110 0x557acba6aca0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da5cf66cb10ff05 server_cookie=3a3746859d11b3c0 in_seq=0 out_seq=0 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.093+0000 7f2ef37fe700 10 client.17046 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.094+0000 7f2ef37fe700 1 -- 172.21.15.19:0/2728490857 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557acb90dba0 con 0x557acb9fd110 2024-05-09T01:32:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.094+0000 7f2ef37fe700 10 client.17046 handle_command_reply: tid=0 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.094+0000 7f2f292cbb80 1 client.17046 shutdown 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.094+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2728490857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb8e0d80 msgr2=0x557acb9f21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.094+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb8e0d80 0x557acb9f21a0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0a6cc0 tx=0x7f2efc091040 comp rx=0 tx=0).stop 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2728490857 shutdown_connections 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557acb9fd110 0x557acba6aca0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557acb9e3480 0x557acb9f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2ef37fe700 0 client.17046 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557acb866520 0x557acb9f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2f292cbb80 1 --2- 172.21.15.19:0/2728490857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557acb8e0d80 0x557acb9f21a0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2728490857 >> 172.21.15.19:0/2728490857 conn(0x557acb869ff0 msgr2=0x557acb9e6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2728490857 shutdown_connections 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2f292cbb80 1 -- 172.21.15.19:0/2728490857 wait complete. 2024-05-09T01:32:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.095+0000 7f2f292cbb80 20 client.17046 trim_cache size 0 max 16384 2024-05-09T01:32:00.098 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:32:00.098 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:32:00.098 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "530f53e6-45fe-4af2-93c3-03ff457d9ace", 2024-05-09T01:32:00.098 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:32:00.098 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f000b2900 msgr2=0x7f2f000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f000b2900 0x7f2f000b4e10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400b2e0 tx=0x7f2f04005940 comp rx=0 tx=0).stop 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f14105df0 msgr2=0x7f2f141036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f14105df0 0x7f2f141036d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00e970 tx=0x7f2efc00ec80 comp rx=0 tx=0).stop 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1904062736 shutdown_connections 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1410e910 0x7f2f14103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f141054a0 0x7f2f14103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f000b2900 0x7f2f000b4e10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 --2- 172.21.15.19:0/1904062736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f14105df0 0x7f2f141036d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1904062736 >> 172.21.15.19:0/1904062736 conn(0x7f2f140fedc0 msgr2=0x7f2f140ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1904062736 shutdown_connections 2024-05-09T01:32:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:00.096+0000 7f2f1a8b5700 1 -- 172.21.15.19:0/1904062736 wait complete. 2024-05-09T01:32:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:00 smithi059 ceph-mon[34351]: pgmap v572: 97 pgs: 97 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:32:00.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:00 smithi019 ceph-mon[30158]: pgmap v572: 97 pgs: 97 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:32:00.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:00 smithi196 ceph-mon[35041]: pgmap v572: 97 pgs: 97 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:32:00.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:32:01.211 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:01.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:01 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:32:01.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:01 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:01.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:01 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:32:01.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:01 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:01 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:32:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:01 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:01 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:32:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:01 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:01.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:01 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:32:01.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:01 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:01.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:01 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:32:01.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:01 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:02.784 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:02 smithi019 ceph-mon[30158]: pgmap v573: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:32:02.784 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:02 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:32:02.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:02 smithi196 ceph-mon[35041]: pgmap v573: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:32:02.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:02 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:32:02.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:02 smithi059 ceph-mon[34351]: pgmap v573: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:32:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:02 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:32:03.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.020+0000 7f98e562c700 1 -- 172.21.15.19:0/2238706095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0108740 msgr2=0x7f98e01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.020+0000 7f98e562c700 1 --2- 172.21.15.19:0/2238706095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0108740 0x7f98e01100a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f98d800b0d0 tx=0x7f98d800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.020+0000 7f98e562c700 1 -- 172.21.15.19:0/2238706095 shutdown_connections 2024-05-09T01:32:03.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.020+0000 7f98e562c700 1 --2- 172.21.15.19:0/2238706095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e0108740 0x7f98e01100a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.020+0000 7f98e562c700 1 --2- 172.21.15.19:0/2238706095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98e01034c0 0x7f98e01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.020+0000 7f98e562c700 1 --2- 172.21.15.19:0/2238706095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e0107df0 0x7f98e0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.020+0000 7f98e562c700 1 -- 172.21.15.19:0/2238706095 >> 172.21.15.19:0/2238706095 conn(0x7f98e00fedf0 msgr2=0x7f98e0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:03.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.020+0000 7f98e562c700 1 -- 172.21.15.19:0/2238706095 shutdown_connections 2024-05-09T01:32:03.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.020+0000 7f98e562c700 1 -- 172.21.15.19:0/2238706095 wait complete. 2024-05-09T01:32:03.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.021+0000 7f98e562c700 1 Processor -- start 2024-05-09T01:32:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.021+0000 7f98e562c700 1 -- start start 2024-05-09T01:32:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.021+0000 7f98e562c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e01034c0 0x7f98e019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.021+0000 7f98e562c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98e0107df0 0x7f98e019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.021+0000 7f98e562c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e0108740 0x7f98e019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.021+0000 7f98e562c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0113370 con 0x7f98e01034c0 2024-05-09T01:32:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.021+0000 7f98e562c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01131f0 con 0x7f98e0108740 2024-05-09T01:32:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.021+0000 7f98e562c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01134f0 con 0x7f98e0107df0 2024-05-09T01:32:03.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e01034c0 0x7f98e019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e01034c0 0x7f98e019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53622/0 (socket says 172.21.15.19:53622) 2024-05-09T01:32:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98deffd700 1 -- 172.21.15.19:0/3653404457 learned_addr learned my addr 172.21.15.19:0/3653404457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98de7fc700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98e0107df0 0x7f98e019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98df7fe700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e0108740 0x7f98e019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98deffd700 1 -- 172.21.15.19:0/3653404457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98e0107df0 msgr2=0x7f98e019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98deffd700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98e0107df0 0x7f98e019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98deffd700 1 -- 172.21.15.19:0/3653404457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e0108740 msgr2=0x7f98e019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98deffd700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e0108740 0x7f98e019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98deffd700 1 -- 172.21.15.19:0/3653404457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e019fca0 con 0x7f98e01034c0 2024-05-09T01:32:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98deffd700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e01034c0 0x7f98e019add0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e9a0 tx=0x7f98d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98c7fff700 1 -- 172.21.15.19:0/3653404457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d000ca10 con 0x7f98e01034c0 2024-05-09T01:32:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98e562c700 1 -- 172.21.15.19:0/3653404457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e019ff90 con 0x7f98e01034c0 2024-05-09T01:32:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98c7fff700 1 -- 172.21.15.19:0/3653404457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d000cbb0 con 0x7f98e01034c0 2024-05-09T01:32:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.022+0000 7f98c7fff700 1 -- 172.21.15.19:0/3653404457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0018680 con 0x7f98e01034c0 2024-05-09T01:32:03.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.023+0000 7f98e562c700 1 -- 172.21.15.19:0/3653404457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e01a78b0 con 0x7f98e01034c0 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.026+0000 7f98c7fff700 1 -- 172.21.15.19:0/3653404457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f98d0018820 con 0x7f98e01034c0 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98f4042b80 1 Processor -- start 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98c7fff700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c80b2880 0x7f98c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98f4042b80 1 -- start start 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98de7fc700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c80b2880 0x7f98c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98c7fff700 1 -- 172.21.15.19:0/3653404457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f98d00e2480 con 0x7f98e01034c0 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f88954d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f889552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f88959810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98f4042b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564f88872ff0 con 0x564f888bd160 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98f4042b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f888117f0 con 0x564f887d34a0 2024-05-09T01:32:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.027+0000 7f98f4042b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f8886efb0 con 0x564f88959400 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f88954d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f88954d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53628/0 (socket says 172.21.15.19:53628) 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98deffd700 1 -- 172.21.15.19:0/1183435671 learned_addr learned my addr 172.21.15.19:0/1183435671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98de7fc700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c80b2880 0x7f98c80b4d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f98e0109890 tx=0x7f98d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98de7fc700 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f889552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98df7fe700 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f88959810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98deffd700 1 -- 172.21.15.19:0/1183435671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 msgr2=0x564f88959810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98deffd700 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f88959810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98deffd700 1 -- 172.21.15.19:0/1183435671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 msgr2=0x564f889552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98deffd700 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f889552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98deffd700 1 -- 172.21.15.19:0/1183435671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f88809370 con 0x564f888bd160 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98de7fc700 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f889552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98df7fe700 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f88959810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98deffd700 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f88954d50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f98d00a5aa0 tx=0x7f98d00b2310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0daa23ce05538a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98c57fa700 1 -- 172.21.15.19:0/1183435671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00addd0 con 0x564f888bd160 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98c57fa700 1 -- 172.21.15.19:0/1183435671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d00b2880 con 0x564f888bd160 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.028+0000 7f98c57fa700 1 -- 172.21.15.19:0/1183435671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00b2610 con 0x564f888bd160 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.029+0000 7f98f4042b80 1 -- 172.21.15.19:0/1183435671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 msgr2=0x564f88954d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.029+0000 7f98f4042b80 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f88954d50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f98d00a5aa0 tx=0x7f98d00b2310 comp rx=0 tx=0).stop 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.029+0000 7f98f4042b80 1 -- 172.21.15.19:0/1183435671 shutdown_connections 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.029+0000 7f98f4042b80 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f88959810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.029+0000 7f98f4042b80 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f88954d50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.029+0000 7f98f4042b80 1 --2- 172.21.15.19:0/1183435671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f889552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.029+0000 7f98f4042b80 1 -- 172.21.15.19:0/1183435671 >> 172.21.15.19:0/1183435671 conn(0x564f888336a0 msgr2=0x564f8894b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.029+0000 7f98f4042b80 1 -- 172.21.15.19:0/1183435671 shutdown_connections 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.029+0000 7f98f4042b80 1 -- 172.21.15.19:0/1183435671 wait complete. 2024-05-09T01:32:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98f4042b80 1 -- start start 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98f4042b80 1 Processor -- start 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f8895f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98f4042b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564f886eeba0 con 0x564f888bd160 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98f4042b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f8882d150 con 0x564f887d34a0 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98f4042b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f8895a1b0 con 0x564f88959400 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98f4042b80 10 client.0 _ll_register_callbacks cb 0x564f886e46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98de7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.030+0000 7f98de7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53644/0 (socket says 172.21.15.19:53644) 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98de7fc700 1 -- 172.21.15.19:0/3565211881 learned_addr learned my addr 172.21.15.19:0/3565211881 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98deffd700 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98df7fe700 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f8895f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98de7fc700 1 -- 172.21.15.19:0/3565211881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 msgr2=0x564f8895f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98de7fc700 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f8895f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98de7fc700 1 -- 172.21.15.19:0/3565211881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 msgr2=0x564f8895e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98de7fc700 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98de7fc700 1 -- 172.21.15.19:0/3565211881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f886ee490 con 0x564f888bd160 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98deffd700 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98de7fc700 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895ebb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f98d401e600 tx=0x7f98d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc24ef50c9266e98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98c2ff5700 1 -- 172.21.15.19:0/3565211881 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4004020 con 0x564f888bd160 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98f4042b80 1 -- 172.21.15.19:0/3565211881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d00b2880 con 0x564f888bd160 2024-05-09T01:32:03.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98c2ff5700 1 -- 172.21.15.19:0/3565211881 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f98d40041c0 con 0x564f888bd160 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98c2ff5700 1 -- 172.21.15.19:0/3565211881 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98d4004400 con 0x564f888bd160 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.031+0000 7f98f4042b80 10 client.17058 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98f4042b80 1 -- 172.21.15.19:0/3565211881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f8896bfb0 con 0x564f888bd160 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98c2ff5700 1 -- 172.21.15.19:0/3565211881 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d00b2880 con 0x564f888bd160 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98c2ff5700 1 -- 172.21.15.19:0/3565211881 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f98d4025030 con 0x564f888bd160 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98f4042b80 10 client.17058 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98f4042b80 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98f4042b80 20 client.17058 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98f4042b80 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f8896e540 0x564f88970960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98f4042b80 4 client.17058 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98f4042b80 1 -- 172.21.15.19:0/3565211881 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f888809d0 con 0x564f8896e540 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.032+0000 7f98deffd700 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f8896e540 0x564f88970960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.033+0000 7f98c2ff5700 10 client.17058 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.033+0000 7f98deffd700 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f8896e540 0x564f88970960 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c94c461638114d7 server_cookie=32bb7af71a1ea7bc in_seq=0 out_seq=0 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.033+0000 7f98c2ff5700 1 -- 172.21.15.19:0/3565211881 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f888809d0 con 0x564f8896e540 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.033+0000 7f98c2ff5700 10 client.17058 handle_command_reply: tid=0 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 client.17058 shutdown 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 -- 172.21.15.19:0/3565211881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 msgr2=0x564f8895ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895ebb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f98d401e600 tx=0x7f98d40047c0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 -- 172.21.15.19:0/3565211881 shutdown_connections 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f8896e540 0x564f88970960 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f8895f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98c2ff5700 0 client.17058 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:03.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 --2- 172.21.15.19:0/3565211881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895ebb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 -- 172.21.15.19:0/3565211881 >> 172.21.15.19:0/3565211881 conn(0x564f888336a0 msgr2=0x564f8894b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:03.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 -- 172.21.15.19:0/3565211881 shutdown_connections 2024-05-09T01:32:03.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 1 -- 172.21.15.19:0/3565211881 wait complete. 2024-05-09T01:32:03.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.034+0000 7f98f4042b80 20 client.17058 trim_cache size 0 max 16384 2024-05-09T01:32:03.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.054+0000 7f98f4042b80 1 Processor -- start 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.054+0000 7f98f4042b80 1 -- start start 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f887d34a0 0x564f889500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f888bd160 0x564f88960430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f88959400 0x564f8896a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98f4042b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01acbd0 con 0x564f88959400 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98f4042b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01ae710 con 0x564f888bd160 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98f4042b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01ad2a0 con 0x564f887d34a0 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f88959400 0x564f8896a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f88959400 0x564f8896a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53652/0 (socket says 172.21.15.19:53652) 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98df7fe700 1 -- 172.21.15.19:0/1604875115 learned_addr learned my addr 172.21.15.19:0/1604875115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98de7fc700 1 --2- 172.21.15.19:0/1604875115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f888bd160 0x564f88960430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98deffd700 1 --2- 172.21.15.19:0/1604875115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f887d34a0 0x564f889500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98df7fe700 1 -- 172.21.15.19:0/1604875115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f887d34a0 msgr2=0x564f889500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98df7fe700 1 --2- 172.21.15.19:0/1604875115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f887d34a0 0x564f889500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98df7fe700 1 -- 172.21.15.19:0/1604875115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f888bd160 msgr2=0x564f88960430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98df7fe700 1 --2- 172.21.15.19:0/1604875115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f888bd160 0x564f88960430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98df7fe700 1 -- 172.21.15.19:0/1604875115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f889589d0 con 0x564f88959400 2024-05-09T01:32:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.055+0000 7f98df7fe700 1 --2- 172.21.15.19:0/1604875115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f88959400 0x564f8896a400 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009d40 tx=0x7f98d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca865194c695f2a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.056+0000 7f98c4ff9700 1 -- 172.21.15.19:0/1604875115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d800baa0 con 0x564f88959400 2024-05-09T01:32:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.056+0000 7f98c4ff9700 1 -- 172.21.15.19:0/1604875115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d800bc40 con 0x564f88959400 2024-05-09T01:32:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.056+0000 7f98c4ff9700 1 -- 172.21.15.19:0/1604875115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8007ca0 con 0x564f88959400 2024-05-09T01:32:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.056+0000 7f98f4042b80 1 -- 172.21.15.19:0/1604875115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f88959400 msgr2=0x564f8896a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.056+0000 7f98f4042b80 1 --2- 172.21.15.19:0/1604875115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f88959400 0x564f8896a400 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009d40 tx=0x7f98d800b040 comp rx=0 tx=0).stop 2024-05-09T01:32:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.056+0000 7f98f4042b80 1 -- 172.21.15.19:0/1604875115 shutdown_connections 2024-05-09T01:32:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.056+0000 7f98f4042b80 1 --2- 172.21.15.19:0/1604875115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f88959400 0x564f8896a400 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.056+0000 7f98f4042b80 1 --2- 172.21.15.19:0/1604875115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f887d34a0 0x564f889500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.056+0000 7f98f4042b80 1 --2- 172.21.15.19:0/1604875115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f888bd160 0x564f88960430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 -- 172.21.15.19:0/1604875115 >> 172.21.15.19:0/1604875115 conn(0x564f888336a0 msgr2=0x564f8895fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 -- 172.21.15.19:0/1604875115 shutdown_connections 2024-05-09T01:32:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 -- 172.21.15.19:0/1604875115 wait complete. 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 -- start start 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 Processor -- start 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f8895c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01ae040 con 0x564f888bd160 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01ad970 con 0x564f887d34a0 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01aab70 con 0x564f88959400 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.057+0000 7f98f4042b80 10 client.0 _ll_register_callbacks cb 0x564f886e46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98de7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98deffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98de7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53664/0 (socket says 172.21.15.19:53664) 2024-05-09T01:32:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98deffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44670/0 (socket says 172.21.15.19:44670) 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98de7fc700 1 -- 172.21.15.19:0/2186752908 learned_addr learned my addr 172.21.15.19:0/2186752908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98df7fe700 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f8895c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98de7fc700 1 -- 172.21.15.19:0/2186752908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 msgr2=0x564f8895c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98de7fc700 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f8895c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98de7fc700 1 -- 172.21.15.19:0/2186752908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 msgr2=0x564f8895b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98de7fc700 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98de7fc700 1 -- 172.21.15.19:0/2186752908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f889d7df0 con 0x564f888bd160 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98deffd700 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98df7fe700 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f8895c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98de7fc700 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895bbc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x564f8894bf10 tx=0x7f98d401ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27decb5288a634f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98c2ff5700 1 -- 172.21.15.19:0/2186752908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4004830 con 0x564f888bd160 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.058+0000 7f98c2ff5700 1 -- 172.21.15.19:0/2186752908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f98d40049d0 con 0x564f888bd160 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98f4042b80 1 -- 172.21.15.19:0/2186752908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d800bc40 con 0x564f888bd160 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98c2ff5700 1 -- 172.21.15.19:0/2186752908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98d4041050 con 0x564f888bd160 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98f4042b80 10 client.17064 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98f4042b80 1 -- 172.21.15.19:0/2186752908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f889d90b0 con 0x564f888bd160 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98c2ff5700 1 -- 172.21.15.19:0/2186752908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d800bc40 con 0x564f888bd160 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98c2ff5700 1 -- 172.21.15.19:0/2186752908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f98d4025070 con 0x564f888bd160 2024-05-09T01:32:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98f4042b80 10 client.17064 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98f4042b80 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98f4042b80 20 client.17064 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98f4042b80 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f889d97f0 0x564f889dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98f4042b80 4 client.17064 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.059+0000 7f98f4042b80 1 -- 172.21.15.19:0/2186752908 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f8887aa70 con 0x564f889d97f0 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.060+0000 7f98deffd700 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f889d97f0 0x564f889dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.060+0000 7f98deffd700 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f889d97f0 0x564f889dbc10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3a619c46d5ab33 server_cookie=452744fb95a7f0bb in_seq=0 out_seq=0 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.060+0000 7f98c2ff5700 10 client.17064 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.060+0000 7f98c2ff5700 1 -- 172.21.15.19:0/2186752908 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564f8887aa70 con 0x564f889d97f0 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.060+0000 7f98c2ff5700 10 client.17064 handle_command_reply: tid=0 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.060+0000 7f98f4042b80 1 client.17064 shutdown 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.060+0000 7f98f4042b80 1 -- 172.21.15.19:0/2186752908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 msgr2=0x564f8895bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.060+0000 7f98f4042b80 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895bbc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x564f8894bf10 tx=0x7f98d401ee90 comp rx=0 tx=0).stop 2024-05-09T01:32:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98f4042b80 1 -- 172.21.15.19:0/2186752908 shutdown_connections 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98f4042b80 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564f889d97f0 0x564f889dbc10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98f4042b80 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f88959400 0x564f8895c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98f4042b80 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f887d34a0 0x564f8895b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98f4042b80 1 --2- 172.21.15.19:0/2186752908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f888bd160 0x564f8895bbc0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98c2ff5700 0 client.17064 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98f4042b80 1 -- 172.21.15.19:0/2186752908 >> 172.21.15.19:0/2186752908 conn(0x564f888336a0 msgr2=0x564f8894b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98f4042b80 1 -- 172.21.15.19:0/2186752908 shutdown_connections 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98f4042b80 1 -- 172.21.15.19:0/2186752908 wait complete. 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98f4042b80 20 client.17064 trim_cache size 0 max 16384 2024-05-09T01:32:03.064 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:32:03.065 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:32:03.065 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:32:03.065 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:32:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 -- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c80b2880 msgr2=0x7f98c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c80b2880 0x7f98c80b4d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f98e0109890 tx=0x7f98d4009260 comp rx=0 tx=0).stop 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 -- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e01034c0 msgr2=0x7f98e019add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e01034c0 0x7f98e019add0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e9a0 tx=0x7f98d000ecb0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 -- 172.21.15.19:0/3653404457 shutdown_connections 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e0108740 0x7f98e019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c80b2880 0x7f98c80b4d90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98e01034c0 0x7f98e019add0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 --2- 172.21.15.19:0/3653404457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98e0107df0 0x7f98e019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.061+0000 7f98e562c700 1 -- 172.21.15.19:0/3653404457 >> 172.21.15.19:0/3653404457 conn(0x7f98e00fedf0 msgr2=0x7f98e0075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.062+0000 7f98e562c700 1 -- 172.21.15.19:0/3653404457 shutdown_connections 2024-05-09T01:32:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:03.062+0000 7f98e562c700 1 -- 172.21.15.19:0/3653404457 wait complete. 2024-05-09T01:32:03.816 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:32:03.817 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:32:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.991+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/3255119141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c64106d70 msgr2=0x7f9c64109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.991+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/3255119141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c64106d70 0x7f9c64109250 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009900 tx=0x7f9c5402a410 comp rx=0 tx=0).stop 2024-05-09T01:32:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.991+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/3255119141 shutdown_connections 2024-05-09T01:32:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.991+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/3255119141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c641098e0 0x7f9c6410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.991+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/3255119141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c64106d70 0x7f9c64109250 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.991+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/3255119141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c64104380 0x7f9c641067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.991+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/3255119141 >> 172.21.15.19:0/3255119141 conn(0x7f9c640fdd00 msgr2=0x7f9c641001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.991+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/3255119141 shutdown_connections 2024-05-09T01:32:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.991+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/3255119141 wait complete. 2024-05-09T01:32:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.992+0000 7f9c6bd3c640 1 Processor -- start 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.992+0000 7f9c6bd3c640 1 -- start start 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6bd3c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c64104380 0x7f9c6419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6bd3c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c64106d70 0x7f9c6419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6bd3c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c641098e0 0x7f9c641a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6bd3c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6410f700 con 0x7f9c641098e0 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6bd3c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6410f580 con 0x7f9c64104380 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6bd3c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6410f880 con 0x7f9c64106d70 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6a2d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c641098e0 0x7f9c641a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6a2d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c641098e0 0x7f9c641a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53696/0 (socket says 172.21.15.19:53696) 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6a2d9640 1 -- 172.21.15.19:0/2638913041 learned_addr learned my addr 172.21.15.19:0/2638913041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c64104380 0x7f9c6419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c692d7640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c64106d70 0x7f9c6419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6a2d9640 1 -- 172.21.15.19:0/2638913041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c64106d70 msgr2=0x7f9c6419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6a2d9640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c64106d70 0x7f9c6419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6a2d9640 1 -- 172.21.15.19:0/2638913041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c64104380 msgr2=0x7f9c6419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6a2d9640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c64104380 0x7f9c6419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6a2d9640 1 -- 172.21.15.19:0/2638913041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c641a1e20 con 0x7f9c641098e0 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.993+0000 7f9c6a2d9640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c641098e0 0x7f9c641a17b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00eb10 tx=0x7f9c5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.994+0000 7f9c5affd640 1 -- 172.21.15.19:0/2638913041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c004b20 con 0x7f9c641098e0 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.994+0000 7f9c5affd640 1 -- 172.21.15.19:0/2638913041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c5c004cc0 con 0x7f9c641098e0 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.994+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/2638913041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c641a2110 con 0x7f9c641098e0 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.994+0000 7f9c5affd640 1 -- 172.21.15.19:0/2638913041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c010670 con 0x7f9c641098e0 2024-05-09T01:32:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.994+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/2638913041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c641a2660 con 0x7f9c641098e0 2024-05-09T01:32:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.997+0000 7f9c72bea740 1 Processor -- start 2024-05-09T01:32:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.997+0000 7f9c5affd640 1 -- 172.21.15.19:0/2638913041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9c5c005020 con 0x7f9c641098e0 2024-05-09T01:32:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.997+0000 7f9c72bea740 1 -- start start 2024-05-09T01:32:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.997+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07bd8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.997+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.997+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 0x561f07bdf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.997+0000 7f9c72bea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561f07b3c7c0 con 0x561f0780b2f0 2024-05-09T01:32:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.997+0000 7f9c72bea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561f07ad80e0 con 0x561f078341b0 2024-05-09T01:32:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.997+0000 7f9c72bea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f07b38780 con 0x561f078134d0 2024-05-09T01:32:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c69ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c692d7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07bd8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c69ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53700/0 (socket says 172.21.15.19:53700) 2024-05-09T01:32:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c69ad8640 1 -- 172.21.15.19:0/3572255920 learned_addr learned my addr 172.21.15.19:0/3572255920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c692d7640 1 --2- 172.21.15.19:0/3572255920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 0x561f07bdf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c69ad8640 1 -- 172.21.15.19:0/3572255920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 msgr2=0x561f07bdf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/3572255920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 0x561f07bdf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c69ad8640 1 -- 172.21.15.19:0/3572255920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 msgr2=0x561f07bd8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/3572255920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07bd8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c69ad8640 1 -- 172.21.15.19:0/3572255920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f07b017c0 con 0x561f0780b2f0 2024-05-09T01:32:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c5affd640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9c500b27e0 0x7f9c500b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c692d7640 1 --2- 172.21.15.19:0/3572255920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07bd8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:32:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/3572255920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd8a80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000bd50 tx=0x7f9c600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23221f071dbe345c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c692d7640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9c500b27e0 0x7f9c500b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c5affd640 1 -- 172.21.15.19:0/2638913041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9c5c0e27c0 con 0x7f9c641098e0 2024-05-09T01:32:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.998+0000 7f9c4ffff640 1 -- 172.21.15.19:0/3572255920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60005220 con 0x561f0780b2f0 2024-05-09T01:32:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.999+0000 7f9c4ffff640 1 -- 172.21.15.19:0/3572255920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c6000be40 con 0x561f0780b2f0 2024-05-09T01:32:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.999+0000 7f9c4ffff640 1 -- 172.21.15.19:0/3572255920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c600117c0 con 0x561f0780b2f0 2024-05-09T01:32:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.999+0000 7f9c72bea740 1 -- 172.21.15.19:0/3572255920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 msgr2=0x561f07bd8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.999+0000 7f9c692d7640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9c500b27e0 0x7f9c500b4cf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9c54000c00 tx=0x7f9c54005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:03.999+0000 7f9c72bea740 1 --2- 172.21.15.19:0/3572255920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd8a80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000bd50 tx=0x7f9c600107a0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.000+0000 7f9c72bea740 1 -- 172.21.15.19:0/3572255920 shutdown_connections 2024-05-09T01:32:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.000+0000 7f9c72bea740 1 --2- 172.21.15.19:0/3572255920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 0x561f07bdf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.000+0000 7f9c72bea740 1 --2- 172.21.15.19:0/3572255920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd8a80 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.000+0000 7f9c72bea740 1 --2- 172.21.15.19:0/3572255920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07bd8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.000+0000 7f9c72bea740 1 -- 172.21.15.19:0/3572255920 >> 172.21.15.19:0/3572255920 conn(0x561f07828300 msgr2=0x561f07bd3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.000+0000 7f9c72bea740 1 -- 172.21.15.19:0/3572255920 shutdown_connections 2024-05-09T01:32:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.000+0000 7f9c72bea740 1 -- 172.21.15.19:0/3572255920 wait complete. 2024-05-09T01:32:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.000+0000 7f9c72bea740 1 -- start start 2024-05-09T01:32:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.000+0000 7f9c72bea740 1 Processor -- start 2024-05-09T01:32:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f0780b2f0 0x561f07be21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078134d0 0x561f07be2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078341b0 0x561f07be9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c72bea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561f07be03d0 con 0x561f078341b0 2024-05-09T01:32:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c72bea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561f07ac39c0 con 0x561f078134d0 2024-05-09T01:32:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c72bea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f07be0550 con 0x561f0780b2f0 2024-05-09T01:32:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c72bea740 10 client.0 _ll_register_callbacks cb 0x561f07af0020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c6a2d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078341b0 0x561f07be9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c69ad8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f0780b2f0 0x561f07be21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c6a2d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078341b0 0x561f07be9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53714/0 (socket says 172.21.15.19:53714) 2024-05-09T01:32:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c69ad8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f0780b2f0 0x561f07be21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:45974/0 (socket says 172.21.15.19:45974) 2024-05-09T01:32:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c6a2d9640 1 -- 172.21.15.19:0/2277696337 learned_addr learned my addr 172.21.15.19:0/2277696337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c692d7640 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078134d0 0x561f07be2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c6a2d9640 1 -- 172.21.15.19:0/2277696337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f0780b2f0 msgr2=0x561f07be21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c6a2d9640 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f0780b2f0 0x561f07be21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c6a2d9640 1 -- 172.21.15.19:0/2277696337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078134d0 msgr2=0x561f07be2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.001+0000 7f9c6a2d9640 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078134d0 0x561f07be2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c6a2d9640 1 -- 172.21.15.19:0/2277696337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f07aea1f0 con 0x561f078341b0 2024-05-09T01:32:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c6a2d9640 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078341b0 0x561f07be9670 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0a5e40 tx=0x7f9c5c0a9710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5d5b2f16a87bb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/2277696337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0aa4c0 con 0x561f078341b0 2024-05-09T01:32:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/2277696337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9c5c0aa230 con 0x561f078341b0 2024-05-09T01:32:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c72bea740 1 -- 172.21.15.19:0/2277696337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561f07b4a3d0 con 0x561f078341b0 2024-05-09T01:32:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/2277696337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c5c0a9dc0 con 0x561f078341b0 2024-05-09T01:32:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c72bea740 10 client.17076 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c72bea740 1 -- 172.21.15.19:0/2277696337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f0789f790 con 0x561f078341b0 2024-05-09T01:32:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/2277696337 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561f07b4a3d0 con 0x561f078341b0 2024-05-09T01:32:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.002+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/2277696337 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9c5c0a5850 con 0x561f078341b0 2024-05-09T01:32:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.003+0000 7f9c72bea740 10 client.17076 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.003+0000 7f9c72bea740 10 client.17076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.003+0000 7f9c72bea740 20 client.17076 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.003+0000 7f9c72bea740 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561f077f2d80 0x561f07befcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.003+0000 7f9c72bea740 4 client.17076 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.003+0000 7f9c72bea740 1 -- 172.21.15.19:0/2277696337 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f07b44240 con 0x561f077f2d80 2024-05-09T01:32:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.003+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561f077f2d80 0x561f07befcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.003+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561f077f2d80 0x561f07befcf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f666ab31c5e2c7 server_cookie=161afc25b2d4ad50 in_seq=0 out_seq=0 2024-05-09T01:32:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.003+0000 7f9c4d7fa640 10 client.17076 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/2277696337 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f07b44240 con 0x561f077f2d80 2024-05-09T01:32:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c4d7fa640 10 client.17076 handle_command_reply: tid=0 2024-05-09T01:32:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c72bea740 1 client.17076 shutdown 2024-05-09T01:32:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c72bea740 1 -- 172.21.15.19:0/2277696337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078341b0 msgr2=0x561f07be9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c72bea740 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078341b0 0x561f07be9670 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0a5e40 tx=0x7f9c5c0a9710 comp rx=0 tx=0).stop 2024-05-09T01:32:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c72bea740 1 -- 172.21.15.19:0/2277696337 shutdown_connections 2024-05-09T01:32:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c72bea740 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561f077f2d80 0x561f07befcf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c72bea740 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078341b0 0x561f07be9670 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c72bea740 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078134d0 0x561f07be2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c72bea740 1 --2- 172.21.15.19:0/2277696337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f0780b2f0 0x561f07be21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c4d7fa640 0 client.17076 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.004+0000 7f9c72bea740 1 -- 172.21.15.19:0/2277696337 >> 172.21.15.19:0/2277696337 conn(0x561f07828300 msgr2=0x561f07bd2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.005+0000 7f9c72bea740 1 -- 172.21.15.19:0/2277696337 shutdown_connections 2024-05-09T01:32:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.005+0000 7f9c72bea740 1 -- 172.21.15.19:0/2277696337 wait complete. 2024-05-09T01:32:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.005+0000 7f9c72bea740 20 client.17076 trim_cache size 0 max 16384 2024-05-09T01:32:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.023+0000 7f9c72bea740 1 Processor -- start 2024-05-09T01:32:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.023+0000 7f9c72bea740 1 -- start start 2024-05-09T01:32:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.023+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.023+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 0x561f07be9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.023+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07be9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.023+0000 7f9c72bea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641ab3d0 con 0x561f0780b2f0 2024-05-09T01:32:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.023+0000 7f9c72bea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641acf10 con 0x561f078341b0 2024-05-09T01:32:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.023+0000 7f9c72bea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641abaa0 con 0x561f078134d0 2024-05-09T01:32:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c69ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c6a2d9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07be9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c69ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53724/0 (socket says 172.21.15.19:53724) 2024-05-09T01:32:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c6a2d9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07be9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44718/0 (socket says 172.21.15.19:44718) 2024-05-09T01:32:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c69ad8640 1 -- 172.21.15.19:0/675418007 learned_addr learned my addr 172.21.15.19:0/675418007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c692d7640 1 --2- 172.21.15.19:0/675418007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 0x561f07be9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c69ad8640 1 -- 172.21.15.19:0/675418007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 msgr2=0x561f07be9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/675418007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 0x561f07be9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c69ad8640 1 -- 172.21.15.19:0/675418007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 msgr2=0x561f07be9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/675418007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07be9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c69ad8640 1 -- 172.21.15.19:0/675418007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f07bdc0f0 con 0x561f0780b2f0 2024-05-09T01:32:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.024+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/675418007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd3f60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005d00 tx=0x7f9c60005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b9df40d0fb2edd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.025+0000 7f9c4f7fe640 1 -- 172.21.15.19:0/675418007 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60012240 con 0x561f0780b2f0 2024-05-09T01:32:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.025+0000 7f9c4f7fe640 1 -- 172.21.15.19:0/675418007 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c60028e50 con 0x561f0780b2f0 2024-05-09T01:32:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.025+0000 7f9c4f7fe640 1 -- 172.21.15.19:0/675418007 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60012a90 con 0x561f0780b2f0 2024-05-09T01:32:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.025+0000 7f9c72bea740 1 -- 172.21.15.19:0/675418007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 msgr2=0x561f07bd3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.025+0000 7f9c72bea740 1 --2- 172.21.15.19:0/675418007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd3f60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005d00 tx=0x7f9c60005890 comp rx=0 tx=0).stop 2024-05-09T01:32:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 -- 172.21.15.19:0/675418007 shutdown_connections 2024-05-09T01:32:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 --2- 172.21.15.19:0/675418007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f078341b0 0x561f07be9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 --2- 172.21.15.19:0/675418007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078134d0 0x561f07be9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 --2- 172.21.15.19:0/675418007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f0780b2f0 0x561f07bd3f60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 -- 172.21.15.19:0/675418007 >> 172.21.15.19:0/675418007 conn(0x561f07828300 msgr2=0x561f07bd3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 -- 172.21.15.19:0/675418007 shutdown_connections 2024-05-09T01:32:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 -- 172.21.15.19:0/675418007 wait complete. 2024-05-09T01:32:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 -- start start 2024-05-09T01:32:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 Processor -- start 2024-05-09T01:32:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f0780b2f0 0x561f07be72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.026+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078134d0 0x561f07be7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c72bea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078341b0 0x561f07be8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c72bea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641ac840 con 0x561f078134d0 2024-05-09T01:32:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c72bea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641ac170 con 0x561f0780b2f0 2024-05-09T01:32:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c72bea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641a93b0 con 0x561f078341b0 2024-05-09T01:32:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c72bea740 10 client.0 _ll_register_callbacks cb 0x7f9c5c0a5d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c692d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078134d0 0x561f07be7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c692d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078134d0 0x561f07be7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53736/0 (socket says 172.21.15.19:53736) 2024-05-09T01:32:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c692d7640 1 -- 172.21.15.19:0/1917526886 learned_addr learned my addr 172.21.15.19:0/1917526886 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f0780b2f0 0x561f07be72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c6a2d9640 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078341b0 0x561f07be8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c692d7640 1 -- 172.21.15.19:0/1917526886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078341b0 msgr2=0x561f07be8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c692d7640 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078341b0 0x561f07be8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c692d7640 1 -- 172.21.15.19:0/1917526886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f0780b2f0 msgr2=0x561f07be72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c692d7640 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f0780b2f0 0x561f07be72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c692d7640 1 -- 172.21.15.19:0/1917526886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f079165f0 con 0x561f078134d0 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c692d7640 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078134d0 0x561f07be7820 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005ab0 tx=0x7f9c54031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe91feffcbb48d43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.027+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/1917526886 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54035040 con 0x561f078134d0 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/1917526886 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9c54003dc0 con 0x561f078134d0 2024-05-09T01:32:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c72bea740 1 -- 172.21.15.19:0/1917526886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c60028e50 con 0x561f078134d0 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/1917526886 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c5405b070 con 0x561f078134d0 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c72bea740 10 client.17082 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c72bea740 1 -- 172.21.15.19:0/1917526886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f07b00ed0 con 0x561f078134d0 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/1917526886 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c60028e50 con 0x561f078134d0 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/1917526886 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9c5403a040 con 0x561f078134d0 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c72bea740 10 client.17082 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c72bea740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c72bea740 20 client.17082 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c72bea740 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561f077c81c0 0x561f07c5a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c72bea740 4 client.17082 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.028+0000 7f9c72bea740 1 -- 172.21.15.19:0/1917526886 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f07abf350 con 0x561f077c81c0 2024-05-09T01:32:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.029+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561f077c81c0 0x561f07c5a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.029+0000 7f9c4d7fa640 10 client.17082 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.029+0000 7f9c69ad8640 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561f077c81c0 0x561f07c5a3b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4a548cccc2b088 server_cookie=9e831028e32a14fd in_seq=0 out_seq=0 2024-05-09T01:32:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c4d7fa640 1 -- 172.21.15.19:0/1917526886 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561f07abf350 con 0x561f077c81c0 2024-05-09T01:32:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c4d7fa640 10 client.17082 handle_command_reply: tid=0 2024-05-09T01:32:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c72bea740 1 client.17082 shutdown 2024-05-09T01:32:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c72bea740 1 -- 172.21.15.19:0/1917526886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078134d0 msgr2=0x561f07be7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c72bea740 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078134d0 0x561f07be7820 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005ab0 tx=0x7f9c54031400 comp rx=0 tx=0).stop 2024-05-09T01:32:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c72bea740 1 -- 172.21.15.19:0/1917526886 shutdown_connections 2024-05-09T01:32:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c72bea740 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561f077c81c0 0x561f07c5a3b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c72bea740 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f078341b0 0x561f07be8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c72bea740 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f078134d0 0x561f07be7820 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c4d7fa640 0 client.17082 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c72bea740 1 --2- 172.21.15.19:0/1917526886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561f0780b2f0 0x561f07be72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.044+0000 7f9c72bea740 1 -- 172.21.15.19:0/1917526886 >> 172.21.15.19:0/1917526886 conn(0x561f07828300 msgr2=0x561f07bd2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c72bea740 1 -- 172.21.15.19:0/1917526886 shutdown_connections 2024-05-09T01:32:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c72bea740 1 -- 172.21.15.19:0/1917526886 wait complete. 2024-05-09T01:32:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c72bea740 20 client.17082 trim_cache size 0 max 16384 2024-05-09T01:32:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:32:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:32:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:32:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9c500b27e0 msgr2=0x7f9c500b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9c500b27e0 0x7f9c500b4cf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9c54000c00 tx=0x7f9c54005a20 comp rx=0 tx=0).stop 2024-05-09T01:32:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c641098e0 msgr2=0x7f9c641a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c641098e0 0x7f9c641a17b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00eb10 tx=0x7f9c5c00c2d0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/2638913041 shutdown_connections 2024-05-09T01:32:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9c500b27e0 0x7f9c500b4cf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c641098e0 0x7f9c641a17b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c64106d70 0x7f9c6419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 --2- 172.21.15.19:0/2638913041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c64104380 0x7f9c6419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/2638913041 >> 172.21.15.19:0/2638913041 conn(0x7f9c640fdd00 msgr2=0x7f9c64106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.045+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/2638913041 shutdown_connections 2024-05-09T01:32:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:04.046+0000 7f9c6bd3c640 1 -- 172.21.15.19:0/2638913041 wait complete. 2024-05-09T01:32:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:04 smithi059 ceph-mon[34351]: pgmap v574: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:32:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:04 smithi019 ceph-mon[30158]: pgmap v574: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:32:04.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:04 smithi196 ceph-mon[35041]: pgmap v574: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.758+0000 7efd74299700 1 -- 172.21.15.19:0/403038143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c108f00 msgr2=0x7efd6c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.758+0000 7efd74299700 1 --2- 172.21.15.19:0/403038143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c108f00 0x7efd6c109310 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0099c0 tx=0x7efd5c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.758+0000 7efd74299700 1 -- 172.21.15.19:0/403038143 shutdown_connections 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.758+0000 7efd74299700 1 --2- 172.21.15.19:0/403038143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd6c109850 0x7efd6c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.758+0000 7efd74299700 1 --2- 172.21.15.19:0/403038143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd6c1045d0 0x7efd6c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.758+0000 7efd74299700 1 --2- 172.21.15.19:0/403038143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c108f00 0x7efd6c109310 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.758+0000 7efd74299700 1 -- 172.21.15.19:0/403038143 >> 172.21.15.19:0/403038143 conn(0x7efd6c0ffea0 msgr2=0x7efd6c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.758+0000 7efd74299700 1 -- 172.21.15.19:0/403038143 shutdown_connections 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.758+0000 7efd74299700 1 -- 172.21.15.19:0/403038143 wait complete. 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.759+0000 7efd74299700 1 Processor -- start 2024-05-09T01:32:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.759+0000 7efd74299700 1 -- start start 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd74299700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1045d0 0x7efd6c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd74299700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd6c108f00 0x7efd6c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd74299700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd6c109850 0x7efd6c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd74299700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c114460 con 0x7efd6c1045d0 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd74299700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1142e0 con 0x7efd6c108f00 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd74299700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1145e0 con 0x7efd6c109850 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72035700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1045d0 0x7efd6c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72035700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1045d0 0x7efd6c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53750/0 (socket says 172.21.15.19:53750) 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72035700 1 -- 172.21.15.19:0/3359358165 learned_addr learned my addr 172.21.15.19:0/3359358165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd71834700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd6c108f00 0x7efd6c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72836700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd6c109850 0x7efd6c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72035700 1 -- 172.21.15.19:0/3359358165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd6c109850 msgr2=0x7efd6c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72035700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd6c109850 0x7efd6c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72035700 1 -- 172.21.15.19:0/3359358165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd6c108f00 msgr2=0x7efd6c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72035700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd6c108f00 0x7efd6c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72035700 1 -- 172.21.15.19:0/3359358165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c19fc80 con 0x7efd6c1045d0 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72836700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd6c109850 0x7efd6c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd71834700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd6c108f00 0x7efd6c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.760+0000 7efd72035700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1045d0 0x7efd6c19adb0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efd6800e9a0 tx=0x7efd6800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.761+0000 7efd637fe700 1 -- 172.21.15.19:0/3359358165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6800cbd0 con 0x7efd6c1045d0 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.761+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c19ff70 con 0x7efd6c1045d0 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.761+0000 7efd637fe700 1 -- 172.21.15.19:0/3359358165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd6800cd70 con 0x7efd6c1045d0 2024-05-09T01:32:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.761+0000 7efd637fe700 1 -- 172.21.15.19:0/3359358165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68018680 con 0x7efd6c1045d0 2024-05-09T01:32:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.761+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c1a7890 con 0x7efd6c1045d0 2024-05-09T01:32:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.762+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6c0512c0 con 0x7efd6c1045d0 2024-05-09T01:32:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.764+0000 7efd637fe700 1 -- 172.21.15.19:0/3359358165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efd68010970 con 0x7efd6c1045d0 2024-05-09T01:32:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.765+0000 7efd637fe700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efd580b2900 0x7efd580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.765+0000 7efd71834700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efd580b2900 0x7efd580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.765+0000 7efd637fe700 1 -- 172.21.15.19:0/3359358165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7efd680e2540 con 0x7efd6c1045d0 2024-05-09T01:32:05.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.765+0000 7efd71834700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efd580b2900 0x7efd580b4e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efd5c000c00 tx=0x7efd5c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:05.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:05.770+0000 7efd637fe700 1 -- 172.21.15.19:0/3359358165 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd680aa240 con 0x7efd6c1045d0 2024-05-09T01:32:06.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.030+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd6c02e5f0 con 0x7efd6c1045d0 2024-05-09T01:32:06.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.031+0000 7efd637fe700 1 -- 172.21.15.19:0/3359358165 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7efd680a9b40 con 0x7efd6c1045d0 2024-05-09T01:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:32:06.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efd580b2900 msgr2=0x7efd580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:06.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efd580b2900 0x7efd580b4e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efd5c000c00 tx=0x7efd5c01a040 comp rx=0 tx=0).stop 2024-05-09T01:32:06.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1045d0 msgr2=0x7efd6c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:06.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1045d0 0x7efd6c19adb0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efd6800e9a0 tx=0x7efd6800ed60 comp rx=0 tx=0).stop 2024-05-09T01:32:06.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 shutdown_connections 2024-05-09T01:32:06.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd6c109850 0x7efd6c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:06.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7efd580b2900 0x7efd580b4e10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:06.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c1045d0 0x7efd6c19adb0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:06.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 --2- 172.21.15.19:0/3359358165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd6c108f00 0x7efd6c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:06.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 >> 172.21.15.19:0/3359358165 conn(0x7efd6c0ffea0 msgr2=0x7efd6c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:06.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 shutdown_connections 2024-05-09T01:32:06.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:06.033+0000 7efd74299700 1 -- 172.21.15.19:0/3359358165 wait complete. 2024-05-09T01:32:06.031 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:32:06.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:06 smithi019 ceph-mon[30158]: pgmap v575: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:32:06.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:06 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3359358165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:32:06.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:32:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:06 smithi059 ceph-mon[34351]: pgmap v575: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:32:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:06 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3359358165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:32:06.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:06 smithi196 ceph-mon[35041]: pgmap v575: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:32:06.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:06 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3359358165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:32:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:08.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:08 smithi019 ceph-mon[30158]: pgmap v576: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:32:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:08 smithi059 ceph-mon[34351]: pgmap v576: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:32:08.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:08 smithi196 ceph-mon[35041]: pgmap v576: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:32:08.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.997+0000 7fc477773700 1 -- 172.21.15.19:0/3341200939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 msgr2=0x7fc4701049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:08.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.997+0000 7fc477773700 1 --2- 172.21.15.19:0/3341200939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 0x7fc4701049e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0099c0 tx=0x7fc46c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:32:08.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.997+0000 7fc477773700 1 -- 172.21.15.19:0/3341200939 shutdown_connections 2024-05-09T01:32:08.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.997+0000 7fc477773700 1 --2- 172.21.15.19:0/3341200939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc470109850 0x7fc4701111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:08.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.997+0000 7fc477773700 1 --2- 172.21.15.19:0/3341200939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 0x7fc4701049e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:08.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.997+0000 7fc477773700 1 --2- 172.21.15.19:0/3341200939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470108f00 0x7fc470109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:08.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.997+0000 7fc477773700 1 -- 172.21.15.19:0/3341200939 >> 172.21.15.19:0/3341200939 conn(0x7fc4700ffea0 msgr2=0x7fc4701022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:08.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.997+0000 7fc477773700 1 -- 172.21.15.19:0/3341200939 shutdown_connections 2024-05-09T01:32:08.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.997+0000 7fc477773700 1 -- 172.21.15.19:0/3341200939 wait complete. 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc477773700 1 Processor -- start 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc477773700 1 -- start start 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc477773700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 0x7fc47019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc477773700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470108f00 0x7fc47019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc477773700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc470109850 0x7fc47019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc477773700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701144f0 con 0x7fc4701045d0 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc477773700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc470114370 con 0x7fc470109850 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc477773700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc470114670 con 0x7fc470108f00 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc47550f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 0x7fc47019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc47550f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 0x7fc47019aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53768/0 (socket says 172.21.15.19:53768) 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc47550f700 1 -- 172.21.15.19:0/3257236370 learned_addr learned my addr 172.21.15.19:0/3257236370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc475d10700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc470109850 0x7fc47019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.998+0000 7fc474d0e700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470108f00 0x7fc47019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc47550f700 1 -- 172.21.15.19:0/3257236370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470108f00 msgr2=0x7fc47019b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc47550f700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470108f00 0x7fc47019b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc47550f700 1 -- 172.21.15.19:0/3257236370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc470109850 msgr2=0x7fc47019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc47550f700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc470109850 0x7fc47019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc47550f700 1 -- 172.21.15.19:0/3257236370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47019fd00 con 0x7fc4701045d0 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc47550f700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 0x7fc47019aee0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00b580 tx=0x7fc46c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc4667fc700 1 -- 172.21.15.19:0/3257236370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c027070 con 0x7fc4701045d0 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc4667fc700 1 -- 172.21.15.19:0/3257236370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc46c004020 con 0x7fc4701045d0 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc477773700 1 -- 172.21.15.19:0/3257236370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47019ff90 con 0x7fc4701045d0 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc4667fc700 1 -- 172.21.15.19:0/3257236370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c022e10 con 0x7fc4701045d0 2024-05-09T01:32:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:08.999+0000 7fc477773700 1 -- 172.21.15.19:0/3257236370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4701a04c0 con 0x7fc4701045d0 2024-05-09T01:32:08.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.002+0000 7fc4667fc700 1 -- 172.21.15.19:0/3257236370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc46c02a070 con 0x7fc4701045d0 2024-05-09T01:32:09.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.003+0000 7fc4667fc700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc45c0b2900 0x7fc45c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.003+0000 7fc474d0e700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc45c0b2900 0x7fc45c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.003+0000 7fc4667fc700 1 -- 172.21.15.19:0/3257236370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc46c0e3180 con 0x7fc4701045d0 2024-05-09T01:32:09.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc486189b80 1 Processor -- start 2024-05-09T01:32:09.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc486189b80 1 -- start start 2024-05-09T01:32:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc486189b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a26c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc486189b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a26ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc486189b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a2666a0 0x55d70a266ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc474d0e700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc45c0b2900 0x7fc45c0b4e10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc4701074f0 tx=0x7fc460009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc486189b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d70a188ff0 con 0x55d70a0e94a0 2024-05-09T01:32:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc486189b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d70a1277f0 con 0x55d70a1d3160 2024-05-09T01:32:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc486189b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d70a184fb0 con 0x55d70a2666a0 2024-05-09T01:32:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc474d0e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a26ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc474d0e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a26ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53776/0 (socket says 172.21.15.19:53776) 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc474d0e700 1 -- 172.21.15.19:0/2987290816 learned_addr learned my addr 172.21.15.19:0/2987290816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc475d10700 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a2666a0 0x55d70a266ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc47550f700 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a26c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc474d0e700 1 -- 172.21.15.19:0/2987290816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a2666a0 msgr2=0x55d70a266ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc474d0e700 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a2666a0 0x55d70a266ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc474d0e700 1 -- 172.21.15.19:0/2987290816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 msgr2=0x55d70a26c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc474d0e700 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a26c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc474d0e700 1 -- 172.21.15.19:0/2987290816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d70a11f370 con 0x55d70a0e94a0 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.004+0000 7fc47550f700 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a26c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.005+0000 7fc474d0e700 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a26ca30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc460009fd0 tx=0x7fc460011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d743f41a01e2b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.005+0000 7fc45b7fe700 1 -- 172.21.15.19:0/2987290816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46001b040 con 0x55d70a0e94a0 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.005+0000 7fc45b7fe700 1 -- 172.21.15.19:0/2987290816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc460024070 con 0x55d70a0e94a0 2024-05-09T01:32:09.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.005+0000 7fc45b7fe700 1 -- 172.21.15.19:0/2987290816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4600136b0 con 0x55d70a0e94a0 2024-05-09T01:32:09.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 -- 172.21.15.19:0/2987290816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 msgr2=0x55d70a26ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:09.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a26ca30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc460009fd0 tx=0x7fc460011000 comp rx=0 tx=0).stop 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 -- 172.21.15.19:0/2987290816 shutdown_connections 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a2666a0 0x55d70a266ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a26c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 --2- 172.21.15.19:0/2987290816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a26ca30 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 -- 172.21.15.19:0/2987290816 >> 172.21.15.19:0/2987290816 conn(0x55d70a1496a0 msgr2=0x55d70a261400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 -- 172.21.15.19:0/2987290816 shutdown_connections 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 -- 172.21.15.19:0/2987290816 wait complete. 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.006+0000 7fc486189b80 1 -- start start 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc486189b80 1 Processor -- start 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc486189b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a0e94a0 0x55d70a2700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc486189b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a2705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc486189b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a2666a0 0x55d70a270b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc486189b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d70a004ba0 con 0x55d70a2666a0 2024-05-09T01:32:09.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc486189b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d70a143150 con 0x55d70a1d3160 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc486189b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d70a2731d0 con 0x55d70a0e94a0 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc486189b80 10 client.0 _ll_register_callbacks cb 0x55d709ffa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc475d10700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a2666a0 0x55d70a270b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc47550f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a0e94a0 0x55d70a2700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc475d10700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a2666a0 0x55d70a270b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53778/0 (socket says 172.21.15.19:53778) 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc47550f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a0e94a0 0x55d70a2700b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:46064/0 (socket says 172.21.15.19:46064) 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc475d10700 1 -- 172.21.15.19:0/2581887770 learned_addr learned my addr 172.21.15.19:0/2581887770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.007+0000 7fc474d0e700 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a2705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc475d10700 1 -- 172.21.15.19:0/2581887770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a0e94a0 msgr2=0x55d70a2700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc475d10700 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a0e94a0 0x55d70a2700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc475d10700 1 -- 172.21.15.19:0/2581887770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 msgr2=0x55d70a2705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc475d10700 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a2705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc475d10700 1 -- 172.21.15.19:0/2581887770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70a004490 con 0x55d70a2666a0 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc475d10700 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a2666a0 0x55d70a270b30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc468012d50 tx=0x7fc468010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea72e3768fa23a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc458ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc458ff9700 1 -- 172.21.15.19:0/2581887770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46801a070 con 0x55d70a2666a0 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc458ff9700 1 -- 172.21.15.19:0/2581887770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc468010c90 con 0x55d70a2666a0 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc486189b80 1 -- 172.21.15.19:0/2581887770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc460024070 con 0x55d70a2666a0 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc458ff9700 1 -- 172.21.15.19:0/2581887770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc468025050 con 0x55d70a2666a0 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc486189b80 10 client.17100 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.008+0000 7fc486189b80 1 -- 172.21.15.19:0/2581887770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d70a27dbb0 con 0x55d70a2666a0 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.009+0000 7fc458ff9700 1 -- 172.21.15.19:0/2581887770 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc460024070 con 0x55d70a2666a0 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.009+0000 7fc458ff9700 1 -- 172.21.15.19:0/2581887770 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc4680143a0 con 0x55d70a2666a0 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.009+0000 7fc486189b80 10 client.17100 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.009+0000 7fc486189b80 10 client.17100 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.009+0000 7fc486189b80 20 client.17100 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.009+0000 7fc486189b80 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55d70a280140 0x55d70a282590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.009+0000 7fc486189b80 4 client.17100 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:09.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.009+0000 7fc486189b80 1 -- 172.21.15.19:0/2581887770 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d70a1969d0 con 0x55d70a280140 2024-05-09T01:32:09.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.009+0000 7fc47550f700 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55d70a280140 0x55d70a282590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.010+0000 7fc47550f700 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55d70a280140 0x55d70a282590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee561248f0406ab6 server_cookie=53c607f48d51c6b4 in_seq=0 out_seq=0 2024-05-09T01:32:09.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.010+0000 7fc458ff9700 10 client.17100 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.010+0000 7fc458ff9700 1 -- 172.21.15.19:0/2581887770 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d70a1969d0 con 0x55d70a280140 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.010+0000 7fc458ff9700 10 client.17100 handle_command_reply: tid=0 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.010+0000 7fc486189b80 1 client.17100 shutdown 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 -- 172.21.15.19:0/2581887770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a2666a0 msgr2=0x55d70a270b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a2666a0 0x55d70a270b30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc468012d50 tx=0x7fc468010520 comp rx=0 tx=0).stop 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 -- 172.21.15.19:0/2581887770 shutdown_connections 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55d70a280140 0x55d70a282590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a2666a0 0x55d70a270b30 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a0e94a0 0x55d70a2700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc458ff9700 0 client.17100 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 --2- 172.21.15.19:0/2581887770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a2705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 -- 172.21.15.19:0/2581887770 >> 172.21.15.19:0/2581887770 conn(0x55d70a1496a0 msgr2=0x55d70a2611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:09.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 -- 172.21.15.19:0/2581887770 shutdown_connections 2024-05-09T01:32:09.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 1 -- 172.21.15.19:0/2581887770 wait complete. 2024-05-09T01:32:09.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.011+0000 7fc486189b80 20 client.17100 trim_cache size 0 max 16384 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc486189b80 1 Processor -- start 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc486189b80 1 -- start start 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc486189b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1496a0 0x55d70a2660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc486189b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a270970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc486189b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a268e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc486189b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701b0d40 con 0x55d70a0e94a0 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc486189b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701b2880 con 0x55d70a1d3160 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc486189b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701b1410 con 0x55d70a1496a0 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc474d0e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a270970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc474d0e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a270970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53780/0 (socket says 172.21.15.19:53780) 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.026+0000 7fc474d0e700 1 -- 172.21.15.19:0/2122113305 learned_addr learned my addr 172.21.15.19:0/2122113305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc475d10700 1 --2- 172.21.15.19:0/2122113305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a268e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc47550f700 1 --2- 172.21.15.19:0/2122113305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1496a0 0x55d70a2660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc474d0e700 1 -- 172.21.15.19:0/2122113305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1496a0 msgr2=0x55d70a2660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc474d0e700 1 --2- 172.21.15.19:0/2122113305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1496a0 0x55d70a2660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc474d0e700 1 -- 172.21.15.19:0/2122113305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 msgr2=0x55d70a268e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc474d0e700 1 --2- 172.21.15.19:0/2122113305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a268e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc474d0e700 1 -- 172.21.15.19:0/2122113305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d70a272950 con 0x55d70a0e94a0 2024-05-09T01:32:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc474d0e700 1 --2- 172.21.15.19:0/2122113305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a270970 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc460006e20 tx=0x7fc46000e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407f56d24e3dd775 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc45affd700 1 -- 172.21.15.19:0/2122113305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46000ee60 con 0x55d70a0e94a0 2024-05-09T01:32:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc45affd700 1 -- 172.21.15.19:0/2122113305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc460024040 con 0x55d70a0e94a0 2024-05-09T01:32:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.027+0000 7fc45affd700 1 -- 172.21.15.19:0/2122113305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460029b90 con 0x55d70a0e94a0 2024-05-09T01:32:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 -- 172.21.15.19:0/2122113305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 msgr2=0x55d70a270970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 --2- 172.21.15.19:0/2122113305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a270970 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc460006e20 tx=0x7fc46000e960 comp rx=0 tx=0).stop 2024-05-09T01:32:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 -- 172.21.15.19:0/2122113305 shutdown_connections 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 --2- 172.21.15.19:0/2122113305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a1d3160 0x55d70a268e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 --2- 172.21.15.19:0/2122113305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1496a0 0x55d70a2660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 --2- 172.21.15.19:0/2122113305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a0e94a0 0x55d70a270970 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 -- 172.21.15.19:0/2122113305 >> 172.21.15.19:0/2122113305 conn(0x55d70a266e10 msgr2=0x55d70a271760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 -- 172.21.15.19:0/2122113305 shutdown_connections 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 -- 172.21.15.19:0/2122113305 wait complete. 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.028+0000 7fc486189b80 1 -- start start 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc486189b80 1 Processor -- start 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc486189b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a0e94a0 0x55d70a26cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc486189b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a1496a0 0x55d70a26d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc486189b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1d3160 0x55d70a26d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc486189b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701b21b0 con 0x55d70a1496a0 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc486189b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701b1ae0 con 0x55d70a0e94a0 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc486189b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701aece0 con 0x55d70a1d3160 2024-05-09T01:32:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc486189b80 10 client.0 _ll_register_callbacks cb 0x55d709ffa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc474d0e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a1496a0 0x55d70a26d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc474d0e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a1496a0 0x55d70a26d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53796/0 (socket says 172.21.15.19:53796) 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc474d0e700 1 -- 172.21.15.19:0/1535691435 learned_addr learned my addr 172.21.15.19:0/1535691435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc47550f700 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a0e94a0 0x55d70a26cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.029+0000 7fc475d10700 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1d3160 0x55d70a26d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc474d0e700 1 -- 172.21.15.19:0/1535691435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1d3160 msgr2=0x55d70a26d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc474d0e700 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1d3160 0x55d70a26d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc474d0e700 1 -- 172.21.15.19:0/1535691435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a0e94a0 msgr2=0x55d70a26cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc474d0e700 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a0e94a0 0x55d70a26cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc474d0e700 1 -- 172.21.15.19:0/1535691435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70a2767d0 con 0x55d70a1496a0 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc474d0e700 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a1496a0 0x55d70a26d4b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc4600097f0 tx=0x7fc46002acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a83acd18b808c80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc458ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc458ff9700 1 -- 172.21.15.19:0/1535691435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460042060 con 0x55d70a1496a0 2024-05-09T01:32:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc458ff9700 1 -- 172.21.15.19:0/1535691435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc460015500 con 0x55d70a1496a0 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc486189b80 1 -- 172.21.15.19:0/1535691435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc460024040 con 0x55d70a1496a0 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc458ff9700 1 -- 172.21.15.19:0/1535691435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc460015700 con 0x55d70a1496a0 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc486189b80 10 client.17106 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc486189b80 1 -- 172.21.15.19:0/1535691435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d70a2eab60 con 0x55d70a1496a0 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.030+0000 7fc458ff9700 1 -- 172.21.15.19:0/1535691435 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc460024040 con 0x55d70a1496a0 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.031+0000 7fc458ff9700 1 -- 172.21.15.19:0/1535691435 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc46001e030 con 0x55d70a1496a0 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.031+0000 7fc486189b80 10 client.17106 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.031+0000 7fc486189b80 10 client.17106 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.031+0000 7fc486189b80 20 client.17106 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.031+0000 7fc486189b80 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55d70a2666a0 0x55d70a2ecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.031+0000 7fc486189b80 4 client.17106 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.031+0000 7fc486189b80 1 -- 172.21.15.19:0/1535691435 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d70a190a70 con 0x55d70a2666a0 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.031+0000 7fc47550f700 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55d70a2666a0 0x55d70a2ecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc47550f700 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55d70a2666a0 0x55d70a2ecec0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=610fb9459a5bca65 server_cookie=70b89a280c40c36e in_seq=0 out_seq=0 2024-05-09T01:32:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc458ff9700 10 client.17106 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:32:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc458ff9700 1 -- 172.21.15.19:0/1535691435 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d70a190a70 con 0x55d70a2666a0 2024-05-09T01:32:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc458ff9700 10 client.17106 handle_command_reply: tid=0 2024-05-09T01:32:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 client.17106 shutdown 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 -- 172.21.15.19:0/1535691435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a1496a0 msgr2=0x55d70a26d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a1496a0 0x55d70a26d4b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc4600097f0 tx=0x7fc46002acb0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 -- 172.21.15.19:0/1535691435 shutdown_connections 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55d70a2666a0 0x55d70a2ecec0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d70a1d3160 0x55d70a26d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc458ff9700 0 client.17106 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d70a0e94a0 0x55d70a26cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 --2- 172.21.15.19:0/1535691435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d70a1496a0 0x55d70a26d4b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 -- 172.21.15.19:0/1535691435 >> 172.21.15.19:0/1535691435 conn(0x55d70a266e10 msgr2=0x55d70a261310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 -- 172.21.15.19:0/1535691435 shutdown_connections 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 1 -- 172.21.15.19:0/1535691435 wait complete. 2024-05-09T01:32:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.032+0000 7fc486189b80 20 client.17106 trim_cache size 0 max 16384 2024-05-09T01:32:09.037 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:32:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 -- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc45c0b2900 msgr2=0x7fc45c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc45c0b2900 0x7fc45c0b4e10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc4701074f0 tx=0x7fc460009190 comp rx=0 tx=0).stop 2024-05-09T01:32:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 -- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 msgr2=0x7fc47019aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 0x7fc47019aee0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00b580 tx=0x7fc46c0047b0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 -- 172.21.15.19:0/3257236370 shutdown_connections 2024-05-09T01:32:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc470109850 0x7fc47019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc45c0b2900 0x7fc45c0b4e10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4701045d0 0x7fc47019aee0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 --2- 172.21.15.19:0/3257236370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470108f00 0x7fc47019b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 -- 172.21.15.19:0/3257236370 >> 172.21.15.19:0/3257236370 conn(0x7fc4700ffea0 msgr2=0x7fc470102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 -- 172.21.15.19:0/3257236370 shutdown_connections 2024-05-09T01:32:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:09.033+0000 7fc477773700 1 -- 172.21.15.19:0/3257236370 wait complete. 2024-05-09T01:32:09.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:32:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:10.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:10 smithi059 ceph-mon[34351]: pgmap v577: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:32:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:10 smithi019 ceph-mon[30158]: pgmap v577: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:32:10.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:10 smithi196 ceph-mon[35041]: pgmap v577: 97 pgs: 97 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:32:11.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.788+0000 7f70298c3700 1 -- 172.21.15.19:0/3383505615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 msgr2=0x7f70241111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:11.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.788+0000 7f70298c3700 1 --2- 172.21.15.19:0/3383505615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 0x7f70241111b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f701c00b0d0 tx=0x7f701c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.788+0000 7f70298c3700 1 -- 172.21.15.19:0/3383505615 shutdown_connections 2024-05-09T01:32:11.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.788+0000 7f70298c3700 1 --2- 172.21.15.19:0/3383505615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 0x7f70241111b0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.788+0000 7f70298c3700 1 --2- 172.21.15.19:0/3383505615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241045d0 0x7f70241049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.788+0000 7f70298c3700 1 --2- 172.21.15.19:0/3383505615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024108f00 0x7f7024109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.788+0000 7f70298c3700 1 -- 172.21.15.19:0/3383505615 >> 172.21.15.19:0/3383505615 conn(0x7f70240ffea0 msgr2=0x7f70241022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:11.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.788+0000 7f70298c3700 1 -- 172.21.15.19:0/3383505615 shutdown_connections 2024-05-09T01:32:11.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.788+0000 7f70298c3700 1 -- 172.21.15.19:0/3383505615 wait complete. 2024-05-09T01:32:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.789+0000 7f70298c3700 1 Processor -- start 2024-05-09T01:32:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.789+0000 7f70298c3700 1 -- start start 2024-05-09T01:32:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70298c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241045d0 0x7f702419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70298c3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024108f00 0x7f702419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70298c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 0x7f702419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70298c3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7024114480 con 0x7f7024109850 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70298c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7024114300 con 0x7f70241045d0 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70298c3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7024114600 con 0x7f7024108f00 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 0x7f702419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 0x7f702419f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52672/0 (socket says 172.21.15.19:52672) 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70237fe700 1 -- 172.21.15.19:0/3897094049 learned_addr learned my addr 172.21.15.19:0/3897094049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70227fc700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024108f00 0x7f702419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f7022ffd700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241045d0 0x7f702419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70237fe700 1 -- 172.21.15.19:0/3897094049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024108f00 msgr2=0x7f702419b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70237fe700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024108f00 0x7f702419b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70237fe700 1 -- 172.21.15.19:0/3897094049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241045d0 msgr2=0x7f702419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70237fe700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241045d0 0x7f702419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70237fe700 1 -- 172.21.15.19:0/3897094049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702419fc40 con 0x7f7024109850 2024-05-09T01:32:11.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.790+0000 7f70237fe700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 0x7f702419f5a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f701c0095c0 tx=0x7f701c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.791+0000 7f70288c1700 1 -- 172.21.15.19:0/3897094049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c01f070 con 0x7f7024109850 2024-05-09T01:32:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.791+0000 7f70298c3700 1 -- 172.21.15.19:0/3897094049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702419fed0 con 0x7f7024109850 2024-05-09T01:32:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.791+0000 7f70288c1700 1 -- 172.21.15.19:0/3897094049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c007e20 con 0x7f7024109850 2024-05-09T01:32:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.791+0000 7f70288c1700 1 -- 172.21.15.19:0/3897094049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c01b4a0 con 0x7f7024109850 2024-05-09T01:32:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.791+0000 7f70298c3700 1 -- 172.21.15.19:0/3897094049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70241a03b0 con 0x7f7024109850 2024-05-09T01:32:11.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.793+0000 7f70288c1700 1 -- 172.21.15.19:0/3897094049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f701c022070 con 0x7f7024109850 2024-05-09T01:32:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.795+0000 7f70288c1700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70100b2880 0x7f70100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.795+0000 7f7022ffd700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70100b2880 0x7f70100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.795+0000 7f70288c1700 1 -- 172.21.15.19:0/3897094049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f701c00f020 con 0x7f7024109850 2024-05-09T01:32:11.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.795+0000 7f70382d9b80 1 Processor -- start 2024-05-09T01:32:11.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.795+0000 7f70382d9b80 1 -- start start 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.795+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b7684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.795+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b7646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b765150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f70382d9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b686ff0 con 0x55ac9b6d1160 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f70382d9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b6257f0 con 0x55ac9b764c80 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f70382d9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b682fb0 con 0x55ac9b5e74a0 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b7684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b7684f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52678/0 (socket says 172.21.15.19:52678) 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 -- 172.21.15.19:0/385675592 learned_addr learned my addr 172.21.15.19:0/385675592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f70237fe700 1 --2- 172.21.15.19:0/385675592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b765150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70100b2880 0x7f70100b4d90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7014005950 tx=0x7f701400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:11.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f70227fc700 1 --2- 172.21.15.19:0/385675592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b7646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 -- 172.21.15.19:0/385675592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 msgr2=0x55ac9b7646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 --2- 172.21.15.19:0/385675592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b7646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 -- 172.21.15.19:0/385675592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 msgr2=0x55ac9b765150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 --2- 172.21.15.19:0/385675592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b765150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 -- 172.21.15.19:0/385675592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9b61d370 con 0x55ac9b6d1160 2024-05-09T01:32:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.796+0000 7f7022ffd700 1 --2- 172.21.15.19:0/385675592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b7684f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7014010fa0 tx=0x7f7014013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ab6c1899e199ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.797+0000 7f7019ffb700 1 -- 172.21.15.19:0/385675592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701400be60 con 0x55ac9b6d1160 2024-05-09T01:32:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.797+0000 7f7019ffb700 1 -- 172.21.15.19:0/385675592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7014011db0 con 0x55ac9b6d1160 2024-05-09T01:32:11.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.797+0000 7f7019ffb700 1 -- 172.21.15.19:0/385675592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014026800 con 0x55ac9b6d1160 2024-05-09T01:32:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.797+0000 7f70382d9b80 1 -- 172.21.15.19:0/385675592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 msgr2=0x55ac9b7684f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.797+0000 7f70382d9b80 1 --2- 172.21.15.19:0/385675592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b7684f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7014010fa0 tx=0x7f7014013000 comp rx=0 tx=0).stop 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.798+0000 7f70382d9b80 1 -- 172.21.15.19:0/385675592 shutdown_connections 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.798+0000 7f70382d9b80 1 --2- 172.21.15.19:0/385675592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b765150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.798+0000 7f70382d9b80 1 --2- 172.21.15.19:0/385675592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b7684f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.798+0000 7f70382d9b80 1 --2- 172.21.15.19:0/385675592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b7646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.798+0000 7f70382d9b80 1 -- 172.21.15.19:0/385675592 >> 172.21.15.19:0/385675592 conn(0x55ac9b6476a0 msgr2=0x55ac9b75f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.798+0000 7f70382d9b80 1 -- 172.21.15.19:0/385675592 shutdown_connections 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.798+0000 7f70382d9b80 1 -- 172.21.15.19:0/385675592 wait complete. 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.798+0000 7f70382d9b80 1 -- start start 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70382d9b80 1 Processor -- start 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b776450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b776990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b776ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70382d9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b502ba0 con 0x55ac9b6d1160 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70382d9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b641150 con 0x55ac9b764c80 2024-05-09T01:32:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70382d9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b779850 con 0x55ac9b5e74a0 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70382d9b80 10 client.0 _ll_register_callbacks cb 0x55ac9b4f86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f7022ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b776450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b776990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f7022ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b776450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40340/0 (socket says 172.21.15.19:40340) 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f7022ffd700 1 -- 172.21.15.19:0/858707766 learned_addr learned my addr 172.21.15.19:0/858707766 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f70237fe700 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b776ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f7022ffd700 1 -- 172.21.15.19:0/858707766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 msgr2=0x55ac9b776ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f7022ffd700 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b776ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f7022ffd700 1 -- 172.21.15.19:0/858707766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 msgr2=0x55ac9b776990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f7022ffd700 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b776990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.799+0000 7f7022ffd700 1 -- 172.21.15.19:0/858707766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9b502490 con 0x55ac9b5e74a0 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.800+0000 7f70227fc700 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b776990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.800+0000 7f7022ffd700 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b776450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7014005350 tx=0x7f7014005380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=348876810416fd79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.800+0000 7f7002ffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.800+0000 7f7002ffd700 1 -- 172.21.15.19:0/858707766 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701401ecd0 con 0x55ac9b5e74a0 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.800+0000 7f70382d9b80 1 -- 172.21.15.19:0/858707766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7014011db0 con 0x55ac9b5e74a0 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.800+0000 7f7002ffd700 1 -- 172.21.15.19:0/858707766 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f701401e630 con 0x55ac9b5e74a0 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.800+0000 7f7002ffd700 1 -- 172.21.15.19:0/858707766 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7014005690 con 0x55ac9b5e74a0 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.800+0000 7f70382d9b80 10 client.26509 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.800+0000 7f70382d9b80 1 -- 172.21.15.19:0/858707766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac9b7843a0 con 0x55ac9b5e74a0 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.801+0000 7f7002ffd700 1 -- 172.21.15.19:0/858707766 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7014011db0 con 0x55ac9b5e74a0 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.801+0000 7f7002ffd700 1 -- 172.21.15.19:0/858707766 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7014031420 con 0x55ac9b5e74a0 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.801+0000 7f70382d9b80 10 client.26509 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.801+0000 7f70382d9b80 10 client.26509 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.801+0000 7f70382d9b80 20 client.26509 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.801+0000 7f70382d9b80 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac9b7868d0 0x55ac9b788cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.801+0000 7f70382d9b80 4 client.26509 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.801+0000 7f70382d9b80 1 -- 172.21.15.19:0/858707766 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac9b6949d0 con 0x55ac9b7868d0 2024-05-09T01:32:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.801+0000 7f70227fc700 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac9b7868d0 0x55ac9b788cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.802+0000 7f7002ffd700 10 client.26509 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:32:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.802+0000 7f70227fc700 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac9b7868d0 0x55ac9b788cf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=82cac4519114c5c3 server_cookie=f9364292cdc44942 in_seq=0 out_seq=0 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.802+0000 7f7002ffd700 1 -- 172.21.15.19:0/858707766 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac9b6949d0 con 0x55ac9b7868d0 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.802+0000 7f7002ffd700 10 client.26509 handle_command_reply: tid=0 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.802+0000 7f70382d9b80 1 client.26509 shutdown 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.802+0000 7f70382d9b80 1 -- 172.21.15.19:0/858707766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 msgr2=0x55ac9b776450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.802+0000 7f70382d9b80 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b776450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7014005350 tx=0x7f7014005380 comp rx=0 tx=0).stop 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f70382d9b80 1 -- 172.21.15.19:0/858707766 shutdown_connections 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f70382d9b80 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac9b7868d0 0x55ac9b788cf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f70382d9b80 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b776ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f70382d9b80 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b776450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f7002ffd700 0 client.26509 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:32:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f70382d9b80 1 --2- 172.21.15.19:0/858707766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b776990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f70382d9b80 1 -- 172.21.15.19:0/858707766 >> 172.21.15.19:0/858707766 conn(0x55ac9b6476a0 msgr2=0x55ac9b75f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f70382d9b80 1 -- 172.21.15.19:0/858707766 shutdown_connections 2024-05-09T01:32:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f70382d9b80 1 -- 172.21.15.19:0/858707766 wait complete. 2024-05-09T01:32:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.803+0000 7f70382d9b80 20 client.26509 trim_cache size 0 max 16384 2024-05-09T01:32:11.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.818+0000 7f70382d9b80 1 Processor -- start 2024-05-09T01:32:11.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.818+0000 7f70382d9b80 1 -- start start 2024-05-09T01:32:11.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.818+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b5e74a0 0x55ac9b778530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.818+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b6d1160 0x55ac9b776d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.818+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b764c80 0x55ac9b782760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.818+0000 7f70382d9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70241ac9c0 con 0x55ac9b764c80 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.818+0000 7f70382d9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70241ae500 con 0x55ac9b5e74a0 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.818+0000 7f70382d9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70241ad090 con 0x55ac9b6d1160 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b764c80 0x55ac9b782760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b764c80 0x55ac9b782760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52704/0 (socket says 172.21.15.19:52704) 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70237fe700 1 -- 172.21.15.19:0/3817053690 learned_addr learned my addr 172.21.15.19:0/3817053690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f7022ffd700 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b5e74a0 0x55ac9b778530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70227fc700 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b6d1160 0x55ac9b776d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70237fe700 1 -- 172.21.15.19:0/3817053690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b6d1160 msgr2=0x55ac9b776d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70237fe700 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b6d1160 0x55ac9b776d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70237fe700 1 -- 172.21.15.19:0/3817053690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b5e74a0 msgr2=0x55ac9b778530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70237fe700 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b5e74a0 0x55ac9b778530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70237fe700 1 -- 172.21.15.19:0/3817053690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9b7749c0 con 0x55ac9b764c80 2024-05-09T01:32:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70227fc700 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b6d1160 0x55ac9b776d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70237fe700 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b764c80 0x55ac9b782760 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f701c00d010 tx=0x7f701c0aa310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeccdb17db59ac5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70197fa700 1 -- 172.21.15.19:0/3817053690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0c4070 con 0x55ac9b764c80 2024-05-09T01:32:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70197fa700 1 -- 172.21.15.19:0/3817053690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c0aa5b0 con 0x55ac9b764c80 2024-05-09T01:32:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.819+0000 7f70197fa700 1 -- 172.21.15.19:0/3817053690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0bfe60 con 0x55ac9b764c80 2024-05-09T01:32:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.820+0000 7f70382d9b80 1 -- 172.21.15.19:0/3817053690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b764c80 msgr2=0x55ac9b782760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.820+0000 7f70382d9b80 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b764c80 0x55ac9b782760 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f701c00d010 tx=0x7f701c0aa310 comp rx=0 tx=0).stop 2024-05-09T01:32:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.820+0000 7f70382d9b80 1 -- 172.21.15.19:0/3817053690 shutdown_connections 2024-05-09T01:32:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.820+0000 7f70382d9b80 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b764c80 0x55ac9b782760 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.820+0000 7f70382d9b80 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b5e74a0 0x55ac9b778530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.820+0000 7f70382d9b80 1 --2- 172.21.15.19:0/3817053690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b6d1160 0x55ac9b776d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.820+0000 7f70382d9b80 1 -- 172.21.15.19:0/3817053690 >> 172.21.15.19:0/3817053690 conn(0x55ac9b6476a0 msgr2=0x55ac9b7753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.820+0000 7f70382d9b80 1 -- 172.21.15.19:0/3817053690 shutdown_connections 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.820+0000 7f70382d9b80 1 -- 172.21.15.19:0/3817053690 wait complete. 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70382d9b80 1 -- start start 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70382d9b80 1 Processor -- start 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b77ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b77efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70382d9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b77a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70382d9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70241ade30 con 0x55ac9b6d1160 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70382d9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70241ad760 con 0x55ac9b764c80 2024-05-09T01:32:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70382d9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70241aa960 con 0x55ac9b5e74a0 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70382d9b80 10 client.0 _ll_register_callbacks cb 0x55ac9b4f86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b77efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b77efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52720/0 (socket says 172.21.15.19:52720) 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70227fc700 1 -- 172.21.15.19:0/4153470989 learned_addr learned my addr 172.21.15.19:0/4153470989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.821+0000 7f70237fe700 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b77a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f7022ffd700 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b77ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70227fc700 1 -- 172.21.15.19:0/4153470989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 msgr2=0x55ac9b77ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70227fc700 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b77ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70227fc700 1 -- 172.21.15.19:0/4153470989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 msgr2=0x55ac9b77a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70227fc700 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b77a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70227fc700 1 -- 172.21.15.19:0/4153470989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9b77a6b0 con 0x55ac9b6d1160 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f7022ffd700 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b77ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70237fe700 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b77a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70227fc700 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b77efa0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f700c0041d0 tx=0x7f700c00bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3b2068eaad77f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f7002ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f7002ffd700 1 -- 172.21.15.19:0/4153470989 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c023060 con 0x55ac9b6d1160 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70382d9b80 1 -- 172.21.15.19:0/4153470989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f701c0aa5b0 con 0x55ac9b6d1160 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f7002ffd700 1 -- 172.21.15.19:0/4153470989 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f700c01d430 con 0x55ac9b6d1160 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f7002ffd700 1 -- 172.21.15.19:0/4153470989 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f700c003e70 con 0x55ac9b6d1160 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70382d9b80 10 client.17124 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.822+0000 7f70382d9b80 1 -- 172.21.15.19:0/4153470989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac9b7f0950 con 0x55ac9b6d1160 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.823+0000 7f7002ffd700 1 -- 172.21.15.19:0/4153470989 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c0aa5b0 con 0x55ac9b6d1160 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.823+0000 7f7002ffd700 1 -- 172.21.15.19:0/4153470989 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f700c011ae0 con 0x55ac9b6d1160 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.823+0000 7f70382d9b80 10 client.17124 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.823+0000 7f70382d9b80 10 client.17124 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.823+0000 7f70382d9b80 20 client.17124 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.823+0000 7f70382d9b80 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac9b7f1010 0x55ac9b7f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.823+0000 7f70382d9b80 4 client.17124 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.823+0000 7f70382d9b80 1 -- 172.21.15.19:0/4153470989 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac9b68ea70 con 0x55ac9b7f1010 2024-05-09T01:32:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.823+0000 7f7022ffd700 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac9b7f1010 0x55ac9b7f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f7022ffd700 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac9b7f1010 0x55ac9b7f3430 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a13f3aa1f75bfb9 server_cookie=1d1ef460420064eb in_seq=0 out_seq=0 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f7002ffd700 10 client.17124 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f7002ffd700 1 -- 172.21.15.19:0/4153470989 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac9b68ea70 con 0x55ac9b7f1010 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f7002ffd700 10 client.17124 handle_command_reply: tid=0 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 client.17124 shutdown 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 -- 172.21.15.19:0/4153470989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 msgr2=0x55ac9b77efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b77efa0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f700c0041d0 tx=0x7f700c00bc30 comp rx=0 tx=0).stop 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 -- 172.21.15.19:0/4153470989 shutdown_connections 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55ac9b7f1010 0x55ac9b7f3430 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9b764c80 0x55ac9b77a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac9b5e74a0 0x55ac9b77ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f7002ffd700 0 client.17124 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:32:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 --2- 172.21.15.19:0/4153470989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9b6d1160 0x55ac9b77efa0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 -- 172.21.15.19:0/4153470989 >> 172.21.15.19:0/4153470989 conn(0x55ac9b6476a0 msgr2=0x55ac9b75f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 -- 172.21.15.19:0/4153470989 shutdown_connections 2024-05-09T01:32:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.824+0000 7f70382d9b80 1 -- 172.21.15.19:0/4153470989 wait complete. 2024-05-09T01:32:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70382d9b80 20 client.17124 trim_cache size 0 max 16384 2024-05-09T01:32:11.827 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:32:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 -- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70100b2880 msgr2=0x7f70100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70100b2880 0x7f70100b4d90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7014005950 tx=0x7f701400b410 comp rx=0 tx=0).stop 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 -- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 msgr2=0x7f702419f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 0x7f702419f5a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f701c0095c0 tx=0x7f701c007a20 comp rx=0 tx=0).stop 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 -- 172.21.15.19:0/3897094049 shutdown_connections 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f70100b2880 0x7f70100b4d90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024109850 0x7f702419f5a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241045d0 0x7f702419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 --2- 172.21.15.19:0/3897094049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024108f00 0x7f702419b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 -- 172.21.15.19:0/3897094049 >> 172.21.15.19:0/3897094049 conn(0x7f70240ffea0 msgr2=0x7f7024108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 -- 172.21.15.19:0/3897094049 shutdown_connections 2024-05-09T01:32:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:11.825+0000 7f70298c3700 1 -- 172.21.15.19:0/3897094049 wait complete. 2024-05-09T01:32:12.499 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:32:12.768 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:12 smithi019 ceph-mon[30158]: pgmap v578: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:32:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:12 smithi059 ceph-mon[34351]: pgmap v578: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:32:12.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:12 smithi196 ceph-mon[35041]: pgmap v578: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:32:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.496+0000 7f0322087700 1 -- 172.21.15.19:0/3120099913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c108f00 msgr2=0x7f031c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.496+0000 7f0322087700 1 --2- 172.21.15.19:0/3120099913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c108f00 0x7f031c109310 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f03040099c0 tx=0x7f0304009cd0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.496+0000 7f0322087700 1 -- 172.21.15.19:0/3120099913 shutdown_connections 2024-05-09T01:32:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.496+0000 7f0322087700 1 --2- 172.21.15.19:0/3120099913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f031c109850 0x7f031c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.496+0000 7f0322087700 1 --2- 172.21.15.19:0/3120099913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031c1045d0 0x7f031c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.496+0000 7f0322087700 1 --2- 172.21.15.19:0/3120099913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c108f00 0x7f031c109310 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.496+0000 7f0322087700 1 -- 172.21.15.19:0/3120099913 >> 172.21.15.19:0/3120099913 conn(0x7f031c0ffea0 msgr2=0x7f031c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.496+0000 7f0322087700 1 -- 172.21.15.19:0/3120099913 shutdown_connections 2024-05-09T01:32:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.496+0000 7f0322087700 1 -- 172.21.15.19:0/3120099913 wait complete. 2024-05-09T01:32:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.497+0000 7f0322087700 1 Processor -- start 2024-05-09T01:32:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.497+0000 7f0322087700 1 -- start start 2024-05-09T01:32:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f0322087700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f031c1045d0 0x7f031c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f0322087700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031c108f00 0x7f031c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f0322087700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c109850 0x7f031c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f0322087700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f031c114460 con 0x7f031c109850 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f0322087700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1142e0 con 0x7f031c1045d0 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f0322087700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1145e0 con 0x7f031c108f00 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f031bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c109850 0x7f031c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f031bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c109850 0x7f031c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52734/0 (socket says 172.21.15.19:52734) 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f031bfff700 1 -- 172.21.15.19:0/512217638 learned_addr learned my addr 172.21.15.19:0/512217638 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f0312ffd700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031c108f00 0x7f031c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f031b7fe700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f031c1045d0 0x7f031c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f031bfff700 1 -- 172.21.15.19:0/512217638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031c108f00 msgr2=0x7f031c19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f031bfff700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031c108f00 0x7f031c19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f031bfff700 1 -- 172.21.15.19:0/512217638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f031c1045d0 msgr2=0x7f031c19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f031bfff700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f031c1045d0 0x7f031c19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f031bfff700 1 -- 172.21.15.19:0/512217638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c19fcf0 con 0x7f031c109850 2024-05-09T01:32:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.498+0000 7f0312ffd700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031c108f00 0x7f031c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.499+0000 7f031bfff700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c109850 0x7f031c19f700 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f031400c900 tx=0x7f031400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.499+0000 7f03197fa700 1 -- 172.21.15.19:0/512217638 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314007b00 con 0x7f031c109850 2024-05-09T01:32:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.499+0000 7f03197fa700 1 -- 172.21.15.19:0/512217638 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031400ce50 con 0x7f031c109850 2024-05-09T01:32:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.499+0000 7f0322087700 1 -- 172.21.15.19:0/512217638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c19ffe0 con 0x7f031c109850 2024-05-09T01:32:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.499+0000 7f03197fa700 1 -- 172.21.15.19:0/512217638 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314018660 con 0x7f031c109850 2024-05-09T01:32:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.499+0000 7f0322087700 1 -- 172.21.15.19:0/512217638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c1a04a0 con 0x7f031c109850 2024-05-09T01:32:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.501+0000 7f03197fa700 1 -- 172.21.15.19:0/512217638 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f03140213f0 con 0x7f031c109850 2024-05-09T01:32:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.502+0000 7f03197fa700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03080b2880 0x7f03080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.502+0000 7f031b7fe700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03080b2880 0x7f03080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.502+0000 7f0330a9db80 1 Processor -- start 2024-05-09T01:32:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.502+0000 7f03197fa700 1 -- 172.21.15.19:0/512217638 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03140e2190 con 0x7f031c109850 2024-05-09T01:32:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f0330a9db80 1 -- start start 2024-05-09T01:32:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f031b7fe700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03080b2880 0x7f03080b4d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f030c009550 tx=0x7f030c0091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415ec160 0x5564416854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415024a0 0x556441685a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 0x55644167fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f0330a9db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5564415a1ff0 con 0x5564415ec160 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f0330a9db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5564415407f0 con 0x5564415024a0 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f0330a9db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55644159dfb0 con 0x55644167f6a0 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415ec160 0x5564416854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415ec160 0x5564416854f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52748/0 (socket says 172.21.15.19:52748) 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f031b7fe700 1 -- 172.21.15.19:0/673911614 learned_addr learned my addr 172.21.15.19:0/673911614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f0312ffd700 1 --2- 172.21.15.19:0/673911614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415024a0 0x556441685a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.503+0000 7f031bfff700 1 --2- 172.21.15.19:0/673911614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 0x55644167fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.504+0000 7f031b7fe700 1 -- 172.21.15.19:0/673911614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 msgr2=0x55644167fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.504+0000 7f031b7fe700 1 --2- 172.21.15.19:0/673911614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 0x55644167fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.504+0000 7f031b7fe700 1 -- 172.21.15.19:0/673911614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415024a0 msgr2=0x556441685a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.504+0000 7f031b7fe700 1 --2- 172.21.15.19:0/673911614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415024a0 0x556441685a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.504+0000 7f031b7fe700 1 -- 172.21.15.19:0/673911614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556441538370 con 0x5564415ec160 2024-05-09T01:32:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.504+0000 7f031b7fe700 1 --2- 172.21.15.19:0/673911614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415ec160 0x5564416854f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f030c0094c0 tx=0x7f030c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0a4545526e6a3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.504+0000 7f0311ffb700 1 -- 172.21.15.19:0/673911614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c01c050 con 0x5564415ec160 2024-05-09T01:32:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.504+0000 7f0311ffb700 1 -- 172.21.15.19:0/673911614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f030c026070 con 0x5564415ec160 2024-05-09T01:32:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.504+0000 7f0311ffb700 1 -- 172.21.15.19:0/673911614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c015440 con 0x5564415ec160 2024-05-09T01:32:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 -- 172.21.15.19:0/673911614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415ec160 msgr2=0x5564416854f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 --2- 172.21.15.19:0/673911614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415ec160 0x5564416854f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f030c0094c0 tx=0x7f030c013000 comp rx=0 tx=0).stop 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 -- 172.21.15.19:0/673911614 shutdown_connections 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 --2- 172.21.15.19:0/673911614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 0x55644167fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 --2- 172.21.15.19:0/673911614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415ec160 0x5564416854f0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 --2- 172.21.15.19:0/673911614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415024a0 0x556441685a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 -- 172.21.15.19:0/673911614 >> 172.21.15.19:0/673911614 conn(0x5564415626a0 msgr2=0x55644167a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 -- 172.21.15.19:0/673911614 shutdown_connections 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 -- 172.21.15.19:0/673911614 wait complete. 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.505+0000 7f0330a9db80 1 -- start start 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f0330a9db80 1 Processor -- start 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441689140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564415ec160 0x556441689680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55644167f6a0 0x556441689bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f0330a9db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55644141dba0 con 0x5564415024a0 2024-05-09T01:32:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f0330a9db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55644155c150 con 0x55644167f6a0 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f0330a9db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55644168c1d0 con 0x5564415ec160 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f0330a9db80 10 client.0 _ll_register_callbacks cb 0x5564414136c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441689140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441689140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52762/0 (socket says 172.21.15.19:52762) 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f031b7fe700 1 -- 172.21.15.19:0/2014785509 learned_addr learned my addr 172.21.15.19:0/2014785509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f031bfff700 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55644167f6a0 0x556441689bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f0312ffd700 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564415ec160 0x556441689680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f031b7fe700 1 -- 172.21.15.19:0/2014785509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564415ec160 msgr2=0x556441689680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.506+0000 7f031b7fe700 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564415ec160 0x556441689680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f031b7fe700 1 -- 172.21.15.19:0/2014785509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55644167f6a0 msgr2=0x556441689bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f031b7fe700 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55644167f6a0 0x556441689bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f031b7fe700 1 -- 172.21.15.19:0/2014785509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55644141d490 con 0x5564415024a0 2024-05-09T01:32:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f031b7fe700 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441689140 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f030c021ca0 tx=0x7f030c0216c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c1b150afba20e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f02faffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f02faffd700 1 -- 172.21.15.19:0/2014785509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c004bc0 con 0x5564415024a0 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f02faffd700 1 -- 172.21.15.19:0/2014785509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f030c004d60 con 0x5564415024a0 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f0330a9db80 1 -- 172.21.15.19:0/2014785509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030c026070 con 0x5564415024a0 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f02faffd700 1 -- 172.21.15.19:0/2014785509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f030c01c030 con 0x5564415024a0 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f0330a9db80 10 client.17136 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f0330a9db80 1 -- 172.21.15.19:0/2014785509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556441696b10 con 0x5564415024a0 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.507+0000 7f02faffd700 1 -- 172.21.15.19:0/2014785509 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f030c026070 con 0x5564415024a0 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.508+0000 7f02faffd700 1 -- 172.21.15.19:0/2014785509 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f030c015600 con 0x5564415024a0 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.508+0000 7f0330a9db80 10 client.17136 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.508+0000 7f0330a9db80 10 client.17136 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.508+0000 7f0330a9db80 20 client.17136 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.508+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5564416990f0 0x55644169b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.508+0000 7f0330a9db80 4 client.17136 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.508+0000 7f0330a9db80 1 -- 172.21.15.19:0/2014785509 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564415af9d0 con 0x5564416990f0 2024-05-09T01:32:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.508+0000 7f0312ffd700 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5564416990f0 0x55644169b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.509+0000 7f02faffd700 10 client.17136 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:32:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.509+0000 7f0312ffd700 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5564416990f0 0x55644169b510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63277456d9f19274 server_cookie=9a896d4f7eabb18a in_seq=0 out_seq=0 2024-05-09T01:32:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.509+0000 7f02faffd700 1 -- 172.21.15.19:0/2014785509 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564415af9d0 con 0x5564416990f0 2024-05-09T01:32:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f02faffd700 10 client.17136 handle_command_reply: tid=0 2024-05-09T01:32:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 client.17136 shutdown 2024-05-09T01:32:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 -- 172.21.15.19:0/2014785509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 msgr2=0x556441689140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441689140 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f030c021ca0 tx=0x7f030c0216c0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 -- 172.21.15.19:0/2014785509 shutdown_connections 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5564416990f0 0x55644169b510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55644167f6a0 0x556441689bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441689140 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2014785509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564415ec160 0x556441689680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 -- 172.21.15.19:0/2014785509 >> 172.21.15.19:0/2014785509 conn(0x5564415626a0 msgr2=0x55644167a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f02faffd700 0 client.17136 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 -- 172.21.15.19:0/2014785509 shutdown_connections 2024-05-09T01:32:14.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 1 -- 172.21.15.19:0/2014785509 wait complete. 2024-05-09T01:32:14.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.510+0000 7f0330a9db80 20 client.17136 trim_cache size 0 max 16384 2024-05-09T01:32:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.525+0000 7f0330a9db80 1 Processor -- start 2024-05-09T01:32:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f0330a9db80 1 -- start start 2024-05-09T01:32:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441686dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564415ec160 0x556441688460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55644167f6a0 0x55644168b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f0330a9db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1aec70 con 0x5564415024a0 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f0330a9db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1b07b0 con 0x55644167f6a0 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f0330a9db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1af340 con 0x5564415ec160 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441686dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441686dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52766/0 (socket says 172.21.15.19:52766) 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f031b7fe700 1 -- 172.21.15.19:0/2417585119 learned_addr learned my addr 172.21.15.19:0/2417585119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f031b7fe700 1 -- 172.21.15.19:0/2417585119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564415ec160 msgr2=0x556441688460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f031b7fe700 1 --2- 172.21.15.19:0/2417585119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564415ec160 0x556441688460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f031b7fe700 1 -- 172.21.15.19:0/2417585119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55644167f6a0 msgr2=0x55644168b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f031b7fe700 1 --2- 172.21.15.19:0/2417585119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55644167f6a0 0x55644168b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f031b7fe700 1 -- 172.21.15.19:0/2417585119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556441688f10 con 0x5564415024a0 2024-05-09T01:32:14.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.526+0000 7f031b7fe700 1 --2- 172.21.15.19:0/2417585119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441686dd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f030c015aa0 tx=0x7f030c003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5f89e5ed3044de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:14.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f03117fa700 1 -- 172.21.15.19:0/2417585119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c015440 con 0x5564415024a0 2024-05-09T01:32:14.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f03117fa700 1 -- 172.21.15.19:0/2417585119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f030c026040 con 0x5564415024a0 2024-05-09T01:32:14.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f03117fa700 1 -- 172.21.15.19:0/2417585119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0053e0 con 0x5564415024a0 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f0330a9db80 1 -- 172.21.15.19:0/2417585119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 msgr2=0x556441686dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2417585119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441686dd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f030c015aa0 tx=0x7f030c003cc0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f0330a9db80 1 -- 172.21.15.19:0/2417585119 shutdown_connections 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2417585119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55644167f6a0 0x55644168b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2417585119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x556441686dd0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f0330a9db80 1 --2- 172.21.15.19:0/2417585119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564415ec160 0x556441688460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.527+0000 7f0330a9db80 1 -- 172.21.15.19:0/2417585119 >> 172.21.15.19:0/2417585119 conn(0x5564415626a0 msgr2=0x5564416848a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.528+0000 7f0330a9db80 1 -- 172.21.15.19:0/2417585119 shutdown_connections 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.528+0000 7f0330a9db80 1 -- 172.21.15.19:0/2417585119 wait complete. 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.528+0000 7f0330a9db80 1 -- start start 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.528+0000 7f0330a9db80 1 Processor -- start 2024-05-09T01:32:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.528+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x55644169e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.528+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415ec160 0x55644169b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f0330a9db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 0x55644169b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f0330a9db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1b00e0 con 0x5564415024a0 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f0330a9db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1afa10 con 0x5564415ec160 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f0330a9db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1acc10 con 0x55644167f6a0 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f0330a9db80 10 client.0 _ll_register_callbacks cb 0x5564414136c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x55644169e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x55644169e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52770/0 (socket says 172.21.15.19:52770) 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031b7fe700 1 -- 172.21.15.19:0/3192611319 learned_addr learned my addr 172.21.15.19:0/3192611319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031bfff700 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 0x55644169b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f0312ffd700 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415ec160 0x55644169b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031b7fe700 1 -- 172.21.15.19:0/3192611319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 msgr2=0x55644169b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:14.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031b7fe700 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 0x55644169b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031b7fe700 1 -- 172.21.15.19:0/3192611319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415ec160 msgr2=0x55644169b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031b7fe700 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415ec160 0x55644169b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031b7fe700 1 -- 172.21.15.19:0/3192611319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55644169be80 con 0x5564415024a0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f031b7fe700 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x55644169e840 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f030c015b50 tx=0x7f030c009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0960c805406b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.529+0000 7f02faffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f02faffd700 1 -- 172.21.15.19:0/3192611319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c004bc0 con 0x5564415024a0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f0330a9db80 1 -- 172.21.15.19:0/3192611319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030c026040 con 0x5564415024a0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f02faffd700 1 -- 172.21.15.19:0/3192611319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f030c004d60 con 0x5564415024a0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f02faffd700 1 -- 172.21.15.19:0/3192611319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f030c005040 con 0x5564415024a0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f0330a9db80 10 client.17142 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f0330a9db80 1 -- 172.21.15.19:0/3192611319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556441702620 con 0x5564415024a0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f02faffd700 1 -- 172.21.15.19:0/3192611319 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f030c026040 con 0x5564415024a0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f02faffd700 1 -- 172.21.15.19:0/3192611319 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f030c047740 con 0x5564415024a0 2024-05-09T01:32:14.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f0330a9db80 10 client.17142 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f0330a9db80 10 client.17142 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f0330a9db80 20 client.17142 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f0330a9db80 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556441702be0 0x556441705000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f0330a9db80 4 client.17142 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.530+0000 7f0330a9db80 1 -- 172.21.15.19:0/3192611319 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564415a9a70 con 0x556441702be0 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0312ffd700 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556441702be0 0x556441705000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f02faffd700 10 client.17142 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0312ffd700 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556441702be0 0x556441705000 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5284afadc07991a server_cookie=f0b42829c4a524e in_seq=0 out_seq=0 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f02faffd700 1 -- 172.21.15.19:0/3192611319 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564415a9a70 con 0x556441702be0 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f02faffd700 10 client.17142 handle_command_reply: tid=0 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0330a9db80 1 client.17142 shutdown 2024-05-09T01:32:14.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0330a9db80 1 -- 172.21.15.19:0/3192611319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 msgr2=0x55644169e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0330a9db80 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x55644169e840 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f030c015b50 tx=0x7f030c009320 comp rx=0 tx=0).stop 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0330a9db80 1 -- 172.21.15.19:0/3192611319 shutdown_connections 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0330a9db80 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55644167f6a0 0x55644169b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0330a9db80 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564415024a0 0x55644169e840 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0330a9db80 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x556441702be0 0x556441705000 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0330a9db80 1 --2- 172.21.15.19:0/3192611319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564415ec160 0x55644169b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f0330a9db80 1 -- 172.21.15.19:0/3192611319 >> 172.21.15.19:0/3192611319 conn(0x5564415626a0 msgr2=0x556441682e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.531+0000 7f02faffd700 0 client.17142 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0330a9db80 1 -- 172.21.15.19:0/3192611319 shutdown_connections 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0330a9db80 1 -- 172.21.15.19:0/3192611319 wait complete. 2024-05-09T01:32:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0330a9db80 20 client.17142 trim_cache size 0 max 16384 2024-05-09T01:32:14.535 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:32:14.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 -- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03080b2880 msgr2=0x7f03080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:14.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03080b2880 0x7f03080b4d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f030c009550 tx=0x7f030c0091f0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 -- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c109850 msgr2=0x7f031c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:14.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c109850 0x7f031c19f700 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f031400c900 tx=0x7f031400cc10 comp rx=0 tx=0).stop 2024-05-09T01:32:14.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 -- 172.21.15.19:0/512217638 shutdown_connections 2024-05-09T01:32:14.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03080b2880 0x7f03080b4d90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f031c109850 0x7f031c19f700 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f031c1045d0 0x7f031c19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 --2- 172.21.15.19:0/512217638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031c108f00 0x7f031c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:14.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 -- 172.21.15.19:0/512217638 >> 172.21.15.19:0/512217638 conn(0x7f031c0ffea0 msgr2=0x7f031c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:14.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 -- 172.21.15.19:0/512217638 shutdown_connections 2024-05-09T01:32:14.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:14.532+0000 7f0322087700 1 -- 172.21.15.19:0/512217638 wait complete. 2024-05-09T01:32:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:14 smithi059 ceph-mon[34351]: pgmap v579: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:32:14.877 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:14 smithi019 ceph-mon[30158]: pgmap v579: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:32:14.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:14 smithi196 ceph-mon[35041]: pgmap v579: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:32:15.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:32:15.544 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:16.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:16 smithi019 ceph-mon[30158]: pgmap v580: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:32:16.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:16 smithi059 ceph-mon[34351]: pgmap v580: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:32:16.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:16 smithi196 ceph-mon[35041]: pgmap v580: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:32:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 -- 172.21.15.19:0/1136032846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 msgr2=0x7f2408103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 --2- 172.21.15.19:0/1136032846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 0x7f2408103860 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f2404009900 tx=0x7f240402a410 comp rx=0 tx=0).stop 2024-05-09T01:32:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 -- 172.21.15.19:0/1136032846 shutdown_connections 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 --2- 172.21.15.19:0/1136032846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2408108aa0 0x7f240810aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 --2- 172.21.15.19:0/1136032846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2408103da0 0x7f24081041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 --2- 172.21.15.19:0/1136032846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 0x7f2408103860 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 -- 172.21.15.19:0/1136032846 >> 172.21.15.19:0/1136032846 conn(0x7f24080fdd30 msgr2=0x7f2408100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 -- 172.21.15.19:0/1136032846 shutdown_connections 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 -- 172.21.15.19:0/1136032846 wait complete. 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.152+0000 7f2410196640 1 Processor -- start 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f2410196640 1 -- start start 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f2410196640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 0x7f240819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f2410196640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2408103da0 0x7f240819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f2410196640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2408108aa0 0x7f24081a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f2410196640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f240810e640 con 0x7f2408102b60 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f2410196640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f240810e4c0 con 0x7f2408108aa0 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f2410196640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f240810e7c0 con 0x7f2408103da0 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240df32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 0x7f240819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240df32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 0x7f240819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52788/0 (socket says 172.21.15.19:52788) 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240df32640 1 -- 172.21.15.19:0/3814577043 learned_addr learned my addr 172.21.15.19:0/3814577043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240d731640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2408103da0 0x7f240819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240e733640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2408108aa0 0x7f24081a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240df32640 1 -- 172.21.15.19:0/3814577043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2408103da0 msgr2=0x7f240819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240df32640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2408103da0 0x7f240819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240df32640 1 -- 172.21.15.19:0/3814577043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2408108aa0 msgr2=0x7f24081a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240df32640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2408108aa0 0x7f24081a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240df32640 1 -- 172.21.15.19:0/3814577043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24081a1e20 con 0x7f2408102b60 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240e733640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2408108aa0 0x7f24081a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.153+0000 7f240df32640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 0x7f240819a390 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2404006010 tx=0x7f2404004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.154+0000 7f23feffd640 1 -- 172.21.15.19:0/3814577043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240403f030 con 0x7f2408102b60 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.154+0000 7f23feffd640 1 -- 172.21.15.19:0/3814577043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2404003aa0 con 0x7f2408102b60 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.154+0000 7f2410196640 1 -- 172.21.15.19:0/3814577043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24081a20b0 con 0x7f2408102b60 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.154+0000 7f23feffd640 1 -- 172.21.15.19:0/3814577043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404035070 con 0x7f2408102b60 2024-05-09T01:32:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.154+0000 7f2410196640 1 -- 172.21.15.19:0/3814577043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24081a25a0 con 0x7f2408102b60 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.156+0000 7f241704b740 1 Processor -- start 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.156+0000 7f241704b740 1 -- start start 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f241704b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 0x55a969e10500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f23feffd640 1 -- 172.21.15.19:0/3814577043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2404042030 con 0x7f2408102b60 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f241704b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e10a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f241704b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a4b4d0 0x55a969e17980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f241704b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a969d747c0 con 0x55a969a432f0 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f241704b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a969d100e0 con 0x55a969a4b4d0 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f241704b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a969d70780 con 0x55a969a6c1b0 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240df32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e10a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240df32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e10a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52790/0 (socket says 172.21.15.19:52790) 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240df32640 1 -- 172.21.15.19:0/853666529 learned_addr learned my addr 172.21.15.19:0/853666529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240d731640 1 --2- 172.21.15.19:0/853666529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a4b4d0 0x55a969e17980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240e733640 1 --2- 172.21.15.19:0/853666529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 0x55a969e10500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240df32640 1 -- 172.21.15.19:0/853666529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 msgr2=0x55a969e10500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240df32640 1 --2- 172.21.15.19:0/853666529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 0x55a969e10500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240df32640 1 -- 172.21.15.19:0/853666529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a4b4d0 msgr2=0x55a969e17980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240df32640 1 --2- 172.21.15.19:0/853666529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a4b4d0 0x55a969e17980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.157+0000 7f240df32640 1 -- 172.21.15.19:0/853666529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a969d397c0 con 0x55a969a432f0 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.158+0000 7f240df32640 1 --2- 172.21.15.19:0/853666529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e10a80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2404108b80 tx=0x7f2404108c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d350fb9d86d3a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.158+0000 7f23feffd640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f23f40b27e0 0x7f23f40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.158+0000 7f23f3fff640 1 -- 172.21.15.19:0/853666529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24040c96f0 con 0x55a969a432f0 2024-05-09T01:32:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.158+0000 7f240e733640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f23f40b27e0 0x7f23f40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.158+0000 7f23f3fff640 1 -- 172.21.15.19:0/853666529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2404112020 con 0x55a969a432f0 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.158+0000 7f23feffd640 1 -- 172.21.15.19:0/3814577043 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2404102980 con 0x7f2408102b60 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.158+0000 7f23f3fff640 1 -- 172.21.15.19:0/853666529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24040c98d0 con 0x55a969a432f0 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.159+0000 7f241704b740 1 -- 172.21.15.19:0/853666529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 msgr2=0x55a969e10a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.159+0000 7f240e733640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f23f40b27e0 0x7f23f40b4cf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55a969e116d0 tx=0x7f24000086a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.159+0000 7f241704b740 1 --2- 172.21.15.19:0/853666529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e10a80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2404108b80 tx=0x7f2404108c70 comp rx=0 tx=0).stop 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.159+0000 7f241704b740 1 -- 172.21.15.19:0/853666529 shutdown_connections 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.159+0000 7f241704b740 1 --2- 172.21.15.19:0/853666529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a4b4d0 0x55a969e17980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.159+0000 7f241704b740 1 --2- 172.21.15.19:0/853666529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e10a80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.159+0000 7f241704b740 1 --2- 172.21.15.19:0/853666529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 0x55a969e10500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.159+0000 7f241704b740 1 -- 172.21.15.19:0/853666529 >> 172.21.15.19:0/853666529 conn(0x55a969a60300 msgr2=0x55a969e0be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.160+0000 7f241704b740 1 -- 172.21.15.19:0/853666529 shutdown_connections 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.160+0000 7f241704b740 1 -- 172.21.15.19:0/853666529 wait complete. 2024-05-09T01:32:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.160+0000 7f241704b740 1 -- start start 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.160+0000 7f241704b740 1 Processor -- start 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f241704b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 0x55a969e19ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f241704b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 0x55a969e1a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f241704b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a6c1b0 0x55a969e21390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f241704b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a969e18190 con 0x55a969a6c1b0 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f241704b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a969cfb9c0 con 0x55a969a432f0 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f241704b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a969e18310 con 0x55a969a4b4d0 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f241704b740 10 client.0 _ll_register_callbacks cb 0x55a969d28020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240e733640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a6c1b0 0x55a969e21390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240e733640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a6c1b0 0x55a969e21390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52800/0 (socket says 172.21.15.19:52800) 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240e733640 1 -- 172.21.15.19:0/865271463 learned_addr learned my addr 172.21.15.19:0/865271463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240df32640 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 0x55a969e19ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240d731640 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 0x55a969e1a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240df32640 1 -- 172.21.15.19:0/865271463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 msgr2=0x55a969e1a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240df32640 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 0x55a969e1a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240df32640 1 -- 172.21.15.19:0/865271463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a6c1b0 msgr2=0x55a969e21390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240df32640 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a6c1b0 0x55a969e21390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.161+0000 7f240df32640 1 -- 172.21.15.19:0/865271463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a969d221f0 con 0x55a969a432f0 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f240df32640 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 0x55a969e19ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2404086ec0 tx=0x7f240402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f524a7469e4ee05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f23f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f23f17fa640 1 -- 172.21.15.19:0/865271463 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404111060 con 0x55a969a432f0 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f23f17fa640 1 -- 172.21.15.19:0/865271463 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f240403b460 con 0x55a969a432f0 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f241704b740 1 -- 172.21.15.19:0/865271463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2404112020 con 0x55a969a432f0 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f23f17fa640 1 -- 172.21.15.19:0/865271463 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2404049030 con 0x55a969a432f0 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f241704b740 10 client.26585 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f241704b740 1 -- 172.21.15.19:0/865271463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a969ad7790 con 0x55a969a432f0 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f23f17fa640 1 -- 172.21.15.19:0/865271463 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2404112020 con 0x55a969a432f0 2024-05-09T01:32:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f23f17fa640 1 -- 172.21.15.19:0/865271463 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2404003c50 con 0x55a969a432f0 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f241704b740 10 client.26585 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f241704b740 10 client.26585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f241704b740 20 client.26585 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.162+0000 7f241704b740 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a969a2ad80 0x55a969e27470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.163+0000 7f241704b740 4 client.26585 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.163+0000 7f241704b740 1 -- 172.21.15.19:0/865271463 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a969d823d0 con 0x55a969a2ad80 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.163+0000 7f240d731640 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a969a2ad80 0x55a969e27470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.163+0000 7f240d731640 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a969a2ad80 0x55a969e27470 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae12897f4dbabd1a server_cookie=638bdd45d8aa34f1 in_seq=0 out_seq=0 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.163+0000 7f23f17fa640 10 client.26585 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f23f17fa640 1 -- 172.21.15.19:0/865271463 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a969d823d0 con 0x55a969a2ad80 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f23f17fa640 10 client.26585 handle_command_reply: tid=0 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f241704b740 1 client.26585 shutdown 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f241704b740 1 -- 172.21.15.19:0/865271463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 msgr2=0x55a969e19ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f241704b740 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 0x55a969e19ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2404086ec0 tx=0x7f240402a410 comp rx=0 tx=0).stop 2024-05-09T01:32:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f241704b740 1 -- 172.21.15.19:0/865271463 shutdown_connections 2024-05-09T01:32:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f241704b740 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a969a2ad80 0x55a969e27470 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f241704b740 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a6c1b0 0x55a969e21390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f241704b740 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 0x55a969e1a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f23f17fa640 0 client.26585 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f241704b740 1 --2- 172.21.15.19:0/865271463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 0x55a969e19ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.164+0000 7f241704b740 1 -- 172.21.15.19:0/865271463 >> 172.21.15.19:0/865271463 conn(0x55a969a60300 msgr2=0x55a969e0a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.165+0000 7f241704b740 1 -- 172.21.15.19:0/865271463 shutdown_connections 2024-05-09T01:32:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.165+0000 7f241704b740 1 -- 172.21.15.19:0/865271463 wait complete. 2024-05-09T01:32:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.165+0000 7f241704b740 20 client.26585 trim_cache size 0 max 16384 2024-05-09T01:32:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f241704b740 1 Processor -- start 2024-05-09T01:32:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f241704b740 1 -- start start 2024-05-09T01:32:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f241704b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e14c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f241704b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 0x55a969e1a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f241704b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a6c1b0 0x55a969e2bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f241704b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24081ab3c0 con 0x55a969a432f0 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f241704b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24081acf00 con 0x55a969a6c1b0 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f241704b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24081aba90 con 0x55a969a4b4d0 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f240df32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e14c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f240df32640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e14c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52812/0 (socket says 172.21.15.19:52812) 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f240df32640 1 -- 172.21.15.19:0/113066817 learned_addr learned my addr 172.21.15.19:0/113066817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f240e733640 1 --2- 172.21.15.19:0/113066817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a6c1b0 0x55a969e2bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.185+0000 7f240d731640 1 --2- 172.21.15.19:0/113066817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 0x55a969e1a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.186+0000 7f240df32640 1 -- 172.21.15.19:0/113066817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 msgr2=0x55a969e1a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.186+0000 7f240df32640 1 --2- 172.21.15.19:0/113066817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 0x55a969e1a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.186+0000 7f240df32640 1 -- 172.21.15.19:0/113066817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a6c1b0 msgr2=0x55a969e2bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.186+0000 7f240df32640 1 --2- 172.21.15.19:0/113066817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a6c1b0 0x55a969e2bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.186+0000 7f240df32640 1 -- 172.21.15.19:0/113066817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a969b4e5f0 con 0x55a969a432f0 2024-05-09T01:32:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.186+0000 7f240df32640 1 --2- 172.21.15.19:0/113066817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e14c90 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f24040c9c80 tx=0x7f2404031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5e54b784a11f7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.186+0000 7f23f37fe640 1 -- 172.21.15.19:0/113066817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24040e4070 con 0x55a969a432f0 2024-05-09T01:32:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.186+0000 7f23f37fe640 1 -- 172.21.15.19:0/113066817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24040ca680 con 0x55a969a432f0 2024-05-09T01:32:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.186+0000 7f23f37fe640 1 -- 172.21.15.19:0/113066817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24040c9ac0 con 0x55a969a432f0 2024-05-09T01:32:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.187+0000 7f241704b740 1 -- 172.21.15.19:0/113066817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 msgr2=0x55a969e14c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.187+0000 7f241704b740 1 --2- 172.21.15.19:0/113066817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e14c90 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f24040c9c80 tx=0x7f2404031400 comp rx=0 tx=0).stop 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.187+0000 7f241704b740 1 -- 172.21.15.19:0/113066817 shutdown_connections 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.187+0000 7f241704b740 1 --2- 172.21.15.19:0/113066817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a6c1b0 0x55a969e2bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.187+0000 7f241704b740 1 --2- 172.21.15.19:0/113066817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a4b4d0 0x55a969e1a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.187+0000 7f241704b740 1 --2- 172.21.15.19:0/113066817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a432f0 0x55a969e14c90 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.187+0000 7f241704b740 1 -- 172.21.15.19:0/113066817 >> 172.21.15.19:0/113066817 conn(0x55a969a60300 msgr2=0x55a969e19ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.187+0000 7f241704b740 1 -- 172.21.15.19:0/113066817 shutdown_connections 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.187+0000 7f241704b740 1 -- 172.21.15.19:0/113066817 wait complete. 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f241704b740 1 -- start start 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f241704b740 1 Processor -- start 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f241704b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 0x55a969e1dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f241704b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a4b4d0 0x55a969e1e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f241704b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 0x55a969e1f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f241704b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24081ac830 con 0x55a969a4b4d0 2024-05-09T01:32:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f241704b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24081ac160 con 0x55a969a432f0 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f241704b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24081a93a0 con 0x55a969a6c1b0 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f241704b740 10 client.0 _ll_register_callbacks cb 0x7f23f80095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.188+0000 7f240d731640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a4b4d0 0x55a969e1e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240d731640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a4b4d0 0x55a969e1e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52822/0 (socket says 172.21.15.19:52822) 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240d731640 1 -- 172.21.15.19:0/381481619 learned_addr learned my addr 172.21.15.19:0/381481619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240df32640 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 0x55a969e1dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240e733640 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 0x55a969e1f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240d731640 1 -- 172.21.15.19:0/381481619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 msgr2=0x55a969e1f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240d731640 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 0x55a969e1f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240d731640 1 -- 172.21.15.19:0/381481619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 msgr2=0x55a969e1dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240d731640 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 0x55a969e1dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240d731640 1 -- 172.21.15.19:0/381481619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a969d38ed0 con 0x55a969a4b4d0 2024-05-09T01:32:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f240d731640 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a4b4d0 0x55a969e1e190 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x55a969e0cc20 tx=0x7f23f8012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212acdc57dac30e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f23f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f23f17fa640 1 -- 172.21.15.19:0/381481619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8021580 con 0x55a969a4b4d0 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f23f17fa640 1 -- 172.21.15.19:0/381481619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f23f8021720 con 0x55a969a4b4d0 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f241704b740 1 -- 172.21.15.19:0/381481619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24040ca680 con 0x55a969a4b4d0 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f23f17fa640 1 -- 172.21.15.19:0/381481619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23f8030070 con 0x55a969a4b4d0 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.189+0000 7f241704b740 10 client.17160 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f241704b740 1 -- 172.21.15.19:0/381481619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a969e20840 con 0x55a969a4b4d0 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f23f17fa640 1 -- 172.21.15.19:0/381481619 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24040ca680 con 0x55a969a4b4d0 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f23f17fa640 1 -- 172.21.15.19:0/381481619 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f23f8011440 con 0x55a969a4b4d0 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f241704b740 10 client.17160 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f241704b740 10 client.17160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f241704b740 20 client.17160 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f241704b740 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a969a001c0 0x55a969e95f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f241704b740 4 client.17160 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:32:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f241704b740 1 -- 172.21.15.19:0/381481619 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a969d7c240 con 0x55a969a001c0 2024-05-09T01:32:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.190+0000 7f240df32640 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a969a001c0 0x55a969e95f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.191+0000 7f240df32640 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a969a001c0 0x55a969e95f30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=518c9fa18857236e server_cookie=d3eab05ada18f52b in_seq=0 out_seq=0 2024-05-09T01:32:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.191+0000 7f23f17fa640 10 client.17160 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.209+0000 7f23f17fa640 1 -- 172.21.15.19:0/381481619 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a969d7c240 con 0x55a969a001c0 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.209+0000 7f23f17fa640 10 client.17160 handle_command_reply: tid=0 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.209+0000 7f241704b740 1 client.17160 shutdown 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.209+0000 7f241704b740 1 -- 172.21.15.19:0/381481619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a4b4d0 msgr2=0x55a969e1e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.209+0000 7f241704b740 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a4b4d0 0x55a969e1e190 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x55a969e0cc20 tx=0x7f23f8012a40 comp rx=0 tx=0).stop 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f241704b740 1 -- 172.21.15.19:0/381481619 shutdown_connections 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f241704b740 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a969a001c0 0x55a969e95f30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f241704b740 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a969a6c1b0 0x55a969e1f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f241704b740 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a969a4b4d0 0x55a969e1e190 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f241704b740 1 --2- 172.21.15.19:0/381481619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a969a432f0 0x55a969e1dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f23f17fa640 0 client.17160 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f241704b740 1 -- 172.21.15.19:0/381481619 >> 172.21.15.19:0/381481619 conn(0x55a969a60300 msgr2=0x55a969e0ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f241704b740 1 -- 172.21.15.19:0/381481619 shutdown_connections 2024-05-09T01:32:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f241704b740 1 -- 172.21.15.19:0/381481619 wait complete. 2024-05-09T01:32:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f241704b740 20 client.17160 trim_cache size 0 max 16384 2024-05-09T01:32:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:32:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:32:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:32:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 -- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f23f40b27e0 msgr2=0x7f23f40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f23f40b27e0 0x7f23f40b4cf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55a969e116d0 tx=0x7f24000086a0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 -- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 msgr2=0x7f240819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 0x7f240819a390 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2404006010 tx=0x7f2404004270 comp rx=0 tx=0).stop 2024-05-09T01:32:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 -- 172.21.15.19:0/3814577043 shutdown_connections 2024-05-09T01:32:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f23f40b27e0 0x7f23f40b4cf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2408108aa0 0x7f24081a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2408103da0 0x7f240819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 --2- 172.21.15.19:0/3814577043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2408102b60 0x7f240819a390 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.210+0000 7f2410196640 1 -- 172.21.15.19:0/3814577043 >> 172.21.15.19:0/3814577043 conn(0x7f24080fdd30 msgr2=0x7f240810a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.211+0000 7f2410196640 1 -- 172.21.15.19:0/3814577043 shutdown_connections 2024-05-09T01:32:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:17.211+0000 7f2410196640 1 -- 172.21.15.19:0/3814577043 wait complete. 2024-05-09T01:32:17.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.214+0000 7f3d54926700 1 -- 172.21.15.19:0/3853717767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50107df0 msgr2=0x7f3d50108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.214+0000 7f3d54926700 1 --2- 172.21.15.19:0/3853717767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50107df0 0x7f3d50108200 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3d440099c0 tx=0x7f3d44009cd0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.214+0000 7f3d54926700 1 -- 172.21.15.19:0/3853717767 shutdown_connections 2024-05-09T01:32:17.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.214+0000 7f3d54926700 1 --2- 172.21.15.19:0/3853717767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d50108740 0x7f3d501100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.214+0000 7f3d54926700 1 --2- 172.21.15.19:0/3853717767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d501034c0 0x7f3d501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.214+0000 7f3d54926700 1 --2- 172.21.15.19:0/3853717767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50107df0 0x7f3d50108200 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.214+0000 7f3d54926700 1 -- 172.21.15.19:0/3853717767 >> 172.21.15.19:0/3853717767 conn(0x7f3d500fedc0 msgr2=0x7f3d501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.214+0000 7f3d54926700 1 -- 172.21.15.19:0/3853717767 shutdown_connections 2024-05-09T01:32:17.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.214+0000 7f3d54926700 1 -- 172.21.15.19:0/3853717767 wait complete. 2024-05-09T01:32:17.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.215+0000 7f3d54926700 1 Processor -- start 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.215+0000 7f3d54926700 1 -- start start 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.215+0000 7f3d54926700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d501034c0 0x7f3d5019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.215+0000 7f3d54926700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d50107df0 0x7f3d5019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d54926700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50108740 0x7f3d5019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d54926700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d501133a0 con 0x7f3d50108740 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d54926700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d50113220 con 0x7f3d501034c0 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d54926700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d50113520 con 0x7f3d50107df0 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50108740 0x7f3d5019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50108740 0x7f3d5019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52840/0 (socket says 172.21.15.19:52840) 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d501034c0 0x7f3d5019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/4064073969 learned_addr learned my addr 172.21.15.19:0/4064073969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4dd9b700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d50107df0 0x7f3d5019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/4064073969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d50107df0 msgr2=0x7f3d5019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d50107df0 0x7f3d5019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/4064073969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d501034c0 msgr2=0x7f3d5019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d501034c0 0x7f3d5019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/4064073969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5019fcd0 con 0x7f3d50108740 2024-05-09T01:32:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d501034c0 0x7f3d5019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:32:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.216+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50108740 0x7f3d5019f630 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800ada0 tx=0x7f3d4800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.217+0000 7f3d3f7fe700 1 -- 172.21.15.19:0/4064073969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48017020 con 0x7f3d50108740 2024-05-09T01:32:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.217+0000 7f3d54926700 1 -- 172.21.15.19:0/4064073969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5019ffc0 con 0x7f3d50108740 2024-05-09T01:32:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.217+0000 7f3d3f7fe700 1 -- 172.21.15.19:0/4064073969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4800cd80 con 0x7f3d50108740 2024-05-09T01:32:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.217+0000 7f3d3f7fe700 1 -- 172.21.15.19:0/4064073969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48005260 con 0x7f3d50108740 2024-05-09T01:32:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.217+0000 7f3d54926700 1 -- 172.21.15.19:0/4064073969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d501a78e0 con 0x7f3d50108740 2024-05-09T01:32:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.220+0000 7f3d3f7fe700 1 -- 172.21.15.19:0/4064073969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3d4800c8d0 con 0x7f3d50108740 2024-05-09T01:32:17.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.221+0000 7f3d3f7fe700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d380b2880 0x7f3d380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.222+0000 7f3d3f7fe700 1 -- 172.21.15.19:0/4064073969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3d480e19d0 con 0x7f3d50108740 2024-05-09T01:32:17.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.222+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d380b2880 0x7f3d380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.222+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d380b2880 0x7f3d380b4d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3d40007fd0 tx=0x7f3d400073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.222+0000 7f3d6333cb80 1 Processor -- start 2024-05-09T01:32:17.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.222+0000 7f3d6333cb80 1 -- start start 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55793742a160 0x5579374c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579373404a0 0x5579374c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5579373dfff0 con 0x5579374c6400 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55793737e7f0 con 0x55793742a160 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5579373dbfb0 con 0x5579373404a0 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52856/0 (socket says 172.21.15.19:52856) 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/3405342833 learned_addr learned my addr 172.21.15.19:0/3405342833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4dd9b700 1 --2- 172.21.15.19:0/3405342833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579373404a0 0x5579374c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/3405342833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55793742a160 0x5579374c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/3405342833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579373404a0 msgr2=0x5579374c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/3405342833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579373404a0 0x5579374c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/3405342833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55793742a160 msgr2=0x5579374c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/3405342833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55793742a160 0x5579374c1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.223+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/3405342833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557937376370 con 0x5579374c6400 2024-05-09T01:32:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.224+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/3405342833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c6810 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800ad70 tx=0x7f3d480a8a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65b4c1fee5c3016 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.224+0000 7f3d3cff9700 1 -- 172.21.15.19:0/3405342833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d480a9700 con 0x5579374c6400 2024-05-09T01:32:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.224+0000 7f3d3cff9700 1 -- 172.21.15.19:0/3405342833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d480a9000 con 0x5579374c6400 2024-05-09T01:32:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.224+0000 7f3d3cff9700 1 -- 172.21.15.19:0/3405342833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d480b1940 con 0x5579374c6400 2024-05-09T01:32:17.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 -- 172.21.15.19:0/3405342833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 msgr2=0x5579374c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/3405342833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c6810 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800ad70 tx=0x7f3d480a8a60 comp rx=0 tx=0).stop 2024-05-09T01:32:17.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 -- 172.21.15.19:0/3405342833 shutdown_connections 2024-05-09T01:32:17.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/3405342833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c6810 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/3405342833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55793742a160 0x5579374c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/3405342833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5579373404a0 0x5579374c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 -- 172.21.15.19:0/3405342833 >> 172.21.15.19:0/3405342833 conn(0x5579373a06a0 msgr2=0x5579374b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 -- 172.21.15.19:0/3405342833 shutdown_connections 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 -- 172.21.15.19:0/3405342833 wait complete. 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.225+0000 7f3d6333cb80 1 -- start start 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d6333cb80 1 Processor -- start 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55793725bba0 con 0x5579373404a0 2024-05-09T01:32:17.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55793739a150 con 0x5579374c6400 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5579374c7150 con 0x55793742a160 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d6333cb80 10 client.0 _ll_register_callbacks cb 0x5579372516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52864/0 (socket says 172.21.15.19:52864) 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4e59c700 1 -- 172.21.15.19:0/2677259847 learned_addr learned my addr 172.21.15.19:0/2677259847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4dd9b700 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4e59c700 1 -- 172.21.15.19:0/2677259847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 msgr2=0x5579374cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4e59c700 1 -- 172.21.15.19:0/2677259847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 msgr2=0x5579374cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4e59c700 1 -- 172.21.15.19:0/2677259847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793725b490 con 0x5579373404a0 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.226+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374cb660 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3d40011010 tx=0x7f3d400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a871f978039d3ea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d35ffb700 1 -- 172.21.15.19:0/2677259847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40024070 con 0x5579373404a0 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d35ffb700 1 -- 172.21.15.19:0/2677259847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3d40011440 con 0x5579373404a0 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d6333cb80 1 -- 172.21.15.19:0/2677259847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d480a9000 con 0x5579373404a0 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d35ffb700 1 -- 172.21.15.19:0/2677259847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d40030050 con 0x5579373404a0 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d6333cb80 10 client.17172 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d6333cb80 1 -- 172.21.15.19:0/2677259847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579374d8f70 con 0x5579373404a0 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d35ffb700 1 -- 172.21.15.19:0/2677259847 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d480a9000 con 0x5579373404a0 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d35ffb700 1 -- 172.21.15.19:0/2677259847 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3d40020710 con 0x5579373404a0 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.227+0000 7f3d6333cb80 10 client.17172 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.228+0000 7f3d6333cb80 10 client.17172 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.228+0000 7f3d6333cb80 20 client.17172 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.228+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5579374db550 0x5579374dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.228+0000 7f3d6333cb80 4 client.17172 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:17.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.228+0000 7f3d6333cb80 1 -- 172.21.15.19:0/2677259847 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579373ed9d0 con 0x5579374db550 2024-05-09T01:32:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.228+0000 7f3d4dd9b700 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5579374db550 0x5579374dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.228+0000 7f3d4dd9b700 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5579374db550 0x5579374dd9f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2f87df6e44d99c3 server_cookie=1746f7640b1c086f in_seq=0 out_seq=0 2024-05-09T01:32:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.228+0000 7f3d35ffb700 10 client.17172 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:32:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d35ffb700 1 -- 172.21.15.19:0/2677259847 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5579373ed9d0 con 0x5579374db550 2024-05-09T01:32:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d35ffb700 10 client.17172 handle_command_reply: tid=0 2024-05-09T01:32:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 client.17172 shutdown 2024-05-09T01:32:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 -- 172.21.15.19:0/2677259847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 msgr2=0x5579374cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374cb660 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3d40011010 tx=0x7f3d400047c0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 -- 172.21.15.19:0/2677259847 shutdown_connections 2024-05-09T01:32:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5579374db550 0x5579374dd9f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374cb660 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/2677259847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 -- 172.21.15.19:0/2677259847 >> 172.21.15.19:0/2677259847 conn(0x5579373a06a0 msgr2=0x5579374b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d35ffb700 0 client.17172 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:32:17.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 -- 172.21.15.19:0/2677259847 shutdown_connections 2024-05-09T01:32:17.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.229+0000 7f3d6333cb80 1 -- 172.21.15.19:0/2677259847 wait complete. 2024-05-09T01:32:17.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.230+0000 7f3d6333cb80 20 client.17172 trim_cache size 0 max 16384 2024-05-09T01:32:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d6333cb80 1 Processor -- start 2024-05-09T01:32:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d6333cb80 1 -- start start 2024-05-09T01:32:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d501aca00 con 0x5579373404a0 2024-05-09T01:32:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d501ae540 con 0x5579374c6400 2024-05-09T01:32:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d501ad0d0 con 0x55793742a160 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d4e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d4e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52874/0 (socket says 172.21.15.19:52874) 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.245+0000 7f3d4e59c700 1 -- 172.21.15.19:0/483176301 learned_addr learned my addr 172.21.15.19:0/483176301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4dd9b700 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4e59c700 1 -- 172.21.15.19:0/483176301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 msgr2=0x5579374cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4e59c700 1 -- 172.21.15.19:0/483176301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 msgr2=0x5579374bc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374bc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4e59c700 1 -- 172.21.15.19:0/483176301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579374b9ec0 con 0x5579373404a0 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4dd9b700 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374bc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374ca9a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3d40000c00 tx=0x7f3d400283b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754874b0faaa0eb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d37fff700 1 -- 172.21.15.19:0/483176301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40038020 con 0x5579373404a0 2024-05-09T01:32:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d37fff700 1 -- 172.21.15.19:0/483176301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d40015070 con 0x5579373404a0 2024-05-09T01:32:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.246+0000 7f3d37fff700 1 -- 172.21.15.19:0/483176301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40024070 con 0x5579373404a0 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 -- 172.21.15.19:0/483176301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 msgr2=0x5579374ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374ca9a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3d40000c00 tx=0x7f3d400283b0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 -- 172.21.15.19:0/483176301 shutdown_connections 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579374c6400 0x5579374bc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579373404a0 0x5579374ca9a0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/483176301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 -- 172.21.15.19:0/483176301 >> 172.21.15.19:0/483176301 conn(0x5579373a06a0 msgr2=0x5579374c5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 -- 172.21.15.19:0/483176301 shutdown_connections 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 -- 172.21.15.19:0/483176301 wait complete. 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 -- start start 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.247+0000 7f3d6333cb80 1 Processor -- start 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579373404a0 0x5579374c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374c8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d6333cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d501ade70 con 0x5579374c6400 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d501ad7a0 con 0x5579373404a0 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d6333cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d501aa9c0 con 0x55793742a160 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d6333cb80 10 client.0 _ll_register_callbacks cb 0x5579372516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52890/0 (socket says 172.21.15.19:52890) 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/129874281 learned_addr learned my addr 172.21.15.19:0/129874281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4dd9b700 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579373404a0 0x5579374c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/129874281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 msgr2=0x5579374c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/129874281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579373404a0 msgr2=0x5579374c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579373404a0 0x5579374c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4ed9d700 1 -- 172.21.15.19:0/129874281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579374b1f60 con 0x5579374c6400 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4dd9b700 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d4ed9d700 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c8540 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3d480a6270 tx=0x7f3d480a9490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23710b539779a84a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.248+0000 7f3d35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d35ffb700 1 -- 172.21.15.19:0/129874281 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d480a9000 con 0x5579374c6400 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d35ffb700 1 -- 172.21.15.19:0/129874281 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3d480b1b50 con 0x5579374c6400 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d6333cb80 1 -- 172.21.15.19:0/129874281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d40015070 con 0x5579374c6400 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d35ffb700 1 -- 172.21.15.19:0/129874281 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d480a9700 con 0x5579374c6400 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d6333cb80 10 client.17178 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d6333cb80 1 -- 172.21.15.19:0/129874281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579374d3710 con 0x5579374c6400 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d35ffb700 1 -- 172.21.15.19:0/129874281 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d40015070 con 0x5579374c6400 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d35ffb700 1 -- 172.21.15.19:0/129874281 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3d48012070 con 0x5579374c6400 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d6333cb80 10 client.17178 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d6333cb80 10 client.17178 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d6333cb80 20 client.17178 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5579374d3af0 0x557937547d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d6333cb80 4 client.17178 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.249+0000 7f3d6333cb80 1 -- 172.21.15.19:0/129874281 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579373e7a70 con 0x5579374d3af0 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5579374d3af0 0x557937547d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d35ffb700 10 client.17178 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d4e59c700 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5579374d3af0 0x557937547d70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bdf289b204b68d2 server_cookie=43cd5a9e2ebc57d1 in_seq=0 out_seq=0 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d35ffb700 1 -- 172.21.15.19:0/129874281 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579373e7a70 con 0x5579374d3af0 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d35ffb700 10 client.17178 handle_command_reply: tid=0 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d6333cb80 1 client.17178 shutdown 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d6333cb80 1 -- 172.21.15.19:0/129874281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 msgr2=0x5579374c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c8540 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3d480a6270 tx=0x7f3d480a9490 comp rx=0 tx=0).stop 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d6333cb80 1 -- 172.21.15.19:0/129874281 shutdown_connections 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5579374d3af0 0x557937547d70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579374c6400 0x5579374c8540 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579373404a0 0x5579374c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d35ffb700 0 client.17178 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:32:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d6333cb80 1 --2- 172.21.15.19:0/129874281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55793742a160 0x5579374c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.250+0000 7f3d6333cb80 1 -- 172.21.15.19:0/129874281 >> 172.21.15.19:0/129874281 conn(0x5579373a06a0 msgr2=0x5579374c5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d6333cb80 1 -- 172.21.15.19:0/129874281 shutdown_connections 2024-05-09T01:32:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d6333cb80 1 -- 172.21.15.19:0/129874281 wait complete. 2024-05-09T01:32:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d6333cb80 20 client.17178 trim_cache size 0 max 16384 2024-05-09T01:32:17.253 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 -- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d380b2880 msgr2=0x7f3d380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d380b2880 0x7f3d380b4d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3d40007fd0 tx=0x7f3d400073d0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 -- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50108740 msgr2=0x7f3d5019f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50108740 0x7f3d5019f630 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800ada0 tx=0x7f3d4800c330 comp rx=0 tx=0).stop 2024-05-09T01:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 -- 172.21.15.19:0/4064073969 shutdown_connections 2024-05-09T01:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d380b2880 0x7f3d380b4d90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d50108740 0x7f3d5019f630 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d501034c0 0x7f3d5019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 --2- 172.21.15.19:0/4064073969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d50107df0 0x7f3d5019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 -- 172.21.15.19:0/4064073969 >> 172.21.15.19:0/4064073969 conn(0x7f3d500fedc0 msgr2=0x7f3d5006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 -- 172.21.15.19:0/4064073969 shutdown_connections 2024-05-09T01:32:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:17.251+0000 7f3d54926700 1 -- 172.21.15.19:0/4064073969 wait complete. 2024-05-09T01:32:18.002 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:32:18.259 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:18.639 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:18 smithi019 ceph-mon[30158]: pgmap v581: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-05-09T01:32:18.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:18 smithi059 ceph-mon[34351]: pgmap v581: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-05-09T01:32:18.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:18 smithi196 ceph-mon[35041]: pgmap v581: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-05-09T01:32:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.970+0000 7fbfc5999700 1 -- 172.21.15.19:0/1657426376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010e910 msgr2=0x7fbfc0110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.970+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1657426376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010e910 0x7fbfc0110df0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800b0a0 tx=0x7fbfb800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.970+0000 7fbfc5999700 1 -- 172.21.15.19:0/1657426376 shutdown_connections 2024-05-09T01:32:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.970+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1657426376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc010e910 0x7fbfc0110df0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.970+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1657426376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc01054e0 0x7fbfc01058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.970+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1657426376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc0105e30 0x7fbfc010e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.970+0000 7fbfc5999700 1 -- 172.21.15.19:0/1657426376 >> 172.21.15.19:0/1657426376 conn(0x7fbfc00fef30 msgr2=0x7fbfc0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.971+0000 7fbfc5999700 1 -- 172.21.15.19:0/1657426376 shutdown_connections 2024-05-09T01:32:19.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.971+0000 7fbfc5999700 1 -- 172.21.15.19:0/1657426376 wait complete. 2024-05-09T01:32:19.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.971+0000 7fbfc5999700 1 Processor -- start 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfc5999700 1 -- start start 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfc5999700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc01054e0 0x7fbfc019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfc5999700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc0105e30 0x7fbfc019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfc5999700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc010e910 0x7fbfc019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfc5999700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0114470 con 0x7fbfc0105e30 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfc5999700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01142f0 con 0x7fbfc010e910 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfc5999700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01145f0 con 0x7fbfc01054e0 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfb67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc0105e30 0x7fbfc019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfb67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc0105e30 0x7fbfc019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53814/0 (socket says 172.21.15.19:53814) 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfb67fc700 1 -- 172.21.15.19:0/1774278032 learned_addr learned my addr 172.21.15.19:0/1774278032 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfbf7fe700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc010e910 0x7fbfc019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.972+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc01054e0 0x7fbfc019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfb67fc700 1 -- 172.21.15.19:0/1774278032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc01054e0 msgr2=0x7fbfc019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc01054e0 0x7fbfc019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfb67fc700 1 -- 172.21.15.19:0/1774278032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc010e910 msgr2=0x7fbfc019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc010e910 0x7fbfc019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfb67fc700 1 -- 172.21.15.19:0/1774278032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc019fb80 con 0x7fbfc0105e30 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc0105e30 0x7fbfc019b300 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800c920 tx=0x7fbfa800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfc4997700 1 -- 172.21.15.19:0/1774278032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8004a30 con 0x7fbfc0105e30 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfc4997700 1 -- 172.21.15.19:0/1774278032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa80054d0 con 0x7fbfc0105e30 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfc4997700 1 -- 172.21.15.19:0/1774278032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8007c70 con 0x7fbfc0105e30 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfc5999700 1 -- 172.21.15.19:0/1774278032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc019fe70 con 0x7fbfc0105e30 2024-05-09T01:32:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.973+0000 7fbfc5999700 1 -- 172.21.15.19:0/1774278032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc01a0360 con 0x7fbfc0105e30 2024-05-09T01:32:19.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.976+0000 7fbfc4997700 1 -- 172.21.15.19:0/1774278032 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbfa8004c60 con 0x7fbfc0105e30 2024-05-09T01:32:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.977+0000 7fbfc4997700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbfac0b2880 0x7fbfac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.977+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbfac0b2880 0x7fbfac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.977+0000 7fbfd43afb80 1 Processor -- start 2024-05-09T01:32:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.977+0000 7fbfc4997700 1 -- 172.21.15.19:0/1774278032 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbfa80e1fe0 con 0x7fbfc0105e30 2024-05-09T01:32:19.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.977+0000 7fbfd43afb80 1 -- start start 2024-05-09T01:32:19.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.977+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e962240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.977+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7df4a0 0x55756e962780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.977+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e95c400 0x55756e95c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.977+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55756e87f0a0 con 0x55756e859d00 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55756e8249a0 con 0x55756e7df4a0 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55756e87b060 con 0x55756e95c400 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbfac0b2880 0x7fbfac0b4d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000ac50 tx=0x7fbfb000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e962240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e962240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53820/0 (socket says 172.21.15.19:53820) 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 -- 172.21.15.19:0/2867933742 learned_addr learned my addr 172.21.15.19:0/2867933742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/2867933742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7df4a0 0x55756e962780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 -- 172.21.15.19:0/2867933742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e95c400 msgr2=0x55756e95c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/2867933742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e95c400 0x55756e95c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 -- 172.21.15.19:0/2867933742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7df4a0 msgr2=0x55756e962780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/2867933742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7df4a0 0x55756e962780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 -- 172.21.15.19:0/2867933742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756e815500 con 0x55756e859d00 2024-05-09T01:32:19.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/2867933742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7df4a0 0x55756e962780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:19.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/2867933742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e962240 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000ff90 tx=0x7fbfb0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80d2b8511170123 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:19.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfb5ffb700 1 -- 172.21.15.19:0/2867933742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0016070 con 0x55756e859d00 2024-05-09T01:32:19.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfb5ffb700 1 -- 172.21.15.19:0/2867933742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb0014070 con 0x55756e859d00 2024-05-09T01:32:19.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.978+0000 7fbfb5ffb700 1 -- 172.21.15.19:0/2867933742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0025400 con 0x55756e859d00 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.979+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2867933742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 msgr2=0x55756e962240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.979+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/2867933742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e962240 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000ff90 tx=0x7fbfb0012000 comp rx=0 tx=0).stop 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.979+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2867933742 shutdown_connections 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.979+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/2867933742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e95c400 0x55756e95c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.979+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/2867933742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e962240 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.979+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/2867933742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7df4a0 0x55756e962780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.979+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2867933742 >> 172.21.15.19:0/2867933742 conn(0x55756e7e2f70 msgr2=0x55756e957220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.979+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2867933742 shutdown_connections 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.979+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2867933742 wait complete. 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.980+0000 7fbfd43afb80 1 -- start start 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.980+0000 7fbfd43afb80 1 Processor -- start 2024-05-09T01:32:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.980+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 0x55756e965f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.980+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e859d00 0x55756e9664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.980+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e95c400 0x55756e9669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.980+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55756e95d160 con 0x55756e95c400 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.980+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55756e819b20 con 0x55756e859d00 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.980+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55756e969020 con 0x55756e7df4a0 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfd43afb80 10 client.0 _ll_register_callbacks cb 0x55756e6f06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e95c400 0x55756e9669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e95c400 0x55756e9669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53826/0 (socket says 172.21.15.19:53826) 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbf7fe700 1 -- 172.21.15.19:0/3916661141 learned_addr learned my addr 172.21.15.19:0/3916661141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e859d00 0x55756e9664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 0x55756e965f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbf7fe700 1 -- 172.21.15.19:0/3916661141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 msgr2=0x55756e965f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbf7fe700 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 0x55756e965f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbf7fe700 1 -- 172.21.15.19:0/3916661141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e859d00 msgr2=0x55756e9664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbf7fe700 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e859d00 0x55756e9664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbf7fe700 1 -- 172.21.15.19:0/3916661141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756e6fa490 con 0x55756e95c400 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfbf7fe700 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e95c400 0x55756e9669e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8004c40 tx=0x7fbfb8004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd57fb1540f0fb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbf9effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbf9effd700 1 -- 172.21.15.19:0/3916661141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80079d0 con 0x55756e95c400 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbf9effd700 1 -- 172.21.15.19:0/3916661141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbfb8007b70 con 0x55756e95c400 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.981+0000 7fbfd43afb80 1 -- 172.21.15.19:0/3916661141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb0014070 con 0x55756e95c400 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbf9effd700 1 -- 172.21.15.19:0/3916661141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbfb802c070 con 0x55756e95c400 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbfd43afb80 10 client.17190 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbfd43afb80 1 -- 172.21.15.19:0/3916661141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55756e973930 con 0x55756e95c400 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbf9effd700 1 -- 172.21.15.19:0/3916661141 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb0014070 con 0x55756e95c400 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbf9effd700 1 -- 172.21.15.19:0/3916661141 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbfb8011070 con 0x55756e95c400 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbfd43afb80 10 client.17190 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbfd43afb80 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbfd43afb80 20 client.17190 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55756e975f70 0x55756e978390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbfd43afb80 4 client.17190 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:19.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbfd43afb80 1 -- 172.21.15.19:0/3916661141 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55756e88ca80 con 0x55756e975f70 2024-05-09T01:32:19.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.982+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55756e975f70 0x55756e978390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.983+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55756e975f70 0x55756e978390 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca671fdd4720acfc server_cookie=94892e17a23fbd32 in_seq=0 out_seq=0 2024-05-09T01:32:19.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.983+0000 7fbf9effd700 10 client.17190 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:19.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbf9effd700 1 -- 172.21.15.19:0/3916661141 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55756e88ca80 con 0x55756e975f70 2024-05-09T01:32:19.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbf9effd700 10 client.17190 handle_command_reply: tid=0 2024-05-09T01:32:19.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 client.17190 shutdown 2024-05-09T01:32:19.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 -- 172.21.15.19:0/3916661141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e95c400 msgr2=0x55756e9669e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:19.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e95c400 0x55756e9669e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8004c40 tx=0x7fbfb8004c70 comp rx=0 tx=0).stop 2024-05-09T01:32:19.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 -- 172.21.15.19:0/3916661141 shutdown_connections 2024-05-09T01:32:19.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55756e975f70 0x55756e978390 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e95c400 0x55756e9669e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 0x55756e965f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbf9effd700 0 client.17190 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/3916661141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e859d00 0x55756e9664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 -- 172.21.15.19:0/3916661141 >> 172.21.15.19:0/3916661141 conn(0x55756e7e2f70 msgr2=0x55756e950ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 -- 172.21.15.19:0/3916661141 shutdown_connections 2024-05-09T01:32:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 1 -- 172.21.15.19:0/3916661141 wait complete. 2024-05-09T01:32:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.984+0000 7fbfd43afb80 20 client.17190 trim_cache size 0 max 16384 2024-05-09T01:32:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:19.999+0000 7fbfd43afb80 1 Processor -- start 2024-05-09T01:32:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfd43afb80 1 -- start start 2024-05-09T01:32:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 0x55756e966820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 0x55756e9603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e971dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01aeb80 con 0x55756e859d00 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01b06c0 con 0x55756e7e2f70 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01af250 con 0x55756e7df4a0 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e971dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e971dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53834/0 (socket says 172.21.15.19:53834) 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbf7fe700 1 -- 172.21.15.19:0/2806347957 learned_addr learned my addr 172.21.15.19:0/2806347957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/2806347957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 0x55756e966820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/2806347957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 0x55756e9603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbf7fe700 1 -- 172.21.15.19:0/2806347957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 msgr2=0x55756e9603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbf7fe700 1 --2- 172.21.15.19:0/2806347957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 0x55756e9603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbf7fe700 1 -- 172.21.15.19:0/2806347957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 msgr2=0x55756e966820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbf7fe700 1 --2- 172.21.15.19:0/2806347957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 0x55756e966820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbf7fe700 1 -- 172.21.15.19:0/2806347957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756e968800 con 0x55756e859d00 2024-05-09T01:32:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.000+0000 7fbfbf7fe700 1 --2- 172.21.15.19:0/2806347957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e971dd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80059b0 tx=0x7fbfb8009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72ce40a4baec827 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfb57fa700 1 -- 172.21.15.19:0/2806347957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8035020 con 0x55756e859d00 2024-05-09T01:32:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfb57fa700 1 -- 172.21.15.19:0/2806347957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb8031480 con 0x55756e859d00 2024-05-09T01:32:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfb57fa700 1 -- 172.21.15.19:0/2806347957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8007e00 con 0x55756e859d00 2024-05-09T01:32:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2806347957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 msgr2=0x55756e971dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/2806347957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e971dd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80059b0 tx=0x7fbfb8009dc0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2806347957 shutdown_connections 2024-05-09T01:32:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/2806347957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e859d00 0x55756e971dd0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/2806347957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 0x55756e966820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/2806347957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e7df4a0 0x55756e9603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2806347957 >> 172.21.15.19:0/2806347957 conn(0x55756e957520 msgr2=0x55756e95fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.001+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2806347957 shutdown_connections 2024-05-09T01:32:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.002+0000 7fbfd43afb80 1 -- 172.21.15.19:0/2806347957 wait complete. 2024-05-09T01:32:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.002+0000 7fbfd43afb80 1 -- start start 2024-05-09T01:32:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.002+0000 7fbfd43afb80 1 Processor -- start 2024-05-09T01:32:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.002+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e7df4a0 0x55756e96e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.002+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 0x55756e96e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.002+0000 7fbfd43afb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e859d00 0x55756e97caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01afff0 con 0x55756e7df4a0 2024-05-09T01:32:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01af920 con 0x55756e7e2f70 2024-05-09T01:32:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfd43afb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01acb20 con 0x55756e859d00 2024-05-09T01:32:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfd43afb80 10 client.0 _ll_register_callbacks cb 0x55756e6f06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e7df4a0 0x55756e96e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e7df4a0 0x55756e96e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53842/0 (socket says 172.21.15.19:53842) 2024-05-09T01:32:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbeffd700 1 -- 172.21.15.19:0/4141851096 learned_addr learned my addr 172.21.15.19:0/4141851096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 0x55756e96e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbeffd700 1 -- 172.21.15.19:0/4141851096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e859d00 msgr2=0x55756e97caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbf7fe700 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e859d00 0x55756e97caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e859d00 0x55756e97caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbeffd700 1 -- 172.21.15.19:0/4141851096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 msgr2=0x55756e96e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 0x55756e96e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbeffd700 1 -- 172.21.15.19:0/4141851096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756e97d030 con 0x55756e7df4a0 2024-05-09T01:32:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 0x55756e96e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbf7fe700 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e859d00 0x55756e97caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfbeffd700 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e7df4a0 0x55756e96e100 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0010b10 tx=0x7fbfb0010620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dbaa0e67880ad4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbf9effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbf9effd700 1 -- 172.21.15.19:0/4141851096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0003d20 con 0x55756e7df4a0 2024-05-09T01:32:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbf9effd700 1 -- 172.21.15.19:0/4141851096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbfb0033110 con 0x55756e7df4a0 2024-05-09T01:32:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.003+0000 7fbfd43afb80 1 -- 172.21.15.19:0/4141851096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb8031480 con 0x55756e7df4a0 2024-05-09T01:32:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbf9effd700 1 -- 172.21.15.19:0/4141851096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbfb0033310 con 0x55756e7df4a0 2024-05-09T01:32:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbfd43afb80 10 client.17196 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbfd43afb80 1 -- 172.21.15.19:0/4141851096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55756e9dfe10 con 0x55756e7df4a0 2024-05-09T01:32:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbf9effd700 1 -- 172.21.15.19:0/4141851096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb8031480 con 0x55756e7df4a0 2024-05-09T01:32:20.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbf9effd700 1 -- 172.21.15.19:0/4141851096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbfb003d6a0 con 0x55756e7df4a0 2024-05-09T01:32:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbfd43afb80 10 client.17196 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbfd43afb80 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbfd43afb80 20 client.17196 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55756e95c400 0x55756e9e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbfd43afb80 4 client.17196 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:32:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbfd43afb80 1 -- 172.21.15.19:0/4141851096 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55756e886b20 con 0x55756e95c400 2024-05-09T01:32:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.004+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55756e95c400 0x55756e9e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfb67fc700 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55756e95c400 0x55756e9e2250 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4c5c91217456f6 server_cookie=bec48aadb7cd050 in_seq=0 out_seq=0 2024-05-09T01:32:20.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbf9effd700 10 client.17196 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:20.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbf9effd700 1 -- 172.21.15.19:0/4141851096 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55756e886b20 con 0x55756e95c400 2024-05-09T01:32:20.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbf9effd700 10 client.17196 handle_command_reply: tid=0 2024-05-09T01:32:20.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 client.17196 shutdown 2024-05-09T01:32:20.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 -- 172.21.15.19:0/4141851096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e7df4a0 msgr2=0x55756e96e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:20.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e7df4a0 0x55756e96e100 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0010b10 tx=0x7fbfb0010620 comp rx=0 tx=0).stop 2024-05-09T01:32:20.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 -- 172.21.15.19:0/4141851096 shutdown_connections 2024-05-09T01:32:20.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55756e95c400 0x55756e9e2250 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbf9effd700 0 client.17196 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55756e859d00 0x55756e97caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756e7df4a0 0x55756e96e100 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 --2- 172.21.15.19:0/4141851096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55756e7e2f70 0x55756e96e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 -- 172.21.15.19:0/4141851096 >> 172.21.15.19:0/4141851096 conn(0x55756e957520 msgr2=0x55756e950ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 -- 172.21.15.19:0/4141851096 shutdown_connections 2024-05-09T01:32:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 1 -- 172.21.15.19:0/4141851096 wait complete. 2024-05-09T01:32:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.005+0000 7fbfd43afb80 20 client.17196 trim_cache size 0 max 16384 2024-05-09T01:32:20.015 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:32:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 -- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbfac0b2880 msgr2=0x7fbfac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbfac0b2880 0x7fbfac0b4d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000ac50 tx=0x7fbfb000a380 comp rx=0 tx=0).stop 2024-05-09T01:32:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 -- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc0105e30 msgr2=0x7fbfc019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc0105e30 0x7fbfc019b300 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800c920 tx=0x7fbfa800cc30 comp rx=0 tx=0).stop 2024-05-09T01:32:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 -- 172.21.15.19:0/1774278032 shutdown_connections 2024-05-09T01:32:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc010e910 0x7fbfc019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc01054e0 0x7fbfc019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbfac0b2880 0x7fbfac0b4d90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 --2- 172.21.15.19:0/1774278032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfc0105e30 0x7fbfc019b300 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 -- 172.21.15.19:0/1774278032 >> 172.21.15.19:0/1774278032 conn(0x7fbfc00fef30 msgr2=0x7fbfc00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 -- 172.21.15.19:0/1774278032 shutdown_connections 2024-05-09T01:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:20.006+0000 7fbfc5999700 1 -- 172.21.15.19:0/1774278032 wait complete. 2024-05-09T01:32:20.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:20 smithi019 ceph-mon[30158]: pgmap v582: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:32:20.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:20 smithi059 ceph-mon[34351]: pgmap v582: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:32:20.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:20 smithi196 ceph-mon[35041]: pgmap v582: 97 pgs: 97 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:32:21.789 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:32:21.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:32:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:22 smithi059 ceph-mon[34351]: pgmap v583: 97 pgs: 97 active+clean; 4.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:22.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:22 smithi019 ceph-mon[30158]: pgmap v583: 97 pgs: 97 active+clean; 4.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:22.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:22 smithi196 ceph-mon[35041]: pgmap v583: 97 pgs: 97 active+clean; 4.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.786+0000 7f19ac5f6700 1 -- 172.21.15.19:0/584476634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 msgr2=0x7f19a4103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.786+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/584476634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 0x7f19a4103d70 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1994009990 tx=0x7f1994009ca0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.787+0000 7f19ac5f6700 1 -- 172.21.15.19:0/584476634 shutdown_connections 2024-05-09T01:32:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.787+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/584476634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a41042b0 0x7f19a4110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.787+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/584476634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19a4108200 0x7f19a4108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.787+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/584476634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 0x7f19a4103d70 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.787+0000 7f19ac5f6700 1 -- 172.21.15.19:0/584476634 >> 172.21.15.19:0/584476634 conn(0x7f19a40fe4e0 msgr2=0x7f19a4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.787+0000 7f19ac5f6700 1 -- 172.21.15.19:0/584476634 shutdown_connections 2024-05-09T01:32:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.787+0000 7f19ac5f6700 1 -- 172.21.15.19:0/584476634 wait complete. 2024-05-09T01:32:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.787+0000 7f19ac5f6700 1 Processor -- start 2024-05-09T01:32:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.787+0000 7f19ac5f6700 1 -- start start 2024-05-09T01:32:23.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19ac5f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 0x7f19a419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19ac5f6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19a41042b0 0x7f19a419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19ac5f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a4108200 0x7f19a419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19ac5f6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41133d0 con 0x7f19a4103960 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19ac5f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4113250 con 0x7f19a4108200 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19ac5f6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4113550 con 0x7f19a41042b0 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aa392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 0x7f19a419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aa392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 0x7f19a419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53856/0 (socket says 172.21.15.19:53856) 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aa392700 1 -- 172.21.15.19:0/2070834271 learned_addr learned my addr 172.21.15.19:0/2070834271 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19a9b91700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19a41042b0 0x7f19a419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aab93700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a4108200 0x7f19a419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aa392700 1 -- 172.21.15.19:0/2070834271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19a41042b0 msgr2=0x7f19a419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aa392700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19a41042b0 0x7f19a419c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aa392700 1 -- 172.21.15.19:0/2070834271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a4108200 msgr2=0x7f19a419c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aa392700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a4108200 0x7f19a419c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aa392700 1 -- 172.21.15.19:0/2070834271 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a41a6fe0 con 0x7f19a4103960 2024-05-09T01:32:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.788+0000 7f19aab93700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a4108200 0x7f19a419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:32:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.789+0000 7f19aa392700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 0x7f19a419bc70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f19a0007ee0 tx=0x7f19a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.789+0000 7f199b7fe700 1 -- 172.21.15.19:0/2070834271 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a000dc90 con 0x7f19a4103960 2024-05-09T01:32:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.789+0000 7f19ac5f6700 1 -- 172.21.15.19:0/2070834271 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a41a72d0 con 0x7f19a4103960 2024-05-09T01:32:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.789+0000 7f199b7fe700 1 -- 172.21.15.19:0/2070834271 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a000de30 con 0x7f19a4103960 2024-05-09T01:32:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.789+0000 7f199b7fe700 1 -- 172.21.15.19:0/2070834271 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a00133f0 con 0x7f19a4103960 2024-05-09T01:32:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.789+0000 7f19ac5f6700 1 -- 172.21.15.19:0/2070834271 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a41a7860 con 0x7f19a4103960 2024-05-09T01:32:23.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.793+0000 7f199b7fe700 1 -- 172.21.15.19:0/2070834271 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f19a001a020 con 0x7f19a4103960 2024-05-09T01:32:23.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.793+0000 7f19bb00cb80 1 Processor -- start 2024-05-09T01:32:23.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19bb00cb80 1 -- start start 2024-05-09T01:32:23.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f199b7fe700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19900b2880 0x7f19900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994631e0 0x5635994fc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563599379520 0x5635994fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19aab93700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19900b2880 0x7f19900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994f6720 0x5635994f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563599419070 con 0x5635994f6720 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5635993b7870 con 0x563599379520 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563599415030 con 0x5635994631e0 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f199b7fe700 1 -- 172.21.15.19:0/2070834271 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f19a00e1a20 con 0x7f19a4103960 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19aab93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994f6720 0x5635994f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19aab93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994f6720 0x5635994f6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53862/0 (socket says 172.21.15.19:53862) 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19aab93700 1 -- 172.21.15.19:0/607479220 learned_addr learned my addr 172.21.15.19:0/607479220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19aa392700 1 --2- 172.21.15.19:0/607479220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994631e0 0x5635994fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.794+0000 7f19a9b91700 1 --2- 172.21.15.19:0/607479220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563599379520 0x5635994fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f19aab93700 1 -- 172.21.15.19:0/607479220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994631e0 msgr2=0x5635994fc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f19aab93700 1 --2- 172.21.15.19:0/607479220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994631e0 0x5635994fc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f19aab93700 1 -- 172.21.15.19:0/607479220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563599379520 msgr2=0x5635994fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f19aab93700 1 --2- 172.21.15.19:0/607479220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563599379520 0x5635994fcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f19aab93700 1 -- 172.21.15.19:0/607479220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635993af3f0 con 0x5635994f6720 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f19aab93700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19900b2880 0x7f19900b4d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f199c019d20 tx=0x7f199c009b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f19aab93700 1 --2- 172.21.15.19:0/607479220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994f6720 0x5635994f6b50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f199c0094e0 tx=0x7f199c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1565c64c43e4cf37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f1998ff9700 1 -- 172.21.15.19:0/607479220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c012e70 con 0x5635994f6720 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f1998ff9700 1 -- 172.21.15.19:0/607479220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c0044e0 con 0x5635994f6720 2024-05-09T01:32:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.795+0000 7f1998ff9700 1 -- 172.21.15.19:0/607479220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c004a90 con 0x5635994f6720 2024-05-09T01:32:23.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.796+0000 7f19bb00cb80 1 -- 172.21.15.19:0/607479220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994f6720 msgr2=0x5635994f6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:23.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.796+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/607479220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994f6720 0x5635994f6b50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f199c0094e0 tx=0x7f199c020000 comp rx=0 tx=0).stop 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.796+0000 7f19bb00cb80 1 -- 172.21.15.19:0/607479220 shutdown_connections 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.796+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/607479220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994f6720 0x5635994f6b50 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.796+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/607479220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994631e0 0x5635994fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.796+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/607479220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563599379520 0x5635994fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.796+0000 7f19bb00cb80 1 -- 172.21.15.19:0/607479220 >> 172.21.15.19:0/607479220 conn(0x5635993d9720 msgr2=0x5635994f1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.796+0000 7f19bb00cb80 1 -- 172.21.15.19:0/607479220 shutdown_connections 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.796+0000 7f19bb00cb80 1 -- 172.21.15.19:0/607479220 wait complete. 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19bb00cb80 1 -- start start 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19bb00cb80 1 Processor -- start 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 0x563599500200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x563599500740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 0x563599500c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563599294c20 con 0x5635994631e0 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5635993d31d0 con 0x5635994f6720 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563599503250 con 0x563599379520 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19bb00cb80 10 client.0 _ll_register_callbacks cb 0x56359928a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19a9b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x563599500740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19a9b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x563599500740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53870/0 (socket says 172.21.15.19:53870) 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19aa392700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 0x563599500200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19a9b91700 1 -- 172.21.15.19:0/2526327605 learned_addr learned my addr 172.21.15.19:0/2526327605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.797+0000 7f19aab93700 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 0x563599500c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f19a9b91700 1 -- 172.21.15.19:0/2526327605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 msgr2=0x563599500200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f19a9b91700 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 0x563599500200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f19a9b91700 1 -- 172.21.15.19:0/2526327605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 msgr2=0x563599500c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f19a9b91700 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 0x563599500c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f19a9b91700 1 -- 172.21.15.19:0/2526327605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563599294510 con 0x5635994631e0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f19a9b91700 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x563599500740 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1994000c00 tx=0x7f199400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52203306379d692 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f198dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f198dffb700 1 -- 172.21.15.19:0/2526327605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199400fa40 con 0x5635994631e0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f198dffb700 1 -- 172.21.15.19:0/2526327605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f199400fbe0 con 0x5635994631e0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2526327605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f199c0044e0 con 0x5635994631e0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f198dffb700 1 -- 172.21.15.19:0/2526327605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f199400fde0 con 0x5635994631e0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f19bb00cb80 10 client.17208 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2526327605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56359950dba0 con 0x5635994631e0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.798+0000 7f198dffb700 1 -- 172.21.15.19:0/2526327605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c0044e0 con 0x5635994631e0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.799+0000 7f198dffb700 1 -- 172.21.15.19:0/2526327605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1994003930 con 0x5635994631e0 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.799+0000 7f19bb00cb80 10 client.17208 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.799+0000 7f19bb00cb80 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.799+0000 7f19bb00cb80 20 client.17208 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.799+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563599510130 0x563599512570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.799+0000 7f19bb00cb80 4 client.17208 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.799+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2526327605 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563599426a50 con 0x563599510130 2024-05-09T01:32:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.799+0000 7f19aa392700 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563599510130 0x563599512570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.800+0000 7f198dffb700 10 client.17208 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.800+0000 7f19aa392700 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563599510130 0x563599512570 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e3d21b9e6132b7 server_cookie=331d27bcb1c3a9b7 in_seq=0 out_seq=0 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.800+0000 7f198dffb700 1 -- 172.21.15.19:0/2526327605 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563599426a50 con 0x563599510130 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.800+0000 7f198dffb700 10 client.17208 handle_command_reply: tid=0 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 client.17208 shutdown 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2526327605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 msgr2=0x563599500740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x563599500740 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1994000c00 tx=0x7f199400bde0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2526327605 shutdown_connections 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563599510130 0x563599512570 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 0x563599500c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 0x563599500200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2526327605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x563599500740 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2526327605 >> 172.21.15.19:0/2526327605 conn(0x5635993d9720 msgr2=0x5635994f1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f198dffb700 0 client.17208 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2526327605 shutdown_connections 2024-05-09T01:32:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2526327605 wait complete. 2024-05-09T01:32:23.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.801+0000 7f19bb00cb80 20 client.17208 trim_cache size 0 max 16384 2024-05-09T01:32:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19bb00cb80 1 Processor -- start 2024-05-09T01:32:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19bb00cb80 1 -- start start 2024-05-09T01:32:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 0x563599500bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x56359950c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 0x5635994f5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41acba0 con 0x5635994631e0 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41ae6e0 con 0x5635994f6720 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41ad270 con 0x563599379520 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19a9b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x56359950c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19a9b91700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x56359950c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53880/0 (socket says 172.21.15.19:53880) 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.822+0000 7f19a9b91700 1 -- 172.21.15.19:0/1410884690 learned_addr learned my addr 172.21.15.19:0/1410884690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f19aab93700 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 0x5635994f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f19aa392700 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 0x563599500bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f19a9b91700 1 -- 172.21.15.19:0/1410884690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 msgr2=0x563599500bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f19a9b91700 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 0x563599500bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f19a9b91700 1 -- 172.21.15.19:0/1410884690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 msgr2=0x5635994f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f19a9b91700 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 0x5635994f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f19a9b91700 1 -- 172.21.15.19:0/1410884690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635994f7690 con 0x5635994631e0 2024-05-09T01:32:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f19aa392700 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 0x563599500bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f19a9b91700 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x56359950c090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f19940098e0 tx=0x7f199400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5a2a584bbf9cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f198ffff700 1 -- 172.21.15.19:0/1410884690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994033070 con 0x5635994631e0 2024-05-09T01:32:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f198ffff700 1 -- 172.21.15.19:0/1410884690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199400f740 con 0x5635994631e0 2024-05-09T01:32:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.823+0000 7f198ffff700 1 -- 172.21.15.19:0/1410884690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994005320 con 0x5635994631e0 2024-05-09T01:32:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.824+0000 7f19bb00cb80 1 -- 172.21.15.19:0/1410884690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 msgr2=0x56359950c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.824+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x56359950c090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f19940098e0 tx=0x7f199400b410 comp rx=0 tx=0).stop 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.824+0000 7f19bb00cb80 1 -- 172.21.15.19:0/1410884690 shutdown_connections 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.824+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994f6720 0x5635994f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.824+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563599379520 0x563599500bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.824+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/1410884690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635994631e0 0x56359950c090 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.824+0000 7f19bb00cb80 1 -- 172.21.15.19:0/1410884690 >> 172.21.15.19:0/1410884690 conn(0x5635993d9720 msgr2=0x5635994fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.824+0000 7f19bb00cb80 1 -- 172.21.15.19:0/1410884690 shutdown_connections 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.824+0000 7f19bb00cb80 1 -- 172.21.15.19:0/1410884690 wait complete. 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19bb00cb80 1 -- start start 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19bb00cb80 1 Processor -- start 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563599379520 0x5635995178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994631e0 0x563599517e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19bb00cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994f6720 0x5635995798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41ae010 con 0x563599379520 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41ad940 con 0x5635994631e0 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19bb00cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41aab60 con 0x5635994f6720 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19bb00cb80 10 client.0 _ll_register_callbacks cb 0x56359928a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19aa392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563599379520 0x5635995178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19aa392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563599379520 0x5635995178f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53896/0 (socket says 172.21.15.19:53896) 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.825+0000 7f19aa392700 1 -- 172.21.15.19:0/2697838461 learned_addr learned my addr 172.21.15.19:0/2697838461 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19a9b91700 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994631e0 0x563599517e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19aab93700 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994f6720 0x5635995798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19aa392700 1 -- 172.21.15.19:0/2697838461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994f6720 msgr2=0x5635995798e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19aa392700 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994f6720 0x5635995798e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19aa392700 1 -- 172.21.15.19:0/2697838461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994631e0 msgr2=0x563599517e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19aa392700 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994631e0 0x563599517e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19aa392700 1 -- 172.21.15.19:0/2697838461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563599518650 con 0x563599379520 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19a9b91700 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994631e0 0x563599517e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19aa392700 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563599379520 0x5635995178f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f19a00a5040 tx=0x7f19a00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813c0e2096990700 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f198dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f198dffb700 1 -- 172.21.15.19:0/2697838461 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a00b1e20 con 0x563599379520 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f198dffb700 1 -- 172.21.15.19:0/2697838461 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f19a0004000 con 0x563599379520 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2697838461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f199400f740 con 0x563599379520 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f198dffb700 1 -- 172.21.15.19:0/2697838461 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19a0004200 con 0x563599379520 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19bb00cb80 10 client.17214 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.826+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2697838461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56359957ae70 con 0x563599379520 2024-05-09T01:32:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.827+0000 7f198dffb700 1 -- 172.21.15.19:0/2697838461 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199400f740 con 0x563599379520 2024-05-09T01:32:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.827+0000 7f198dffb700 1 -- 172.21.15.19:0/2697838461 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f19a00ab040 con 0x563599379520 2024-05-09T01:32:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.827+0000 7f19bb00cb80 10 client.17214 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.827+0000 7f19bb00cb80 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.827+0000 7f19bb00cb80 20 client.17214 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.827+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56359957b550 0x56359957d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.827+0000 7f19bb00cb80 4 client.17214 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:32:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.827+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2697838461 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563599420af0 con 0x56359957b550 2024-05-09T01:32:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.827+0000 7f19a9b91700 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56359957b550 0x56359957d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.828+0000 7f198dffb700 10 client.17214 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.828+0000 7f19a9b91700 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56359957b550 0x56359957d970 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e3fcea6053d2c9 server_cookie=497fb8221a25f1d1 in_seq=0 out_seq=0 2024-05-09T01:32:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f198dffb700 1 -- 172.21.15.19:0/2697838461 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563599420af0 con 0x56359957b550 2024-05-09T01:32:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f198dffb700 10 client.17214 handle_command_reply: tid=0 2024-05-09T01:32:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f19bb00cb80 1 client.17214 shutdown 2024-05-09T01:32:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2697838461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563599379520 msgr2=0x5635995178f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563599379520 0x5635995178f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f19a00a5040 tx=0x7f19a00043c0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2697838461 shutdown_connections 2024-05-09T01:32:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56359957b550 0x56359957d970 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635994f6720 0x5635995798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563599379520 0x5635995178f0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f19bb00cb80 1 --2- 172.21.15.19:0/2697838461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635994631e0 0x563599517e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.829+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2697838461 >> 172.21.15.19:0/2697838461 conn(0x5635993d9720 msgr2=0x5635994f5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.830+0000 7f198dffb700 0 client.17214 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.830+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2697838461 shutdown_connections 2024-05-09T01:32:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.830+0000 7f19bb00cb80 1 -- 172.21.15.19:0/2697838461 wait complete. 2024-05-09T01:32:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.830+0000 7f19bb00cb80 20 client.17214 trim_cache size 0 max 16384 2024-05-09T01:32:23.832 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:32:23.832 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:32:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "5e0f6881-ac7c-4345-8fa8-3ad984e7e780", 2024-05-09T01:32:23.833 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:32:23.833 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:32:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 -- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19900b2880 msgr2=0x7f19900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19900b2880 0x7f19900b4d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f199c019d20 tx=0x7f199c009b70 comp rx=0 tx=0).stop 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 -- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 msgr2=0x7f19a419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 0x7f19a419bc70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f19a0007ee0 tx=0x7f19a000d3b0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 -- 172.21.15.19:0/2070834271 shutdown_connections 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a4108200 0x7f19a419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19900b2880 0x7f19900b4d90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19a4103960 0x7f19a419bc70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 --2- 172.21.15.19:0/2070834271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19a41042b0 0x7f19a419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 -- 172.21.15.19:0/2070834271 >> 172.21.15.19:0/2070834271 conn(0x7f19a40fe4e0 msgr2=0x7f19a4107670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 -- 172.21.15.19:0/2070834271 shutdown_connections 2024-05-09T01:32:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:23.831+0000 7f19ac5f6700 1 -- 172.21.15.19:0/2070834271 wait complete. 2024-05-09T01:32:24.632 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:32:24.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:24 smithi019 ceph-mon[30158]: pgmap v584: 97 pgs: 97 active+clean; 4.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-05-09T01:32:24.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:24 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:32:24.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:24 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:24.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:24 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:32:24.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:24 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:24 smithi059 ceph-mon[34351]: pgmap v584: 97 pgs: 97 active+clean; 4.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-05-09T01:32:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:24 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:32:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:24 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:24 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:32:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:24 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:24.885 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:24.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:24 smithi196 ceph-mon[35041]: pgmap v584: 97 pgs: 97 active+clean; 4.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-05-09T01:32:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:24 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:32:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:24 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:24 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:32:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:24 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:32:26.596 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:26 smithi019 ceph-mon[30158]: pgmap v585: 97 pgs: 97 active+clean; 4.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-05-09T01:32:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.617+0000 7fc249995700 1 -- 172.21.15.19:0/2614295318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 msgr2=0x7fc2441111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.617+0000 7fc249995700 1 --2- 172.21.15.19:0/2614295318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 0x7fc2441111b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b0d0 tx=0x7fc23c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.617+0000 7fc249995700 1 -- 172.21.15.19:0/2614295318 shutdown_connections 2024-05-09T01:32:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.617+0000 7fc249995700 1 --2- 172.21.15.19:0/2614295318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 0x7fc2441111b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.617+0000 7fc249995700 1 --2- 172.21.15.19:0/2614295318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2441045d0 0x7fc2441049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.617+0000 7fc249995700 1 --2- 172.21.15.19:0/2614295318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244108f00 0x7fc244109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.617+0000 7fc249995700 1 -- 172.21.15.19:0/2614295318 >> 172.21.15.19:0/2614295318 conn(0x7fc2440ffea0 msgr2=0x7fc2441022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.617+0000 7fc249995700 1 -- 172.21.15.19:0/2614295318 shutdown_connections 2024-05-09T01:32:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.617+0000 7fc249995700 1 -- 172.21.15.19:0/2614295318 wait complete. 2024-05-09T01:32:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc249995700 1 Processor -- start 2024-05-09T01:32:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc249995700 1 -- start start 2024-05-09T01:32:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc249995700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2441045d0 0x7fc244076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc249995700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244108f00 0x7fc244077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc249995700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 0x7fc244077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc249995700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc244114540 con 0x7fc244109850 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc249995700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441143c0 con 0x7fc2441045d0 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc249995700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441146c0 con 0x7fc244108f00 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc2437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 0x7fc244077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc242ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2441045d0 0x7fc244076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc2437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 0x7fc244077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53910/0 (socket says 172.21.15.19:53910) 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc2437fe700 1 -- 172.21.15.19:0/6642138 learned_addr learned my addr 172.21.15.19:0/6642138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.618+0000 7fc2427fc700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244108f00 0x7fc244077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc2437fe700 1 -- 172.21.15.19:0/6642138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244108f00 msgr2=0x7fc244077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc2437fe700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244108f00 0x7fc244077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc2437fe700 1 -- 172.21.15.19:0/6642138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2441045d0 msgr2=0x7fc244076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc2437fe700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2441045d0 0x7fc244076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc2437fe700 1 -- 172.21.15.19:0/6642138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2441a9230 con 0x7fc244109850 2024-05-09T01:32:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc242ffd700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2441045d0 0x7fc244076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:32:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc2437fe700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 0x7fc244077650 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0079a0 tx=0x7fc23c0079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc248993700 1 -- 172.21.15.19:0/6642138 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c01f070 con 0x7fc244109850 2024-05-09T01:32:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc249995700 1 -- 172.21.15.19:0/6642138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2441a94c0 con 0x7fc244109850 2024-05-09T01:32:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc248993700 1 -- 172.21.15.19:0/6642138 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c003c00 con 0x7fc244109850 2024-05-09T01:32:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc248993700 1 -- 172.21.15.19:0/6642138 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c01b690 con 0x7fc244109850 2024-05-09T01:32:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.619+0000 7fc249995700 1 -- 172.21.15.19:0/6642138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2441a9970 con 0x7fc244109850 2024-05-09T01:32:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.622+0000 7fc248993700 1 -- 172.21.15.19:0/6642138 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc23c022070 con 0x7fc244109850 2024-05-09T01:32:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.622+0000 7fc248993700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc22c0b2880 0x7fc22c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc242ffd700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc22c0b2880 0x7fc22c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc248993700 1 -- 172.21.15.19:0/6642138 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc23c00f020 con 0x7fc244109850 2024-05-09T01:32:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2583abb80 1 Processor -- start 2024-05-09T01:32:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2583abb80 1 -- start start 2024-05-09T01:32:26.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 0x5566079614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc242ffd700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc22c0b2880 0x7fc22c0b4d90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc234011490 tx=0x7fc234009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:26.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x556607961a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660795bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2583abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55660787dff0 con 0x55660795b6a0 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2583abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55660781c7f0 con 0x5566078c8160 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2583abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556607879fb0 con 0x5566077de4a0 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660795bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660795bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53916/0 (socket says 172.21.15.19:53916) 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc2437fe700 1 -- 172.21.15.19:0/1918615629 learned_addr learned my addr 172.21.15.19:0/1918615629 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.623+0000 7fc242ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 0x5566079614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc2427fc700 1 --2- 172.21.15.19:0/1918615629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x556607961a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc2437fe700 1 -- 172.21.15.19:0/1918615629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 msgr2=0x556607961a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc2437fe700 1 --2- 172.21.15.19:0/1918615629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x556607961a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc2437fe700 1 -- 172.21.15.19:0/1918615629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 msgr2=0x5566079614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc2437fe700 1 --2- 172.21.15.19:0/1918615629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 0x5566079614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc2437fe700 1 -- 172.21.15.19:0/1918615629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556607814370 con 0x55660795b6a0 2024-05-09T01:32:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc242ffd700 1 --2- 172.21.15.19:0/1918615629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 0x5566079614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:32:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc2437fe700 1 --2- 172.21.15.19:0/1918615629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660795bad0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0aa730 tx=0x7fc23c005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444e8755edf52be0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc229ffb700 1 -- 172.21.15.19:0/1918615629 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0aa540 con 0x55660795b6a0 2024-05-09T01:32:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc229ffb700 1 -- 172.21.15.19:0/1918615629 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c0ab0c0 con 0x55660795b6a0 2024-05-09T01:32:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.624+0000 7fc229ffb700 1 -- 172.21.15.19:0/1918615629 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0a61b0 con 0x55660795b6a0 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.625+0000 7fc2583abb80 1 -- 172.21.15.19:0/1918615629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 msgr2=0x55660795bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.625+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1918615629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660795bad0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0aa730 tx=0x7fc23c005700 comp rx=0 tx=0).stop 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.625+0000 7fc2583abb80 1 -- 172.21.15.19:0/1918615629 shutdown_connections 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.625+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1918615629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660795bad0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.625+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1918615629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 0x5566079614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.625+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1918615629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x556607961a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.625+0000 7fc2583abb80 1 -- 172.21.15.19:0/1918615629 >> 172.21.15.19:0/1918615629 conn(0x55660783e6a0 msgr2=0x556607956400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.625+0000 7fc2583abb80 1 -- 172.21.15.19:0/1918615629 shutdown_connections 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.625+0000 7fc2583abb80 1 -- 172.21.15.19:0/1918615629 wait complete. 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2583abb80 1 -- start start 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2583abb80 1 Processor -- start 2024-05-09T01:32:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x55660796ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566078c8160 0x55660796b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55660795b6a0 0x556607965000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2583abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5566076f9ba0 con 0x5566078c8160 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2583abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556607838150 con 0x55660795b6a0 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2583abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55660795c360 con 0x5566077de4a0 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2583abb80 10 client.0 _ll_register_callbacks cb 0x5566076ef6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2427fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566078c8160 0x55660796b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2427fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566078c8160 0x55660796b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53924/0 (socket says 172.21.15.19:53924) 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc2427fc700 1 -- 172.21.15.19:0/2164149619 learned_addr learned my addr 172.21.15.19:0/2164149619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.626+0000 7fc242ffd700 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x55660796ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2437fe700 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55660795b6a0 0x556607965000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2427fc700 1 -- 172.21.15.19:0/2164149619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 msgr2=0x55660796ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2427fc700 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x55660796ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2427fc700 1 -- 172.21.15.19:0/2164149619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55660795b6a0 msgr2=0x556607965000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2427fc700 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55660795b6a0 0x556607965000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2427fc700 1 -- 172.21.15.19:0/2164149619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566076f9490 con 0x5566078c8160 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2437fe700 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55660795b6a0 0x556607965000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2427fc700 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566078c8160 0x55660796b140 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc238009ed0 tx=0x7fc238011540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0aeb27517ee6fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc222ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc222ffd700 1 -- 172.21.15.19:0/2164149619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238019070 con 0x5566078c8160 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc222ffd700 1 -- 172.21.15.19:0/2164149619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc238015460 con 0x5566078c8160 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2583abb80 1 -- 172.21.15.19:0/2164149619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23c0ab0c0 con 0x5566078c8160 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc222ffd700 1 -- 172.21.15.19:0/2164149619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc238025050 con 0x5566078c8160 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2583abb80 10 client.17226 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.627+0000 7fc2583abb80 1 -- 172.21.15.19:0/2164149619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556607972880 con 0x5566078c8160 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.628+0000 7fc222ffd700 1 -- 172.21.15.19:0/2164149619 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c0ab0c0 con 0x5566078c8160 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.628+0000 7fc222ffd700 1 -- 172.21.15.19:0/2164149619 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc2380147c0 con 0x5566078c8160 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.628+0000 7fc2583abb80 10 client.17226 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.628+0000 7fc2583abb80 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.628+0000 7fc2583abb80 20 client.17226 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.628+0000 7fc2583abb80 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556607974eb0 0x556607977300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.628+0000 7fc2583abb80 4 client.17226 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.628+0000 7fc2583abb80 1 -- 172.21.15.19:0/2164149619 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55660788b9d0 con 0x556607974eb0 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.628+0000 7fc242ffd700 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556607974eb0 0x556607977300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.629+0000 7fc242ffd700 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556607974eb0 0x556607977300 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa18d56900931e8 server_cookie=76408875347d5402 in_seq=0 out_seq=0 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.629+0000 7fc222ffd700 10 client.17226 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.629+0000 7fc222ffd700 1 -- 172.21.15.19:0/2164149619 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55660788b9d0 con 0x556607974eb0 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.629+0000 7fc222ffd700 10 client.17226 handle_command_reply: tid=0 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.629+0000 7fc2583abb80 1 client.17226 shutdown 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.629+0000 7fc2583abb80 1 -- 172.21.15.19:0/2164149619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566078c8160 msgr2=0x55660796b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.629+0000 7fc2583abb80 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566078c8160 0x55660796b140 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc238009ed0 tx=0x7fc238011540 comp rx=0 tx=0).stop 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc2583abb80 1 -- 172.21.15.19:0/2164149619 shutdown_connections 2024-05-09T01:32:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc2583abb80 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x556607974eb0 0x556607977300 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc2583abb80 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55660795b6a0 0x556607965000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc222ffd700 0 client.17226 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc2583abb80 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x55660796ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc2583abb80 1 --2- 172.21.15.19:0/2164149619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566078c8160 0x55660796b140 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc2583abb80 1 -- 172.21.15.19:0/2164149619 >> 172.21.15.19:0/2164149619 conn(0x55660783e6a0 msgr2=0x5566079561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc2583abb80 1 -- 172.21.15.19:0/2164149619 shutdown_connections 2024-05-09T01:32:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc2583abb80 1 -- 172.21.15.19:0/2164149619 wait complete. 2024-05-09T01:32:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.630+0000 7fc2583abb80 20 client.17226 trim_cache size 0 max 16384 2024-05-09T01:32:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.650+0000 7fc2583abb80 1 Processor -- start 2024-05-09T01:32:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.650+0000 7fc2583abb80 1 -- start start 2024-05-09T01:32:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.650+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x556607964fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.650+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 0x55660797caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.650+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660797cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.650+0000 7fc2583abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441b0e50 con 0x55660795b6a0 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.650+0000 7fc2583abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441b2990 con 0x5566078c8160 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.650+0000 7fc2583abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441b1520 con 0x5566077de4a0 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660797cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660797cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53934/0 (socket says 172.21.15.19:53934) 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2437fe700 1 -- 172.21.15.19:0/3359263346 learned_addr learned my addr 172.21.15.19:0/3359263346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc242ffd700 1 --2- 172.21.15.19:0/3359263346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x556607964fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2437fe700 1 -- 172.21.15.19:0/3359263346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 msgr2=0x556607964fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2437fe700 1 --2- 172.21.15.19:0/3359263346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x556607964fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2437fe700 1 -- 172.21.15.19:0/3359263346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 msgr2=0x55660797caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2427fc700 1 --2- 172.21.15.19:0/3359263346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 0x55660797caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2437fe700 1 --2- 172.21.15.19:0/3359263346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 0x55660797caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2437fe700 1 -- 172.21.15.19:0/3359263346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556607963570 con 0x55660795b6a0 2024-05-09T01:32:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2437fe700 1 --2- 172.21.15.19:0/3359263346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660797cfe0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0a6ab0 tx=0x7fc23c0a6a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad50d90df873a704 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:26.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2297fa700 1 -- 172.21.15.19:0/3359263346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0ab0c0 con 0x55660795b6a0 2024-05-09T01:32:26.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2297fa700 1 -- 172.21.15.19:0/3359263346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c0c3070 con 0x55660795b6a0 2024-05-09T01:32:26.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.651+0000 7fc2297fa700 1 -- 172.21.15.19:0/3359263346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0cd0c0 con 0x55660795b6a0 2024-05-09T01:32:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.652+0000 7fc2583abb80 1 -- 172.21.15.19:0/3359263346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 msgr2=0x55660797cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.652+0000 7fc2583abb80 1 --2- 172.21.15.19:0/3359263346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660797cfe0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0a6ab0 tx=0x7fc23c0a6a20 comp rx=0 tx=0).stop 2024-05-09T01:32:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.652+0000 7fc2583abb80 1 -- 172.21.15.19:0/3359263346 shutdown_connections 2024-05-09T01:32:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.652+0000 7fc2583abb80 1 --2- 172.21.15.19:0/3359263346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660797cfe0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.652+0000 7fc2583abb80 1 --2- 172.21.15.19:0/3359263346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566077de4a0 0x556607964fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.652+0000 7fc2583abb80 1 --2- 172.21.15.19:0/3359263346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566078c8160 0x55660797caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.652+0000 7fc2583abb80 1 -- 172.21.15.19:0/3359263346 >> 172.21.15.19:0/3359263346 conn(0x55660783e6a0 msgr2=0x5566079648b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.652+0000 7fc2583abb80 1 -- 172.21.15.19:0/3359263346 shutdown_connections 2024-05-09T01:32:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.652+0000 7fc2583abb80 1 -- 172.21.15.19:0/3359263346 wait complete. 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2583abb80 1 -- start start 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2583abb80 1 Processor -- start 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566077de4a0 0x55660796ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566078c8160 0x55660796f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2583abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660796f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2583abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441b22c0 con 0x55660795b6a0 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2583abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441b1bf0 con 0x5566077de4a0 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2583abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441aedf0 con 0x5566078c8160 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660796f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc242ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566077de4a0 0x55660796ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2437fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660796f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53936/0 (socket says 172.21.15.19:53936) 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2583abb80 10 client.0 _ll_register_callbacks cb 0x5566076ef6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2437fe700 1 -- 172.21.15.19:0/1295057483 learned_addr learned my addr 172.21.15.19:0/1295057483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.653+0000 7fc2427fc700 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566078c8160 0x55660796f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc2437fe700 1 -- 172.21.15.19:0/1295057483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566078c8160 msgr2=0x55660796f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc2437fe700 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566078c8160 0x55660796f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc2437fe700 1 -- 172.21.15.19:0/1295057483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566077de4a0 msgr2=0x55660796ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc2437fe700 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566077de4a0 0x55660796ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc2437fe700 1 -- 172.21.15.19:0/1295057483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566079de690 con 0x55660795b6a0 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc2437fe700 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660796f760 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b870 tx=0x7fc23c0aed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9b73df113b9e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc222ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc222ffd700 1 -- 172.21.15.19:0/1295057483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0b3790 con 0x55660795b6a0 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc222ffd700 1 -- 172.21.15.19:0/1295057483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc23c071170 con 0x55660795b6a0 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc2583abb80 1 -- 172.21.15.19:0/1295057483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23c0c3070 con 0x55660795b6a0 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc222ffd700 1 -- 172.21.15.19:0/1295057483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc23c0b10d0 con 0x55660795b6a0 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc2583abb80 10 client.17232 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.654+0000 7fc2583abb80 1 -- 172.21.15.19:0/1295057483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566079df8e0 con 0x55660795b6a0 2024-05-09T01:32:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.655+0000 7fc222ffd700 1 -- 172.21.15.19:0/1295057483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c0c3070 con 0x55660795b6a0 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.655+0000 7fc222ffd700 1 -- 172.21.15.19:0/1295057483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc23c011070 con 0x55660795b6a0 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.655+0000 7fc2583abb80 10 client.17232 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.655+0000 7fc2583abb80 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.655+0000 7fc2583abb80 20 client.17232 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.655+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5566079dfb50 0x5566079e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.655+0000 7fc2583abb80 4 client.17232 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.655+0000 7fc2583abb80 1 -- 172.21.15.19:0/1295057483 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556607885a70 con 0x5566079dfb50 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.655+0000 7fc242ffd700 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5566079dfb50 0x5566079e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc222ffd700 10 client.17232 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc242ffd700 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5566079dfb50 0x5566079e1f70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7836af34bb7887e0 server_cookie=7f866401220777fd in_seq=0 out_seq=0 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc222ffd700 1 -- 172.21.15.19:0/1295057483 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556607885a70 con 0x5566079dfb50 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc222ffd700 10 client.17232 handle_command_reply: tid=0 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 client.17232 shutdown 2024-05-09T01:32:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 -- 172.21.15.19:0/1295057483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 msgr2=0x55660796f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660796f760 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b870 tx=0x7fc23c0aed70 comp rx=0 tx=0).stop 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 -- 172.21.15.19:0/1295057483 shutdown_connections 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5566079dfb50 0x5566079e1f70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55660795b6a0 0x55660796f760 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566077de4a0 0x55660796ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc222ffd700 0 client.17232 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 --2- 172.21.15.19:0/1295057483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566078c8160 0x55660796f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 -- 172.21.15.19:0/1295057483 >> 172.21.15.19:0/1295057483 conn(0x55660783e6a0 msgr2=0x55660795ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 -- 172.21.15.19:0/1295057483 shutdown_connections 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 1 -- 172.21.15.19:0/1295057483 wait complete. 2024-05-09T01:32:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.656+0000 7fc2583abb80 20 client.17232 trim_cache size 0 max 16384 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout: "5e0f6881-ac7c-4345-8fa8-3ad984e7e780": { 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "5e0f6881-ac7c-4345-8fa8-3ad984e7e780", 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:32:26.659 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:32:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 -- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc22c0b2880 msgr2=0x7fc22c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc22c0b2880 0x7fc22c0b4d90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc234011490 tx=0x7fc234009260 comp rx=0 tx=0).stop 2024-05-09T01:32:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 -- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 msgr2=0x7fc244077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 0x7fc244077650 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0079a0 tx=0x7fc23c0079d0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 -- 172.21.15.19:0/6642138 shutdown_connections 2024-05-09T01:32:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc22c0b2880 0x7fc22c0b4d90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc244109850 0x7fc244077650 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2441045d0 0x7fc244076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 --2- 172.21.15.19:0/6642138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244108f00 0x7fc244077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 -- 172.21.15.19:0/6642138 >> 172.21.15.19:0/6642138 conn(0x7fc2440ffea0 msgr2=0x7fc2441086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 -- 172.21.15.19:0/6642138 shutdown_connections 2024-05-09T01:32:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:26.657+0000 7fc249995700 1 -- 172.21.15.19:0/6642138 wait complete. 2024-05-09T01:32:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:26 smithi059 ceph-mon[34351]: pgmap v585: 97 pgs: 97 active+clean; 4.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-05-09T01:32:26.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:26 smithi196 ceph-mon[35041]: pgmap v585: 97 pgs: 97 active+clean; 4.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-05-09T01:32:27.397 INFO:tasks.cephfs.filesystem:scrub status for tag:5e0f6881-ac7c-4345-8fa8-3ad984e7e780 - {'path': '/', 'tag': '5e0f6881-ac7c-4345-8fa8-3ad984e7e780', 'options': 'recursive,force'} 2024-05-09T01:32:27.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:27 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:32:27.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:27 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:32:27.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:27 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:32:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:28 smithi059 ceph-mon[34351]: pgmap v586: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:28.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:28 smithi019 ceph-mon[30158]: pgmap v586: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:28.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:28 smithi196 ceph-mon[35041]: pgmap v586: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.327+0000 7f726f63b640 1 -- 172.21.15.19:0/1246288374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268103d90 msgr2=0x7f72681041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.327+0000 7f726f63b640 1 --2- 172.21.15.19:0/1246288374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268103d90 0x7f72681041a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7258009900 tx=0x7f725802a410 comp rx=0 tx=0).stop 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.327+0000 7f726f63b640 1 -- 172.21.15.19:0/1246288374 shutdown_connections 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.327+0000 7f726f63b640 1 --2- 172.21.15.19:0/1246288374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268108a90 0x7f726810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.327+0000 7f726f63b640 1 --2- 172.21.15.19:0/1246288374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268103d90 0x7f72681041a0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.327+0000 7f726f63b640 1 --2- 172.21.15.19:0/1246288374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7268102b50 0x7f7268103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.327+0000 7f726f63b640 1 -- 172.21.15.19:0/1246288374 >> 172.21.15.19:0/1246288374 conn(0x7f72680fdd40 msgr2=0x7f72681001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.327+0000 7f726f63b640 1 -- 172.21.15.19:0/1246288374 shutdown_connections 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.327+0000 7f726f63b640 1 -- 172.21.15.19:0/1246288374 wait complete. 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726f63b640 1 Processor -- start 2024-05-09T01:32:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726f63b640 1 -- start start 2024-05-09T01:32:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726f63b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7268102b50 0x7f726819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726f63b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268103d90 0x7f726819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726f63b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268108a90 0x7f72681a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726f63b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f726810e680 con 0x7f7268108a90 2024-05-09T01:32:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726f63b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f726810e500 con 0x7f7268102b50 2024-05-09T01:32:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726f63b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f726810e800 con 0x7f7268103d90 2024-05-09T01:32:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726dbd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268108a90 0x7f72681a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726dbd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268108a90 0x7f72681a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32896/0 (socket says 172.21.15.19:32896) 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726cbd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268103d90 0x7f726819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.328+0000 7f726dbd8640 1 -- 172.21.15.19:0/2303252508 learned_addr learned my addr 172.21.15.19:0/2303252508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f726d3d7640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7268102b50 0x7f726819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f726dbd8640 1 -- 172.21.15.19:0/2303252508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268103d90 msgr2=0x7f726819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f726dbd8640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268103d90 0x7f726819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f726dbd8640 1 -- 172.21.15.19:0/2303252508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7268102b50 msgr2=0x7f726819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f726dbd8640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7268102b50 0x7f726819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f726dbd8640 1 -- 172.21.15.19:0/2303252508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72681a1e80 con 0x7f7268108a90 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f726dbd8640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268108a90 0x7f72681a1810 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f726000eaa0 tx=0x7f726000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f725e7fc640 1 -- 172.21.15.19:0/2303252508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726000cdb0 con 0x7f7268108a90 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f725e7fc640 1 -- 172.21.15.19:0/2303252508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7260005020 con 0x7f7268108a90 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f726f63b640 1 -- 172.21.15.19:0/2303252508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72681a2170 con 0x7f7268108a90 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f725e7fc640 1 -- 172.21.15.19:0/2303252508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72600116d0 con 0x7f7268108a90 2024-05-09T01:32:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.329+0000 7f726f63b640 1 -- 172.21.15.19:0/2303252508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72681a26c0 con 0x7f7268108a90 2024-05-09T01:32:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.332+0000 7f72765c9740 1 Processor -- start 2024-05-09T01:32:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.332+0000 7f725e7fc640 1 -- 172.21.15.19:0/2303252508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f72600054a0 con 0x7f7268108a90 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.332+0000 7f72765c9740 1 -- start start 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f72765c9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 0x563fcbb09500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f72765c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb09a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f72765c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 0x563fcbb10980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f72765c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563fcba6d7c0 con 0x563fcb73c2f0 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f72765c9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563fcba090e0 con 0x563fcb7651b0 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f72765c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563fcba69780 con 0x563fcb7444d0 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726d3d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb09a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726d3d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb09a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32906/0 (socket says 172.21.15.19:32906) 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726d3d7640 1 -- 172.21.15.19:0/1795613619 learned_addr learned my addr 172.21.15.19:0/1795613619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726cbd6640 1 --2- 172.21.15.19:0/1795613619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 0x563fcbb09500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726cbd6640 1 --2- 172.21.15.19:0/1795613619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 0x563fcbb10980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f725e7fc640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72540b27e0 0x7f72540b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726d3d7640 1 -- 172.21.15.19:0/1795613619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 msgr2=0x563fcbb10980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726d3d7640 1 --2- 172.21.15.19:0/1795613619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 0x563fcbb10980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726d3d7640 1 -- 172.21.15.19:0/1795613619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 msgr2=0x563fcbb09500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726d3d7640 1 --2- 172.21.15.19:0/1795613619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 0x563fcbb09500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726d3d7640 1 -- 172.21.15.19:0/1795613619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fcba327c0 con 0x563fcb73c2f0 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726dbd8640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72540b27e0 0x7f72540b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f725e7fc640 1 -- 172.21.15.19:0/2303252508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f72600e2320 con 0x7f7268108a90 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.333+0000 7f726d3d7640 1 --2- 172.21.15.19:0/1795613619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb09a80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f726400bc70 tx=0x7f72640107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aa8658e40bc246 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.334+0000 7f72537fe640 1 -- 172.21.15.19:0/1795613619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264004220 con 0x563fcb73c2f0 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.334+0000 7f72537fe640 1 -- 172.21.15.19:0/1795613619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726400be20 con 0x563fcb73c2f0 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.334+0000 7f72537fe640 1 -- 172.21.15.19:0/1795613619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264004c90 con 0x563fcb73c2f0 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 -- 172.21.15.19:0/1795613619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 msgr2=0x563fcbb09a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 --2- 172.21.15.19:0/1795613619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb09a80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f726400bc70 tx=0x7f72640107a0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f726dbd8640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72540b27e0 0x7f72540b4cf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f72600a9960 tx=0x7f726006efc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 -- 172.21.15.19:0/1795613619 shutdown_connections 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 --2- 172.21.15.19:0/1795613619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 0x563fcbb10980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 --2- 172.21.15.19:0/1795613619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb09a80 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 --2- 172.21.15.19:0/1795613619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 0x563fcbb09500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 -- 172.21.15.19:0/1795613619 >> 172.21.15.19:0/1795613619 conn(0x563fcb759300 msgr2=0x563fcbb04e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 -- 172.21.15.19:0/1795613619 shutdown_connections 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 -- 172.21.15.19:0/1795613619 wait complete. 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 -- start start 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.335+0000 7f72765c9740 1 Processor -- start 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f72765c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb13200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f72765c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 0x563fcbb13740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f72765c9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 0x563fcbb1a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f72765c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563fcbb113d0 con 0x563fcb73c2f0 2024-05-09T01:32:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f72765c9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563fcb9f49c0 con 0x563fcb7651b0 2024-05-09T01:32:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f72765c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563fcbb11550 con 0x563fcb7444d0 2024-05-09T01:32:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f72765c9740 10 client.0 _ll_register_callbacks cb 0x563fcba21020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb13200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb13200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32916/0 (socket says 172.21.15.19:32916) 2024-05-09T01:32:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 -- 172.21.15.19:0/1692656933 learned_addr learned my addr 172.21.15.19:0/1692656933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 0x563fcbb1a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726d3d7640 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 0x563fcbb13740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 -- 172.21.15.19:0/1692656933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 msgr2=0x563fcbb13740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 0x563fcbb13740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 -- 172.21.15.19:0/1692656933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 msgr2=0x563fcbb1a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 0x563fcbb1a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 -- 172.21.15.19:0/1692656933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fcba1b1f0 con 0x563fcb73c2f0 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f726cbd6640 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb13200 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7258009900 tx=0x7f7258009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aacb43ab627be13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.336+0000 7f7250ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f7250ff9640 1 -- 172.21.15.19:0/1692656933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258046040 con 0x563fcb73c2f0 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f7250ff9640 1 -- 172.21.15.19:0/1692656933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7258004300 con 0x563fcb73c2f0 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f72765c9740 1 -- 172.21.15.19:0/1692656933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f726400be20 con 0x563fcb73c2f0 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f7250ff9640 1 -- 172.21.15.19:0/1692656933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7258035040 con 0x563fcb73c2f0 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f72765c9740 10 client.17244 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f72765c9740 1 -- 172.21.15.19:0/1692656933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563fcb7d0790 con 0x563fcb73c2f0 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f7250ff9640 1 -- 172.21.15.19:0/1692656933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726400be20 con 0x563fcb73c2f0 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f7250ff9640 1 -- 172.21.15.19:0/1692656933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f725803c500 con 0x563fcb73c2f0 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f72765c9740 10 client.17244 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f72765c9740 10 client.17244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f72765c9740 20 client.17244 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f72765c9740 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563fcb723d80 0x563fcbb20b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f72765c9740 4 client.17244 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.337+0000 7f72765c9740 1 -- 172.21.15.19:0/1692656933 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fcba7b3d0 con 0x563fcb723d80 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.338+0000 7f726d3d7640 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563fcb723d80 0x563fcbb20b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.338+0000 7f726d3d7640 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563fcb723d80 0x563fcbb20b40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8dd2767fb1b5d0f server_cookie=c35540b916c6b954 in_seq=0 out_seq=0 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.338+0000 7f7250ff9640 10 client.17244 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f7250ff9640 1 -- 172.21.15.19:0/1692656933 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563fcba7b3d0 con 0x563fcb723d80 2024-05-09T01:32:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f7250ff9640 10 client.17244 handle_command_reply: tid=0 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 client.17244 shutdown 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 -- 172.21.15.19:0/1692656933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 msgr2=0x563fcbb13200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb13200 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7258009900 tx=0x7f7258009260 comp rx=0 tx=0).stop 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 -- 172.21.15.19:0/1692656933 shutdown_connections 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563fcb723d80 0x563fcbb20b40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7651b0 0x563fcbb1a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7444d0 0x563fcbb13740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 --2- 172.21.15.19:0/1692656933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb13200 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f7250ff9640 0 client.17244 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 -- 172.21.15.19:0/1692656933 >> 172.21.15.19:0/1692656933 conn(0x563fcb759300 msgr2=0x563fcbb03700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 -- 172.21.15.19:0/1692656933 shutdown_connections 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 1 -- 172.21.15.19:0/1692656933 wait complete. 2024-05-09T01:32:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.339+0000 7f72765c9740 20 client.17244 trim_cache size 0 max 16384 2024-05-09T01:32:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f72765c9740 1 Processor -- start 2024-05-09T01:32:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f72765c9740 1 -- start start 2024-05-09T01:32:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f72765c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb0dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f72765c9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7444d0 0x563fcbb134e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f72765c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 0x563fcbb1a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f72765c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72681ab500 con 0x563fcb73c2f0 2024-05-09T01:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f72765c9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72681ad040 con 0x563fcb7444d0 2024-05-09T01:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f72765c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72681abbd0 con 0x563fcb7651b0 2024-05-09T01:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f726cbd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb0dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f726cbd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb0dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32922/0 (socket says 172.21.15.19:32922) 2024-05-09T01:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f726cbd6640 1 -- 172.21.15.19:0/1484963127 learned_addr learned my addr 172.21.15.19:0/1484963127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f726d3d7640 1 --2- 172.21.15.19:0/1484963127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7444d0 0x563fcbb134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f726cbd6640 1 -- 172.21.15.19:0/1484963127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 msgr2=0x563fcbb1a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f726cbd6640 1 --2- 172.21.15.19:0/1484963127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 0x563fcbb1a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f726cbd6640 1 -- 172.21.15.19:0/1484963127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7444d0 msgr2=0x563fcbb134e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f726cbd6640 1 --2- 172.21.15.19:0/1484963127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7444d0 0x563fcbb134e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.355+0000 7f726cbd6640 1 -- 172.21.15.19:0/1484963127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fcb8475f0 con 0x563fcb73c2f0 2024-05-09T01:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.356+0000 7f726cbd6640 1 --2- 172.21.15.19:0/1484963127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb0dc90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7258006010 tx=0x7f7258031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65323f7f77064967 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.356+0000 7f7252ffd640 1 -- 172.21.15.19:0/1484963127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725803c670 con 0x563fcb73c2f0 2024-05-09T01:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.356+0000 7f7252ffd640 1 -- 172.21.15.19:0/1484963127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f725805a070 con 0x563fcb73c2f0 2024-05-09T01:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.356+0000 7f7252ffd640 1 -- 172.21.15.19:0/1484963127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258003850 con 0x563fcb73c2f0 2024-05-09T01:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 -- 172.21.15.19:0/1484963127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 msgr2=0x563fcbb0dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 --2- 172.21.15.19:0/1484963127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb0dc90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7258006010 tx=0x7f7258031400 comp rx=0 tx=0).stop 2024-05-09T01:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 -- 172.21.15.19:0/1484963127 shutdown_connections 2024-05-09T01:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 --2- 172.21.15.19:0/1484963127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 0x563fcbb1a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 --2- 172.21.15.19:0/1484963127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb7444d0 0x563fcbb134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 --2- 172.21.15.19:0/1484963127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb73c2f0 0x563fcbb0dc90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 -- 172.21.15.19:0/1484963127 >> 172.21.15.19:0/1484963127 conn(0x563fcb759300 msgr2=0x563fcbb12dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 -- 172.21.15.19:0/1484963127 shutdown_connections 2024-05-09T01:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 -- 172.21.15.19:0/1484963127 wait complete. 2024-05-09T01:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 -- start start 2024-05-09T01:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.357+0000 7f72765c9740 1 Processor -- start 2024-05-09T01:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f72765c9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb73c2f0 0x563fcbb18380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f72765c9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb7444d0 0x563fcbb188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f72765c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 0x563fcbb197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f72765c9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72681ac970 con 0x563fcb7444d0 2024-05-09T01:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f72765c9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72681ac2a0 con 0x563fcb73c2f0 2024-05-09T01:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f72765c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72681a94e0 con 0x563fcb7651b0 2024-05-09T01:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f72765c9740 10 client.0 _ll_register_callbacks cb 0x7f7258005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726d3d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb7444d0 0x563fcbb188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726d3d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb7444d0 0x563fcbb188c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32924/0 (socket says 172.21.15.19:32924) 2024-05-09T01:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726cbd6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb73c2f0 0x563fcbb18380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726d3d7640 1 -- 172.21.15.19:0/3153635194 learned_addr learned my addr 172.21.15.19:0/3153635194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726cbd6640 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 0x563fcbb197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726d3d7640 1 -- 172.21.15.19:0/3153635194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 msgr2=0x563fcbb197f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726d3d7640 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 0x563fcbb197f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726d3d7640 1 -- 172.21.15.19:0/3153635194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb73c2f0 msgr2=0x563fcbb18380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726d3d7640 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb73c2f0 0x563fcbb18380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726d3d7640 1 -- 172.21.15.19:0/3153635194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fcba31ed0 con 0x563fcb7444d0 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726cbd6640 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 0x563fcbb197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f726d3d7640 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb7444d0 0x563fcbb188c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x563fcbb0a8e0 tx=0x7f726400b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef31ba5396d47a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.358+0000 7f7250ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f7250ff9640 1 -- 172.21.15.19:0/3153635194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72640044d0 con 0x563fcb7444d0 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f7250ff9640 1 -- 172.21.15.19:0/3153635194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f72640283c0 con 0x563fcb7444d0 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f72765c9740 1 -- 172.21.15.19:0/3153635194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f725805a070 con 0x563fcb7444d0 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f7250ff9640 1 -- 172.21.15.19:0/3153635194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7264037050 con 0x563fcb7444d0 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f72765c9740 10 client.17250 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f72765c9740 1 -- 172.21.15.19:0/3153635194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563fcbb1af00 con 0x563fcb7444d0 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f7250ff9640 1 -- 172.21.15.19:0/3153635194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f725805a070 con 0x563fcb7444d0 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f7250ff9640 1 -- 172.21.15.19:0/3153635194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7264003a40 con 0x563fcb7444d0 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f72765c9740 10 client.17250 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f72765c9740 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f72765c9740 20 client.17250 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f72765c9740 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563fcb6f91c0 0x563fcbb8b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f72765c9740 4 client.17250 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f72765c9740 1 -- 172.21.15.19:0/3153635194 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fcba75240 con 0x563fcb6f91c0 2024-05-09T01:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.359+0000 7f726cbd6640 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563fcb6f91c0 0x563fcbb8b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.360+0000 7f726cbd6640 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563fcb6f91c0 0x563fcbb8b200 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c136b70ad99571f server_cookie=362dc544952dce2c in_seq=0 out_seq=0 2024-05-09T01:32:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.360+0000 7f7250ff9640 10 client.17250 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.376+0000 7f7250ff9640 1 -- 172.21.15.19:0/3153635194 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563fcba75240 con 0x563fcb6f91c0 2024-05-09T01:32:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.376+0000 7f7250ff9640 10 client.17250 handle_command_reply: tid=0 2024-05-09T01:32:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.376+0000 7f72765c9740 1 client.17250 shutdown 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 -- 172.21.15.19:0/3153635194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb7444d0 msgr2=0x563fcbb188c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb7444d0 0x563fcbb188c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x563fcbb0a8e0 tx=0x7f726400b930 comp rx=0 tx=0).stop 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 -- 172.21.15.19:0/3153635194 shutdown_connections 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563fcb6f91c0 0x563fcbb8b200 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fcb7651b0 0x563fcbb197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563fcb7444d0 0x563fcbb188c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f7250ff9640 0 client.17250 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 --2- 172.21.15.19:0/3153635194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563fcb73c2f0 0x563fcbb18380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 -- 172.21.15.19:0/3153635194 >> 172.21.15.19:0/3153635194 conn(0x563fcb759300 msgr2=0x563fcbb03700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 -- 172.21.15.19:0/3153635194 shutdown_connections 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 1 -- 172.21.15.19:0/3153635194 wait complete. 2024-05-09T01:32:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.377+0000 7f72765c9740 20 client.17250 trim_cache size 0 max 16384 2024-05-09T01:32:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:32:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:32:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:32:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 -- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72540b27e0 msgr2=0x7f72540b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72540b27e0 0x7f72540b4cf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f72600a9960 tx=0x7f726006efc0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 -- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268108a90 msgr2=0x7f72681a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268108a90 0x7f72681a1810 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f726000eaa0 tx=0x7f726000ef60 comp rx=0 tx=0).stop 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 -- 172.21.15.19:0/2303252508 shutdown_connections 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f72540b27e0 0x7f72540b4cf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7268108a90 0x7f72681a1810 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268103d90 0x7f726819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 --2- 172.21.15.19:0/2303252508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7268102b50 0x7f726819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 -- 172.21.15.19:0/2303252508 >> 172.21.15.19:0/2303252508 conn(0x7f72680fdd40 msgr2=0x7f726806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 -- 172.21.15.19:0/2303252508 shutdown_connections 2024-05-09T01:32:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:30.378+0000 7f726f63b640 1 -- 172.21.15.19:0/2303252508 wait complete. 2024-05-09T01:32:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:30 smithi059 ceph-mon[34351]: pgmap v587: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-05-09T01:32:30.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:30 smithi019 ceph-mon[30158]: pgmap v587: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-05-09T01:32:30.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:30 smithi196 ceph-mon[35041]: pgmap v587: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-05-09T01:32:32.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:32 smithi059 ceph-mon[34351]: pgmap v588: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-09T01:32:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:32 smithi019 ceph-mon[30158]: pgmap v588: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-09T01:32:32.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:32 smithi196 ceph-mon[35041]: pgmap v588: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-09T01:32:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:34 smithi059 ceph-mon[34351]: pgmap v589: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:32:34.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:34 smithi019 ceph-mon[30158]: pgmap v589: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:32:34.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:34 smithi196 ceph-mon[35041]: pgmap v589: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:32:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:36 smithi059 ceph-mon[34351]: pgmap v590: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:32:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:36 smithi019 ceph-mon[30158]: pgmap v590: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:32:36.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:36 smithi196 ceph-mon[35041]: pgmap v590: 97 pgs: 97 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-05-09T01:32:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:38 smithi059 ceph-mon[34351]: pgmap v591: 97 pgs: 97 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:32:38.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:38 smithi019 ceph-mon[30158]: pgmap v591: 97 pgs: 97 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:32:38.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:38 smithi196 ceph-mon[35041]: pgmap v591: 97 pgs: 97 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:32:40.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:39 smithi059 ceph-mon[34351]: pgmap v592: 97 pgs: 97 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:32:40.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:39 smithi019 ceph-mon[30158]: pgmap v592: 97 pgs: 97 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:32:40.193 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:39 smithi196 ceph-mon[35041]: pgmap v592: 97 pgs: 97 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:32:42.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:42 smithi019 ceph-mon[30158]: pgmap v593: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:42.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:42 smithi196 ceph-mon[35041]: pgmap v593: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:42 smithi059 ceph-mon[34351]: pgmap v593: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.480+0000 7ff2db2f9640 1 -- 172.21.15.19:0/1405571979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4103d90 msgr2=0x7ff2d41041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.480+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/1405571979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4103d90 0x7ff2d41041a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009900 tx=0x7ff2c402a410 comp rx=0 tx=0).stop 2024-05-09T01:32:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.480+0000 7ff2db2f9640 1 -- 172.21.15.19:0/1405571979 shutdown_connections 2024-05-09T01:32:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.480+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/1405571979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4108a90 0x7ff2d410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.480+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/1405571979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4103d90 0x7ff2d41041a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.480+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/1405571979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4102b50 0x7ff2d4103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.480+0000 7ff2db2f9640 1 -- 172.21.15.19:0/1405571979 >> 172.21.15.19:0/1405571979 conn(0x7ff2d40fdd40 msgr2=0x7ff2d41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.480+0000 7ff2db2f9640 1 -- 172.21.15.19:0/1405571979 shutdown_connections 2024-05-09T01:32:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.480+0000 7ff2db2f9640 1 -- 172.21.15.19:0/1405571979 wait complete. 2024-05-09T01:32:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2db2f9640 1 Processor -- start 2024-05-09T01:32:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2db2f9640 1 -- start start 2024-05-09T01:32:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2db2f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4102b50 0x7ff2d406af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2db2f9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d90 0x7ff2d41a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2db2f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4108a90 0x7ff2d41a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2db2f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d410e680 con 0x7ff2d4102b50 2024-05-09T01:32:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2db2f9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d410e500 con 0x7ff2d4103d90 2024-05-09T01:32:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2db2f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d410e800 con 0x7ff2d4108a90 2024-05-09T01:32:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d9095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4102b50 0x7ff2d406af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d9095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4102b50 0x7ff2d406af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59046/0 (socket says 172.21.15.19:59046) 2024-05-09T01:32:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d9095640 1 -- 172.21.15.19:0/3231001371 learned_addr learned my addr 172.21.15.19:0/3231001371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d9896640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4108a90 0x7ff2d41a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d8894640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d90 0x7ff2d41a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d9095640 1 -- 172.21.15.19:0/3231001371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4108a90 msgr2=0x7ff2d41a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d9095640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4108a90 0x7ff2d41a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d9095640 1 -- 172.21.15.19:0/3231001371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d90 msgr2=0x7ff2d41a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d9095640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d90 0x7ff2d41a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.481+0000 7ff2d9095640 1 -- 172.21.15.19:0/3231001371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d41a4070 con 0x7ff2d4102b50 2024-05-09T01:32:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.482+0000 7ff2d9095640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4102b50 0x7ff2d406af60 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0005fd0 tx=0x7ff2d000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.482+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/3231001371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0017070 con 0x7ff2d4102b50 2024-05-09T01:32:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.482+0000 7ff2db2f9640 1 -- 172.21.15.19:0/3231001371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d41a4360 con 0x7ff2d4102b50 2024-05-09T01:32:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.482+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/3231001371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d0009c30 con 0x7ff2d4102b50 2024-05-09T01:32:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.482+0000 7ff2db2f9640 1 -- 172.21.15.19:0/3231001371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d41a4900 con 0x7ff2d4102b50 2024-05-09T01:32:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.482+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/3231001371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0003d10 con 0x7ff2d4102b50 2024-05-09T01:32:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.484+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/3231001371 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff2d0003eb0 con 0x7ff2d4102b50 2024-05-09T01:32:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.485+0000 7ff2ca7fc640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27e0 0x7ff2c00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.485+0000 7ff2e21ab740 1 Processor -- start 2024-05-09T01:32:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2e21ab740 1 -- start start 2024-05-09T01:32:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d8894640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27e0 0x7ff2c00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2ca7fc640 1 -- 172.21.15.19:0/3231001371 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff2d00e2020 con 0x7ff2d4102b50 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7477ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f74783980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557f746e07c0 con 0x557f743d81b0 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557f7467c0e0 con 0x557f743b74d0 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f746dc780 con 0x557f743af2f0 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59058/0 (socket says 172.21.15.19:59058) 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9095640 1 -- 172.21.15.19:0/814202272 learned_addr learned my addr 172.21.15.19:0/814202272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d8894640 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7477ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9896640 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f74783980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d8894640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27e0 0x7ff2c00b4cf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4000c00 tx=0x7ff2c4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9095640 1 -- 172.21.15.19:0/814202272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 msgr2=0x557f7477ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9095640 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7477ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9095640 1 -- 172.21.15.19:0/814202272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 msgr2=0x557f74783980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9095640 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f74783980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9095640 1 -- 172.21.15.19:0/814202272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f746a57c0 con 0x557f743d81b0 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.486+0000 7ff2d9896640 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f74783980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.487+0000 7ff2d9095640 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477c500 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009610 tx=0x7ff2d00a8e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8e74833acd5982 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.487+0000 7ff2bf7fe640 1 -- 172.21.15.19:0/814202272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00b2420 con 0x557f743d81b0 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.487+0000 7ff2bf7fe640 1 -- 172.21.15.19:0/814202272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00035f0 con 0x557f743d81b0 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.487+0000 7ff2bf7fe640 1 -- 172.21.15.19:0/814202272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004400 con 0x557f743d81b0 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 -- 172.21.15.19:0/814202272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 msgr2=0x557f7477c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477c500 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009610 tx=0x7ff2d00a8e40 comp rx=0 tx=0).stop 2024-05-09T01:32:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 -- 172.21.15.19:0/814202272 shutdown_connections 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f74783980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7477ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/814202272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477c500 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 -- 172.21.15.19:0/814202272 >> 172.21.15.19:0/814202272 conn(0x557f743cc300 msgr2=0x557f74777e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 -- 172.21.15.19:0/814202272 shutdown_connections 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 -- 172.21.15.19:0/814202272 wait complete. 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 -- start start 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 Processor -- start 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.488+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743af2f0 0x557f74786180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f747866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743d81b0 0x557f7478d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557f74784310 con 0x557f743af2f0 2024-05-09T01:32:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557f746679c0 con 0x557f743b74d0 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f74784490 con 0x557f743d81b0 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2e21ab740 10 client.0 _ll_register_callbacks cb 0x557f74694020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743af2f0 0x557f74786180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9095640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743af2f0 0x557f74786180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59062/0 (socket says 172.21.15.19:59062) 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9095640 1 -- 172.21.15.19:0/1761965688 learned_addr learned my addr 172.21.15.19:0/1761965688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d8894640 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f747866c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9896640 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743d81b0 0x557f7478d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9095640 1 -- 172.21.15.19:0/1761965688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743d81b0 msgr2=0x557f7478d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9095640 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743d81b0 0x557f7478d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9095640 1 -- 172.21.15.19:0/1761965688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 msgr2=0x557f747866c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9095640 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f747866c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9095640 1 -- 172.21.15.19:0/1761965688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7468e1f0 con 0x557f743af2f0 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2d9095640 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743af2f0 0x557f74786180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0000c00 tx=0x7ff2d00adaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa42a278706a732c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2bcff9640 1 -- 172.21.15.19:0/1761965688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00b2420 con 0x557f743af2f0 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2bcff9640 1 -- 172.21.15.19:0/1761965688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff2d00a5bb0 con 0x557f743af2f0 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.489+0000 7ff2e21ab740 1 -- 172.21.15.19:0/1761965688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d00035f0 con 0x557f743af2f0 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2bcff9640 1 -- 172.21.15.19:0/1761965688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2d00043b0 con 0x557f743af2f0 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2e21ab740 10 client.17262 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2e21ab740 1 -- 172.21.15.19:0/1761965688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f74443790 con 0x557f743af2f0 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2bcff9640 1 -- 172.21.15.19:0/1761965688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00035f0 con 0x557f743af2f0 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2bcff9640 1 -- 172.21.15.19:0/1761965688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff2d00a3ce0 con 0x557f743af2f0 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2e21ab740 10 client.17262 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2e21ab740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2e21ab740 20 client.17262 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557f74396d80 0x557f747936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2e21ab740 4 client.17262 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2e21ab740 1 -- 172.21.15.19:0/1761965688 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f746ee3d0 con 0x557f74396d80 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.490+0000 7ff2d8894640 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557f74396d80 0x557f747936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.491+0000 7ff2bcff9640 10 client.17262 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.491+0000 7ff2d8894640 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557f74396d80 0x557f747936e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe2a10f37ed957bb server_cookie=eaf57c7009a70b85 in_seq=0 out_seq=0 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2bcff9640 1 -- 172.21.15.19:0/1761965688 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f746ee3d0 con 0x557f74396d80 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2bcff9640 10 client.17262 handle_command_reply: tid=0 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 client.17262 shutdown 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 -- 172.21.15.19:0/1761965688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743af2f0 msgr2=0x557f74786180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743af2f0 0x557f74786180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0000c00 tx=0x7ff2d00adaf0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 -- 172.21.15.19:0/1761965688 shutdown_connections 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557f74396d80 0x557f747936e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743d81b0 0x557f7478d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f747866c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2bcff9640 0 client.17262 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/1761965688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743af2f0 0x557f74786180 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 -- 172.21.15.19:0/1761965688 >> 172.21.15.19:0/1761965688 conn(0x557f743cc300 msgr2=0x557f74776700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 -- 172.21.15.19:0/1761965688 shutdown_connections 2024-05-09T01:32:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 1 -- 172.21.15.19:0/1761965688 wait complete. 2024-05-09T01:32:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.492+0000 7ff2e21ab740 20 client.17262 trim_cache size 0 max 16384 2024-05-09T01:32:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.507+0000 7ff2e21ab740 1 Processor -- start 2024-05-09T01:32:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.507+0000 7ff2e21ab740 1 -- start start 2024-05-09T01:32:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7477ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f7477b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41ad760 con 0x557f743d81b0 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41af2a0 con 0x557f743b74d0 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41ade30 con 0x557f743af2f0 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9896640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9896640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59076/0 (socket says 172.21.15.19:59076) 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9896640 1 -- 172.21.15.19:0/142109401 learned_addr learned my addr 172.21.15.19:0/142109401 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d8894640 1 --2- 172.21.15.19:0/142109401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f7477b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9095640 1 --2- 172.21.15.19:0/142109401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7477ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9896640 1 -- 172.21.15.19:0/142109401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 msgr2=0x557f7477ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9896640 1 --2- 172.21.15.19:0/142109401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7477ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9896640 1 -- 172.21.15.19:0/142109401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 msgr2=0x557f7477b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9896640 1 --2- 172.21.15.19:0/142109401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f7477b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9896640 1 -- 172.21.15.19:0/142109401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f744ba5f0 con 0x557f743d81b0 2024-05-09T01:32:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2d9896640 1 --2- 172.21.15.19:0/142109401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477b710 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009ec0 tx=0x7ff2cc013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c45f472587896a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2beffd640 1 -- 172.21.15.19:0/142109401 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc020070 con 0x557f743d81b0 2024-05-09T01:32:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2beffd640 1 -- 172.21.15.19:0/142109401 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc011b00 con 0x557f743d81b0 2024-05-09T01:32:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.508+0000 7ff2beffd640 1 -- 172.21.15.19:0/142109401 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc01bc20 con 0x557f743d81b0 2024-05-09T01:32:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.509+0000 7ff2e21ab740 1 -- 172.21.15.19:0/142109401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 msgr2=0x557f7477b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.509+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/142109401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477b710 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009ec0 tx=0x7ff2cc013830 comp rx=0 tx=0).stop 2024-05-09T01:32:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.509+0000 7ff2e21ab740 1 -- 172.21.15.19:0/142109401 shutdown_connections 2024-05-09T01:32:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.509+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/142109401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743d81b0 0x557f7477b710 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.509+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/142109401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743b74d0 0x557f7477b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.509+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/142109401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7477ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.509+0000 7ff2e21ab740 1 -- 172.21.15.19:0/142109401 >> 172.21.15.19:0/142109401 conn(0x557f743cc300 msgr2=0x557f747777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.509+0000 7ff2e21ab740 1 -- 172.21.15.19:0/142109401 shutdown_connections 2024-05-09T01:32:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.509+0000 7ff2e21ab740 1 -- 172.21.15.19:0/142109401 wait complete. 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2e21ab740 1 -- start start 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2e21ab740 1 Processor -- start 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7478af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743b74d0 0x557f7478b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2e21ab740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743d81b0 0x557f7478c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41aebd0 con 0x557f743b74d0 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41ae500 con 0x557f743d81b0 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2e21ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41ab740 con 0x557f743af2f0 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2e21ab740 10 client.0 _ll_register_callbacks cb 0x7ff2d0013070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2d8894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743b74d0 0x557f7478b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2d8894640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743b74d0 0x557f7478b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59078/0 (socket says 172.21.15.19:59078) 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2d8894640 1 -- 172.21.15.19:0/4239010012 learned_addr learned my addr 172.21.15.19:0/4239010012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2d9896640 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743d81b0 0x557f7478c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.510+0000 7ff2d9095640 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7478af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2d8894640 1 -- 172.21.15.19:0/4239010012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 msgr2=0x557f7478af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2d8894640 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7478af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2d8894640 1 -- 172.21.15.19:0/4239010012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743d81b0 msgr2=0x557f7478c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2d8894640 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743d81b0 0x557f7478c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2d8894640 1 -- 172.21.15.19:0/4239010012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f746a4ed0 con 0x557f743b74d0 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2d9095640 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7478af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2d8894640 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743b74d0 0x557f7478b470 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4031500 tx=0x7ff2c402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cce1ca10c3c71cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2bcff9640 1 -- 172.21.15.19:0/4239010012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4035050 con 0x557f743b74d0 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2bcff9640 1 -- 172.21.15.19:0/4239010012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff2c4004450 con 0x557f743b74d0 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2e21ab740 1 -- 172.21.15.19:0/4239010012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2cc011b00 con 0x557f743b74d0 2024-05-09T01:32:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2bcff9640 1 -- 172.21.15.19:0/4239010012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2c4060070 con 0x557f743b74d0 2024-05-09T01:32:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2e21ab740 10 client.17268 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2e21ab740 1 -- 172.21.15.19:0/4239010012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f7478daf0 con 0x557f743b74d0 2024-05-09T01:32:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.511+0000 7ff2bcff9640 1 -- 172.21.15.19:0/4239010012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc011b00 con 0x557f743b74d0 2024-05-09T01:32:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.512+0000 7ff2bcff9640 1 -- 172.21.15.19:0/4239010012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff2c4004cd0 con 0x557f743b74d0 2024-05-09T01:32:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.512+0000 7ff2e21ab740 10 client.17268 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.512+0000 7ff2e21ab740 10 client.17268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.512+0000 7ff2e21ab740 20 client.17268 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:43.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.512+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557f7436c1c0 0x557f747fdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:43.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.512+0000 7ff2e21ab740 4 client.17268 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:32:43.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.512+0000 7ff2e21ab740 1 -- 172.21.15.19:0/4239010012 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f746e8240 con 0x557f7436c1c0 2024-05-09T01:32:43.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.512+0000 7ff2d9095640 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557f7436c1c0 0x557f747fdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:43.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.512+0000 7ff2bcff9640 10 client.17268 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:43.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.513+0000 7ff2d9095640 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557f7436c1c0 0x557f747fdb70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd37be36d878912 server_cookie=533d373062f12180 in_seq=0 out_seq=0 2024-05-09T01:32:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.531+0000 7ff2bcff9640 1 -- 172.21.15.19:0/4239010012 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557f746e8240 con 0x557f7436c1c0 2024-05-09T01:32:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.531+0000 7ff2bcff9640 10 client.17268 handle_command_reply: tid=0 2024-05-09T01:32:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.531+0000 7ff2e21ab740 1 client.17268 shutdown 2024-05-09T01:32:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 -- 172.21.15.19:0/4239010012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743b74d0 msgr2=0x557f7478b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743b74d0 0x557f7478b470 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4031500 tx=0x7ff2c402a410 comp rx=0 tx=0).stop 2024-05-09T01:32:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 -- 172.21.15.19:0/4239010012 shutdown_connections 2024-05-09T01:32:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557f7436c1c0 0x557f747fdb70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f743d81b0 0x557f7478c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2bcff9640 0 client.17268 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f743b74d0 0x557f7478b470 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 --2- 172.21.15.19:0/4239010012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f743af2f0 0x557f7478af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 -- 172.21.15.19:0/4239010012 >> 172.21.15.19:0/4239010012 conn(0x557f743cc300 msgr2=0x557f7477ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 -- 172.21.15.19:0/4239010012 shutdown_connections 2024-05-09T01:32:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 1 -- 172.21.15.19:0/4239010012 wait complete. 2024-05-09T01:32:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2e21ab740 20 client.17268 trim_cache size 0 max 16384 2024-05-09T01:32:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:32:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:32:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:32:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2db2f9640 1 -- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27e0 msgr2=0x7ff2c00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27e0 0x7ff2c00b4cf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4000c00 tx=0x7ff2c4031400 comp rx=0 tx=0).stop 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2db2f9640 1 -- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4102b50 msgr2=0x7ff2d406af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.532+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4102b50 0x7ff2d406af60 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0005fd0 tx=0x7ff2d000dde0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.533+0000 7ff2db2f9640 1 -- 172.21.15.19:0/3231001371 shutdown_connections 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.533+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff2c00b27e0 0x7ff2c00b4cf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.533+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d4108a90 0x7ff2d41a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.533+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d4103d90 0x7ff2d41a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.533+0000 7ff2db2f9640 1 --2- 172.21.15.19:0/3231001371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d4102b50 0x7ff2d406af60 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.533+0000 7ff2db2f9640 1 -- 172.21.15.19:0/3231001371 >> 172.21.15.19:0/3231001371 conn(0x7ff2d40fdd40 msgr2=0x7ff2d4104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.533+0000 7ff2db2f9640 1 -- 172.21.15.19:0/3231001371 shutdown_connections 2024-05-09T01:32:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:43.533+0000 7ff2db2f9640 1 -- 172.21.15.19:0/3231001371 wait complete. 2024-05-09T01:32:44.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:44 smithi019 ceph-mon[30158]: pgmap v594: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:32:44.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:44 smithi196 ceph-mon[35041]: pgmap v594: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:32:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:44 smithi059 ceph-mon[34351]: pgmap v594: 97 pgs: 97 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:32:46.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:46 smithi019 ceph-mon[30158]: pgmap v595: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:32:46.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:46 smithi196 ceph-mon[35041]: pgmap v595: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:32:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:46 smithi059 ceph-mon[34351]: pgmap v595: 97 pgs: 97 active+clean; 4.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:32:48.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:48 smithi019 ceph-mon[30158]: pgmap v596: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:32:48.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:48 smithi196 ceph-mon[35041]: pgmap v596: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:32:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:48 smithi059 ceph-mon[34351]: pgmap v596: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:32:50.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:50 smithi019 ceph-mon[30158]: pgmap v597: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:32:50.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:50 smithi196 ceph-mon[35041]: pgmap v597: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:32:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:50 smithi059 ceph-mon[34351]: pgmap v597: 97 pgs: 97 active+clean; 4.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:32:52.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:52 smithi019 ceph-mon[30158]: pgmap v598: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:52.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:52 smithi196 ceph-mon[35041]: pgmap v598: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:52.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:52 smithi059 ceph-mon[34351]: pgmap v598: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:251/600: open fd = 3 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:252/600: open fd = 3 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:253/600: open fd = 3 2024-05-09T01:32:54.455 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:254/600: open fd = 3 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:255/600: open fd = 3 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:256/600: open fd = 3 2024-05-09T01:32:54.456 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:257/600: open fd = 3 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:258/600: open fd = 3 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:259/600: open fd = 3 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.457 INFO:tasks.workunit.client.0.smithi019.stdout:260/600: open fd = 3 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:261/600: open fd = 3 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:262/600: open fd = 3 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:263/600: open fd = 3 2024-05-09T01:32:54.458 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:264/600: open fd = 3 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:265/600: open fd = 3 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:266/600: open fd = 3 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.459 INFO:tasks.workunit.client.0.smithi019.stdout:267/600: open fd = 3 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:268/600: open fd = 3 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:269/600: open fd = 3 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:270/600: open fd = 3 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.460 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:271/600: open fd = 3 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:272/600: open fd = 3 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:273/600: open fd = 3 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.461 INFO:tasks.workunit.client.0.smithi019.stdout:274/600: open fd = 3 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:275/600: open fd = 3 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:276/600: open fd = 3 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:277/600: open fd = 3 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.462 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:278/600: open fd = 3 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:279/600: open fd = 3 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:280/600: open fd = 3 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.463 INFO:tasks.workunit.client.0.smithi019.stdout:281/600: open fd = 3 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:282/600: open fd = 3 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:283/600: open fd = 3 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:284/600: open fd = 3 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.464 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:285/600: open fd = 3 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:286/600: open fd = 3 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:287/600: open fd = 3 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.465 INFO:tasks.workunit.client.0.smithi019.stdout:288/600: open fd = 3 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:289/600: open fd = 3 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:290/600: open fd = 3 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:291/600: open fd = 3 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.466 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:292/600: open fd = 3 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:293/600: open fd = 3 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:294/600: open fd = 3 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:295/600: open fd = 3 2024-05-09T01:32:54.467 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:296/600: open fd = 3 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:297/600: open fd = 3 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:298/600: open fd = 3 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.468 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:299/600: open fd = 3 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:300/600: open fd = 3 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:301/600: open fd = 3 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.469 INFO:tasks.workunit.client.0.smithi019.stdout:302/600: open fd = 3 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:303/600: open fd = 3 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:304/600: open fd = 3 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:305/600: open fd = 3 2024-05-09T01:32:54.470 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:306/600: open fd = 3 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:307/600: open fd = 3 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:308/600: open fd = 3 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.471 INFO:tasks.workunit.client.0.smithi019.stdout:309/600: open fd = 3 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:310/600: open fd = 3 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:311/600: open fd = 3 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:312/600: open fd = 3 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.472 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:32:54.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:54 smithi019 ceph-mon[30158]: pgmap v599: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:32:54.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:54 smithi196 ceph-mon[35041]: pgmap v599: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:32:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:54 smithi059 ceph-mon[34351]: pgmap v599: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:32:55.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:32:55.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:32:55.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:32:55.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:32:55.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:32:55.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:32:55.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:32:55.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:32:55.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:32:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:32:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:32:55.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:32:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.646+0000 7f035770d640 1 -- 172.21.15.19:0/1842074123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350103c60 msgr2=0x7f0350104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.646+0000 7f035770d640 1 --2- 172.21.15.19:0/1842074123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350103c60 0x7f0350104960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f0348009900 tx=0x7f034802a410 comp rx=0 tx=0).stop 2024-05-09T01:32:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.646+0000 7f035770d640 1 -- 172.21.15.19:0/1842074123 shutdown_connections 2024-05-09T01:32:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.646+0000 7f035770d640 1 --2- 172.21.15.19:0/1842074123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0350109ba0 0x7f035010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.646+0000 7f035770d640 1 --2- 172.21.15.19:0/1842074123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0350104ea0 0x7f03501052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.646+0000 7f035770d640 1 --2- 172.21.15.19:0/1842074123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350103c60 0x7f0350104960 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.646+0000 7f035770d640 1 -- 172.21.15.19:0/1842074123 >> 172.21.15.19:0/1842074123 conn(0x7f03500fee70 msgr2=0x7f0350101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.647+0000 7f035770d640 1 -- 172.21.15.19:0/1842074123 shutdown_connections 2024-05-09T01:32:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.647+0000 7f035770d640 1 -- 172.21.15.19:0/1842074123 wait complete. 2024-05-09T01:32:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.647+0000 7f035770d640 1 Processor -- start 2024-05-09T01:32:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.647+0000 7f035770d640 1 -- start start 2024-05-09T01:32:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.647+0000 7f035770d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0350103c60 0x7f035019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.647+0000 7f035770d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0350104ea0 0x7f035019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.647+0000 7f035770d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350109ba0 0x7f03501a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f035770d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f035010f740 con 0x7f0350109ba0 2024-05-09T01:32:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f035770d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f035010f5c0 con 0x7f0350103c60 2024-05-09T01:32:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f035770d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f035010f8c0 con 0x7f0350104ea0 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0355caa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350109ba0 0x7f03501a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0355caa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350109ba0 0x7f03501a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54784/0 (socket says 172.21.15.19:54784) 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0355caa640 1 -- 172.21.15.19:0/1418402318 learned_addr learned my addr 172.21.15.19:0/1418402318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f03554a9640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0350103c60 0x7f035019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0354ca8640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0350104ea0 0x7f035019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0355caa640 1 -- 172.21.15.19:0/1418402318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0350104ea0 msgr2=0x7f035019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0355caa640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0350104ea0 0x7f035019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0355caa640 1 -- 172.21.15.19:0/1418402318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0350103c60 msgr2=0x7f035019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0355caa640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0350103c60 0x7f035019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0355caa640 1 -- 172.21.15.19:0/1418402318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03501a1ec0 con 0x7f0350109ba0 2024-05-09T01:32:56.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:56 smithi019 ceph-mon[30158]: pgmap v600: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f0355caa640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350109ba0 0x7f03501a1850 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f0340009fd0 tx=0x7f034000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f034e7fc640 1 -- 172.21.15.19:0/1418402318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340017070 con 0x7f0350109ba0 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.648+0000 7f034e7fc640 1 -- 172.21.15.19:0/1418402318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0340003e20 con 0x7f0350109ba0 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.649+0000 7f035770d640 1 -- 172.21.15.19:0/1418402318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03501a21b0 con 0x7f0350109ba0 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.649+0000 7f034e7fc640 1 -- 172.21.15.19:0/1418402318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340010460 con 0x7f0350109ba0 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.649+0000 7f035770d640 1 -- 172.21.15.19:0/1418402318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03501a26b0 con 0x7f0350109ba0 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.651+0000 7f035e5c4740 1 Processor -- start 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f034e7fc640 1 -- 172.21.15.19:0/1418402318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0340003970 con 0x7f0350109ba0 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f035e5c4740 1 -- start start 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcf61b0 0x558a5c09e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c09ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 0x558a5c0a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f035e5c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a5bffe7c0 con 0x558a5bcf61b0 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f035e5c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558a5bf9a0e0 con 0x558a5bccd2f0 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f035e5c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a5bffa780 con 0x558a5bcd54d0 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f0354ca8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcf61b0 0x558a5c09e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f0354ca8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcf61b0 0x558a5c09e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54800/0 (socket says 172.21.15.19:54800) 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f03554a9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c09ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.652+0000 7f0354ca8640 1 -- 172.21.15.19:0/2029954479 learned_addr learned my addr 172.21.15.19:0/2029954479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f0354ca8640 1 --2- 172.21.15.19:0/2029954479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 0x558a5c0a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f0354ca8640 1 -- 172.21.15.19:0/2029954479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 msgr2=0x558a5c0a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f0354ca8640 1 --2- 172.21.15.19:0/2029954479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 0x558a5c0a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f0354ca8640 1 -- 172.21.15.19:0/2029954479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 msgr2=0x558a5c09ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f0354ca8640 1 --2- 172.21.15.19:0/2029954479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c09ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f0354ca8640 1 -- 172.21.15.19:0/2029954479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a5bfc37c0 con 0x558a5bcf61b0 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f03554a9640 1 --2- 172.21.15.19:0/2029954479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c09ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f034e7fc640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03380b27e0 0x7f03380b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f0354ca8640 1 --2- 172.21.15.19:0/2029954479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcf61b0 0x558a5c09e910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f03440135a0 tx=0x7f0344009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444ad8d50a98f938 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f03554a9640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03380b27e0 0x7f03380b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f03377fe640 1 -- 172.21.15.19:0/2029954479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034400f580 con 0x558a5bcf61b0 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f034e7fc640 1 -- 172.21.15.19:0/1418402318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03400e1f40 con 0x7f0350109ba0 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f03377fe640 1 -- 172.21.15.19:0/2029954479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034400fbd0 con 0x558a5bcf61b0 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.653+0000 7f03377fe640 1 -- 172.21.15.19:0/2029954479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344010b50 con 0x558a5bcf61b0 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f035e5c4740 1 -- 172.21.15.19:0/2029954479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcf61b0 msgr2=0x558a5c09e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2029954479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcf61b0 0x558a5c09e910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f03440135a0 tx=0x7f0344009850 comp rx=0 tx=0).stop 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f03554a9640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03380b27e0 0x7f03380b4cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0348005ab0 tx=0x7f0348031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f035e5c4740 1 -- 172.21.15.19:0/2029954479 shutdown_connections 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2029954479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 0x558a5c0a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2029954479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c09ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2029954479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcf61b0 0x558a5c09e910 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f035e5c4740 1 -- 172.21.15.19:0/2029954479 >> 172.21.15.19:0/2029954479 conn(0x558a5bcea300 msgr2=0x558a5c095e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f035e5c4740 1 -- 172.21.15.19:0/2029954479 shutdown_connections 2024-05-09T01:32:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.654+0000 7f035e5c4740 1 -- 172.21.15.19:0/2029954479 wait complete. 2024-05-09T01:32:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.655+0000 7f035e5c4740 1 -- start start 2024-05-09T01:32:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.655+0000 7f035e5c4740 1 Processor -- start 2024-05-09T01:32:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.655+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.655+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.655+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcf61b0 0x558a5c0afb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.655+0000 7f035e5c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a5c0a6780 con 0x558a5bcd54d0 2024-05-09T01:32:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.655+0000 7f035e5c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558a5bf859c0 con 0x558a5bccd2f0 2024-05-09T01:32:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.655+0000 7f035e5c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a5c0a6900 con 0x558a5bcf61b0 2024-05-09T01:32:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.655+0000 7f035e5c4740 10 client.0 _ll_register_callbacks cb 0x558a5bfb2020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0354ca8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0354ca8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0a8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54806/0 (socket says 172.21.15.19:54806) 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0354ca8640 1 -- 172.21.15.19:0/1791833012 learned_addr learned my addr 172.21.15.19:0/1791833012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0355caa640 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcf61b0 0x558a5c0afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f03554a9640 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0354ca8640 1 -- 172.21.15.19:0/1791833012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcf61b0 msgr2=0x558a5c0afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0354ca8640 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcf61b0 0x558a5c0afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0354ca8640 1 -- 172.21.15.19:0/1791833012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 msgr2=0x558a5c0a8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0354ca8640 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0a8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0354ca8640 1 -- 172.21.15.19:0/1791833012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a5bfac1f0 con 0x558a5bcd54d0 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f03554a9640 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0a8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0354ca8640 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0a8bb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0344014fd0 tx=0x7f0344005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49231392175bf99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0334ff9640 1 -- 172.21.15.19:0/1791833012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344004050 con 0x558a5bcd54d0 2024-05-09T01:32:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0334ff9640 1 -- 172.21.15.19:0/1791833012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03440041f0 con 0x558a5bcd54d0 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f035e5c4740 1 -- 172.21.15.19:0/1791833012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f034400fbd0 con 0x558a5bcd54d0 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.656+0000 7f0334ff9640 1 -- 172.21.15.19:0/1791833012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0344030070 con 0x558a5bcd54d0 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f035e5c4740 10 client.17280 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f035e5c4740 1 -- 172.21.15.19:0/1791833012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a5bd61790 con 0x558a5bcd54d0 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f0334ff9640 1 -- 172.21.15.19:0/1791833012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034400fbd0 con 0x558a5bcd54d0 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f0334ff9640 1 -- 172.21.15.19:0/1791833012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0344005040 con 0x558a5bcd54d0 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f035e5c4740 10 client.17280 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f035e5c4740 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f035e5c4740 20 client.17280 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f035e5c4740 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558a5bcb4d80 0x558a5c0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f035e5c4740 4 client.17280 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f035e5c4740 1 -- 172.21.15.19:0/1791833012 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a5c00c3d0 con 0x558a5bcb4d80 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.657+0000 7f03554a9640 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558a5bcb4d80 0x558a5c0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.658+0000 7f0334ff9640 10 client.17280 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.658+0000 7f03554a9640 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558a5bcb4d80 0x558a5c0b5db0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916d4884f5b5249 server_cookie=53f8931e79247f0c in_seq=0 out_seq=0 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f0334ff9640 1 -- 172.21.15.19:0/1791833012 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a5c00c3d0 con 0x558a5bcb4d80 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f0334ff9640 10 client.17280 handle_command_reply: tid=0 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 client.17280 shutdown 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 -- 172.21.15.19:0/1791833012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 msgr2=0x558a5c0a8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0a8bb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0344014fd0 tx=0x7f0344005d90 comp rx=0 tx=0).stop 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 -- 172.21.15.19:0/1791833012 shutdown_connections 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558a5bcb4d80 0x558a5c0b5db0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcf61b0 0x558a5c0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0a8bb0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 --2- 172.21.15.19:0/1791833012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0a8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f0334ff9640 0 client.17280 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 -- 172.21.15.19:0/1791833012 >> 172.21.15.19:0/1791833012 conn(0x558a5bcea300 msgr2=0x558a5c094700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 -- 172.21.15.19:0/1791833012 shutdown_connections 2024-05-09T01:32:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 1 -- 172.21.15.19:0/1791833012 wait complete. 2024-05-09T01:32:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.659+0000 7f035e5c4740 20 client.17280 trim_cache size 0 max 16384 2024-05-09T01:32:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f035e5c4740 1 Processor -- start 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f035e5c4740 1 -- start start 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcea300 0x558a5c094e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f035e5c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03501a9220 con 0x558a5bcd54d0 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f035e5c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03501aad60 con 0x558a5bccd2f0 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f035e5c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03501a98f0 con 0x558a5bcea300 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0355caa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0355caa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54814/0 (socket says 172.21.15.19:54814) 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0355caa640 1 -- 172.21.15.19:0/3428138789 learned_addr learned my addr 172.21.15.19:0/3428138789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0355caa640 1 -- 172.21.15.19:0/3428138789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcea300 msgr2=0x558a5c094e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0355caa640 1 --2- 172.21.15.19:0/3428138789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcea300 0x558a5c094e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0355caa640 1 -- 172.21.15.19:0/3428138789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 msgr2=0x558a5c0afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0355caa640 1 --2- 172.21.15.19:0/3428138789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0355caa640 1 -- 172.21.15.19:0/3428138789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a5bdd85f0 con 0x558a5bcd54d0 2024-05-09T01:32:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0355caa640 1 --2- 172.21.15.19:0/3428138789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0aff70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0340005c40 tx=0x7f0340003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94dfc7bc8be7963d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.675+0000 7f0336ffd640 1 -- 172.21.15.19:0/3428138789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340020040 con 0x558a5bcd54d0 2024-05-09T01:32:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f0336ffd640 1 -- 172.21.15.19:0/3428138789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400b23a0 con 0x558a5bcd54d0 2024-05-09T01:32:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f0336ffd640 1 -- 172.21.15.19:0/3428138789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034008fdc0 con 0x558a5bcd54d0 2024-05-09T01:32:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f035e5c4740 1 -- 172.21.15.19:0/3428138789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 msgr2=0x558a5c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f035e5c4740 1 --2- 172.21.15.19:0/3428138789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0aff70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0340005c40 tx=0x7f0340003db0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f035e5c4740 1 -- 172.21.15.19:0/3428138789 shutdown_connections 2024-05-09T01:32:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f035e5c4740 1 --2- 172.21.15.19:0/3428138789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcd54d0 0x558a5c0aff70 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f035e5c4740 1 --2- 172.21.15.19:0/3428138789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f035e5c4740 1 --2- 172.21.15.19:0/3428138789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcea300 0x558a5c094e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f035e5c4740 1 -- 172.21.15.19:0/3428138789 >> 172.21.15.19:0/3428138789 conn(0x558a5c0a3340 msgr2=0x558a5c094700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f035e5c4740 1 -- 172.21.15.19:0/3428138789 shutdown_connections 2024-05-09T01:32:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.676+0000 7f035e5c4740 1 -- 172.21.15.19:0/3428138789 wait complete. 2024-05-09T01:32:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f035e5c4740 1 -- start start 2024-05-09T01:32:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f035e5c4740 1 Processor -- start 2024-05-09T01:32:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 0x558a5c0addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f035e5c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcea300 0x558a5c0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f035e5c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03501aa690 con 0x558a5bcea300 2024-05-09T01:32:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f035e5c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03501a9fc0 con 0x558a5bccd2f0 2024-05-09T01:32:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f035e5c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03501a7200 con 0x558a5bcd54d0 2024-05-09T01:32:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f035e5c4740 10 client.0 _ll_register_callbacks cb 0x7f0348009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f0355caa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcea300 0x558a5c0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f0355caa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcea300 0x558a5c0aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54822/0 (socket says 172.21.15.19:54822) 2024-05-09T01:32:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f0355caa640 1 -- 172.21.15.19:0/2414570840 learned_addr learned my addr 172.21.15.19:0/2414570840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f03554a9640 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.677+0000 7f0354ca8640 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 0x558a5c0addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0355caa640 1 -- 172.21.15.19:0/2414570840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 msgr2=0x558a5c0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0355caa640 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 0x558a5c0addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0355caa640 1 -- 172.21.15.19:0/2414570840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 msgr2=0x558a5c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0355caa640 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0355caa640 1 -- 172.21.15.19:0/2414570840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a5bfc2ed0 con 0x558a5bcea300 2024-05-09T01:32:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0355caa640 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcea300 0x558a5c0aed20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f03400a6860 tx=0x7f03400a5c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae067aa01ea211f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0334ff9640 1 -- 172.21.15.19:0/2414570840 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340020070 con 0x558a5bcea300 2024-05-09T01:32:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0334ff9640 1 -- 172.21.15.19:0/2414570840 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f034008fd60 con 0x558a5bcea300 2024-05-09T01:32:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f035e5c4740 1 -- 172.21.15.19:0/2414570840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03400b23a0 con 0x558a5bcea300 2024-05-09T01:32:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0334ff9640 1 -- 172.21.15.19:0/2414570840 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03400c9070 con 0x558a5bcea300 2024-05-09T01:32:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f035e5c4740 10 client.17286 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f035e5c4740 1 -- 172.21.15.19:0/2414570840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a5c0b03a0 con 0x558a5bcea300 2024-05-09T01:32:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.678+0000 7f0334ff9640 1 -- 172.21.15.19:0/2414570840 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400b23a0 con 0x558a5bcea300 2024-05-09T01:32:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f0334ff9640 1 -- 172.21.15.19:0/2414570840 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f03400a83e0 con 0x558a5bcea300 2024-05-09T01:32:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f035e5c4740 10 client.17286 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f035e5c4740 10 client.17286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f035e5c4740 20 client.17286 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558a5bcf61b0 0x558a5c1206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f035e5c4740 4 client.17286 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:32:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f035e5c4740 1 -- 172.21.15.19:0/2414570840 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a5c006240 con 0x558a5bcf61b0 2024-05-09T01:32:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f03554a9640 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558a5bcf61b0 0x558a5c1206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f0334ff9640 10 client.17286 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.679+0000 7f03554a9640 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558a5bcf61b0 0x558a5c1206b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b602c2b78672f76e server_cookie=cc3cf9a5bd633c53 in_seq=0 out_seq=0 2024-05-09T01:32:56.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:56 smithi196 ceph-mon[35041]: pgmap v600: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:32:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f0334ff9640 1 -- 172.21.15.19:0/2414570840 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558a5c006240 con 0x558a5bcf61b0 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f0334ff9640 10 client.17286 handle_command_reply: tid=0 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 client.17286 shutdown 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 -- 172.21.15.19:0/2414570840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcea300 msgr2=0x558a5c0aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcea300 0x558a5c0aed20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f03400a6860 tx=0x7f03400a5c50 comp rx=0 tx=0).stop 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 -- 172.21.15.19:0/2414570840 shutdown_connections 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558a5bcf61b0 0x558a5c1206b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a5bcea300 0x558a5c0aed20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a5bcd54d0 0x558a5c0addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 --2- 172.21.15.19:0/2414570840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a5bccd2f0 0x558a5c0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f0334ff9640 0 client.17286 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 -- 172.21.15.19:0/2414570840 >> 172.21.15.19:0/2414570840 conn(0x558a5c0a3340 msgr2=0x558a5c095680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 -- 172.21.15.19:0/2414570840 shutdown_connections 2024-05-09T01:32:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 1 -- 172.21.15.19:0/2414570840 wait complete. 2024-05-09T01:32:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.698+0000 7f035e5c4740 20 client.17286 trim_cache size 0 max 16384 2024-05-09T01:32:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:32:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:32:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:32:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 -- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03380b27e0 msgr2=0x7f03380b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03380b27e0 0x7f03380b4cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0348005ab0 tx=0x7f0348031400 comp rx=0 tx=0).stop 2024-05-09T01:32:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 -- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350109ba0 msgr2=0x7f03501a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350109ba0 0x7f03501a1850 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f0340009fd0 tx=0x7f034000b260 comp rx=0 tx=0).stop 2024-05-09T01:32:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 -- 172.21.15.19:0/1418402318 shutdown_connections 2024-05-09T01:32:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03380b27e0 0x7f03380b4cf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0350109ba0 0x7f03501a1850 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0350104ea0 0x7f035019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 --2- 172.21.15.19:0/1418402318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0350103c60 0x7f035019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 -- 172.21.15.19:0/1418402318 >> 172.21.15.19:0/1418402318 conn(0x7f03500fee70 msgr2=0x7f0350105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 -- 172.21.15.19:0/1418402318 shutdown_connections 2024-05-09T01:32:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:32:56.699+0000 7f035770d640 1 -- 172.21.15.19:0/1418402318 wait complete. 2024-05-09T01:32:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:56 smithi059 ceph-mon[34351]: pgmap v600: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:32:57.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:32:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:32:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:32:58 smithi019 ceph-mon[30158]: pgmap v601: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-05-09T01:32:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:32:58 smithi196 ceph-mon[35041]: pgmap v601: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-05-09T01:32:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:32:58 smithi059 ceph-mon[34351]: pgmap v601: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-05-09T01:32:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.444+0000 7fb96463b700 1 -- 172.21.15.19:0/2152191935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 msgr2=0x7fb95c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.444+0000 7fb96463b700 1 --2- 172.21.15.19:0/2152191935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 0x7fb95c1100a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb9540099c0 tx=0x7fb954009cd0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.445+0000 7fb96463b700 1 -- 172.21.15.19:0/2152191935 shutdown_connections 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.445+0000 7fb96463b700 1 --2- 172.21.15.19:0/2152191935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 0x7fb95c1100a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.445+0000 7fb96463b700 1 --2- 172.21.15.19:0/2152191935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c1034c0 0x7fb95c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.445+0000 7fb96463b700 1 --2- 172.21.15.19:0/2152191935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb95c107df0 0x7fb95c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.445+0000 7fb96463b700 1 -- 172.21.15.19:0/2152191935 >> 172.21.15.19:0/2152191935 conn(0x7fb95c0fedc0 msgr2=0x7fb95c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.445+0000 7fb96463b700 1 -- 172.21.15.19:0/2152191935 shutdown_connections 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.445+0000 7fb96463b700 1 -- 172.21.15.19:0/2152191935 wait complete. 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb96463b700 1 Processor -- start 2024-05-09T01:32:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb96463b700 1 -- start start 2024-05-09T01:32:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb96463b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c1034c0 0x7fb95c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb96463b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb95c107df0 0x7fb95c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb96463b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 0x7fb95c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb96463b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c113390 con 0x7fb95c108740 2024-05-09T01:32:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb96463b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c113210 con 0x7fb95c107df0 2024-05-09T01:32:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb96463b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c113510 con 0x7fb95c1034c0 2024-05-09T01:32:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb962bd8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 0x7fb95c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb962bd8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 0x7fb95c1a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54842/0 (socket says 172.21.15.19:54842) 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb962bd8700 1 -- 172.21.15.19:0/4064655052 learned_addr learned my addr 172.21.15.19:0/4064655052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.446+0000 7fb961bd6700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb95c107df0 0x7fb95c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb9623d7700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c1034c0 0x7fb95c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb962bd8700 1 -- 172.21.15.19:0/4064655052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c1034c0 msgr2=0x7fb95c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb962bd8700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c1034c0 0x7fb95c19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb962bd8700 1 -- 172.21.15.19:0/4064655052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb95c107df0 msgr2=0x7fb95c19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb962bd8700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb95c107df0 0x7fb95c19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb962bd8700 1 -- 172.21.15.19:0/4064655052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c1a1de0 con 0x7fb95c108740 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb962bd8700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 0x7fb95c1a17f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb954005910 tx=0x7fb954009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb95b7fe700 1 -- 172.21.15.19:0/4064655052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954027070 con 0x7fb95c108740 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb96463b700 1 -- 172.21.15.19:0/4064655052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c1a2070 con 0x7fb95c108740 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb95b7fe700 1 -- 172.21.15.19:0/4064655052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb954003f20 con 0x7fb95c108740 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb96463b700 1 -- 172.21.15.19:0/4064655052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c1a25a0 con 0x7fb95c108740 2024-05-09T01:32:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.447+0000 7fb95b7fe700 1 -- 172.21.15.19:0/4064655052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954022c50 con 0x7fb95c108740 2024-05-09T01:32:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.451+0000 7fb95b7fe700 1 -- 172.21.15.19:0/4064655052 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb95402a070 con 0x7fb95c108740 2024-05-09T01:32:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.451+0000 7fb973051b80 1 Processor -- start 2024-05-09T01:32:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.451+0000 7fb973051b80 1 -- start start 2024-05-09T01:32:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb95b7fe700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9440b2880 0x7fb9440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb961bd6700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9440b2880 0x7fb9440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb973051b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 0x55e63f8944f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb95b7fe700 1 -- 172.21.15.19:0/4064655052 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb9540e2eb0 con 0x7fb95c108740 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb973051b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7114a0 0x55e63f894a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb973051b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f88ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb973051b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e63f7b0ff0 con 0x55e63f7114a0 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb973051b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e63f74f7f0 con 0x55e63f7fb160 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb973051b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e63f7acfb0 con 0x55e63f88e6a0 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb962bd8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f88ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb961bd6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7114a0 0x55e63f894a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb962bd8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f88ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:43768/0 (socket says 172.21.15.19:43768) 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb962bd8700 1 -- 172.21.15.19:0/4227193078 learned_addr learned my addr 172.21.15.19:0/4227193078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.452+0000 7fb9623d7700 1 --2- 172.21.15.19:0/4227193078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 0x55e63f8944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb962bd8700 1 -- 172.21.15.19:0/4227193078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 msgr2=0x55e63f8944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb962bd8700 1 --2- 172.21.15.19:0/4227193078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 0x55e63f8944f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb962bd8700 1 -- 172.21.15.19:0/4227193078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7114a0 msgr2=0x55e63f894a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb962bd8700 1 --2- 172.21.15.19:0/4227193078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7114a0 0x55e63f894a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb962bd8700 1 -- 172.21.15.19:0/4227193078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e63f747370 con 0x55e63f88e6a0 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb961bd6700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9440b2880 0x7fb9440b4d90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb95c06c310 tx=0x7fb9500134e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb962bd8700 1 --2- 172.21.15.19:0/4227193078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f88ead0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9540a65c0 tx=0x7fb954009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e03aafaed8f342d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb958ff9700 1 -- 172.21.15.19:0/4227193078 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540aa540 con 0x55e63f88e6a0 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb958ff9700 1 -- 172.21.15.19:0/4227193078 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9540e0e50 con 0x55e63f88e6a0 2024-05-09T01:32:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.453+0000 7fb958ff9700 1 -- 172.21.15.19:0/4227193078 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954022560 con 0x55e63f88e6a0 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 -- 172.21.15.19:0/4227193078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 msgr2=0x55e63f88ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 --2- 172.21.15.19:0/4227193078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f88ead0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9540a65c0 tx=0x7fb954009310 comp rx=0 tx=0).stop 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 -- 172.21.15.19:0/4227193078 shutdown_connections 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 --2- 172.21.15.19:0/4227193078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f88ead0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 --2- 172.21.15.19:0/4227193078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 0x55e63f8944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 --2- 172.21.15.19:0/4227193078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7114a0 0x55e63f894a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 -- 172.21.15.19:0/4227193078 >> 172.21.15.19:0/4227193078 conn(0x55e63f7716a0 msgr2=0x55e63f889400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 -- 172.21.15.19:0/4227193078 shutdown_connections 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 -- 172.21.15.19:0/4227193078 wait complete. 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.454+0000 7fb973051b80 1 -- start start 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb973051b80 1 Processor -- start 2024-05-09T01:32:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb973051b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 0x55e63f89dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb973051b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f89e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb973051b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f898110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb973051b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e63f62cba0 con 0x55e63f7fb160 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb973051b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e63f76b150 con 0x55e63f7114a0 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb973051b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e63f88f360 con 0x55e63f88e6a0 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb973051b80 10 client.0 _ll_register_callbacks cb 0x55e63f6226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb961bd6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f89e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb961bd6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f89e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54854/0 (socket says 172.21.15.19:54854) 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb961bd6700 1 -- 172.21.15.19:0/1418161568 learned_addr learned my addr 172.21.15.19:0/1418161568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb962bd8700 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f898110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb9623d7700 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 0x55e63f89dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb961bd6700 1 -- 172.21.15.19:0/1418161568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 msgr2=0x55e63f898110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb961bd6700 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f898110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb961bd6700 1 -- 172.21.15.19:0/1418161568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 msgr2=0x55e63f89dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb961bd6700 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 0x55e63f89dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.455+0000 7fb961bd6700 1 -- 172.21.15.19:0/1418161568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e63f62c490 con 0x55e63f7fb160 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.456+0000 7fb961bd6700 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f89e250 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb950011a80 tx=0x7fb95001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8b53030bc1ed64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.456+0000 7fb941ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.456+0000 7fb941ffb700 1 -- 172.21.15.19:0/1418161568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950003d10 con 0x55e63f7fb160 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.456+0000 7fb941ffb700 1 -- 172.21.15.19:0/1418161568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb950020430 con 0x55e63f7fb160 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.456+0000 7fb973051b80 1 -- 172.21.15.19:0/1418161568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9540e0e50 con 0x55e63f7fb160 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.456+0000 7fb941ffb700 1 -- 172.21.15.19:0/1418161568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb95000ad30 con 0x55e63f7fb160 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.456+0000 7fb973051b80 10 client.17298 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.456+0000 7fb973051b80 1 -- 172.21.15.19:0/1418161568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e63f8a5930 con 0x55e63f7fb160 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.456+0000 7fb941ffb700 1 -- 172.21.15.19:0/1418161568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9540e0e50 con 0x55e63f7fb160 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.457+0000 7fb941ffb700 1 -- 172.21.15.19:0/1418161568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb95001f070 con 0x55e63f7fb160 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.457+0000 7fb973051b80 10 client.17298 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.457+0000 7fb973051b80 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.457+0000 7fb973051b80 20 client.17298 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.457+0000 7fb973051b80 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e63f8a7f70 0x55e63f8aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.457+0000 7fb973051b80 4 client.17298 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:32:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.457+0000 7fb973051b80 1 -- 172.21.15.19:0/1418161568 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e63f7be9d0 con 0x55e63f8a7f70 2024-05-09T01:32:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.457+0000 7fb9623d7700 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e63f8a7f70 0x55e63f8aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.458+0000 7fb9623d7700 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e63f8a7f70 0x55e63f8aa3b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe79e8cf716351a5 server_cookie=5aa27dbdd951f67d in_seq=0 out_seq=0 2024-05-09T01:32:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.458+0000 7fb941ffb700 10 client.17298 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.458+0000 7fb941ffb700 1 -- 172.21.15.19:0/1418161568 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e63f7be9d0 con 0x55e63f8a7f70 2024-05-09T01:32:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.458+0000 7fb941ffb700 10 client.17298 handle_command_reply: tid=0 2024-05-09T01:32:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.458+0000 7fb973051b80 1 client.17298 shutdown 2024-05-09T01:32:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.458+0000 7fb973051b80 1 -- 172.21.15.19:0/1418161568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 msgr2=0x55e63f89e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.458+0000 7fb973051b80 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f89e250 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb950011a80 tx=0x7fb95001b000 comp rx=0 tx=0).stop 2024-05-09T01:32:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb973051b80 1 -- 172.21.15.19:0/1418161568 shutdown_connections 2024-05-09T01:32:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb973051b80 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e63f8a7f70 0x55e63f8aa3b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb973051b80 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f898110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb973051b80 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 0x55e63f89dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb973051b80 1 --2- 172.21.15.19:0/1418161568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f89e250 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb941ffb700 0 client.17298 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb973051b80 1 -- 172.21.15.19:0/1418161568 >> 172.21.15.19:0/1418161568 conn(0x55e63f7716a0 msgr2=0x55e63f8891c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb973051b80 1 -- 172.21.15.19:0/1418161568 shutdown_connections 2024-05-09T01:32:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb973051b80 1 -- 172.21.15.19:0/1418161568 wait complete. 2024-05-09T01:32:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.459+0000 7fb973051b80 20 client.17298 trim_cache size 0 max 16384 2024-05-09T01:32:59.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.479+0000 7fb973051b80 1 Processor -- start 2024-05-09T01:32:59.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.479+0000 7fb973051b80 1 -- start start 2024-05-09T01:32:59.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.479+0000 7fb973051b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f7114a0 0x55e63f893820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.479+0000 7fb973051b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 0x55e63f8afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.479+0000 7fb973051b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f88e6a0 0x55e63f8b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.479+0000 7fb973051b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1b0ce0 con 0x55e63f88e6a0 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.479+0000 7fb973051b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1b2820 con 0x55e63f7fb160 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.479+0000 7fb973051b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1b13b0 con 0x55e63f7114a0 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.479+0000 7fb962bd8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f88e6a0 0x55e63f8b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb962bd8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f88e6a0 0x55e63f8b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54860/0 (socket says 172.21.15.19:54860) 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb962bd8700 1 -- 172.21.15.19:0/4154906301 learned_addr learned my addr 172.21.15.19:0/4154906301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb961bd6700 1 --2- 172.21.15.19:0/4154906301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 0x55e63f8afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb9623d7700 1 --2- 172.21.15.19:0/4154906301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f7114a0 0x55e63f893820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb962bd8700 1 -- 172.21.15.19:0/4154906301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f7114a0 msgr2=0x55e63f893820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb962bd8700 1 --2- 172.21.15.19:0/4154906301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f7114a0 0x55e63f893820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb962bd8700 1 -- 172.21.15.19:0/4154906301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 msgr2=0x55e63f8afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb962bd8700 1 --2- 172.21.15.19:0/4154906301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 0x55e63f8afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb962bd8700 1 -- 172.21.15.19:0/4154906301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e63f897430 con 0x55e63f88e6a0 2024-05-09T01:32:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb962bd8700 1 --2- 172.21.15.19:0/4154906301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f88e6a0 0x55e63f8b00b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb9540a77b0 tx=0x7fb9540aa3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87fe1d7858bfd5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb943fff700 1 -- 172.21.15.19:0/4154906301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540b3310 con 0x55e63f88e6a0 2024-05-09T01:32:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb943fff700 1 -- 172.21.15.19:0/4154906301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9540e0e50 con 0x55e63f88e6a0 2024-05-09T01:32:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.480+0000 7fb943fff700 1 -- 172.21.15.19:0/4154906301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540b1be0 con 0x55e63f88e6a0 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 -- 172.21.15.19:0/4154906301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f88e6a0 msgr2=0x55e63f8b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 --2- 172.21.15.19:0/4154906301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f88e6a0 0x55e63f8b00b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb9540a77b0 tx=0x7fb9540aa3a0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 -- 172.21.15.19:0/4154906301 shutdown_connections 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 --2- 172.21.15.19:0/4154906301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f88e6a0 0x55e63f8b00b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 --2- 172.21.15.19:0/4154906301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f7114a0 0x55e63f893820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 --2- 172.21.15.19:0/4154906301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7fb160 0x55e63f8afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 -- 172.21.15.19:0/4154906301 >> 172.21.15.19:0/4154906301 conn(0x55e63f7716a0 msgr2=0x55e63f893110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 -- 172.21.15.19:0/4154906301 shutdown_connections 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 -- 172.21.15.19:0/4154906301 wait complete. 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 -- start start 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.481+0000 7fb973051b80 1 Processor -- start 2024-05-09T01:32:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb973051b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 0x55e63f8a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb973051b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f8a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb973051b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f8a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb973051b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1b2150 con 0x55e63f7fb160 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb973051b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1b1a80 con 0x55e63f7114a0 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb973051b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1aec80 con 0x55e63f88e6a0 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb973051b80 10 client.0 _ll_register_callbacks cb 0x55e63f6226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb961bd6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f8a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb961bd6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f8a4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54870/0 (socket says 172.21.15.19:54870) 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb961bd6700 1 -- 172.21.15.19:0/2030703838 learned_addr learned my addr 172.21.15.19:0/2030703838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb9623d7700 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 0x55e63f8a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb962bd8700 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f8a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb961bd6700 1 -- 172.21.15.19:0/2030703838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 msgr2=0x55e63f8a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb961bd6700 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f8a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb961bd6700 1 -- 172.21.15.19:0/2030703838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 msgr2=0x55e63f8a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb961bd6700 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 0x55e63f8a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb961bd6700 1 -- 172.21.15.19:0/2030703838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e63f8a2760 con 0x55e63f7fb160 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb961bd6700 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f8a4130 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb95000a790 tx=0x7fb95000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75113bb9a8c03a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.482+0000 7fb941ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb941ffb700 1 -- 172.21.15.19:0/2030703838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950003d70 con 0x55e63f7fb160 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb941ffb700 1 -- 172.21.15.19:0/2030703838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb950005040 con 0x55e63f7fb160 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb973051b80 1 -- 172.21.15.19:0/2030703838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9540e0e50 con 0x55e63f7fb160 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb941ffb700 1 -- 172.21.15.19:0/2030703838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb950044070 con 0x55e63f7fb160 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb973051b80 10 client.17304 fetch_fsmap learned FSMap version 28 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb973051b80 1 -- 172.21.15.19:0/2030703838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e63f913030 con 0x55e63f7fb160 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb941ffb700 1 -- 172.21.15.19:0/2030703838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9540e0e50 con 0x55e63f7fb160 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb941ffb700 1 -- 172.21.15.19:0/2030703838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb95001f040 con 0x55e63f7fb160 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb973051b80 10 client.17304 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb973051b80 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:32:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb973051b80 20 client.17304 populate_metadata read hostname 'smithi019' 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb973051b80 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e63f9136e0 0x55e63f915b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb973051b80 4 client.17304 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.483+0000 7fb973051b80 1 -- 172.21.15.19:0/2030703838 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e63f7b8a70 con 0x55e63f9136e0 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb9623d7700 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e63f9136e0 0x55e63f915b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb941ffb700 10 client.17304 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb9623d7700 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e63f9136e0 0x55e63f915b00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b36c93b46ddc3e server_cookie=68f584c0f9e8d6b2 in_seq=0 out_seq=0 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb941ffb700 1 -- 172.21.15.19:0/2030703838 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e63f7b8a70 con 0x55e63f9136e0 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb941ffb700 10 client.17304 handle_command_reply: tid=0 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb973051b80 1 client.17304 shutdown 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb973051b80 1 -- 172.21.15.19:0/2030703838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 msgr2=0x55e63f8a4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb973051b80 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f8a4130 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb95000a790 tx=0x7fb95000ef40 comp rx=0 tx=0).stop 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb973051b80 1 -- 172.21.15.19:0/2030703838 shutdown_connections 2024-05-09T01:32:59.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb973051b80 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e63f9136e0 0x55e63f915b00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb973051b80 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e63f88e6a0 0x55e63f8a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb973051b80 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e63f7114a0 0x55e63f8a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb973051b80 1 --2- 172.21.15.19:0/2030703838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e63f7fb160 0x55e63f8a4130 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.484+0000 7fb973051b80 1 -- 172.21.15.19:0/2030703838 >> 172.21.15.19:0/2030703838 conn(0x55e63f7716a0 msgr2=0x55e63f893110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb941ffb700 0 client.17304 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:32:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb973051b80 1 -- 172.21.15.19:0/2030703838 shutdown_connections 2024-05-09T01:32:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb973051b80 1 -- 172.21.15.19:0/2030703838 wait complete. 2024-05-09T01:32:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb973051b80 20 client.17304 trim_cache size 0 max 16384 2024-05-09T01:32:59.486 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:32:59.487 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:32:59.487 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:32:59.487 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:32:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 -- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9440b2880 msgr2=0x7fb9440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9440b2880 0x7fb9440b4d90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb95c06c310 tx=0x7fb9500134e0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 -- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 msgr2=0x7fb95c1a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:32:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 0x7fb95c1a17f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb954005910 tx=0x7fb954009580 comp rx=0 tx=0).stop 2024-05-09T01:32:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 -- 172.21.15.19:0/4064655052 shutdown_connections 2024-05-09T01:32:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb9440b2880 0x7fb9440b4d90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb95c108740 0x7fb95c1a17f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c1034c0 0x7fb95c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 --2- 172.21.15.19:0/4064655052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb95c107df0 0x7fb95c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:32:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 -- 172.21.15.19:0/4064655052 >> 172.21.15.19:0/4064655052 conn(0x7fb95c0fedc0 msgr2=0x7fb95c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:32:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 -- 172.21.15.19:0/4064655052 shutdown_connections 2024-05-09T01:32:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:32:59.485+0000 7fb96463b700 1 -- 172.21.15.19:0/4064655052 wait complete. 2024-05-09T01:33:00.251 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:33:00.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:33:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:00.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:00 smithi019 ceph-mon[30158]: pgmap v602: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:33:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:00 smithi196 ceph-mon[35041]: pgmap v602: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:33:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:00 smithi059 ceph-mon[34351]: pgmap v602: 97 pgs: 97 active+clean; 4.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:33:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.325+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/617387099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0107df0 msgr2=0x7fe5a0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.325+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/617387099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0107df0 0x7fe5a0108200 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe5940099c0 tx=0x7fe594009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.326+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/617387099 shutdown_connections 2024-05-09T01:33:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.326+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/617387099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5a0108740 0x7fe5a01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.326+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/617387099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a01034c0 0x7fe5a01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.326+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/617387099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0107df0 0x7fe5a0108200 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.326+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/617387099 >> 172.21.15.19:0/617387099 conn(0x7fe5a00fedf0 msgr2=0x7fe5a0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.326+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/617387099 shutdown_connections 2024-05-09T01:33:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.326+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/617387099 wait complete. 2024-05-09T01:33:02.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe5a4b6a700 1 Processor -- start 2024-05-09T01:33:02.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe5a4b6a700 1 -- start start 2024-05-09T01:33:02.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe5a4b6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a01034c0 0x7fe5a0199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:02.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe5a4b6a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a0107df0 0x7fe5a019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:02.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe5a4b6a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5a0108740 0x7fe5a019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:02.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe5a4b6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0113370 con 0x7fe5a01034c0 2024-05-09T01:33:02.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe5a4b6a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01131f0 con 0x7fe5a0108740 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe5a4b6a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01134f0 con 0x7fe5a0107df0 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe59e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a01034c0 0x7fe5a0199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe59e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a01034c0 0x7fe5a0199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54058/0 (socket says 172.21.15.19:54058) 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe59e59c700 1 -- 172.21.15.19:0/2321269345 learned_addr learned my addr 172.21.15.19:0/2321269345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe59dd9b700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a0107df0 0x7fe5a019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.327+0000 7fe59ed9d700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5a0108740 0x7fe5a019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe59e59c700 1 -- 172.21.15.19:0/2321269345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a0107df0 msgr2=0x7fe5a019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe59e59c700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a0107df0 0x7fe5a019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe59e59c700 1 -- 172.21.15.19:0/2321269345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5a0108740 msgr2=0x7fe5a019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe59e59c700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5a0108740 0x7fe5a019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe59e59c700 1 -- 172.21.15.19:0/2321269345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a019ec70 con 0x7fe5a01034c0 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe59e59c700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a01034c0 0x7fe5a0199da0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe59000d980 tx=0x7fe59000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe58f7fe700 1 -- 172.21.15.19:0/2321269345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590017070 con 0x7fe5a01034c0 2024-05-09T01:33:02.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe58f7fe700 1 -- 172.21.15.19:0/2321269345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe590005020 con 0x7fe5a01034c0 2024-05-09T01:33:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a019ef60 con 0x7fe5a01034c0 2024-05-09T01:33:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe58f7fe700 1 -- 172.21.15.19:0/2321269345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59000fa10 con 0x7fe5a01034c0 2024-05-09T01:33:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.328+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a01a6880 con 0x7fe5a01034c0 2024-05-09T01:33:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.329+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a00512c0 con 0x7fe5a01034c0 2024-05-09T01:33:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.331+0000 7fe58f7fe700 1 -- 172.21.15.19:0/2321269345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe590004a30 con 0x7fe5a01034c0 2024-05-09T01:33:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.331+0000 7fe58f7fe700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5880b2900 0x7fe5880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.332+0000 7fe58f7fe700 1 -- 172.21.15.19:0/2321269345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe5900e2450 con 0x7fe5a01034c0 2024-05-09T01:33:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.332+0000 7fe59dd9b700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5880b2900 0x7fe5880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.332+0000 7fe59dd9b700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5880b2900 0x7fe5880b4e10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe594006010 tx=0x7fe594000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:02.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.335+0000 7fe58f7fe700 1 -- 172.21.15.19:0/2321269345 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe590020020 con 0x7fe5a01034c0 2024-05-09T01:33:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.602+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5a002e5f0 con 0x7fe5a01034c0 2024-05-09T01:33:02.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:02 smithi019 ceph-mon[30158]: pgmap v603: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-05-09T01:33:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.603+0000 7fe58f7fe700 1 -- 172.21.15.19:0/2321269345 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fe5900aa1d0 con 0x7fe5a01034c0 2024-05-09T01:33:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:33:02.600 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5880b2900 msgr2=0x7fe5880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5880b2900 0x7fe5880b4e10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe594006010 tx=0x7fe594000bc0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a01034c0 msgr2=0x7fe5a0199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a01034c0 0x7fe5a0199da0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe59000d980 tx=0x7fe59000dc90 comp rx=0 tx=0).stop 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 shutdown_connections 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5a0108740 0x7fe5a019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5880b2900 0x7fe5880b4e10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a01034c0 0x7fe5a0199da0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 --2- 172.21.15.19:0/2321269345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a0107df0 0x7fe5a019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 >> 172.21.15.19:0/2321269345 conn(0x7fe5a00fedf0 msgr2=0x7fe5a0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:02.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 shutdown_connections 2024-05-09T01:33:02.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:02.605+0000 7fe5a4b6a700 1 -- 172.21.15.19:0/2321269345 wait complete. 2024-05-09T01:33:02.603 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:33:02.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:02 smithi059 ceph-mon[34351]: pgmap v603: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-05-09T01:33:02.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:02 smithi196 ceph-mon[35041]: pgmap v603: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-05-09T01:33:03.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:33:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:03.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:03 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2321269345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:33:03.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:03 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2321269345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:33:03.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:03 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2321269345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:33:04.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:04 smithi019 ceph-mon[30158]: pgmap v604: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:04 smithi196 ceph-mon[35041]: pgmap v604: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:04 smithi059 ceph-mon[34351]: pgmap v604: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:05.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.394+0000 7ff07e292700 1 -- 172.21.15.19:0/584833409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 msgr2=0x7ff0781049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:05.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.394+0000 7ff07e292700 1 --2- 172.21.15.19:0/584833409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 0x7ff0781049f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff0680099c0 tx=0x7ff068009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.394+0000 7ff07e292700 1 -- 172.21.15.19:0/584833409 shutdown_connections 2024-05-09T01:33:05.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.394+0000 7ff07e292700 1 --2- 172.21.15.19:0/584833409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff078109860 0x7ff0781111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.394+0000 7ff07e292700 1 --2- 172.21.15.19:0/584833409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 0x7ff0781049f0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.394+0000 7ff07e292700 1 --2- 172.21.15.19:0/584833409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff078108f10 0x7ff078109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.394+0000 7ff07e292700 1 -- 172.21.15.19:0/584833409 >> 172.21.15.19:0/584833409 conn(0x7ff0780ffe70 msgr2=0x7ff0781022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:05.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.394+0000 7ff07e292700 1 -- 172.21.15.19:0/584833409 shutdown_connections 2024-05-09T01:33:05.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.394+0000 7ff07e292700 1 -- 172.21.15.19:0/584833409 wait complete. 2024-05-09T01:33:05.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff07e292700 1 Processor -- start 2024-05-09T01:33:05.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff07e292700 1 -- start start 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff07e292700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 0x7ff07819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff07e292700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff078108f10 0x7ff07819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff07e292700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff078109860 0x7ff07819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff07e292700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff078114490 con 0x7ff0781045e0 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff07e292700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff078114310 con 0x7ff078109860 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff07e292700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff078114610 con 0x7ff078108f10 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff077fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 0x7ff07819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff077fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 0x7ff07819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54088/0 (socket says 172.21.15.19:54088) 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.395+0000 7ff077fff700 1 -- 172.21.15.19:0/1941824969 learned_addr learned my addr 172.21.15.19:0/1941824969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff07c82f700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff078109860 0x7ff07819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff06ffff700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff078108f10 0x7ff07819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff077fff700 1 -- 172.21.15.19:0/1941824969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff078108f10 msgr2=0x7ff07819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff077fff700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff078108f10 0x7ff07819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff077fff700 1 -- 172.21.15.19:0/1941824969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff078109860 msgr2=0x7ff07819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:05.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff077fff700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff078109860 0x7ff07819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff077fff700 1 -- 172.21.15.19:0/1941824969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07819fcc0 con 0x7ff0781045e0 2024-05-09T01:33:05.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff077fff700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 0x7ff07819ae40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff068006010 tx=0x7ff0680055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:05.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff075ffb700 1 -- 172.21.15.19:0/1941824969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068026070 con 0x7ff0781045e0 2024-05-09T01:33:05.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff07e292700 1 -- 172.21.15.19:0/1941824969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07819ff50 con 0x7ff0781045e0 2024-05-09T01:33:05.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff075ffb700 1 -- 172.21.15.19:0/1941824969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff06800f460 con 0x7ff0781045e0 2024-05-09T01:33:05.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff075ffb700 1 -- 172.21.15.19:0/1941824969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068009e60 con 0x7ff0781045e0 2024-05-09T01:33:05.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.396+0000 7ff07e292700 1 -- 172.21.15.19:0/1941824969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0781a78d0 con 0x7ff0781045e0 2024-05-09T01:33:05.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.399+0000 7ff075ffb700 1 -- 172.21.15.19:0/1941824969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff068029070 con 0x7ff0781045e0 2024-05-09T01:33:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff075ffb700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0640b2880 0x7ff0640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff06ffff700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0640b2880 0x7ff0640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff075ffb700 1 -- 172.21.15.19:0/1941824969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff0680e2b40 con 0x7ff0781045e0 2024-05-09T01:33:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff08cca8b80 1 Processor -- start 2024-05-09T01:33:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff08cca8b80 1 -- start start 2024-05-09T01:33:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff06ffff700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0640b2880 0x7ff0640b4d90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff060005fd0 tx=0x7ff060005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c052c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c0526810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e9c04490a0 con 0x55e9c0423d00 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e9c03ee9a0 con 0x55e9c03a94a0 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.400+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9c0445060 con 0x55e9c0526400 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff077fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff077fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54094/0 (socket says 172.21.15.19:54094) 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff077fff700 1 -- 172.21.15.19:0/403857442 learned_addr learned my addr 172.21.15.19:0/403857442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff06ffff700 1 --2- 172.21.15.19:0/403857442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c052c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff07c82f700 1 --2- 172.21.15.19:0/403857442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c0526810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff077fff700 1 -- 172.21.15.19:0/403857442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 msgr2=0x55e9c0526810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff077fff700 1 --2- 172.21.15.19:0/403857442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c0526810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff077fff700 1 -- 172.21.15.19:0/403857442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 msgr2=0x55e9c052c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff077fff700 1 --2- 172.21.15.19:0/403857442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c052c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff077fff700 1 -- 172.21.15.19:0/403857442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c03df500 con 0x55e9c0423d00 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff077fff700 1 --2- 172.21.15.19:0/403857442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052c240 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff0680aa100 tx=0x7ff0680a9bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1b32ee14ef939c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff06e7fc700 1 -- 172.21.15.19:0/403857442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0680aa870 con 0x55e9c0423d00 2024-05-09T01:33:05.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff06e7fc700 1 -- 172.21.15.19:0/403857442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0680aa170 con 0x55e9c0423d00 2024-05-09T01:33:05.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.401+0000 7ff06e7fc700 1 -- 172.21.15.19:0/403857442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0680aa3d0 con 0x55e9c0423d00 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.402+0000 7ff08cca8b80 1 -- 172.21.15.19:0/403857442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 msgr2=0x55e9c052c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.402+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/403857442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052c240 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff0680aa100 tx=0x7ff0680a9bb0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.402+0000 7ff08cca8b80 1 -- 172.21.15.19:0/403857442 shutdown_connections 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.402+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/403857442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c0526810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.402+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/403857442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052c240 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.402+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/403857442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c052c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.402+0000 7ff08cca8b80 1 -- 172.21.15.19:0/403857442 >> 172.21.15.19:0/403857442 conn(0x55e9c03acf70 msgr2=0x55e9c0521220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.402+0000 7ff08cca8b80 1 -- 172.21.15.19:0/403857442 shutdown_connections 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.402+0000 7ff08cca8b80 1 -- 172.21.15.19:0/403857442 wait complete. 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.403+0000 7ff08cca8b80 1 -- start start 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.403+0000 7ff08cca8b80 1 Processor -- start 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.403+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c0535b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.403+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0423d00 0x55e9c0536040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.403+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0526400 0x55e9c052ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e9c0526fb0 con 0x55e9c0526400 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e9c03e3b20 con 0x55e9c03a94a0 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9c0527130 con 0x55e9c0423d00 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff08cca8b80 10 client.0 _ll_register_callbacks cb 0x55e9c02ba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff07c82f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0526400 0x55e9c052ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff07c82f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0526400 0x55e9c052ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54108/0 (socket says 172.21.15.19:54108) 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff07c82f700 1 -- 172.21.15.19:0/2328696155 learned_addr learned my addr 172.21.15.19:0/2328696155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff06ffff700 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0423d00 0x55e9c0536040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff077fff700 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c0535b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff07c82f700 1 -- 172.21.15.19:0/2328696155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0423d00 msgr2=0x55e9c0536040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff07c82f700 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0423d00 0x55e9c0536040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff07c82f700 1 -- 172.21.15.19:0/2328696155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 msgr2=0x55e9c0535b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff07c82f700 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c0535b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff07c82f700 1 -- 172.21.15.19:0/2328696155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c02c4490 con 0x55e9c0526400 2024-05-09T01:33:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff07c82f700 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0526400 0x55e9c052ff00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff070011f60 tx=0x7ff070014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1098f0518f2f7ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.404+0000 7ff0577fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff0577fe700 1 -- 172.21.15.19:0/2328696155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070014ba0 con 0x55e9c0526400 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff0577fe700 1 -- 172.21.15.19:0/2328696155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff070014d40 con 0x55e9c0526400 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff08cca8b80 1 -- 172.21.15.19:0/2328696155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0680aa170 con 0x55e9c0526400 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff0577fe700 1 -- 172.21.15.19:0/2328696155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff070003a00 con 0x55e9c0526400 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff08cca8b80 10 client.17322 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff08cca8b80 1 -- 172.21.15.19:0/2328696155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9c053d780 con 0x55e9c0526400 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff0577fe700 1 -- 172.21.15.19:0/2328696155 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0680aa170 con 0x55e9c0526400 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff0577fe700 1 -- 172.21.15.19:0/2328696155 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff070007500 con 0x55e9c0526400 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff08cca8b80 10 client.17322 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff08cca8b80 10 client.17322 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff08cca8b80 20 client.17322 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55e9c053fdc0 0x55e9c05421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff08cca8b80 4 client.17322 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.405+0000 7ff08cca8b80 1 -- 172.21.15.19:0/2328696155 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9c0456a80 con 0x55e9c053fdc0 2024-05-09T01:33:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.406+0000 7ff077fff700 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55e9c053fdc0 0x55e9c05421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.406+0000 7ff0577fe700 10 client.17322 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.406+0000 7ff077fff700 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55e9c053fdc0 0x55e9c05421e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7b9621918c96fcd server_cookie=d9cefca3e11fe01 in_seq=0 out_seq=0 2024-05-09T01:33:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.407+0000 7ff0577fe700 1 -- 172.21.15.19:0/2328696155 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9c0456a80 con 0x55e9c053fdc0 2024-05-09T01:33:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.407+0000 7ff0577fe700 10 client.17322 handle_command_reply: tid=0 2024-05-09T01:33:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.407+0000 7ff08cca8b80 1 client.17322 shutdown 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.407+0000 7ff08cca8b80 1 -- 172.21.15.19:0/2328696155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0526400 msgr2=0x55e9c052ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.407+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0526400 0x55e9c052ff00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff070011f60 tx=0x7ff070014600 comp rx=0 tx=0).stop 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.407+0000 7ff08cca8b80 1 -- 172.21.15.19:0/2328696155 shutdown_connections 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.407+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55e9c053fdc0 0x55e9c05421e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.407+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0526400 0x55e9c052ff00 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.408+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c0535b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.408+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/2328696155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0423d00 0x55e9c0536040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.408+0000 7ff08cca8b80 1 -- 172.21.15.19:0/2328696155 >> 172.21.15.19:0/2328696155 conn(0x55e9c03acf70 msgr2=0x55e9c051aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.408+0000 7ff0577fe700 0 client.17322 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.408+0000 7ff08cca8b80 1 -- 172.21.15.19:0/2328696155 shutdown_connections 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.408+0000 7ff08cca8b80 1 -- 172.21.15.19:0/2328696155 wait complete. 2024-05-09T01:33:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.408+0000 7ff08cca8b80 20 client.17322 trim_cache size 0 max 16384 2024-05-09T01:33:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.423+0000 7ff08cca8b80 1 Processor -- start 2024-05-09T01:33:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.423+0000 7ff08cca8b80 1 -- start start 2024-05-09T01:33:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.423+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c0521250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.423+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.423+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c052d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.423+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781aec30 con 0x55e9c0423d00 2024-05-09T01:33:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.423+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781b0770 con 0x55e9c03a94a0 2024-05-09T01:33:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.423+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781af300 con 0x55e9c0526400 2024-05-09T01:33:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54110/0 (socket says 172.21.15.19:54110) 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06ffff700 1 -- 172.21.15.19:0/4256096545 learned_addr learned my addr 172.21.15.19:0/4256096545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff07c82f700 1 --2- 172.21.15.19:0/4256096545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c052d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff077fff700 1 --2- 172.21.15.19:0/4256096545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c0521250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06ffff700 1 -- 172.21.15.19:0/4256096545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 msgr2=0x55e9c052d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06ffff700 1 --2- 172.21.15.19:0/4256096545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c052d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06ffff700 1 -- 172.21.15.19:0/4256096545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 msgr2=0x55e9c0521250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06ffff700 1 --2- 172.21.15.19:0/4256096545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c0521250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06ffff700 1 -- 172.21.15.19:0/4256096545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c05363b0 con 0x55e9c0423d00 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06ffff700 1 --2- 172.21.15.19:0/4256096545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052ef00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff078101420 tx=0x7ff060012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833a8e4465bc3dce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06dffb700 1 -- 172.21.15.19:0/4256096545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060011bb0 con 0x55e9c0423d00 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06dffb700 1 -- 172.21.15.19:0/4256096545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff060011d50 con 0x55e9c0423d00 2024-05-09T01:33:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.424+0000 7ff06dffb700 1 -- 172.21.15.19:0/4256096545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06001e710 con 0x55e9c0423d00 2024-05-09T01:33:05.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.425+0000 7ff08cca8b80 1 -- 172.21.15.19:0/4256096545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 msgr2=0x55e9c052ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:05.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.425+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/4256096545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052ef00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff078101420 tx=0x7ff060012000 comp rx=0 tx=0).stop 2024-05-09T01:33:05.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.425+0000 7ff08cca8b80 1 -- 172.21.15.19:0/4256096545 shutdown_connections 2024-05-09T01:33:05.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.425+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/4256096545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c052d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.425+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/4256096545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c03a94a0 0x55e9c0521250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.425+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/4256096545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c0423d00 0x55e9c052ef00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.425+0000 7ff08cca8b80 1 -- 172.21.15.19:0/4256096545 >> 172.21.15.19:0/4256096545 conn(0x55e9c03acf70 msgr2=0x55e9c052e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.425+0000 7ff08cca8b80 1 -- 172.21.15.19:0/4256096545 shutdown_connections 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.425+0000 7ff08cca8b80 1 -- 172.21.15.19:0/4256096545 wait complete. 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff08cca8b80 1 -- start start 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff08cca8b80 1 Processor -- start 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c03a94a0 0x55e9c05a2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c0423d00 0x55e9c05a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff08cca8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c0539430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781b00a0 con 0x55e9c03a94a0 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781af9d0 con 0x55e9c0423d00 2024-05-09T01:33:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff08cca8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781acbd0 con 0x55e9c0526400 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff08cca8b80 10 client.0 _ll_register_callbacks cb 0x55e9c02ba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff077fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c03a94a0 0x55e9c05a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff077fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c03a94a0 0x55e9c05a2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54112/0 (socket says 172.21.15.19:54112) 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.426+0000 7ff077fff700 1 -- 172.21.15.19:0/29709586 learned_addr learned my addr 172.21.15.19:0/29709586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff06ffff700 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c0423d00 0x55e9c05a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff07c82f700 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c0539430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff077fff700 1 -- 172.21.15.19:0/29709586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 msgr2=0x55e9c0539430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff077fff700 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c0539430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff077fff700 1 -- 172.21.15.19:0/29709586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c0423d00 msgr2=0x55e9c05a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff077fff700 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c0423d00 0x55e9c05a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff077fff700 1 -- 172.21.15.19:0/29709586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c0539aa0 con 0x55e9c03a94a0 2024-05-09T01:33:05.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff077fff700 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c03a94a0 0x55e9c05a2930 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff0680098e0 tx=0x7ff068009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea852443ec198df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff0577fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff0577fe700 1 -- 172.21.15.19:0/29709586 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0680aa870 con 0x55e9c03a94a0 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff0577fe700 1 -- 172.21.15.19:0/29709586 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff0680ae160 con 0x55e9c03a94a0 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff08cca8b80 1 -- 172.21.15.19:0/29709586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff060011d50 con 0x55e9c03a94a0 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff0577fe700 1 -- 172.21.15.19:0/29709586 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff06800f950 con 0x55e9c03a94a0 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff08cca8b80 10 client.17328 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.427+0000 7ff08cca8b80 1 -- 172.21.15.19:0/29709586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9c05aa000 con 0x55e9c03a94a0 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.428+0000 7ff0577fe700 1 -- 172.21.15.19:0/29709586 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff060011d50 con 0x55e9c03a94a0 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.428+0000 7ff0577fe700 1 -- 172.21.15.19:0/29709586 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff0680aa460 con 0x55e9c03a94a0 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.428+0000 7ff08cca8b80 10 client.17328 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.428+0000 7ff08cca8b80 10 client.17328 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.428+0000 7ff08cca8b80 20 client.17328 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:05.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.428+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55e9c05aa4b0 0x55e9c05ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.428+0000 7ff08cca8b80 4 client.17328 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.428+0000 7ff08cca8b80 1 -- 172.21.15.19:0/29709586 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9c0450b20 con 0x55e9c05aa4b0 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.428+0000 7ff06ffff700 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55e9c05aa4b0 0x55e9c05ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff0577fe700 10 client.17328 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff06ffff700 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55e9c05aa4b0 0x55e9c05ac8d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e72557036c55b0bf server_cookie=101eaf4fdc94ef50 in_seq=0 out_seq=0 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff0577fe700 1 -- 172.21.15.19:0/29709586 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9c0450b20 con 0x55e9c05aa4b0 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff0577fe700 10 client.17328 handle_command_reply: tid=0 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 client.17328 shutdown 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 -- 172.21.15.19:0/29709586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c03a94a0 msgr2=0x55e9c05a2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c03a94a0 0x55e9c05a2930 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff0680098e0 tx=0x7ff068009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 -- 172.21.15.19:0/29709586 shutdown_connections 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55e9c05aa4b0 0x55e9c05ac8d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9c0526400 0x55e9c0539430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9c03a94a0 0x55e9c05a2930 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff0577fe700 0 client.17328 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 --2- 172.21.15.19:0/29709586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9c0423d00 0x55e9c05a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 -- 172.21.15.19:0/29709586 >> 172.21.15.19:0/29709586 conn(0x55e9c03acf70 msgr2=0x55e9c052b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 -- 172.21.15.19:0/29709586 shutdown_connections 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 1 -- 172.21.15.19:0/29709586 wait complete. 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.429+0000 7ff08cca8b80 20 client.17328 trim_cache size 0 max 16384 2024-05-09T01:33:05.433 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 -- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0640b2880 msgr2=0x7ff0640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0640b2880 0x7ff0640b4d90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff060005fd0 tx=0x7ff060005f60 comp rx=0 tx=0).stop 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 -- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 msgr2=0x7ff07819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 0x7ff07819ae40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff068006010 tx=0x7ff0680055f0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 -- 172.21.15.19:0/1941824969 shutdown_connections 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff078109860 0x7ff07819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0640b2880 0x7ff0640b4d90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0781045e0 0x7ff07819ae40 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 --2- 172.21.15.19:0/1941824969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff078108f10 0x7ff07819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 -- 172.21.15.19:0/1941824969 >> 172.21.15.19:0/1941824969 conn(0x7ff0780ffe70 msgr2=0x7ff078100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 -- 172.21.15.19:0/1941824969 shutdown_connections 2024-05-09T01:33:05.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:05.430+0000 7ff07e292700 1 -- 172.21.15.19:0/1941824969 wait complete. 2024-05-09T01:33:06.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:33:06.400 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:06.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:06 smithi019 ceph-mon[30158]: pgmap v605: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:33:06.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:06 smithi196 ceph-mon[35041]: pgmap v605: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:33:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:06 smithi059 ceph-mon[34351]: pgmap v605: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:33:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.177+0000 7f5f68f64700 1 -- 172.21.15.19:0/3940058703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64108f20 msgr2=0x7f5f64109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.177+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3940058703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64108f20 0x7f5f64109330 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5f580099c0 tx=0x7f5f58009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.177+0000 7f5f68f64700 1 -- 172.21.15.19:0/3940058703 shutdown_connections 2024-05-09T01:33:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.177+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3940058703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f64109870 0x7f5f641111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.177+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3940058703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f641045f0 0x7f5f64104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.177+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3940058703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64108f20 0x7f5f64109330 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.177+0000 7f5f68f64700 1 -- 172.21.15.19:0/3940058703 >> 172.21.15.19:0/3940058703 conn(0x7f5f64100040 msgr2=0x7f5f64102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.178+0000 7f5f68f64700 1 -- 172.21.15.19:0/3940058703 shutdown_connections 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.178+0000 7f5f68f64700 1 -- 172.21.15.19:0/3940058703 wait complete. 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f68f64700 1 Processor -- start 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f68f64700 1 -- start start 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f68f64700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f641045f0 0x7f5f64075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f68f64700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f64108f20 0x7f5f64076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f68f64700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64109870 0x7f5f64076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f68f64700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641144d0 con 0x7f5f64109870 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f68f64700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f64114350 con 0x7f5f64108f20 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f68f64700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f64114650 con 0x7f5f641045f0 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f62d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64109870 0x7f5f64076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f62d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64109870 0x7f5f64076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54122/0 (socket says 172.21.15.19:54122) 2024-05-09T01:33:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f62d9d700 1 -- 172.21.15.19:0/3608622649 learned_addr learned my addr 172.21.15.19:0/3608622649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f64108f20 0x7f5f64076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f6259c700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f641045f0 0x7f5f64075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f62d9d700 1 -- 172.21.15.19:0/3608622649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f641045f0 msgr2=0x7f5f64075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f62d9d700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f641045f0 0x7f5f64075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f62d9d700 1 -- 172.21.15.19:0/3608622649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f64108f20 msgr2=0x7f5f64076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f62d9d700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f64108f20 0x7f5f64076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.179+0000 7f5f62d9d700 1 -- 172.21.15.19:0/3608622649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f641a92a0 con 0x7f5f64109870 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.180+0000 7f5f62d9d700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64109870 0x7f5f64076540 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c007ee0 tx=0x7f5f5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.180+0000 7f5f537fe700 1 -- 172.21.15.19:0/3608622649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00dd20 con 0x7f5f64109870 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.180+0000 7f5f537fe700 1 -- 172.21.15.19:0/3608622649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5c004020 con 0x7f5f64109870 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.180+0000 7f5f537fe700 1 -- 172.21.15.19:0/3608622649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c015660 con 0x7f5f64109870 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.180+0000 7f5f68f64700 1 -- 172.21.15.19:0/3608622649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f641a9590 con 0x7f5f64109870 2024-05-09T01:33:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.180+0000 7f5f68f64700 1 -- 172.21.15.19:0/3608622649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f641a9a50 con 0x7f5f64109870 2024-05-09T01:33:08.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.183+0000 7f5f537fe700 1 -- 172.21.15.19:0/3608622649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5f5c0041d0 con 0x7f5f64109870 2024-05-09T01:33:08.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f537fe700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f4c0b2880 0x7f5f4c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f537fe700 1 -- 172.21.15.19:0/3608622649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5f5c0e1b80 con 0x7f5f64109870 2024-05-09T01:33:08.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f6259c700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f4c0b2880 0x7f5f4c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f7797ab80 1 Processor -- start 2024-05-09T01:33:08.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f7797ab80 1 -- start start 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f6259c700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f4c0b2880 0x7f5f4c0b4d90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5f54005950 tx=0x7f5f5400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594ced00 0x55e6595d7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594544a0 0x55e6595d7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 0x55e6595d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e6594f40a0 con 0x55e6594544a0 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e6594999a0 con 0x55e6595d1400 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.184+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e6594f0060 con 0x55e6594ced00 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f61d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594544a0 0x55e6595d7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f61d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594544a0 0x55e6595d7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54130/0 (socket says 172.21.15.19:54130) 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f61d9b700 1 -- 172.21.15.19:0/872346977 learned_addr learned my addr 172.21.15.19:0/872346977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f62d9d700 1 --2- 172.21.15.19:0/872346977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 0x55e6595d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f6259c700 1 --2- 172.21.15.19:0/872346977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594ced00 0x55e6595d7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f61d9b700 1 -- 172.21.15.19:0/872346977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594ced00 msgr2=0x55e6595d7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/872346977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594ced00 0x55e6595d7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f61d9b700 1 -- 172.21.15.19:0/872346977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 msgr2=0x55e6595d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/872346977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 0x55e6595d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f61d9b700 1 -- 172.21.15.19:0/872346977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65948a500 con 0x55e6594544a0 2024-05-09T01:33:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/872346977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594544a0 0x55e6595d7780 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b580 tx=0x7f5f580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b341c2197757ecef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f50ff9700 1 -- 172.21.15.19:0/872346977 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58003a60 con 0x55e6594544a0 2024-05-09T01:33:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f50ff9700 1 -- 172.21.15.19:0/872346977 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f58005520 con 0x55e6594544a0 2024-05-09T01:33:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.185+0000 7f5f50ff9700 1 -- 172.21.15.19:0/872346977 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800bca0 con 0x55e6594544a0 2024-05-09T01:33:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.186+0000 7f5f7797ab80 1 -- 172.21.15.19:0/872346977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594544a0 msgr2=0x55e6595d7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.186+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/872346977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594544a0 0x55e6595d7780 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b580 tx=0x7f5f580095c0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.186+0000 7f5f7797ab80 1 -- 172.21.15.19:0/872346977 shutdown_connections 2024-05-09T01:33:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.186+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/872346977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 0x55e6595d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.186+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/872346977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594ced00 0x55e6595d7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.186+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/872346977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594544a0 0x55e6595d7780 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.186+0000 7f5f7797ab80 1 -- 172.21.15.19:0/872346977 >> 172.21.15.19:0/872346977 conn(0x55e659457f70 msgr2=0x55e6595cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 -- 172.21.15.19:0/872346977 shutdown_connections 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 -- 172.21.15.19:0/872346977 wait complete. 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 -- start start 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 Processor -- start 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 0x55e6595daf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 0x55e6595db9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e6595d20a0 con 0x55e6594ced00 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e65948eb20 con 0x55e6594544a0 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e6595d2220 con 0x55e6595d1400 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f7797ab80 10 client.0 _ll_register_callbacks cb 0x55e6593656c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.187+0000 7f5f61d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f61d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595db4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54144/0 (socket says 172.21.15.19:54144) 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f61d9b700 1 -- 172.21.15.19:0/172244370 learned_addr learned my addr 172.21.15.19:0/172244370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f6259c700 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 0x55e6595daf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f62d9d700 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 0x55e6595db9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f61d9b700 1 -- 172.21.15.19:0/172244370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 msgr2=0x55e6595db9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 0x55e6595db9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f61d9b700 1 -- 172.21.15.19:0/172244370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 msgr2=0x55e6595daf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 0x55e6595daf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f61d9b700 1 -- 172.21.15.19:0/172244370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65936f490 con 0x55e6594ced00 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595db4a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009350 tx=0x7f5f58005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89dc1f697ddff50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f49ffb700 1 -- 172.21.15.19:0/172244370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58005980 con 0x55e6594ced00 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f49ffb700 1 -- 172.21.15.19:0/172244370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5f580050d0 con 0x55e6594ced00 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f7797ab80 1 -- 172.21.15.19:0/172244370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f58005520 con 0x55e6594ced00 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f49ffb700 1 -- 172.21.15.19:0/172244370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f580052d0 con 0x55e6594ced00 2024-05-09T01:33:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f7797ab80 10 client.17340 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.188+0000 7f5f7797ab80 1 -- 172.21.15.19:0/172244370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e6595e88b0 con 0x55e6594ced00 2024-05-09T01:33:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.189+0000 7f5f49ffb700 1 -- 172.21.15.19:0/172244370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f58005520 con 0x55e6594ced00 2024-05-09T01:33:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.189+0000 7f5f49ffb700 1 -- 172.21.15.19:0/172244370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5f5800f910 con 0x55e6594ced00 2024-05-09T01:33:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.189+0000 7f5f7797ab80 10 client.17340 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.189+0000 7f5f7797ab80 10 client.17340 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:33:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.189+0000 7f5f7797ab80 20 client.17340 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.189+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e6595ead80 0x55e6595ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.189+0000 7f5f7797ab80 4 client.17340 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.189+0000 7f5f7797ab80 1 -- 172.21.15.19:0/172244370 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e659501a80 con 0x55e6595ead80 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.189+0000 7f5f6259c700 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e6595ead80 0x55e6595ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.190+0000 7f5f49ffb700 10 client.17340 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.190+0000 7f5f6259c700 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e6595ead80 0x55e6595ed1a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90eab845802734c8 server_cookie=22c6531509d86a0a in_seq=0 out_seq=0 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f49ffb700 1 -- 172.21.15.19:0/172244370 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e659501a80 con 0x55e6595ead80 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f49ffb700 10 client.17340 handle_command_reply: tid=0 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 client.17340 shutdown 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 -- 172.21.15.19:0/172244370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 msgr2=0x55e6595db4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595db4a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009350 tx=0x7f5f58005810 comp rx=0 tx=0).stop 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 -- 172.21.15.19:0/172244370 shutdown_connections 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e6595ead80 0x55e6595ed1a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 0x55e6595db9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 0x55e6595daf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/172244370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595db4a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 -- 172.21.15.19:0/172244370 >> 172.21.15.19:0/172244370 conn(0x55e659457f70 msgr2=0x55e6595c5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f49ffb700 0 client.17340 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 -- 172.21.15.19:0/172244370 shutdown_connections 2024-05-09T01:33:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 1 -- 172.21.15.19:0/172244370 wait complete. 2024-05-09T01:33:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.191+0000 7f5f7797ab80 20 client.17340 trim_cache size 0 max 16384 2024-05-09T01:33:08.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.206+0000 7f5f7797ab80 1 Processor -- start 2024-05-09T01:33:08.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.206+0000 7f5f7797ab80 1 -- start start 2024-05-09T01:33:08.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 0x55e6595e7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595da730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 0x55e6595d53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641b0e30 con 0x55e6594ced00 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641b2970 con 0x55e6594544a0 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641b1500 con 0x55e6595d1400 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f61d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595da730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f61d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595da730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54158/0 (socket says 172.21.15.19:54158) 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f61d9b700 1 -- 172.21.15.19:0/2387263094 learned_addr learned my addr 172.21.15.19:0/2387263094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f6259c700 1 --2- 172.21.15.19:0/2387263094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 0x55e6595e7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f62d9d700 1 --2- 172.21.15.19:0/2387263094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 0x55e6595d53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f61d9b700 1 -- 172.21.15.19:0/2387263094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 msgr2=0x55e6595d53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/2387263094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 0x55e6595d53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f61d9b700 1 -- 172.21.15.19:0/2387263094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 msgr2=0x55e6595e7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/2387263094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 0x55e6595e7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f61d9b700 1 -- 172.21.15.19:0/2387263094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6595dd800 con 0x55e6594ced00 2024-05-09T01:33:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.207+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/2387263094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595da730 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f5f58006010 tx=0x7f5f58009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cba42123da806f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f4bfff700 1 -- 172.21.15.19:0/2387263094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58043070 con 0x55e6594ced00 2024-05-09T01:33:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f4bfff700 1 -- 172.21.15.19:0/2387263094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f58030070 con 0x55e6594ced00 2024-05-09T01:33:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f4bfff700 1 -- 172.21.15.19:0/2387263094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58022a60 con 0x55e6594ced00 2024-05-09T01:33:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f7797ab80 1 -- 172.21.15.19:0/2387263094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 msgr2=0x55e6595da730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/2387263094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595da730 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f5f58006010 tx=0x7f5f58009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f7797ab80 1 -- 172.21.15.19:0/2387263094 shutdown_connections 2024-05-09T01:33:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/2387263094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6595d1400 0x55e6595d53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/2387263094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6594544a0 0x55e6595e7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/2387263094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595da730 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f7797ab80 1 -- 172.21.15.19:0/2387263094 >> 172.21.15.19:0/2387263094 conn(0x55e659457f70 msgr2=0x55e6595d4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f7797ab80 1 -- 172.21.15.19:0/2387263094 shutdown_connections 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.208+0000 7f5f7797ab80 1 -- 172.21.15.19:0/2387263094 wait complete. 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f7797ab80 1 -- start start 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f7797ab80 1 Processor -- start 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594544a0 0x55e6595f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595ecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f7797ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 0x55e6595ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641b22a0 con 0x55e6594ced00 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641b1bd0 con 0x55e6595d1400 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f7797ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641aedd0 con 0x55e6594544a0 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f7797ab80 10 client.0 _ll_register_callbacks cb 0x55e6593656c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f61d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595ecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f61d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595ecd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54164/0 (socket says 172.21.15.19:54164) 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.209+0000 7f5f61d9b700 1 -- 172.21.15.19:0/84537697 learned_addr learned my addr 172.21.15.19:0/84537697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f6259c700 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594544a0 0x55e6595f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f62d9d700 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 0x55e6595ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f61d9b700 1 -- 172.21.15.19:0/84537697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594544a0 msgr2=0x55e6595f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594544a0 0x55e6595f0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f61d9b700 1 -- 172.21.15.19:0/84537697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 msgr2=0x55e6595ed330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 0x55e6595ed330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f61d9b700 1 -- 172.21.15.19:0/84537697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6595edaa0 con 0x55e6594ced00 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f62d9d700 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 0x55e6595ed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f61d9b700 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595ecd90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800baf0 tx=0x7f5f58005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f012a603705b8fe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f49ffb700 1 -- 172.21.15.19:0/84537697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58043070 con 0x55e6594ced00 2024-05-09T01:33:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f49ffb700 1 -- 172.21.15.19:0/84537697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5f58003a60 con 0x55e6594ced00 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f7797ab80 1 -- 172.21.15.19:0/84537697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f58030070 con 0x55e6594ced00 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f49ffb700 1 -- 172.21.15.19:0/84537697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f5804d070 con 0x55e6594ced00 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f7797ab80 10 client.17346 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f7797ab80 1 -- 172.21.15.19:0/84537697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e659654240 con 0x55e6594ced00 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.210+0000 7f5f49ffb700 1 -- 172.21.15.19:0/84537697 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f58030070 con 0x55e6594ced00 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f49ffb700 1 -- 172.21.15.19:0/84537697 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5f5800f510 con 0x55e6594ced00 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f7797ab80 10 client.17346 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f7797ab80 10 client.17346 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f7797ab80 20 client.17346 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e6596547d0 0x55e659656bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f7797ab80 4 client.17346 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f7797ab80 1 -- 172.21.15.19:0/84537697 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6594fbb20 con 0x55e6596547d0 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f6259c700 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e6596547d0 0x55e659656bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f6259c700 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e6596547d0 0x55e659656bf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=14763f5c6c9afce1 server_cookie=38efda753220042b in_seq=0 out_seq=0 2024-05-09T01:33:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.211+0000 7f5f49ffb700 10 client.17346 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f49ffb700 1 -- 172.21.15.19:0/84537697 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6594fbb20 con 0x55e6596547d0 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f49ffb700 10 client.17346 handle_command_reply: tid=0 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 client.17346 shutdown 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 -- 172.21.15.19:0/84537697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 msgr2=0x55e6595ecd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595ecd90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800baf0 tx=0x7f5f58005180 comp rx=0 tx=0).stop 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 -- 172.21.15.19:0/84537697 shutdown_connections 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e6596547d0 0x55e659656bf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6595d1400 0x55e6595ed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6594544a0 0x55e6595f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f49ffb700 0 client.17346 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 --2- 172.21.15.19:0/84537697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6594ced00 0x55e6595ecd90 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 -- 172.21.15.19:0/84537697 >> 172.21.15.19:0/84537697 conn(0x55e659457f70 msgr2=0x55e6595c5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 -- 172.21.15.19:0/84537697 shutdown_connections 2024-05-09T01:33:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 1 -- 172.21.15.19:0/84537697 wait complete. 2024-05-09T01:33:08.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.212+0000 7f5f7797ab80 20 client.17346 trim_cache size 0 max 16384 2024-05-09T01:33:08.215 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 -- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f4c0b2880 msgr2=0x7f5f4c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f4c0b2880 0x7f5f4c0b4d90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5f54005950 tx=0x7f5f5400b410 comp rx=0 tx=0).stop 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 -- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64109870 msgr2=0x7f5f64076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64109870 0x7f5f64076540 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c007ee0 tx=0x7f5f5c00d3b0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 -- 172.21.15.19:0/3608622649 shutdown_connections 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5f4c0b2880 0x7f5f4c0b4d90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f64109870 0x7f5f64076540 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f641045f0 0x7f5f64075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 --2- 172.21.15.19:0/3608622649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f64108f20 0x7f5f64076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 -- 172.21.15.19:0/3608622649 >> 172.21.15.19:0/3608622649 conn(0x7f5f64100040 msgr2=0x7f5f6410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 -- 172.21.15.19:0/3608622649 shutdown_connections 2024-05-09T01:33:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:08.213+0000 7f5f68f64700 1 -- 172.21.15.19:0/3608622649 wait complete. 2024-05-09T01:33:08.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:08 smithi019 ceph-mon[30158]: pgmap v606: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:33:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:08 smithi196 ceph-mon[35041]: pgmap v606: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:33:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:08 smithi059 ceph-mon[34351]: pgmap v606: 97 pgs: 97 active+clean; 4.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:33:09.005 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:33:09.257 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.800+0000 7f439abb4640 1 -- 172.21.15.19:0/575397496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394109bb0 msgr2=0x7f439410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.800+0000 7f439abb4640 1 --2- 172.21.15.19:0/575397496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394109bb0 0x7f439410bfd0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f438c009ca0 tx=0x7f438c02a8c0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.800+0000 7f439abb4640 1 -- 172.21.15.19:0/575397496 shutdown_connections 2024-05-09T01:33:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.800+0000 7f439abb4640 1 --2- 172.21.15.19:0/575397496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394109bb0 0x7f439410bfd0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.800+0000 7f439abb4640 1 --2- 172.21.15.19:0/575397496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4394104eb0 0x7f43941052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.800+0000 7f439abb4640 1 --2- 172.21.15.19:0/575397496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4394103c70 0x7f4394104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.800+0000 7f439abb4640 1 -- 172.21.15.19:0/575397496 >> 172.21.15.19:0/575397496 conn(0x7f43940fee40 msgr2=0x7f4394101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.800+0000 7f439abb4640 1 -- 172.21.15.19:0/575397496 shutdown_connections 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.800+0000 7f439abb4640 1 -- 172.21.15.19:0/575397496 wait complete. 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f439abb4640 1 Processor -- start 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f439abb4640 1 -- start start 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f439abb4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4394103c70 0x7f439419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f439abb4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394104eb0 0x7f439419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f439abb4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4394109bb0 0x7f43941a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f439abb4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f439410f7a0 con 0x7f4394104eb0 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f439abb4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f439410f620 con 0x7f4394103c70 2024-05-09T01:33:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f439abb4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f439410f920 con 0x7f4394109bb0 2024-05-09T01:33:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f438bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394104eb0 0x7f439419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f438bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394104eb0 0x7f439419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42462/0 (socket says 172.21.15.19:42462) 2024-05-09T01:33:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f438bfff640 1 -- 172.21.15.19:0/3863843042 learned_addr learned my addr 172.21.15.19:0/3863843042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f4398950640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4394103c70 0x7f439419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f4399151640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4394109bb0 0x7f43941a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f438bfff640 1 -- 172.21.15.19:0/3863843042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4394109bb0 msgr2=0x7f43941a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f438bfff640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4394109bb0 0x7f43941a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f438bfff640 1 -- 172.21.15.19:0/3863843042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4394103c70 msgr2=0x7f439419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f438bfff640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4394103c70 0x7f439419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f438bfff640 1 -- 172.21.15.19:0/3863843042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43941a1e50 con 0x7f4394104eb0 2024-05-09T01:33:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.801+0000 7f4398950640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4394103c70 0x7f439419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.802+0000 7f438bfff640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394104eb0 0x7f439419a850 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f438400e9f0 tx=0x7f438400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.802+0000 7f4389ffb640 1 -- 172.21.15.19:0/3863843042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438400cd80 con 0x7f4394104eb0 2024-05-09T01:33:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.802+0000 7f4389ffb640 1 -- 172.21.15.19:0/3863843042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43840108e0 con 0x7f4394104eb0 2024-05-09T01:33:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.802+0000 7f439abb4640 1 -- 172.21.15.19:0/3863843042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43941a2140 con 0x7f4394104eb0 2024-05-09T01:33:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.802+0000 7f4389ffb640 1 -- 172.21.15.19:0/3863843042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384018980 con 0x7f4394104eb0 2024-05-09T01:33:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.802+0000 7f439abb4640 1 -- 172.21.15.19:0/3863843042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43941a2690 con 0x7f4394104eb0 2024-05-09T01:33:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f43a1a61740 1 Processor -- start 2024-05-09T01:33:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f43a1a61740 1 -- start start 2024-05-09T01:33:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f4389ffb640 1 -- 172.21.15.19:0/3863843042 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4384010430 con 0x7f4394104eb0 2024-05-09T01:33:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345671b0 0x55743490b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55743453e2f0 0x55743490ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x557434912980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f43a1a61740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55743486f7c0 con 0x5574345464d0 2024-05-09T01:33:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f43a1a61740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55743480b0e0 con 0x55743453e2f0 2024-05-09T01:33:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f43a1a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55743486b780 con 0x5574345671b0 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f438bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x557434912980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f4399151640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345671b0 0x55743490b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f438bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x557434912980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42466/0 (socket says 172.21.15.19:42466) 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f4399151640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345671b0 0x55743490b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50588/0 (socket says 172.21.15.19:50588) 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f438bfff640 1 -- 172.21.15.19:0/3864753267 learned_addr learned my addr 172.21.15.19:0/3864753267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.805+0000 7f4398950640 1 --2- 172.21.15.19:0/3864753267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55743453e2f0 0x55743490ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f438bfff640 1 -- 172.21.15.19:0/3864753267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345671b0 msgr2=0x55743490b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f438bfff640 1 --2- 172.21.15.19:0/3864753267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345671b0 0x55743490b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f438bfff640 1 -- 172.21.15.19:0/3864753267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55743453e2f0 msgr2=0x55743490ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f438bfff640 1 --2- 172.21.15.19:0/3864753267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55743453e2f0 0x55743490ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f438bfff640 1 -- 172.21.15.19:0/3864753267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574348347c0 con 0x5574345464d0 2024-05-09T01:33:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f4399151640 1 --2- 172.21.15.19:0/3864753267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345671b0 0x55743490b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f438bfff640 1 --2- 172.21.15.19:0/3864753267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x557434912980 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f43840e8c10 tx=0x7f43840ea690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a8e51bf81142b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f4389ffb640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f437c0b27e0 0x7f437c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f437affd640 1 -- 172.21.15.19:0/3864753267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43840f1070 con 0x5574345464d0 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f4399151640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f437c0b27e0 0x7f437c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f437affd640 1 -- 172.21.15.19:0/3864753267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4384010720 con 0x5574345464d0 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f437affd640 1 -- 172.21.15.19:0/3864753267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43840a9420 con 0x5574345464d0 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.806+0000 7f4389ffb640 1 -- 172.21.15.19:0/3863843042 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f43840e26b0 con 0x7f4394104eb0 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 -- 172.21.15.19:0/3864753267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 msgr2=0x557434912980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f4399151640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f437c0b27e0 0x7f437c0b4cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55743490c6d0 tx=0x7f438c031af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 --2- 172.21.15.19:0/3864753267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x557434912980 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f43840e8c10 tx=0x7f43840ea690 comp rx=0 tx=0).stop 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 -- 172.21.15.19:0/3864753267 shutdown_connections 2024-05-09T01:33:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 --2- 172.21.15.19:0/3864753267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x557434912980 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 --2- 172.21.15.19:0/3864753267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55743453e2f0 0x55743490ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 --2- 172.21.15.19:0/3864753267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345671b0 0x55743490b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 -- 172.21.15.19:0/3864753267 >> 172.21.15.19:0/3864753267 conn(0x55743455b300 msgr2=0x557434906e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 -- 172.21.15.19:0/3864753267 shutdown_connections 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 -- 172.21.15.19:0/3864753267 wait complete. 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.807+0000 7f43a1a61740 1 -- start start 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f43a1a61740 1 Processor -- start 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 0x5574349151d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345464d0 0x557434915710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345671b0 0x55743491c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f43a1a61740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5574349133d0 con 0x5574345671b0 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f43a1a61740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5574347f69c0 con 0x5574345464d0 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f43a1a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557434913550 con 0x55743453e2f0 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f43a1a61740 10 client.0 _ll_register_callbacks cb 0x557434823020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f4399151640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345671b0 0x55743491c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f4399151640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345671b0 0x55743491c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42482/0 (socket says 172.21.15.19:42482) 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f4399151640 1 -- 172.21.15.19:0/2634114226 learned_addr learned my addr 172.21.15.19:0/2634114226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f4398950640 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 0x5574349151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f438bfff640 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345464d0 0x557434915710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f4399151640 1 -- 172.21.15.19:0/2634114226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 msgr2=0x5574349151d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f4399151640 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 0x5574349151d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f4399151640 1 -- 172.21.15.19:0/2634114226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345464d0 msgr2=0x557434915710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.808+0000 7f4399151640 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345464d0 0x557434915710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f4399151640 1 -- 172.21.15.19:0/2634114226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55743481d1f0 con 0x5574345671b0 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f4398950640 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 0x5574349151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f438bfff640 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345464d0 0x557434915710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f4399151640 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345671b0 0x55743491c670 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f438c005da0 tx=0x7f438c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148523020f667b47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f4373fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f4373fff640 1 -- 172.21.15.19:0/2634114226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c004310 con 0x5574345671b0 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f4373fff640 1 -- 172.21.15.19:0/2634114226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f438c0044b0 con 0x5574345671b0 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f43a1a61740 1 -- 172.21.15.19:0/2634114226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55743487d3d0 con 0x5574345671b0 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f4373fff640 1 -- 172.21.15.19:0/2634114226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f438c0046b0 con 0x5574345671b0 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f43a1a61740 10 client.17358 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f43a1a61740 1 -- 172.21.15.19:0/2634114226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574345d2790 con 0x5574345671b0 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.809+0000 7f4373fff640 1 -- 172.21.15.19:0/2634114226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55743487d3d0 con 0x5574345671b0 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f4373fff640 1 -- 172.21.15.19:0/2634114226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f438c051070 con 0x5574345671b0 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f43a1a61740 10 client.17358 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f43a1a61740 10 client.17358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f43a1a61740 20 client.17358 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557434525d80 0x557434922c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f43a1a61740 4 client.17358 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f43a1a61740 1 -- 172.21.15.19:0/2634114226 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557434877240 con 0x557434525d80 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f4398950640 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557434525d80 0x557434922c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f4373fff640 10 client.17358 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.810+0000 7f4398950640 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557434525d80 0x557434922c20 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae9e33ef6e65110 server_cookie=f4602402d63bbeac in_seq=0 out_seq=0 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f4373fff640 1 -- 172.21.15.19:0/2634114226 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557434877240 con 0x557434525d80 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f4373fff640 10 client.17358 handle_command_reply: tid=0 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f43a1a61740 1 client.17358 shutdown 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f43a1a61740 1 -- 172.21.15.19:0/2634114226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345671b0 msgr2=0x55743491c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345671b0 0x55743491c670 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f438c005da0 tx=0x7f438c02a8c0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f43a1a61740 1 -- 172.21.15.19:0/2634114226 shutdown_connections 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x557434525d80 0x557434922c20 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345671b0 0x55743491c670 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345464d0 0x557434915710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f4373fff640 0 client.17358 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2634114226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 0x5574349151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.811+0000 7f43a1a61740 1 -- 172.21.15.19:0/2634114226 >> 172.21.15.19:0/2634114226 conn(0x55743455b300 msgr2=0x557434909da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.812+0000 7f43a1a61740 1 -- 172.21.15.19:0/2634114226 shutdown_connections 2024-05-09T01:33:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.812+0000 7f43a1a61740 1 -- 172.21.15.19:0/2634114226 wait complete. 2024-05-09T01:33:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.812+0000 7f43a1a61740 20 client.17358 trim_cache size 0 max 16384 2024-05-09T01:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f43a1a61740 1 Processor -- start 2024-05-09T01:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f43a1a61740 1 -- start start 2024-05-09T01:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 0x55743490f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x55743491c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 0x55743490fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f43a1a61740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43941ab480 con 0x5574345464d0 2024-05-09T01:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f43a1a61740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43941acfc0 con 0x5574345671b0 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f43a1a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43941abb50 con 0x55743453e2f0 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f438bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x55743491c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f438bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x55743491c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42484/0 (socket says 172.21.15.19:42484) 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f438bfff640 1 -- 172.21.15.19:0/361710567 learned_addr learned my addr 172.21.15.19:0/361710567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f4399151640 1 --2- 172.21.15.19:0/361710567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 0x55743490fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f4398950640 1 --2- 172.21.15.19:0/361710567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 0x55743490f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f438bfff640 1 -- 172.21.15.19:0/361710567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 msgr2=0x55743490f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f438bfff640 1 --2- 172.21.15.19:0/361710567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 0x55743490f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f438bfff640 1 -- 172.21.15.19:0/361710567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 msgr2=0x55743490fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f438bfff640 1 --2- 172.21.15.19:0/361710567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 0x55743490fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f438bfff640 1 -- 172.21.15.19:0/361710567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574346495f0 con 0x5574345464d0 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.827+0000 7f438bfff640 1 --2- 172.21.15.19:0/361710567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x55743491c590 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f43840a99b0 tx=0x7f43840e8f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4711d556acbb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f437a7fc640 1 -- 172.21.15.19:0/361710567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384020c40 con 0x5574345464d0 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f437a7fc640 1 -- 172.21.15.19:0/361710567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43840b2450 con 0x5574345464d0 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f437a7fc640 1 -- 172.21.15.19:0/361710567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384020e60 con 0x5574345464d0 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f43a1a61740 1 -- 172.21.15.19:0/361710567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 msgr2=0x55743491c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f43a1a61740 1 --2- 172.21.15.19:0/361710567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x55743491c590 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f43840a99b0 tx=0x7f43840e8f10 comp rx=0 tx=0).stop 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f43a1a61740 1 -- 172.21.15.19:0/361710567 shutdown_connections 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f43a1a61740 1 --2- 172.21.15.19:0/361710567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 0x55743490fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f43a1a61740 1 --2- 172.21.15.19:0/361710567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574345464d0 0x55743491c590 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f43a1a61740 1 --2- 172.21.15.19:0/361710567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55743453e2f0 0x55743490f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f43a1a61740 1 -- 172.21.15.19:0/361710567 >> 172.21.15.19:0/361710567 conn(0x55743455b300 msgr2=0x55743490b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.828+0000 7f43a1a61740 1 -- 172.21.15.19:0/361710567 shutdown_connections 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 1 -- 172.21.15.19:0/361710567 wait complete. 2024-05-09T01:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 1 -- start start 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 1 Processor -- start 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55743453e2f0 0x5574349196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345464d0 0x557434919be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 0x557434988fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43941ac8f0 con 0x55743453e2f0 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43941ac220 con 0x5574345671b0 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43941a9460 con 0x5574345464d0 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f43a1a61740 10 client.0 _ll_register_callbacks cb 0x7f438c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f4398950640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55743453e2f0 0x5574349196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f4398950640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55743453e2f0 0x5574349196a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42488/0 (socket says 172.21.15.19:42488) 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.829+0000 7f4398950640 1 -- 172.21.15.19:0/2068735277 learned_addr learned my addr 172.21.15.19:0/2068735277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4399151640 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 0x557434988fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f438bfff640 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345464d0 0x557434919be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4398950640 1 -- 172.21.15.19:0/2068735277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345464d0 msgr2=0x557434919be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4398950640 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345464d0 0x557434919be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4398950640 1 -- 172.21.15.19:0/2068735277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 msgr2=0x557434988fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4398950640 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 0x557434988fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4398950640 1 -- 172.21.15.19:0/2068735277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557434833ed0 con 0x55743453e2f0 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4398950640 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55743453e2f0 0x5574349196a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4390005fd0 tx=0x7f4390011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75acecd6012fb8cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4373fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4373fff640 1 -- 172.21.15.19:0/2068735277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439001d970 con 0x55743453e2f0 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4373fff640 1 -- 172.21.15.19:0/2068735277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4390025400 con 0x55743453e2f0 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f43a1a61740 1 -- 172.21.15.19:0/2068735277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55743490f0f0 con 0x55743453e2f0 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4373fff640 1 -- 172.21.15.19:0/2068735277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4390025600 con 0x55743453e2f0 2024-05-09T01:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f43a1a61740 10 client.17364 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f43a1a61740 1 -- 172.21.15.19:0/2068735277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55743498a6a0 con 0x55743453e2f0 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.830+0000 7f4373fff640 1 -- 172.21.15.19:0/2068735277 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55743490f0f0 con 0x55743453e2f0 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f4373fff640 1 -- 172.21.15.19:0/2068735277 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4390030860 con 0x55743453e2f0 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f43a1a61740 10 client.17364 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f43a1a61740 10 client.17364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f43a1a61740 20 client.17364 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574344fb1c0 0x55743498d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f43a1a61740 4 client.17364 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f43a1a61740 1 -- 172.21.15.19:0/2068735277 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574347f2350 con 0x5574344fb1c0 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f438bfff640 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574344fb1c0 0x55743498d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f438bfff640 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574344fb1c0 0x55743498d070 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a07f93a9b01652 server_cookie=a86e87aa3f61ef19 in_seq=0 out_seq=0 2024-05-09T01:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.831+0000 7f4373fff640 10 client.17364 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f4373fff640 1 -- 172.21.15.19:0/2068735277 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5574347f2350 con 0x5574344fb1c0 2024-05-09T01:33:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f4373fff640 10 client.17364 handle_command_reply: tid=0 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 client.17364 shutdown 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 -- 172.21.15.19:0/2068735277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55743453e2f0 msgr2=0x5574349196a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55743453e2f0 0x5574349196a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4390005fd0 tx=0x7f4390011a30 comp rx=0 tx=0).stop 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 -- 172.21.15.19:0/2068735277 shutdown_connections 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5574344fb1c0 0x55743498d070 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574345671b0 0x557434988fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574345464d0 0x557434919be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f4373fff640 0 client.17364 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 --2- 172.21.15.19:0/2068735277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55743453e2f0 0x5574349196a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 -- 172.21.15.19:0/2068735277 >> 172.21.15.19:0/2068735277 conn(0x55743455b300 msgr2=0x5574349128d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 -- 172.21.15.19:0/2068735277 shutdown_connections 2024-05-09T01:33:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 1 -- 172.21.15.19:0/2068735277 wait complete. 2024-05-09T01:33:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.850+0000 7f43a1a61740 20 client.17364 trim_cache size 0 max 16384 2024-05-09T01:33:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:33:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:33:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:33:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 -- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f437c0b27e0 msgr2=0x7f437c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f437c0b27e0 0x7f437c0b4cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55743490c6d0 tx=0x7f438c031af0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 -- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394104eb0 msgr2=0x7f439419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394104eb0 0x7f439419a850 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f438400e9f0 tx=0x7f438400eeb0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 -- 172.21.15.19:0/3863843042 shutdown_connections 2024-05-09T01:33:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f437c0b27e0 0x7f437c0b4cf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4394109bb0 0x7f43941a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394104eb0 0x7f439419a850 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 --2- 172.21.15.19:0/3863843042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4394103c70 0x7f439419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 -- 172.21.15.19:0/3863843042 >> 172.21.15.19:0/3863843042 conn(0x7f43940fee40 msgr2=0x7f4394105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 -- 172.21.15.19:0/3863843042 shutdown_connections 2024-05-09T01:33:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:09.851+0000 7f439abb4640 1 -- 172.21.15.19:0/3863843042 wait complete. 2024-05-09T01:33:10.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:10 smithi019 ceph-mon[30158]: pgmap v607: 97 pgs: 97 active+clean; 4.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-05-09T01:33:10.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:10 smithi196 ceph-mon[35041]: pgmap v607: 97 pgs: 97 active+clean; 4.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-05-09T01:33:10.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:10 smithi059 ceph-mon[34351]: pgmap v607: 97 pgs: 97 active+clean; 4.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-05-09T01:33:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.000+0000 7fbc31854700 1 -- 172.21.15.19:0/2559079946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c108740 msgr2=0x7fbc2c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.000+0000 7fbc31854700 1 --2- 172.21.15.19:0/2559079946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c108740 0x7fbc2c1100a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400b0d0 tx=0x7fbc2400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:33:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.001+0000 7fbc31854700 1 -- 172.21.15.19:0/2559079946 shutdown_connections 2024-05-09T01:33:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.001+0000 7fbc31854700 1 --2- 172.21.15.19:0/2559079946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c108740 0x7fbc2c1100a0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.001+0000 7fbc31854700 1 --2- 172.21.15.19:0/2559079946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c1034c0 0x7fbc2c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.001+0000 7fbc31854700 1 --2- 172.21.15.19:0/2559079946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c107df0 0x7fbc2c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.001+0000 7fbc31854700 1 -- 172.21.15.19:0/2559079946 >> 172.21.15.19:0/2559079946 conn(0x7fbc2c0fedc0 msgr2=0x7fbc2c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.001+0000 7fbc31854700 1 -- 172.21.15.19:0/2559079946 shutdown_connections 2024-05-09T01:33:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.001+0000 7fbc31854700 1 -- 172.21.15.19:0/2559079946 wait complete. 2024-05-09T01:33:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.001+0000 7fbc31854700 1 Processor -- start 2024-05-09T01:33:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc31854700 1 -- start start 2024-05-09T01:33:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc31854700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1034c0 0x7fbc2c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc31854700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c107df0 0x7fbc2c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc31854700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c108740 0x7fbc2c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc31854700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1133e0 con 0x7fbc2c107df0 2024-05-09T01:33:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc31854700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c113260 con 0x7fbc2c108740 2024-05-09T01:33:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc31854700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c113560 con 0x7fbc2c1034c0 2024-05-09T01:33:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c107df0 0x7fbc2c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc2affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1034c0 0x7fbc2c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c107df0 0x7fbc2c19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42512/0 (socket says 172.21.15.19:42512) 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc227fc700 1 -- 172.21.15.19:0/3358932889 learned_addr learned my addr 172.21.15.19:0/3358932889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.002+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c108740 0x7fbc2c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc2affd700 1 -- 172.21.15.19:0/3358932889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c108740 msgr2=0x7fbc2c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc2affd700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c108740 0x7fbc2c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc2affd700 1 -- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c107df0 msgr2=0x7fbc2c19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc2affd700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c107df0 0x7fbc2c19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc2affd700 1 -- 172.21.15.19:0/3358932889 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c19a1c0 con 0x7fbc2c1034c0 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c108740 0x7fbc2c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc227fc700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c107df0 0x7fbc2c19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc2affd700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1034c0 0x7fbc2c19f8a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d950 tx=0x7fbc1c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc30852700 1 -- 172.21.15.19:0/3358932889 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c016070 con 0x7fbc2c1034c0 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc31854700 1 -- 172.21.15.19:0/3358932889 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c19a4b0 con 0x7fbc2c1034c0 2024-05-09T01:33:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc30852700 1 -- 172.21.15.19:0/3358932889 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc1c00fd40 con 0x7fbc2c1034c0 2024-05-09T01:33:11.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc30852700 1 -- 172.21.15.19:0/3358932889 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00e3b0 con 0x7fbc2c1034c0 2024-05-09T01:33:11.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.003+0000 7fbc31854700 1 -- 172.21.15.19:0/3358932889 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c06b610 con 0x7fbc2c1034c0 2024-05-09T01:33:11.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.006+0000 7fbc30852700 1 -- 172.21.15.19:0/3358932889 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbc1c019070 con 0x7fbc2c1034c0 2024-05-09T01:33:11.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc30852700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc180b2880 0x7fbc180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc4026ab80 1 Processor -- start 2024-05-09T01:33:11.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc4026ab80 1 -- start start 2024-05-09T01:33:11.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc227fc700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc180b2880 0x7fbc180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc30852700 1 -- 172.21.15.19:0/3358932889 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbc1c0e1cc0 con 0x7fbc2c1034c0 2024-05-09T01:33:11.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 0x558b2484c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b246c94a0 0x558b2484ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b248466a0 0x558b24846ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558b24768ff0 con 0x558b248466a0 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b247077f0 con 0x558b246c94a0 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b24764fb0 con 0x558b247b3160 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b248466a0 0x558b24846ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b248466a0 0x558b24846ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42520/0 (socket says 172.21.15.19:42520) 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc2b7fe700 1 -- 172.21.15.19:0/3038345786 learned_addr learned my addr 172.21.15.19:0/3038345786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc227fc700 1 --2- 172.21.15.19:0/3038345786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b246c94a0 0x558b2484ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.008+0000 7fbc2affd700 1 --2- 172.21.15.19:0/3038345786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 0x558b2484c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc2b7fe700 1 -- 172.21.15.19:0/3038345786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 msgr2=0x558b2484c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:11.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/3038345786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 0x558b2484c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc2b7fe700 1 -- 172.21.15.19:0/3038345786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b246c94a0 msgr2=0x558b2484ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/3038345786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b246c94a0 0x558b2484ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc2b7fe700 1 -- 172.21.15.19:0/3038345786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b246ff370 con 0x558b248466a0 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc227fc700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc180b2880 0x7fbc180b4d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x558b2484bfb0 tx=0x7fbc14009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/3038345786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b248466a0 0x558b24846ad0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009520 tx=0x7fbc24009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427df9572f02e2f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc21ffb700 1 -- 172.21.15.19:0/3038345786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24003a00 con 0x558b248466a0 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc21ffb700 1 -- 172.21.15.19:0/3038345786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc24003ba0 con 0x558b248466a0 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.009+0000 7fbc21ffb700 1 -- 172.21.15.19:0/3038345786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24005440 con 0x558b248466a0 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.010+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3038345786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b248466a0 msgr2=0x558b24846ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.010+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3038345786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b248466a0 0x558b24846ad0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009520 tx=0x7fbc24009550 comp rx=0 tx=0).stop 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.010+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3038345786 shutdown_connections 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.010+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3038345786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b248466a0 0x558b24846ad0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.010+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3038345786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 0x558b2484c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.010+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3038345786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b246c94a0 0x558b2484ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.010+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3038345786 >> 172.21.15.19:0/3038345786 conn(0x558b247296a0 msgr2=0x558b24841400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.010+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3038345786 shutdown_connections 2024-05-09T01:33:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.010+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3038345786 wait complete. 2024-05-09T01:33:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.011+0000 7fbc4026ab80 1 -- start start 2024-05-09T01:33:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.011+0000 7fbc4026ab80 1 Processor -- start 2024-05-09T01:33:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.011+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b246c94a0 0x558b248501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.011+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b247b3160 0x558b24850710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.011+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b248466a0 0x558b24850c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.011+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558b245e4ba0 con 0x558b247b3160 2024-05-09T01:33:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.011+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b24723150 con 0x558b248466a0 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b248531d0 con 0x558b246c94a0 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc4026ab80 10 client.0 _ll_register_callbacks cb 0x558b245da6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b247b3160 0x558b24850710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b247b3160 0x558b24850710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42528/0 (socket says 172.21.15.19:42528) 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc227fc700 1 -- 172.21.15.19:0/4277737962 learned_addr learned my addr 172.21.15.19:0/4277737962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc2affd700 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b246c94a0 0x558b248501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b248466a0 0x558b24850c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc227fc700 1 -- 172.21.15.19:0/4277737962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b246c94a0 msgr2=0x558b248501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc227fc700 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b246c94a0 0x558b248501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc227fc700 1 -- 172.21.15.19:0/4277737962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b248466a0 msgr2=0x558b24850c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc227fc700 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b248466a0 0x558b24850c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc227fc700 1 -- 172.21.15.19:0/4277737962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b245e4490 con 0x558b247b3160 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b248466a0 0x558b24850c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc227fc700 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b247b3160 0x558b24850710 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fbc14015390 tx=0x7fbc14009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9301db6df1d51a22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc0affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.012+0000 7fbc0affd700 1 -- 172.21.15.19:0/4277737962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400ee90 con 0x558b247b3160 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc0affd700 1 -- 172.21.15.19:0/4277737962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbc14018a10 con 0x558b247b3160 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc4026ab80 1 -- 172.21.15.19:0/4277737962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc24003ba0 con 0x558b247b3160 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc0affd700 1 -- 172.21.15.19:0/4277737962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc14018c10 con 0x558b247b3160 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc4026ab80 10 client.17376 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc4026ab80 1 -- 172.21.15.19:0/4277737962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b2485dbc0 con 0x558b247b3160 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc0affd700 1 -- 172.21.15.19:0/4277737962 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc24003ba0 con 0x558b247b3160 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc0affd700 1 -- 172.21.15.19:0/4277737962 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbc140223f0 con 0x558b247b3160 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc4026ab80 10 client.17376 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc4026ab80 10 client.17376 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc4026ab80 20 client.17376 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x558b248602a0 0x558b248626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc4026ab80 4 client.17376 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.013+0000 7fbc4026ab80 1 -- 172.21.15.19:0/4277737962 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b247769d0 con 0x558b248602a0 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.014+0000 7fbc2affd700 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x558b248602a0 0x558b248626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.014+0000 7fbc2affd700 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x558b248602a0 0x558b248626e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=746ee22be5cba465 server_cookie=83e6d56d3ff289c in_seq=0 out_seq=0 2024-05-09T01:33:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.014+0000 7fbc0affd700 10 client.17376 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc0affd700 1 -- 172.21.15.19:0/4277737962 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b247769d0 con 0x558b248602a0 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc0affd700 10 client.17376 handle_command_reply: tid=0 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 client.17376 shutdown 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 -- 172.21.15.19:0/4277737962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b247b3160 msgr2=0x558b24850710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b247b3160 0x558b24850710 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fbc14015390 tx=0x7fbc14009240 comp rx=0 tx=0).stop 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 -- 172.21.15.19:0/4277737962 shutdown_connections 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x558b248602a0 0x558b248626e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b248466a0 0x558b24850c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b246c94a0 0x558b248501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/4277737962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b247b3160 0x558b24850710 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 -- 172.21.15.19:0/4277737962 >> 172.21.15.19:0/4277737962 conn(0x558b247296a0 msgr2=0x558b248411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc0affd700 0 client.17376 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 -- 172.21.15.19:0/4277737962 shutdown_connections 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.015+0000 7fbc4026ab80 1 -- 172.21.15.19:0/4277737962 wait complete. 2024-05-09T01:33:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.016+0000 7fbc4026ab80 20 client.17376 trim_cache size 0 max 16384 2024-05-09T01:33:11.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.031+0000 7fbc4026ab80 1 Processor -- start 2024-05-09T01:33:11.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.031+0000 7fbc4026ab80 1 -- start start 2024-05-09T01:33:11.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.031+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247296a0 0x558b24850a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.031+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24849e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.031+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247b3160 0x558b2484a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1aec30 con 0x558b246c94a0 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1b0770 con 0x558b247b3160 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1af300 con 0x558b247296a0 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24849e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24849e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42530/0 (socket says 172.21.15.19:42530) 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc227fc700 1 -- 172.21.15.19:0/2191536092 learned_addr learned my addr 172.21.15.19:0/2191536092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc2affd700 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247296a0 0x558b24850a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247b3160 0x558b2484a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc227fc700 1 -- 172.21.15.19:0/2191536092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247296a0 msgr2=0x558b24850a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc227fc700 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247296a0 0x558b24850a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc227fc700 1 -- 172.21.15.19:0/2191536092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247b3160 msgr2=0x558b2484a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc227fc700 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247b3160 0x558b2484a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc227fc700 1 -- 172.21.15.19:0/2191536092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b24847610 con 0x558b246c94a0 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc2affd700 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247296a0 0x558b24850a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc227fc700 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24849e00 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fbc14022a00 tx=0x7fbc14018ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2b5f01202aae4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.032+0000 7fbc217fa700 1 -- 172.21.15.19:0/2191536092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14022c80 con 0x558b246c94a0 2024-05-09T01:33:11.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.033+0000 7fbc217fa700 1 -- 172.21.15.19:0/2191536092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc14027070 con 0x558b246c94a0 2024-05-09T01:33:11.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.033+0000 7fbc217fa700 1 -- 172.21.15.19:0/2191536092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1402dd80 con 0x558b246c94a0 2024-05-09T01:33:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.033+0000 7fbc4026ab80 1 -- 172.21.15.19:0/2191536092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 msgr2=0x558b24849e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.033+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24849e00 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fbc14022a00 tx=0x7fbc14018ba0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.033+0000 7fbc4026ab80 1 -- 172.21.15.19:0/2191536092 shutdown_connections 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.033+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247b3160 0x558b2484a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.033+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247296a0 0x558b24850a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.033+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/2191536092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24849e00 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.033+0000 7fbc4026ab80 1 -- 172.21.15.19:0/2191536092 >> 172.21.15.19:0/2191536092 conn(0x558b2484ccb0 msgr2=0x558b2484eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.034+0000 7fbc4026ab80 1 -- 172.21.15.19:0/2191536092 shutdown_connections 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.034+0000 7fbc4026ab80 1 -- 172.21.15.19:0/2191536092 wait complete. 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.034+0000 7fbc4026ab80 1 -- start start 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.034+0000 7fbc4026ab80 1 Processor -- start 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.034+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24858350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.034+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247296a0 0x558b248588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc4026ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 0x558b248c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1b00a0 con 0x558b246c94a0 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1af9d0 con 0x558b247296a0 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc4026ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1acbf0 con 0x558b247b3160 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc4026ab80 10 client.0 _ll_register_callbacks cb 0x558b245da6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24858350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24858350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42540/0 (socket says 172.21.15.19:42540) 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2affd700 1 -- 172.21.15.19:0/3869207622 learned_addr learned my addr 172.21.15.19:0/3869207622 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc227fc700 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247296a0 0x558b248588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 0x558b248c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2affd700 1 -- 172.21.15.19:0/3869207622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 msgr2=0x558b248c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2affd700 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 0x558b248c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2affd700 1 -- 172.21.15.19:0/3869207622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247296a0 msgr2=0x558b248588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2affd700 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247296a0 0x558b248588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2affd700 1 -- 172.21.15.19:0/3869207622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b248c61b0 con 0x558b246c94a0 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2b7fe700 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 0x558b248c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc2affd700 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24858350 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0df740 tx=0x7fbc1c0a9150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c6f11b0ee65005 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc0affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc0affd700 1 -- 172.21.15.19:0/3869207622 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c06d060 con 0x558b246c94a0 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc0affd700 1 -- 172.21.15.19:0/3869207622 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbc1c0a94c0 con 0x558b246c94a0 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.035+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3869207622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc14027070 con 0x558b246c94a0 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc0affd700 1 -- 172.21.15.19:0/3869207622 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc1c020040 con 0x558b246c94a0 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc4026ab80 10 client.17382 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3869207622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b248678d0 con 0x558b246c94a0 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc0affd700 1 -- 172.21.15.19:0/3869207622 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc14027070 con 0x558b246c94a0 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc0affd700 1 -- 172.21.15.19:0/3869207622 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbc1c0b1c30 con 0x558b246c94a0 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc4026ab80 10 client.17382 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc4026ab80 10 client.17382 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc4026ab80 20 client.17382 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x558b248466a0 0x558b248d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc4026ab80 4 client.17382 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.036+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3869207622 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b24770a70 con 0x558b248466a0 2024-05-09T01:33:11.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc227fc700 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x558b248466a0 0x558b248d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:11.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc0affd700 10 client.17382 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:11.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc227fc700 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x558b248466a0 0x558b248d0f90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67eb684b006c3985 server_cookie=5de814f8e7d2e1e6 in_seq=0 out_seq=0 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc0affd700 1 -- 172.21.15.19:0/3869207622 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b24770a70 con 0x558b248466a0 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc0affd700 10 client.17382 handle_command_reply: tid=0 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc4026ab80 1 client.17382 shutdown 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3869207622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 msgr2=0x558b24858350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24858350 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0df740 tx=0x7fbc1c0a9150 comp rx=0 tx=0).stop 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3869207622 shutdown_connections 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b247b3160 0x558b248c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b246c94a0 0x558b24858350 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x558b248466a0 0x558b248d0f90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc4026ab80 1 --2- 172.21.15.19:0/3869207622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b247296a0 0x558b248588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc0affd700 0 client.17382 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.037+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3869207622 >> 172.21.15.19:0/3869207622 conn(0x558b2484ccb0 msgr2=0x558b24845690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3869207622 shutdown_connections 2024-05-09T01:33:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc4026ab80 1 -- 172.21.15.19:0/3869207622 wait complete. 2024-05-09T01:33:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc4026ab80 20 client.17382 trim_cache size 0 max 16384 2024-05-09T01:33:11.039 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 -- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc180b2880 msgr2=0x7fbc180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc180b2880 0x7fbc180b4d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x558b2484bfb0 tx=0x7fbc14009f90 comp rx=0 tx=0).stop 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 -- 172.21.15.19:0/3358932889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1034c0 msgr2=0x7fbc2c19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1034c0 0x7fbc2c19f8a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d950 tx=0x7fbc1c00dc60 comp rx=0 tx=0).stop 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 -- 172.21.15.19:0/3358932889 shutdown_connections 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c108740 0x7fbc2c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1034c0 0x7fbc2c19f8a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbc180b2880 0x7fbc180b4d90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 --2- 172.21.15.19:0/3358932889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc2c107df0 0x7fbc2c19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.038+0000 7fbc31854700 1 -- 172.21.15.19:0/3358932889 >> 172.21.15.19:0/3358932889 conn(0x7fbc2c0fedc0 msgr2=0x7fbc2c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.039+0000 7fbc31854700 1 -- 172.21.15.19:0/3358932889 shutdown_connections 2024-05-09T01:33:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:11.039+0000 7fbc31854700 1 -- 172.21.15.19:0/3358932889 wait complete. 2024-05-09T01:33:11.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:33:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:12.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:12 smithi019 ceph-mon[30158]: pgmap v608: 97 pgs: 97 active+clean; 707 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-05-09T01:33:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:12 smithi196 ceph-mon[35041]: pgmap v608: 97 pgs: 97 active+clean; 707 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-05-09T01:33:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:12 smithi059 ceph-mon[34351]: pgmap v608: 97 pgs: 97 active+clean; 707 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-05-09T01:33:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.710+0000 7f5494d81700 1 -- 172.21.15.19:0/3934670021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490107e20 msgr2=0x7f5490108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.710+0000 7f5494d81700 1 --2- 172.21.15.19:0/3934670021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490107e20 0x7f5490108230 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f54780099c0 tx=0x7f5478009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.710+0000 7f5494d81700 1 -- 172.21.15.19:0/3934670021 shutdown_connections 2024-05-09T01:33:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.710+0000 7f5494d81700 1 --2- 172.21.15.19:0/3934670021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5490108770 0x7f54901100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.710+0000 7f5494d81700 1 --2- 172.21.15.19:0/3934670021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5490103540 0x7f5490103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.710+0000 7f5494d81700 1 --2- 172.21.15.19:0/3934670021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490107e20 0x7f5490108230 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.710+0000 7f5494d81700 1 -- 172.21.15.19:0/3934670021 >> 172.21.15.19:0/3934670021 conn(0x7f5490078e50 msgr2=0x7f5490079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.710+0000 7f5494d81700 1 -- 172.21.15.19:0/3934670021 shutdown_connections 2024-05-09T01:33:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.710+0000 7f5494d81700 1 -- 172.21.15.19:0/3934670021 wait complete. 2024-05-09T01:33:13.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f5494d81700 1 Processor -- start 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f5494d81700 1 -- start start 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f5494d81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5490103540 0x7f549019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f5494d81700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5490107e20 0x7f549019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f5494d81700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490108770 0x7f54901a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f5494d81700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5490113380 con 0x7f5490108770 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f5494d81700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5490113200 con 0x7f5490103540 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f5494d81700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5490113500 con 0x7f5490107e20 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f548ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490108770 0x7f54901a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f548ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490108770 0x7f54901a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42558/0 (socket says 172.21.15.19:42558) 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f548ed9d700 1 -- 172.21.15.19:0/1989475821 learned_addr learned my addr 172.21.15.19:0/1989475821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f548e59c700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5490103540 0x7f549019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.711+0000 7f5485d9b700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5490107e20 0x7f549019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f548ed9d700 1 -- 172.21.15.19:0/1989475821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5490107e20 msgr2=0x7f549019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f548ed9d700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5490107e20 0x7f549019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f548ed9d700 1 -- 172.21.15.19:0/1989475821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5490103540 msgr2=0x7f549019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f548ed9d700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5490103540 0x7f549019d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f548ed9d700 1 -- 172.21.15.19:0/1989475821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54901a1f10 con 0x7f5490108770 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f548e59c700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5490103540 0x7f549019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f548ed9d700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490108770 0x7f54901a1870 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5488007ee0 tx=0x7f548800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f5487fff700 1 -- 172.21.15.19:0/1989475821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548800dd20 con 0x7f5490108770 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f5494d81700 1 -- 172.21.15.19:0/1989475821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54901a2200 con 0x7f5490108770 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f5494d81700 1 -- 172.21.15.19:0/1989475821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54901a9b20 con 0x7f5490108770 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f5487fff700 1 -- 172.21.15.19:0/1989475821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5488004020 con 0x7f5490108770 2024-05-09T01:33:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.712+0000 7f5487fff700 1 -- 172.21.15.19:0/1989475821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488015640 con 0x7f5490108770 2024-05-09T01:33:13.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.715+0000 7f5487fff700 1 -- 172.21.15.19:0/1989475821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f54880157e0 con 0x7f5490108770 2024-05-09T01:33:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.716+0000 7f5487fff700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f547c0b2880 0x7f547c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.716+0000 7f54a3797b80 1 Processor -- start 2024-05-09T01:33:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.716+0000 7f548e59c700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f547c0b2880 0x7f547c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.716+0000 7f5487fff700 1 -- 172.21.15.19:0/1989475821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f54880e1af0 con 0x7f5490108770 2024-05-09T01:33:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.716+0000 7f54a3797b80 1 -- start start 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137573240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 0x560137573780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f547c0b2880 0x7f547c0b4d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5480005950 tx=0x7f548000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 0x56013756d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f54a3797b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5601374900a0 con 0x56013746ad00 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f54a3797b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5601374359a0 con 0x5601373f04a0 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f54a3797b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56013748c060 con 0x56013756d400 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137573240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137573240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42562/0 (socket says 172.21.15.19:42562) 2024-05-09T01:33:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 -- 172.21.15.19:0/470475856 learned_addr learned my addr 172.21.15.19:0/470475856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f5485d9b700 1 --2- 172.21.15.19:0/470475856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 0x560137573780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548ed9d700 1 --2- 172.21.15.19:0/470475856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 0x56013756d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 -- 172.21.15.19:0/470475856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 msgr2=0x56013756d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 --2- 172.21.15.19:0/470475856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 0x56013756d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 -- 172.21.15.19:0/470475856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 msgr2=0x560137573780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 --2- 172.21.15.19:0/470475856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 0x560137573780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 -- 172.21.15.19:0/470475856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560137426500 con 0x56013746ad00 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.717+0000 7f548e59c700 1 --2- 172.21.15.19:0/470475856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137573240 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5480011790 tx=0x7f5480013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a9e2a38615ea28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.718+0000 7f5484d99700 1 -- 172.21.15.19:0/470475856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480010ea0 con 0x56013746ad00 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.718+0000 7f5484d99700 1 -- 172.21.15.19:0/470475856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548001ead0 con 0x56013746ad00 2024-05-09T01:33:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.718+0000 7f5484d99700 1 -- 172.21.15.19:0/470475856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548001cc10 con 0x56013746ad00 2024-05-09T01:33:13.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 -- 172.21.15.19:0/470475856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 msgr2=0x560137573240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:13.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 --2- 172.21.15.19:0/470475856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137573240 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5480011790 tx=0x7f5480013000 comp rx=0 tx=0).stop 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 -- 172.21.15.19:0/470475856 shutdown_connections 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 --2- 172.21.15.19:0/470475856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 0x56013756d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 --2- 172.21.15.19:0/470475856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137573240 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 --2- 172.21.15.19:0/470475856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 0x560137573780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 -- 172.21.15.19:0/470475856 >> 172.21.15.19:0/470475856 conn(0x5601373f3f70 msgr2=0x560137568220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 -- 172.21.15.19:0/470475856 shutdown_connections 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 -- 172.21.15.19:0/470475856 wait complete. 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 -- start start 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.719+0000 7f54a3797b80 1 Processor -- start 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 0x560137576ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013746ad00 0x560137577530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013756d400 0x560137577a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f54a3797b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56013756e160 con 0x56013756d400 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f54a3797b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56013742ab20 con 0x5601373f04a0 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f54a3797b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56013757a020 con 0x56013746ad00 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f54a3797b80 10 client.0 _ll_register_callbacks cb 0x5601373016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013756d400 0x560137577a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013756d400 0x560137577a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42574/0 (socket says 172.21.15.19:42574) 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548ed9d700 1 -- 172.21.15.19:0/2677167819 learned_addr learned my addr 172.21.15.19:0/2677167819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f5485d9b700 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013746ad00 0x560137577530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548e59c700 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 0x560137576ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548ed9d700 1 -- 172.21.15.19:0/2677167819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013746ad00 msgr2=0x560137577530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548ed9d700 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013746ad00 0x560137577530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548ed9d700 1 -- 172.21.15.19:0/2677167819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 msgr2=0x560137576ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548ed9d700 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 0x560137576ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548ed9d700 1 -- 172.21.15.19:0/2677167819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56013730b490 con 0x56013756d400 2024-05-09T01:33:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f548ed9d700 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013756d400 0x560137577a70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f54880a8fd0 tx=0x7f54880094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56048745ae894b11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.720+0000 7f5475ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f5475ffb700 1 -- 172.21.15.19:0/2677167819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54880ad440 con 0x56013756d400 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f5475ffb700 1 -- 172.21.15.19:0/2677167819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f54880a9500 con 0x56013756d400 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f54a3797b80 1 -- 172.21.15.19:0/2677167819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f548001ead0 con 0x56013756d400 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f5475ffb700 1 -- 172.21.15.19:0/2677167819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54880b1ef0 con 0x56013756d400 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f54a3797b80 10 client.17394 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f54a3797b80 1 -- 172.21.15.19:0/2677167819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560137584a10 con 0x56013756d400 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f5475ffb700 1 -- 172.21.15.19:0/2677167819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548001ead0 con 0x56013756d400 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f5475ffb700 1 -- 172.21.15.19:0/2677167819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f54880b1a60 con 0x56013756d400 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f54a3797b80 10 client.17394 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f54a3797b80 10 client.17394 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f54a3797b80 20 client.17394 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f54a3797b80 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560137586fa0 0x5601375893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f54a3797b80 4 client.17394 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.721+0000 7f54a3797b80 1 -- 172.21.15.19:0/2677167819 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56013749da80 con 0x560137586fa0 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.722+0000 7f548e59c700 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560137586fa0 0x5601375893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.722+0000 7f5475ffb700 10 client.17394 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.722+0000 7f548e59c700 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560137586fa0 0x5601375893e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1c44a2cf4b58d9b server_cookie=ad90a0b9ae102158 in_seq=0 out_seq=0 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f5475ffb700 1 -- 172.21.15.19:0/2677167819 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56013749da80 con 0x560137586fa0 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f5475ffb700 10 client.17394 handle_command_reply: tid=0 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 client.17394 shutdown 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 -- 172.21.15.19:0/2677167819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013756d400 msgr2=0x560137577a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013756d400 0x560137577a70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f54880a8fd0 tx=0x7f54880094b0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 -- 172.21.15.19:0/2677167819 shutdown_connections 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x560137586fa0 0x5601375893e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013756d400 0x560137577a70 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5601373f04a0 0x560137576ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 --2- 172.21.15.19:0/2677167819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013746ad00 0x560137577530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 -- 172.21.15.19:0/2677167819 >> 172.21.15.19:0/2677167819 conn(0x5601373f3f70 msgr2=0x560137561ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f5475ffb700 0 client.17394 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:33:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 -- 172.21.15.19:0/2677167819 shutdown_connections 2024-05-09T01:33:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 1 -- 172.21.15.19:0/2677167819 wait complete. 2024-05-09T01:33:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.723+0000 7f54a3797b80 20 client.17394 trim_cache size 0 max 16384 2024-05-09T01:33:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f54a3797b80 1 Processor -- start 2024-05-09T01:33:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f54a3797b80 1 -- start start 2024-05-09T01:33:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601373f04a0 0x560137572400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137575f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013756d400 0x560137562700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f54a3797b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f547000d420 con 0x56013746ad00 2024-05-09T01:33:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f54a3797b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f547000ef60 con 0x56013756d400 2024-05-09T01:33:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f54a3797b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f547000daf0 con 0x5601373f04a0 2024-05-09T01:33:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f5485d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137575f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f5485d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137575f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42582/0 (socket says 172.21.15.19:42582) 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f5485d9b700 1 -- 172.21.15.19:0/384084819 learned_addr learned my addr 172.21.15.19:0/384084819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f548ed9d700 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013756d400 0x560137562700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.739+0000 7f548e59c700 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601373f04a0 0x560137572400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f5485d9b700 1 -- 172.21.15.19:0/384084819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601373f04a0 msgr2=0x560137572400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f5485d9b700 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601373f04a0 0x560137572400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f5485d9b700 1 -- 172.21.15.19:0/384084819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013756d400 msgr2=0x560137562700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f5485d9b700 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013756d400 0x560137562700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f5485d9b700 1 -- 172.21.15.19:0/384084819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560137579840 con 0x56013746ad00 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f548e59c700 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601373f04a0 0x560137572400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f5485d9b700 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137575f50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5478009990 tx=0x7f5478005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35dd9a0bf80bea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f5477fff700 1 -- 172.21.15.19:0/384084819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478028020 con 0x56013746ad00 2024-05-09T01:33:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f5477fff700 1 -- 172.21.15.19:0/384084819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54780095c0 con 0x56013746ad00 2024-05-09T01:33:13.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.740+0000 7f5477fff700 1 -- 172.21.15.19:0/384084819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547800bd20 con 0x56013746ad00 2024-05-09T01:33:13.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 -- 172.21.15.19:0/384084819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 msgr2=0x560137575f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:13.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137575f50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5478009990 tx=0x7f5478005040 comp rx=0 tx=0).stop 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 -- 172.21.15.19:0/384084819 shutdown_connections 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013756d400 0x560137562700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601373f04a0 0x560137572400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 --2- 172.21.15.19:0/384084819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56013746ad00 0x560137575f50 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 -- 172.21.15.19:0/384084819 >> 172.21.15.19:0/384084819 conn(0x5601373f3f70 msgr2=0x560137561ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 -- 172.21.15.19:0/384084819 shutdown_connections 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 -- 172.21.15.19:0/384084819 wait complete. 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 -- start start 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.741+0000 7f54a3797b80 1 Processor -- start 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601373f04a0 0x56013757f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013746ad00 0x56013757ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f54a3797b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 0x56013757b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f54a3797b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f547000e890 con 0x5601373f04a0 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f54a3797b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f547000e1c0 con 0x56013746ad00 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f54a3797b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f547000b3c0 con 0x56013756d400 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f54a3797b80 10 client.0 _ll_register_callbacks cb 0x5601373016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601373f04a0 0x56013757f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601373f04a0 0x56013757f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42594/0 (socket says 172.21.15.19:42594) 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548e59c700 1 -- 172.21.15.19:0/897252613 learned_addr learned my addr 172.21.15.19:0/897252613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f5485d9b700 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013746ad00 0x56013757ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548ed9d700 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 0x56013757b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548e59c700 1 -- 172.21.15.19:0/897252613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 msgr2=0x56013757b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548e59c700 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 0x56013757b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548e59c700 1 -- 172.21.15.19:0/897252613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013746ad00 msgr2=0x56013757ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548e59c700 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013746ad00 0x56013757ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548e59c700 1 -- 172.21.15.19:0/897252613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56013757b900 con 0x5601373f04a0 2024-05-09T01:33:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f548e59c700 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601373f04a0 0x56013757f820 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5480011790 tx=0x7f548001c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2f74c2fa7837f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f5475ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.742+0000 7f5475ffb700 1 -- 172.21.15.19:0/897252613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480039070 con 0x5601373f04a0 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f5475ffb700 1 -- 172.21.15.19:0/897252613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f548001cc30 con 0x5601373f04a0 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f54a3797b80 1 -- 172.21.15.19:0/897252613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560137497b20 con 0x5601373f04a0 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f5475ffb700 1 -- 172.21.15.19:0/897252613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5480045050 con 0x5601373f04a0 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f54a3797b80 10 client.17400 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f54a3797b80 1 -- 172.21.15.19:0/897252613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5601375f1760 con 0x5601373f04a0 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f5475ffb700 1 -- 172.21.15.19:0/897252613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560137497b20 con 0x5601373f04a0 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f5475ffb700 1 -- 172.21.15.19:0/897252613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5480005380 con 0x5601373f04a0 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f54a3797b80 10 client.17400 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f54a3797b80 10 client.17400 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f54a3797b80 20 client.17400 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f54a3797b80 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5601375f1b40 0x5601375f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f54a3797b80 4 client.17400 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.743+0000 7f54a3797b80 1 -- 172.21.15.19:0/897252613 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56013730b970 con 0x5601375f1b40 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.744+0000 7f5485d9b700 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5601375f1b40 0x5601375f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.744+0000 7f5475ffb700 10 client.17400 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.744+0000 7f5485d9b700 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5601375f1b40 0x5601375f3f60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88d05270a669bde6 server_cookie=bd16d64d58466a0d in_seq=0 out_seq=0 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.744+0000 7f5475ffb700 1 -- 172.21.15.19:0/897252613 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56013730b970 con 0x5601375f1b40 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.744+0000 7f5475ffb700 10 client.17400 handle_command_reply: tid=0 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.744+0000 7f54a3797b80 1 client.17400 shutdown 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 -- 172.21.15.19:0/897252613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601373f04a0 msgr2=0x56013757f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601373f04a0 0x56013757f820 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5480011790 tx=0x7f548001c680 comp rx=0 tx=0).stop 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 -- 172.21.15.19:0/897252613 shutdown_connections 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5601375f1b40 0x5601375f3f60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56013756d400 0x56013757b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601373f04a0 0x56013757f820 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 --2- 172.21.15.19:0/897252613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56013746ad00 0x56013757ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 -- 172.21.15.19:0/897252613 >> 172.21.15.19:0/897252613 conn(0x5601373f3f70 msgr2=0x560137571460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f5475ffb700 0 client.17400 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:33:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 -- 172.21.15.19:0/897252613 shutdown_connections 2024-05-09T01:33:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 1 -- 172.21.15.19:0/897252613 wait complete. 2024-05-09T01:33:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f54a3797b80 20 client.17400 trim_cache size 0 max 16384 2024-05-09T01:33:13.747 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f5494d81700 1 -- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f547c0b2880 msgr2=0x7f547c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.745+0000 7f5494d81700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f547c0b2880 0x7f547c0b4d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5480005950 tx=0x7f548000b410 comp rx=0 tx=0).stop 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 -- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490108770 msgr2=0x7f54901a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490108770 0x7f54901a1870 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5488007ee0 tx=0x7f548800d3b0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 -- 172.21.15.19:0/1989475821 shutdown_connections 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f547c0b2880 0x7f547c0b4d90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5490108770 0x7f54901a1870 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5490103540 0x7f549019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 --2- 172.21.15.19:0/1989475821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5490107e20 0x7f549019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 -- 172.21.15.19:0/1989475821 >> 172.21.15.19:0/1989475821 conn(0x7f5490078e50 msgr2=0x7f549010ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 -- 172.21.15.19:0/1989475821 shutdown_connections 2024-05-09T01:33:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:13.746+0000 7f5494d81700 1 -- 172.21.15.19:0/1989475821 wait complete. 2024-05-09T01:33:14.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:33:14.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:14 smithi196 ceph-mon[35041]: pgmap v609: 97 pgs: 97 active+clean; 707 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:33:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:14 smithi059 ceph-mon[34351]: pgmap v609: 97 pgs: 97 active+clean; 707 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:33:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:14 smithi019 ceph-mon[30158]: pgmap v609: 97 pgs: 97 active+clean; 707 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:33:16.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.555+0000 7f249cf60700 1 -- 172.21.15.19:0/55339131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 msgr2=0x7f24981038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.555+0000 7f249cf60700 1 --2- 172.21.15.19:0/55339131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 0x7f24981038f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f248800b0d0 tx=0x7f248800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.555+0000 7f249cf60700 1 -- 172.21.15.19:0/55339131 shutdown_connections 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.555+0000 7f249cf60700 1 --2- 172.21.15.19:0/55339131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2498108760 0x7f24981100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.555+0000 7f249cf60700 1 --2- 172.21.15.19:0/55339131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 0x7f24981038f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.555+0000 7f249cf60700 1 --2- 172.21.15.19:0/55339131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2498107e10 0x7f2498108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.555+0000 7f249cf60700 1 -- 172.21.15.19:0/55339131 >> 172.21.15.19:0/55339131 conn(0x7f24980fef50 msgr2=0x7f2498101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.555+0000 7f249cf60700 1 -- 172.21.15.19:0/55339131 shutdown_connections 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.555+0000 7f249cf60700 1 -- 172.21.15.19:0/55339131 wait complete. 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249cf60700 1 Processor -- start 2024-05-09T01:33:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249cf60700 1 -- start start 2024-05-09T01:33:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249cf60700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 0x7f249819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249cf60700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2498107e10 0x7f249819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249cf60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2498108760 0x7f249819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249cf60700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24981144e0 con 0x7f24981034e0 2024-05-09T01:33:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249cf60700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2498114360 con 0x7f2498108760 2024-05-09T01:33:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249cf60700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2498114660 con 0x7f2498107e10 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 0x7f249819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 0x7f249819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42612/0 (socket says 172.21.15.19:42612) 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f249659c700 1 -- 172.21.15.19:0/2905963993 learned_addr learned my addr 172.21.15.19:0/2905963993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.556+0000 7f2495d9b700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2498107e10 0x7f249819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f2496d9d700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2498108760 0x7f249819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f249659c700 1 -- 172.21.15.19:0/2905963993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2498107e10 msgr2=0x7f249819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f249659c700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2498107e10 0x7f249819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f249659c700 1 -- 172.21.15.19:0/2905963993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2498108760 msgr2=0x7f249819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f249659c700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2498108760 0x7f249819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f249659c700 1 -- 172.21.15.19:0/2905963993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249819fce0 con 0x7f24981034e0 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f249659c700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 0x7f249819ae10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2488007a20 tx=0x7f2488007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f248f7fe700 1 -- 172.21.15.19:0/2905963993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248801f070 con 0x7f24981034e0 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f248f7fe700 1 -- 172.21.15.19:0/2905963993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2488004c50 con 0x7f24981034e0 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f249cf60700 1 -- 172.21.15.19:0/2905963993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249819ff70 con 0x7f24981034e0 2024-05-09T01:33:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f248f7fe700 1 -- 172.21.15.19:0/2905963993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248801b720 con 0x7f24981034e0 2024-05-09T01:33:16.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.557+0000 7f249cf60700 1 -- 172.21.15.19:0/2905963993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24981a0450 con 0x7f24981034e0 2024-05-09T01:33:16.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.560+0000 7f248f7fe700 1 -- 172.21.15.19:0/2905963993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2488022070 con 0x7f24981034e0 2024-05-09T01:33:16.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.561+0000 7f248f7fe700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24840b2880 0x7f24840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.561+0000 7f2495d9b700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24840b2880 0x7f24840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.561+0000 7f248f7fe700 1 -- 172.21.15.19:0/2905963993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f248800f020 con 0x7f24981034e0 2024-05-09T01:33:16.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.561+0000 7f24ab976b80 1 Processor -- start 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.561+0000 7f24ab976b80 1 -- start start 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.561+0000 7f2495d9b700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24840b2880 0x7f24840b4d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2498102a10 tx=0x7f248000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.561+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x5594859844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485984a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948597ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f24ab976b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5594858a0ff0 con 0x55948597e6a0 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f24ab976b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55948583f7f0 con 0x5594858014a0 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f24ab976b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55948589cfb0 con 0x5594858eb160 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948597ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948597ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42628/0 (socket says 172.21.15.19:42628) 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2496d9d700 1 -- 172.21.15.19:0/3647389971 learned_addr learned my addr 172.21.15.19:0/3647389971 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2495d9b700 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485984a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f249659c700 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x5594859844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2496d9d700 1 -- 172.21.15.19:0/3647389971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 msgr2=0x5594859844f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2496d9d700 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x5594859844f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2496d9d700 1 -- 172.21.15.19:0/3647389971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 msgr2=0x559485984a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2496d9d700 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485984a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2496d9d700 1 -- 172.21.15.19:0/3647389971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559485837370 con 0x55948597e6a0 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2495d9b700 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485984a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f2496d9d700 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948597ead0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2490009c80 tx=0x7f2490009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b72dee63776c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f248cff9700 1 -- 172.21.15.19:0/3647389971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490010c90 con 0x55948597e6a0 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.562+0000 7f248cff9700 1 -- 172.21.15.19:0/3647389971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2490010e30 con 0x55948597e6a0 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.563+0000 7f248cff9700 1 -- 172.21.15.19:0/3647389971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490011800 con 0x55948597e6a0 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.563+0000 7f24ab976b80 1 -- 172.21.15.19:0/3647389971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 msgr2=0x55948597ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.563+0000 7f24ab976b80 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948597ead0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2490009c80 tx=0x7f2490009f90 comp rx=0 tx=0).stop 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.563+0000 7f24ab976b80 1 -- 172.21.15.19:0/3647389971 shutdown_connections 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.563+0000 7f24ab976b80 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948597ead0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.563+0000 7f24ab976b80 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x5594859844f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.563+0000 7f24ab976b80 1 --2- 172.21.15.19:0/3647389971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485984a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.563+0000 7f24ab976b80 1 -- 172.21.15.19:0/3647389971 >> 172.21.15.19:0/3647389971 conn(0x5594858616a0 msgr2=0x559485979400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 -- 172.21.15.19:0/3647389971 shutdown_connections 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 -- 172.21.15.19:0/3647389971 wait complete. 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 -- start start 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 Processor -- start 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594858014a0 0x5594859880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858eb160 0x559485988620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55948597e6a0 0x559485988b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55948571cba0 con 0x5594858014a0 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55948585b150 con 0x5594858eb160 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f24ab976b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55948598b1d0 con 0x55948597e6a0 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.564+0000 7f249659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594858014a0 0x5594859880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f249659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594858014a0 0x5594859880e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42632/0 (socket says 172.21.15.19:42632) 2024-05-09T01:33:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f249659c700 1 -- 172.21.15.19:0/351210917 learned_addr learned my addr 172.21.15.19:0/351210917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f24ab976b80 10 client.0 _ll_register_callbacks cb 0x5594857126c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f2495d9b700 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858eb160 0x559485988620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f2496d9d700 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55948597e6a0 0x559485988b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f249659c700 1 -- 172.21.15.19:0/351210917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55948597e6a0 msgr2=0x559485988b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f249659c700 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55948597e6a0 0x559485988b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f249659c700 1 -- 172.21.15.19:0/351210917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858eb160 msgr2=0x559485988620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f249659c700 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858eb160 0x559485988620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f249659c700 1 -- 172.21.15.19:0/351210917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55948571c490 con 0x5594858014a0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f249659c700 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594858014a0 0x5594859880e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f24880090a0 tx=0x7f24880a62f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d2a9b8fdc285b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f247dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f247dffb700 1 -- 172.21.15.19:0/351210917 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24880c4060 con 0x5594858014a0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f247dffb700 1 -- 172.21.15.19:0/351210917 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f24880cd2d0 con 0x5594858014a0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f24ab976b80 1 -- 172.21.15.19:0/351210917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2490010e30 con 0x5594858014a0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f247dffb700 1 -- 172.21.15.19:0/351210917 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24880b3190 con 0x5594858014a0 2024-05-09T01:33:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f24ab976b80 10 client.17412 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.565+0000 7f24ab976b80 1 -- 172.21.15.19:0/351210917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559485995b80 con 0x5594858014a0 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.566+0000 7f247dffb700 1 -- 172.21.15.19:0/351210917 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2490010e30 con 0x5594858014a0 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.566+0000 7f247dffb700 1 -- 172.21.15.19:0/351210917 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2488011070 con 0x5594858014a0 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.566+0000 7f24ab976b80 10 client.17412 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.566+0000 7f24ab976b80 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.566+0000 7f24ab976b80 20 client.17412 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.566+0000 7f24ab976b80 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559485998190 0x55948599a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.566+0000 7f24ab976b80 4 client.17412 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.566+0000 7f24ab976b80 1 -- 172.21.15.19:0/351210917 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594858ae9d0 con 0x559485998190 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.566+0000 7f2495d9b700 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559485998190 0x55948599a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.567+0000 7f247dffb700 10 client.17412 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.567+0000 7f2495d9b700 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559485998190 0x55948599a5d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc7cb6ceedae9dd server_cookie=6ed77d70cf7ccf9b in_seq=0 out_seq=0 2024-05-09T01:33:16.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.567+0000 7f247dffb700 1 -- 172.21.15.19:0/351210917 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594858ae9d0 con 0x559485998190 2024-05-09T01:33:16.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.567+0000 7f247dffb700 10 client.17412 handle_command_reply: tid=0 2024-05-09T01:33:16.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.567+0000 7f24ab976b80 1 client.17412 shutdown 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 -- 172.21.15.19:0/351210917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594858014a0 msgr2=0x5594859880e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594858014a0 0x5594859880e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f24880090a0 tx=0x7f24880a62f0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 -- 172.21.15.19:0/351210917 shutdown_connections 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559485998190 0x55948599a5d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55948597e6a0 0x559485988b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f247dffb700 0 client.17412 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5594858014a0 0x5594859880e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 --2- 172.21.15.19:0/351210917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858eb160 0x559485988620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 -- 172.21.15.19:0/351210917 >> 172.21.15.19:0/351210917 conn(0x5594858616a0 msgr2=0x5594859791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 -- 172.21.15.19:0/351210917 shutdown_connections 2024-05-09T01:33:16.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 1 -- 172.21.15.19:0/351210917 wait complete. 2024-05-09T01:33:16.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.568+0000 7f24ab976b80 20 client.17412 trim_cache size 0 max 16384 2024-05-09T01:33:16.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f24ab976b80 1 Processor -- start 2024-05-09T01:33:16.580 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:16 smithi019 ceph-mon[30158]: pgmap v610: 97 pgs: 97 active+clean; 777 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:33:16.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f24ab976b80 1 -- start start 2024-05-09T01:33:16.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485982620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x559485980e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x5594859837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f24ab976b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24981aecd0 con 0x55948597e6a0 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f24ab976b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24981b0810 con 0x5594858014a0 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f24ab976b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24981af3a0 con 0x5594858eb160 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f2496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x5594859837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f2496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x5594859837f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42648/0 (socket says 172.21.15.19:42648) 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f2496d9d700 1 -- 172.21.15.19:0/3263581505 learned_addr learned my addr 172.21.15.19:0/3263581505 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.583+0000 7f249659c700 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485982620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f2495d9b700 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x559485980e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f2496d9d700 1 -- 172.21.15.19:0/3263581505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 msgr2=0x559485980e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f2496d9d700 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x559485980e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f2496d9d700 1 -- 172.21.15.19:0/3263581505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 msgr2=0x559485982620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:16.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f2496d9d700 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485982620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f2496d9d700 1 -- 172.21.15.19:0/3263581505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948597ad00 con 0x55948597e6a0 2024-05-09T01:33:16.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f2495d9b700 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x559485980e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:16.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f249659c700 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485982620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:16.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f2496d9d700 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x5594859837f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x5594859850c0 tx=0x7f2490010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65859364b33236d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:16.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f247ffff700 1 -- 172.21.15.19:0/3263581505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249002b070 con 0x55948597e6a0 2024-05-09T01:33:16.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f247ffff700 1 -- 172.21.15.19:0/3263581505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24900054f0 con 0x55948597e6a0 2024-05-09T01:33:16.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.584+0000 7f247ffff700 1 -- 172.21.15.19:0/3263581505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490003e50 con 0x55948597e6a0 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.585+0000 7f24ab976b80 1 -- 172.21.15.19:0/3263581505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 msgr2=0x5594859837f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.585+0000 7f24ab976b80 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x5594859837f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x5594859850c0 tx=0x7f2490010cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.585+0000 7f24ab976b80 1 -- 172.21.15.19:0/3263581505 shutdown_connections 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.585+0000 7f24ab976b80 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x5594859837f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.585+0000 7f24ab976b80 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x559485982620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.585+0000 7f24ab976b80 1 --2- 172.21.15.19:0/3263581505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x559485980e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.585+0000 7f24ab976b80 1 -- 172.21.15.19:0/3263581505 >> 172.21.15.19:0/3263581505 conn(0x5594858616a0 msgr2=0x5594859830e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.585+0000 7f24ab976b80 1 -- 172.21.15.19:0/3263581505 shutdown_connections 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.585+0000 7f24ab976b80 1 -- 172.21.15.19:0/3263581505 wait complete. 2024-05-09T01:33:16.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f24ab976b80 1 -- start start 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f24ab976b80 1 Processor -- start 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x55948599ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x55948599f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f24ab976b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948599f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f24ab976b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24981b0140 con 0x55948597e6a0 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f24ab976b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24981afa70 con 0x5594858014a0 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f24ab976b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24981acc70 con 0x5594858eb160 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f24ab976b80 10 client.0 _ll_register_callbacks cb 0x5594857126c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f249659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x55948599ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f2496d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948599f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f249659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x55948599ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:58734/0 (socket says 172.21.15.19:58734) 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.586+0000 7f249659c700 1 -- 172.21.15.19:0/1182047727 learned_addr learned my addr 172.21.15.19:0/1182047727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f2495d9b700 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x55948599f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f249659c700 1 -- 172.21.15.19:0/1182047727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 msgr2=0x55948599f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f249659c700 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x55948599f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f249659c700 1 -- 172.21.15.19:0/1182047727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 msgr2=0x55948599f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f249659c700 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948599f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f249659c700 1 -- 172.21.15.19:0/1182047727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55948598f500 con 0x5594858014a0 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f2496d9d700 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948599f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f249659c700 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x55948599ee20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f24880090a0 tx=0x7f2488005950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bdc67c5f6d10186 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f247dffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f247dffb700 1 -- 172.21.15.19:0/1182047727 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24880aa670 con 0x5594858014a0 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f247dffb700 1 -- 172.21.15.19:0/1182047727 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f24880b1210 con 0x5594858014a0 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f24ab976b80 1 -- 172.21.15.19:0/1182047727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24900054f0 con 0x5594858014a0 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f247dffb700 1 -- 172.21.15.19:0/1182047727 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24880b3850 con 0x5594858014a0 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f24ab976b80 10 client.26828 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.587+0000 7f24ab976b80 1 -- 172.21.15.19:0/1182047727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559485990720 con 0x5594858014a0 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.588+0000 7f247dffb700 1 -- 172.21.15.19:0/1182047727 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24900054f0 con 0x5594858014a0 2024-05-09T01:33:16.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.588+0000 7f247dffb700 1 -- 172.21.15.19:0/1182047727 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2488011040 con 0x5594858014a0 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.588+0000 7f24ab976b80 10 client.26828 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.588+0000 7f24ab976b80 10 client.26828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.588+0000 7f24ab976b80 20 client.26828 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.588+0000 7f24ab976b80 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559485990bf0 0x559485a0d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.588+0000 7f24ab976b80 4 client.26828 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.588+0000 7f24ab976b80 1 -- 172.21.15.19:0/1182047727 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594858a8a70 con 0x559485990bf0 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.588+0000 7f2495d9b700 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559485990bf0 0x559485a0d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f2495d9b700 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559485990bf0 0x559485a0d550 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58670afb4869407a server_cookie=4e441cbefa676317 in_seq=0 out_seq=0 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f247dffb700 10 client.26828 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f247dffb700 1 -- 172.21.15.19:0/1182047727 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594858a8a70 con 0x559485990bf0 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f247dffb700 10 client.26828 handle_command_reply: tid=0 2024-05-09T01:33:16.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 client.26828 shutdown 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 -- 172.21.15.19:0/1182047727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 msgr2=0x55948599ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x55948599ee20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f24880090a0 tx=0x7f2488005950 comp rx=0 tx=0).stop 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 -- 172.21.15.19:0/1182047727 shutdown_connections 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559485990bf0 0x559485a0d550 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55948597e6a0 0x55948599f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5594858014a0 0x55948599ee20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f247dffb700 0 client.26828 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 --2- 172.21.15.19:0/1182047727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594858eb160 0x55948599f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 -- 172.21.15.19:0/1182047727 >> 172.21.15.19:0/1182047727 conn(0x5594858616a0 msgr2=0x559485981e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.589+0000 7f24ab976b80 1 -- 172.21.15.19:0/1182047727 shutdown_connections 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f24ab976b80 1 -- 172.21.15.19:0/1182047727 wait complete. 2024-05-09T01:33:16.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f24ab976b80 20 client.26828 trim_cache size 0 max 16384 2024-05-09T01:33:16.592 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:16.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 -- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24840b2880 msgr2=0x7f24840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:16.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24840b2880 0x7f24840b4d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2498102a10 tx=0x7f248000b410 comp rx=0 tx=0).stop 2024-05-09T01:33:16.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 -- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 msgr2=0x7f249819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 0x7f249819ae10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2488007a20 tx=0x7f2488007a50 comp rx=0 tx=0).stop 2024-05-09T01:33:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 -- 172.21.15.19:0/2905963993 shutdown_connections 2024-05-09T01:33:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2498108760 0x7f249819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f24840b2880 0x7f24840b4d90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24981034e0 0x7f249819ae10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 --2- 172.21.15.19:0/2905963993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2498107e10 0x7f249819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.590+0000 7f249cf60700 1 -- 172.21.15.19:0/2905963993 >> 172.21.15.19:0/2905963993 conn(0x7f24980fef50 msgr2=0x7f2498107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.591+0000 7f249cf60700 1 -- 172.21.15.19:0/2905963993 shutdown_connections 2024-05-09T01:33:16.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:16.591+0000 7f249cf60700 1 -- 172.21.15.19:0/2905963993 wait complete. 2024-05-09T01:33:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:16 smithi196 ceph-mon[35041]: pgmap v610: 97 pgs: 97 active+clean; 777 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:33:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:16 smithi059 ceph-mon[34351]: pgmap v610: 97 pgs: 97 active+clean; 777 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:33:18.349 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:33:18.350 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:33:18.618 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:18.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:18 smithi019 ceph-mon[30158]: pgmap v611: 97 pgs: 97 active+clean; 777 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:33:18.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:18 smithi196 ceph-mon[35041]: pgmap v611: 97 pgs: 97 active+clean; 777 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:33:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:18 smithi059 ceph-mon[34351]: pgmap v611: 97 pgs: 97 active+clean; 777 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:33:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.361+0000 7fec71771700 1 -- 172.21.15.19:0/3378827770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c107df0 msgr2=0x7fec6c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.361+0000 7fec71771700 1 --2- 172.21.15.19:0/3378827770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c107df0 0x7fec6c108200 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fec600099c0 tx=0x7fec60009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.361+0000 7fec71771700 1 -- 172.21.15.19:0/3378827770 shutdown_connections 2024-05-09T01:33:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.361+0000 7fec71771700 1 --2- 172.21.15.19:0/3378827770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c108740 0x7fec6c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.361+0000 7fec71771700 1 --2- 172.21.15.19:0/3378827770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6c1034c0 0x7fec6c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.361+0000 7fec71771700 1 --2- 172.21.15.19:0/3378827770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c107df0 0x7fec6c108200 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.361+0000 7fec71771700 1 -- 172.21.15.19:0/3378827770 >> 172.21.15.19:0/3378827770 conn(0x7fec6c0fedf0 msgr2=0x7fec6c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.361+0000 7fec71771700 1 -- 172.21.15.19:0/3378827770 shutdown_connections 2024-05-09T01:33:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.361+0000 7fec71771700 1 -- 172.21.15.19:0/3378827770 wait complete. 2024-05-09T01:33:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec71771700 1 Processor -- start 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec71771700 1 -- start start 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec71771700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c1034c0 0x7fec6c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec71771700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6c107df0 0x7fec6c19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec71771700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c108740 0x7fec6c1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec71771700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c114480 con 0x7fec6c108740 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec71771700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c114300 con 0x7fec6c1034c0 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec71771700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c114600 con 0x7fec6c107df0 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec6affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c1034c0 0x7fec6c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c108740 0x7fec6c1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec6affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c1034c0 0x7fec6c19d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52888/0 (socket says 172.21.15.19:52888) 2024-05-09T01:33:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec6a7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6c107df0 0x7fec6c19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c108740 0x7fec6c1a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55000/0 (socket says 172.21.15.19:55000) 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.362+0000 7fec6affd700 1 -- 172.21.15.19:0/2751280850 learned_addr learned my addr 172.21.15.19:0/2751280850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec6affd700 1 -- 172.21.15.19:0/2751280850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6c107df0 msgr2=0x7fec6c19d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec6affd700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6c107df0 0x7fec6c19d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec6affd700 1 -- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c108740 msgr2=0x7fec6c1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec6affd700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c108740 0x7fec6c1a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec6affd700 1 -- 172.21.15.19:0/2751280850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c1a1ee0 con 0x7fec6c1034c0 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec6b7fe700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c108740 0x7fec6c1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec6affd700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c1034c0 0x7fec6c19d010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d950 tx=0x7fec5c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec53fff700 1 -- 172.21.15.19:0/2751280850 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c016070 con 0x7fec6c1034c0 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec71771700 1 -- 172.21.15.19:0/2751280850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6c1a21d0 con 0x7fec6c1034c0 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec71771700 1 -- 172.21.15.19:0/2751280850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6c1a9af0 con 0x7fec6c1034c0 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec53fff700 1 -- 172.21.15.19:0/2751280850 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c00fd40 con 0x7fec6c1034c0 2024-05-09T01:33:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.363+0000 7fec53fff700 1 -- 172.21.15.19:0/2751280850 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c020060 con 0x7fec6c1034c0 2024-05-09T01:33:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.366+0000 7fec53fff700 1 -- 172.21.15.19:0/2751280850 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fec5c019070 con 0x7fec6c1034c0 2024-05-09T01:33:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec80187b80 1 Processor -- start 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec80187b80 1 -- start start 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec53fff700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fec540b2880 0x7fec540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec80187b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df772c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec80187b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df77800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec6b7fe700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fec540b2880 0x7fec540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec80187b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec80187b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a85de94120 con 0x55a85df71480 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec80187b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a85de39a20 con 0x55a85ddf4520 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec80187b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a85de900e0 con 0x55a85de6ed80 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec53fff700 1 -- 172.21.15.19:0/2751280850 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fec5c0e1c40 con 0x7fec6c1034c0 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.367+0000 7fec6a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df77800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df77800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52904/0 (socket says 172.21.15.19:52904) 2024-05-09T01:33:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6a7fc700 1 -- 172.21.15.19:0/1958561581 learned_addr learned my addr 172.21.15.19:0/1958561581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6a7fc700 1 -- 172.21.15.19:0/1958561581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 msgr2=0x55a85df772c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6a7fc700 1 --2- 172.21.15.19:0/1958561581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df772c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6a7fc700 1 -- 172.21.15.19:0/1958561581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 msgr2=0x55a85df71890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6a7fc700 1 --2- 172.21.15.19:0/1958561581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df71890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6a7fc700 1 -- 172.21.15.19:0/1958561581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85de2a580 con 0x55a85ddf4520 2024-05-09T01:33:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6b7fe700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fec540b2880 0x7fec540b4d90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fec64004230 tx=0x7fec640045f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec6a7fc700 1 --2- 172.21.15.19:0/1958561581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df77800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec600099c0 tx=0x7fec60005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=570304753228ca51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.368+0000 7fec517fa700 1 -- 172.21.15.19:0/1958561581 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60027070 con 0x55a85ddf4520 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec517fa700 1 -- 172.21.15.19:0/1958561581 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec60005920 con 0x55a85ddf4520 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec517fa700 1 -- 172.21.15.19:0/1958561581 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6000b9f0 con 0x55a85ddf4520 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec80187b80 1 -- 172.21.15.19:0/1958561581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 msgr2=0x55a85df77800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec80187b80 1 --2- 172.21.15.19:0/1958561581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df77800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec600099c0 tx=0x7fec60005040 comp rx=0 tx=0).stop 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec80187b80 1 -- 172.21.15.19:0/1958561581 shutdown_connections 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec80187b80 1 --2- 172.21.15.19:0/1958561581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df71890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec80187b80 1 --2- 172.21.15.19:0/1958561581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec80187b80 1 --2- 172.21.15.19:0/1958561581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df77800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec80187b80 1 -- 172.21.15.19:0/1958561581 >> 172.21.15.19:0/1958561581 conn(0x55a85ddf7ff0 msgr2=0x55a85df6c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec80187b80 1 -- 172.21.15.19:0/1958561581 shutdown_connections 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.369+0000 7fec80187b80 1 -- 172.21.15.19:0/1958561581 wait complete. 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.370+0000 7fec80187b80 1 -- start start 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.370+0000 7fec80187b80 1 Processor -- start 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.370+0000 7fec80187b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df7afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.370+0000 7fec80187b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df7b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.370+0000 7fec80187b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df7ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.370+0000 7fec80187b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a85df72180 con 0x55a85df71480 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.370+0000 7fec80187b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a85de2eba0 con 0x55a85ddf4520 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.370+0000 7fec80187b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a85df872e0 con 0x55a85de6ed80 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.370+0000 7fec80187b80 10 client.0 _ll_register_callbacks cb 0x55a85dd05740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df7ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df7ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55014/0 (socket says 172.21.15.19:55014) 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6b7fe700 1 -- 172.21.15.19:0/1872271080 learned_addr learned my addr 172.21.15.19:0/1872271080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6affd700 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df7afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6a7fc700 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df7b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6b7fe700 1 -- 172.21.15.19:0/1872271080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 msgr2=0x55a85df7b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6b7fe700 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df7b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6b7fe700 1 -- 172.21.15.19:0/1872271080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 msgr2=0x55a85df7afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6b7fe700 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df7afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6b7fe700 1 -- 172.21.15.19:0/1872271080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85dd0f510 con 0x55a85df71480 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec6b7fe700 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df7ba40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fec6400efd0 tx=0x7fec64004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf344e83a96ff2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec4eff5700 1 -- 172.21.15.19:0/1872271080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64027030 con 0x55a85df71480 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec4eff5700 1 -- 172.21.15.19:0/1872271080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fec6402c410 con 0x55a85df71480 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.371+0000 7fec80187b80 1 -- 172.21.15.19:0/1872271080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec60005920 con 0x55a85df71480 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec4eff5700 1 -- 172.21.15.19:0/1872271080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec64032070 con 0x55a85df71480 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec80187b80 10 client.17430 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec80187b80 1 -- 172.21.15.19:0/1872271080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a85df88b30 con 0x55a85df71480 2024-05-09T01:33:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec4eff5700 1 -- 172.21.15.19:0/1872271080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec60005920 con 0x55a85df71480 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec4eff5700 1 -- 172.21.15.19:0/1872271080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fec64036460 con 0x55a85df71480 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec80187b80 10 client.17430 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec80187b80 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec80187b80 20 client.17430 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec80187b80 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a85df8b210 0x55a85df8d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec80187b80 4 client.17430 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec80187b80 1 -- 172.21.15.19:0/1872271080 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a85dea1b00 con 0x55a85df8b210 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.372+0000 7fec6affd700 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a85df8b210 0x55a85df8d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.373+0000 7fec6affd700 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a85df8b210 0x55a85df8d690 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cbb0f8dc709f13c server_cookie=428703b651ee7f8 in_seq=0 out_seq=0 2024-05-09T01:33:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.373+0000 7fec4eff5700 10 client.17430 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:20.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.373+0000 7fec4eff5700 1 -- 172.21.15.19:0/1872271080 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a85dea1b00 con 0x55a85df8b210 2024-05-09T01:33:20.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.373+0000 7fec4eff5700 10 client.17430 handle_command_reply: tid=0 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.373+0000 7fec80187b80 1 client.17430 shutdown 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 -- 172.21.15.19:0/1872271080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 msgr2=0x55a85df7ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df7ba40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fec6400efd0 tx=0x7fec64004b40 comp rx=0 tx=0).stop 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 -- 172.21.15.19:0/1872271080 shutdown_connections 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a85df8b210 0x55a85df8d690 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85df71480 0x55a85df7ba40 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85ddf4520 0x55a85df7afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec4eff5700 0 client.17430 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 --2- 172.21.15.19:0/1872271080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df7b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 -- 172.21.15.19:0/1872271080 >> 172.21.15.19:0/1872271080 conn(0x55a85ddf7ff0 msgr2=0x55a85df66070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 -- 172.21.15.19:0/1872271080 shutdown_connections 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 1 -- 172.21.15.19:0/1872271080 wait complete. 2024-05-09T01:33:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.374+0000 7fec80187b80 20 client.17430 trim_cache size 0 max 16384 2024-05-09T01:33:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec80187b80 1 Processor -- start 2024-05-09T01:33:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec80187b80 1 -- start start 2024-05-09T01:33:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec80187b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df7b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec80187b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df92e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec80187b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 0x55a85df93390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec80187b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec5800d420 con 0x55a85ddf4520 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec80187b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec5800ef60 con 0x55a85df71480 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec80187b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec5800daf0 con 0x55a85de6ed80 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df7b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df7b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55024/0 (socket says 172.21.15.19:55024) 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6affd700 1 -- 172.21.15.19:0/1397208919 learned_addr learned my addr 172.21.15.19:0/1397208919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6b7fe700 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 0x55a85df93390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6a7fc700 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df92e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6affd700 1 -- 172.21.15.19:0/1397208919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 msgr2=0x55a85df92e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6affd700 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df92e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6affd700 1 -- 172.21.15.19:0/1397208919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 msgr2=0x55a85df93390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6affd700 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 0x55a85df93390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6affd700 1 -- 172.21.15.19:0/1397208919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85df7dfa0 con 0x55a85ddf4520 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.395+0000 7fec6a7fc700 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df92e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec6affd700 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df7b9b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0a9170 tx=0x7fec5c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c417cf3bddf0b354 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec50ff9700 1 -- 172.21.15.19:0/1397208919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0b2040 con 0x55a85ddf4520 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec50ff9700 1 -- 172.21.15.19:0/1397208919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c08f750 con 0x55a85ddf4520 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec50ff9700 1 -- 172.21.15.19:0/1397208919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0ad120 con 0x55a85ddf4520 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec80187b80 1 -- 172.21.15.19:0/1397208919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 msgr2=0x55a85df7b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec80187b80 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df7b9b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0a9170 tx=0x7fec5c010000 comp rx=0 tx=0).stop 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec80187b80 1 -- 172.21.15.19:0/1397208919 shutdown_connections 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec80187b80 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 0x55a85df93390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec80187b80 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df7b9b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec80187b80 1 --2- 172.21.15.19:0/1397208919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df92e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.396+0000 7fec80187b80 1 -- 172.21.15.19:0/1397208919 >> 172.21.15.19:0/1397208919 conn(0x55a85ddf7ff0 msgr2=0x55a85df666e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 -- 172.21.15.19:0/1397208919 shutdown_connections 2024-05-09T01:33:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 -- 172.21.15.19:0/1397208919 wait complete. 2024-05-09T01:33:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 -- start start 2024-05-09T01:33:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 Processor -- start 2024-05-09T01:33:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df80b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 0x55a85dff5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec5800e890 con 0x55a85ddf4520 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec5800e1c0 con 0x55a85df71480 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec5800b3c0 con 0x55a85de6ed80 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec80187b80 10 client.0 _ll_register_callbacks cb 0x55a85dd05740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df805f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55036/0 (socket says 172.21.15.19:55036) 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.397+0000 7fec6affd700 1 -- 172.21.15.19:0/3943412576 learned_addr learned my addr 172.21.15.19:0/3943412576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec6b7fe700 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 0x55a85dff5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec6a7fc700 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df80b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec6affd700 1 -- 172.21.15.19:0/3943412576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 msgr2=0x55a85df80b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec6affd700 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df80b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec6affd700 1 -- 172.21.15.19:0/3943412576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 msgr2=0x55a85dff5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec6affd700 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 0x55a85dff5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec6affd700 1 -- 172.21.15.19:0/3943412576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85df7bfd0 con 0x55a85ddf4520 2024-05-09T01:33:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec6affd700 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df805f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00ffa0 tx=0x7fec5c005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d898d004ceb59c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec4eff5700 1 -- 172.21.15.19:0/3943412576 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0a9440 con 0x55a85ddf4520 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec4eff5700 1 -- 172.21.15.19:0/3943412576 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fec5c0a95e0 con 0x55a85ddf4520 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec80187b80 1 -- 172.21.15.19:0/3943412576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5c08f750 con 0x55a85ddf4520 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec4eff5700 1 -- 172.21.15.19:0/3943412576 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec5c0b1bb0 con 0x55a85ddf4520 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec80187b80 10 client.17436 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.398+0000 7fec80187b80 1 -- 172.21.15.19:0/3943412576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a85dff6f70 con 0x55a85ddf4520 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.399+0000 7fec4eff5700 1 -- 172.21.15.19:0/3943412576 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c08f750 con 0x55a85ddf4520 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.399+0000 7fec4eff5700 1 -- 172.21.15.19:0/3943412576 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fec5c0a8440 con 0x55a85ddf4520 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.399+0000 7fec80187b80 10 client.17436 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.399+0000 7fec80187b80 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.399+0000 7fec80187b80 20 client.17436 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.399+0000 7fec80187b80 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a85dff7610 0x55a85dff9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.399+0000 7fec80187b80 4 client.17436 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:33:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.399+0000 7fec80187b80 1 -- 172.21.15.19:0/3943412576 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a85de9bba0 con 0x55a85dff7610 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.399+0000 7fec6a7fc700 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a85dff7610 0x55a85dff9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.400+0000 7fec4eff5700 10 client.17436 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.400+0000 7fec6a7fc700 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a85dff7610 0x55a85dff9a30 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c03922f4cd8a9bf server_cookie=bf5be804584e3047 in_seq=0 out_seq=0 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec4eff5700 1 -- 172.21.15.19:0/3943412576 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a85de9bba0 con 0x55a85dff7610 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec4eff5700 10 client.17436 handle_command_reply: tid=0 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 client.17436 shutdown 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 -- 172.21.15.19:0/3943412576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 msgr2=0x55a85df805f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df805f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00ffa0 tx=0x7fec5c005910 comp rx=0 tx=0).stop 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 -- 172.21.15.19:0/3943412576 shutdown_connections 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a85dff7610 0x55a85dff9a30 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a85df71480 0x55a85dff5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a85ddf4520 0x55a85df805f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 --2- 172.21.15.19:0/3943412576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a85de6ed80 0x55a85df80b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 -- 172.21.15.19:0/3943412576 >> 172.21.15.19:0/3943412576 conn(0x55a85ddf7ff0 msgr2=0x55a85df70470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec4eff5700 0 client.17436 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 -- 172.21.15.19:0/3943412576 shutdown_connections 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.401+0000 7fec80187b80 1 -- 172.21.15.19:0/3943412576 wait complete. 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec80187b80 20 client.17436 trim_cache size 0 max 16384 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "bb7acb9c-d41a-4744-809b-7dfed9c8ca7e", 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:33:20.403 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:33:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 -- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fec540b2880 msgr2=0x7fec540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fec540b2880 0x7fec540b4d90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fec64004230 tx=0x7fec640045f0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 -- 172.21.15.19:0/2751280850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c1034c0 msgr2=0x7fec6c19d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c1034c0 0x7fec6c19d010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d950 tx=0x7fec5c00dc60 comp rx=0 tx=0).stop 2024-05-09T01:33:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 -- 172.21.15.19:0/2751280850 shutdown_connections 2024-05-09T01:33:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fec540b2880 0x7fec540b4d90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c108740 0x7fec6c1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c1034c0 0x7fec6c19d010 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 --2- 172.21.15.19:0/2751280850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6c107df0 0x7fec6c19d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 -- 172.21.15.19:0/2751280850 >> 172.21.15.19:0/2751280850 conn(0x7fec6c0fedf0 msgr2=0x7fec6c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 -- 172.21.15.19:0/2751280850 shutdown_connections 2024-05-09T01:33:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:20.402+0000 7fec71771700 1 -- 172.21.15.19:0/2751280850 wait complete. 2024-05-09T01:33:20.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:20 smithi019 ceph-mon[30158]: pgmap v612: 97 pgs: 97 active+clean; 840 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:33:20.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:20 smithi196 ceph-mon[35041]: pgmap v612: 97 pgs: 97 active+clean; 840 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:33:20.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:20 smithi059 ceph-mon[34351]: pgmap v612: 97 pgs: 97 active+clean; 840 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:33:21.217 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:33:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:21.597 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:21 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:33:21.597 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:21 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:21.597 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:21 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:33:21.597 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:21 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:21.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:21 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:33:21.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:21 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:21.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:21 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:33:21.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:21 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:21.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:21 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:33:21.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:21 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:21.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:21 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:33:21.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:21 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:22.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:22 smithi019 ceph-mon[30158]: pgmap v613: 97 pgs: 97 active+clean; 840 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:33:22.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:22 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:33:22.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:22 smithi196 ceph-mon[35041]: pgmap v613: 97 pgs: 97 active+clean; 840 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:33:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:22 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:33:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:22 smithi059 ceph-mon[34351]: pgmap v613: 97 pgs: 97 active+clean; 840 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:33:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:22 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:33:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 -- 172.21.15.19:0/3036845607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c104730 msgr2=0x7f0c8c106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 --2- 172.21.15.19:0/3036845607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c104730 0x7f0c8c106b50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009900 tx=0x7f0c8802a410 comp rx=0 tx=0).stop 2024-05-09T01:33:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 -- 172.21.15.19:0/3036845607 shutdown_connections 2024-05-09T01:33:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 --2- 172.21.15.19:0/3036845607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c1099f0 0x7f0c8c10be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 --2- 172.21.15.19:0/3036845607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c107090 0x7f0c8c1094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 --2- 172.21.15.19:0/3036845607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c104730 0x7f0c8c106b50 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 -- 172.21.15.19:0/3036845607 >> 172.21.15.19:0/3036845607 conn(0x7f0c8c0fdd30 msgr2=0x7f0c8c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 -- 172.21.15.19:0/3036845607 shutdown_connections 2024-05-09T01:33:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 -- 172.21.15.19:0/3036845607 wait complete. 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.959+0000 7f0c9355e640 1 Processor -- start 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c9355e640 1 -- start start 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c9355e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c104730 0x7f0c8c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c9355e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c107090 0x7f0c8c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c9355e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c1099f0 0x7f0c8c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c9355e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c10f7a0 con 0x7f0c8c107090 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c9355e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c10f620 con 0x7f0c8c104730 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c9355e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c10f920 con 0x7f0c8c1099f0 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c90af9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c107090 0x7f0c8c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c90af9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c107090 0x7f0c8c19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55068/0 (socket says 172.21.15.19:55068) 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c90af9640 1 -- 172.21.15.19:0/2195045501 learned_addr learned my addr 172.21.15.19:0/2195045501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c912fa640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c104730 0x7f0c8c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c91afb640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c1099f0 0x7f0c8c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c90af9640 1 -- 172.21.15.19:0/2195045501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c1099f0 msgr2=0x7f0c8c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c90af9640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c1099f0 0x7f0c8c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c90af9640 1 -- 172.21.15.19:0/2195045501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c104730 msgr2=0x7f0c8c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c90af9640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c104730 0x7f0c8c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c90af9640 1 -- 172.21.15.19:0/2195045501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c1a1f70 con 0x7f0c8c107090 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c90af9640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c107090 0x7f0c8c19a970 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00b4a0 tx=0x7f0c7c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.960+0000 7f0c827fc640 1 -- 172.21.15.19:0/2195045501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c014070 con 0x7f0c8c107090 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.961+0000 7f0c827fc640 1 -- 172.21.15.19:0/2195045501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c7c010ac0 con 0x7f0c8c107090 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.961+0000 7f0c9355e640 1 -- 172.21.15.19:0/2195045501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c1a2260 con 0x7f0c8c107090 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.961+0000 7f0c827fc640 1 -- 172.21.15.19:0/2195045501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c017c30 con 0x7f0c8c107090 2024-05-09T01:33:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.961+0000 7f0c9355e640 1 -- 172.21.15.19:0/2195045501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c1a2830 con 0x7f0c8c107090 2024-05-09T01:33:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.963+0000 7f0c827fc640 1 -- 172.21.15.19:0/2195045501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0c7c017070 con 0x7f0c8c107090 2024-05-09T01:33:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.963+0000 7f0c9a40e740 1 Processor -- start 2024-05-09T01:33:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c9a40e740 1 -- start start 2024-05-09T01:33:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 0x562425405500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425405a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x56242540c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5624253697c0 con 0x5624250382f0 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5624253050e0 con 0x5624250404d0 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562425365780 con 0x5624250611b0 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c912fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425405a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c912fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425405a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55074/0 (socket says 172.21.15.19:55074) 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c912fa640 1 -- 172.21.15.19:0/310407499 learned_addr learned my addr 172.21.15.19:0/310407499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c91afb640 1 --2- 172.21.15.19:0/310407499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 0x562425405500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c90af9640 1 --2- 172.21.15.19:0/310407499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x56242540c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c827fc640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0c780b27e0 0x7f0c780b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c912fa640 1 -- 172.21.15.19:0/310407499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 msgr2=0x562425405500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c912fa640 1 --2- 172.21.15.19:0/310407499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 0x562425405500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c912fa640 1 -- 172.21.15.19:0/310407499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 msgr2=0x56242540c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c912fa640 1 --2- 172.21.15.19:0/310407499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x56242540c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.964+0000 7f0c912fa640 1 -- 172.21.15.19:0/310407499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242532e7c0 con 0x5624250382f0 2024-05-09T01:33:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.965+0000 7f0c91afb640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0c780b27e0 0x7f0c780b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.965+0000 7f0c827fc640 1 -- 172.21.15.19:0/2195045501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0c7c0e1930 con 0x7f0c8c107090 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.965+0000 7f0c912fa640 1 --2- 172.21.15.19:0/310407499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425405a80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0c88031500 tx=0x7f0c88031c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdf948637a28318 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.965+0000 7f0c777fe640 1 -- 172.21.15.19:0/310407499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88035030 con 0x5624250382f0 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.965+0000 7f0c91afb640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0c780b27e0 0x7f0c780b4cf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5624254066d0 tx=0x7f0c84004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.965+0000 7f0c777fe640 1 -- 172.21.15.19:0/310407499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c88040070 con 0x5624250382f0 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.965+0000 7f0c777fe640 1 -- 172.21.15.19:0/310407499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88003b20 con 0x5624250382f0 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 -- 172.21.15.19:0/310407499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 msgr2=0x562425405a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/310407499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425405a80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0c88031500 tx=0x7f0c88031c50 comp rx=0 tx=0).stop 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 -- 172.21.15.19:0/310407499 shutdown_connections 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/310407499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x56242540c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/310407499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425405a80 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/310407499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 0x562425405500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 -- 172.21.15.19:0/310407499 >> 172.21.15.19:0/310407499 conn(0x562425055300 msgr2=0x562425400e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 -- 172.21.15.19:0/310407499 shutdown_connections 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 -- 172.21.15.19:0/310407499 wait complete. 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 -- start start 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.966+0000 7f0c9a40e740 1 Processor -- start 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250382f0 0x56242540f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250404d0 0x56242540f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x5624254166a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56242540d370 con 0x5624250611b0 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5624252f09c0 con 0x5624250382f0 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56242540d4f0 con 0x5624250404d0 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c9a40e740 10 client.0 _ll_register_callbacks cb 0x56242531d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c91afb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x5624254166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c91afb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x5624254166a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55082/0 (socket says 172.21.15.19:55082) 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c91afb640 1 -- 172.21.15.19:0/2156958782 learned_addr learned my addr 172.21.15.19:0/2156958782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c912fa640 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250382f0 0x56242540f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c90af9640 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250404d0 0x56242540f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c91afb640 1 -- 172.21.15.19:0/2156958782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250404d0 msgr2=0x56242540f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c91afb640 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250404d0 0x56242540f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c91afb640 1 -- 172.21.15.19:0/2156958782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250382f0 msgr2=0x56242540f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c91afb640 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250382f0 0x56242540f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.967+0000 7f0c91afb640 1 -- 172.21.15.19:0/2156958782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624253171f0 con 0x5624250611b0 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c91afb640 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x5624254166a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0c84013970 tx=0x7f0c84004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da78bb50eec40900 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c74ff9640 1 -- 172.21.15.19:0/2156958782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8401ed70 con 0x5624250611b0 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c74ff9640 1 -- 172.21.15.19:0/2156958782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0c84028470 con 0x5624250611b0 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c9a40e740 1 -- 172.21.15.19:0/2156958782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c88040070 con 0x5624250611b0 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c74ff9640 1 -- 172.21.15.19:0/2156958782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c84004b20 con 0x5624250611b0 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c9a40e740 10 client.17448 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c9a40e740 1 -- 172.21.15.19:0/2156958782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624250cc790 con 0x5624250611b0 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c74ff9640 1 -- 172.21.15.19:0/2156958782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c88040070 con 0x5624250611b0 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c74ff9640 1 -- 172.21.15.19:0/2156958782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0c8400f040 con 0x5624250611b0 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c9a40e740 10 client.17448 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c9a40e740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.968+0000 7f0c9a40e740 20 client.17448 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.969+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56242501fd80 0x56242541cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.969+0000 7f0c9a40e740 4 client.17448 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.969+0000 7f0c9a40e740 1 -- 172.21.15.19:0/2156958782 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624253773d0 con 0x56242501fd80 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.969+0000 7f0c912fa640 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56242501fd80 0x56242541cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.969+0000 7f0c74ff9640 10 client.17448 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.969+0000 7f0c912fa640 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56242501fd80 0x56242541cc10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5708a2aa93bf704 server_cookie=53786b06bf94e13e in_seq=0 out_seq=0 2024-05-09T01:33:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c74ff9640 1 -- 172.21.15.19:0/2156958782 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5624253773d0 con 0x56242501fd80 2024-05-09T01:33:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c74ff9640 10 client.17448 handle_command_reply: tid=0 2024-05-09T01:33:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 client.17448 shutdown 2024-05-09T01:33:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 -- 172.21.15.19:0/2156958782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 msgr2=0x5624254166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x5624254166a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0c84013970 tx=0x7f0c84004ab0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 -- 172.21.15.19:0/2156958782 shutdown_connections 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56242501fd80 0x56242541cc10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x5624254166a0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250404d0 0x56242540f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/2156958782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250382f0 0x56242540f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c74ff9640 0 client.17448 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 -- 172.21.15.19:0/2156958782 >> 172.21.15.19:0/2156958782 conn(0x562425055300 msgr2=0x5624253ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 -- 172.21.15.19:0/2156958782 shutdown_connections 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 1 -- 172.21.15.19:0/2156958782 wait complete. 2024-05-09T01:33:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.970+0000 7f0c9a40e740 20 client.17448 trim_cache size 0 max 16384 2024-05-09T01:33:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c9a40e740 1 Processor -- start 2024-05-09T01:33:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c9a40e740 1 -- start start 2024-05-09T01:33:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425400850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x562425400d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 0x5624254165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1ab5e0 con 0x5624250382f0 2024-05-09T01:33:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1ad120 con 0x5624250404d0 2024-05-09T01:33:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1abcb0 con 0x5624250611b0 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c912fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425400850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c912fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425400850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55090/0 (socket says 172.21.15.19:55090) 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c912fa640 1 -- 172.21.15.19:0/1886485393 learned_addr learned my addr 172.21.15.19:0/1886485393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c90af9640 1 --2- 172.21.15.19:0/1886485393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x562425400d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.987+0000 7f0c91afb640 1 --2- 172.21.15.19:0/1886485393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 0x5624254165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.988+0000 7f0c912fa640 1 -- 172.21.15.19:0/1886485393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 msgr2=0x5624254165c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.988+0000 7f0c912fa640 1 --2- 172.21.15.19:0/1886485393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 0x5624254165c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.988+0000 7f0c912fa640 1 -- 172.21.15.19:0/1886485393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 msgr2=0x562425400d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.988+0000 7f0c912fa640 1 --2- 172.21.15.19:0/1886485393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x562425400d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.988+0000 7f0c912fa640 1 -- 172.21.15.19:0/1886485393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624251435f0 con 0x5624250382f0 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.988+0000 7f0c912fa640 1 --2- 172.21.15.19:0/1886485393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425400850 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0c88031500 tx=0x7f0c8802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c77861d24400e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.988+0000 7f0c76ffd640 1 -- 172.21.15.19:0/1886485393 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88035040 con 0x5624250382f0 2024-05-09T01:33:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.988+0000 7f0c76ffd640 1 -- 172.21.15.19:0/1886485393 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c88040040 con 0x5624250382f0 2024-05-09T01:33:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.988+0000 7f0c76ffd640 1 -- 172.21.15.19:0/1886485393 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8803c780 con 0x5624250382f0 2024-05-09T01:33:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 -- 172.21.15.19:0/1886485393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 msgr2=0x562425400850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/1886485393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425400850 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0c88031500 tx=0x7f0c8802a410 comp rx=0 tx=0).stop 2024-05-09T01:33:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 -- 172.21.15.19:0/1886485393 shutdown_connections 2024-05-09T01:33:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/1886485393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250611b0 0x5624254165c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/1886485393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x562425400d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/1886485393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250382f0 0x562425400850 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 -- 172.21.15.19:0/1886485393 >> 172.21.15.19:0/1886485393 conn(0x562425055300 msgr2=0x5624254096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 -- 172.21.15.19:0/1886485393 shutdown_connections 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 -- 172.21.15.19:0/1886485393 wait complete. 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.989+0000 7f0c9a40e740 1 -- start start 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c9a40e740 1 Processor -- start 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250382f0 0x5624254140e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x562425414620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c9a40e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x562425415550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1aca50 con 0x5624250611b0 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1ac380 con 0x5624250404d0 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c9a40e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1a95c0 con 0x5624250382f0 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c9a40e740 10 client.0 _ll_register_callbacks cb 0x7f0c84009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c91afb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x562425415550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c91afb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x562425415550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55106/0 (socket says 172.21.15.19:55106) 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c91afb640 1 -- 172.21.15.19:0/3565589407 learned_addr learned my addr 172.21.15.19:0/3565589407 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c912fa640 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250382f0 0x5624254140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c90af9640 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x562425414620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c91afb640 1 -- 172.21.15.19:0/3565589407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250382f0 msgr2=0x5624254140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c91afb640 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250382f0 0x5624254140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c91afb640 1 -- 172.21.15.19:0/3565589407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 msgr2=0x562425414620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c91afb640 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x562425414620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.990+0000 7f0c91afb640 1 -- 172.21.15.19:0/3565589407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242532ded0 con 0x5624250611b0 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c91afb640 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x562425415550 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0c840098d0 tx=0x7f0c8401e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b3b33722ab06d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c74ff9640 1 -- 172.21.15.19:0/3565589407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8401f810 con 0x5624250611b0 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c74ff9640 1 -- 172.21.15.19:0/3565589407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0c8401f9b0 con 0x5624250611b0 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c9a40e740 1 -- 172.21.15.19:0/3565589407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c88040040 con 0x5624250611b0 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c74ff9640 1 -- 172.21.15.19:0/3565589407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c8401fbb0 con 0x5624250611b0 2024-05-09T01:33:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c9a40e740 10 client.17454 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c9a40e740 1 -- 172.21.15.19:0/3565589407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562425416c30 con 0x5624250611b0 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c74ff9640 1 -- 172.21.15.19:0/3565589407 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c88040040 con 0x5624250611b0 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c74ff9640 1 -- 172.21.15.19:0/3565589407 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0c84033430 con 0x5624250611b0 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c9a40e740 10 client.17454 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c9a40e740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c9a40e740 20 client.17454 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562424ff51c0 0x562425487250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c9a40e740 4 client.17454 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.991+0000 7f0c9a40e740 1 -- 172.21.15.19:0/3565589407 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562425371240 con 0x562424ff51c0 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.992+0000 7f0c912fa640 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562424ff51c0 0x562425487250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.992+0000 7f0c74ff9640 10 client.17454 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:22.992+0000 7f0c912fa640 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562424ff51c0 0x562425487250 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4674313fd306c9d server_cookie=cb7dddc23adfc14c in_seq=0 out_seq=0 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c74ff9640 1 -- 172.21.15.19:0/3565589407 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562425371240 con 0x562424ff51c0 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c74ff9640 10 client.17454 handle_command_reply: tid=0 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c9a40e740 1 client.17454 shutdown 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c9a40e740 1 -- 172.21.15.19:0/3565589407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 msgr2=0x562425415550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x562425415550 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0c840098d0 tx=0x7f0c8401e8c0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c9a40e740 1 -- 172.21.15.19:0/3565589407 shutdown_connections 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562424ff51c0 0x562425487250 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624250611b0 0x562425415550 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624250404d0 0x562425414620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c74ff9640 0 client.17454 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c9a40e740 1 --2- 172.21.15.19:0/3565589407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624250382f0 0x5624254140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.008+0000 7f0c9a40e740 1 -- 172.21.15.19:0/3565589407 >> 172.21.15.19:0/3565589407 conn(0x562425055300 msgr2=0x5624254007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9a40e740 1 -- 172.21.15.19:0/3565589407 shutdown_connections 2024-05-09T01:33:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9a40e740 1 -- 172.21.15.19:0/3565589407 wait complete. 2024-05-09T01:33:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9a40e740 20 client.17454 trim_cache size 0 max 16384 2024-05-09T01:33:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:33:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:33:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:33:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 -- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0c780b27e0 msgr2=0x7f0c780b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0c780b27e0 0x7f0c780b4cf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5624254066d0 tx=0x7f0c84004a40 comp rx=0 tx=0).stop 2024-05-09T01:33:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 -- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c107090 msgr2=0x7f0c8c19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c107090 0x7f0c8c19a970 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00b4a0 tx=0x7f0c7c00b960 comp rx=0 tx=0).stop 2024-05-09T01:33:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 -- 172.21.15.19:0/2195045501 shutdown_connections 2024-05-09T01:33:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0c780b27e0 0x7f0c780b4cf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c8c1099f0 0x7f0c8c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c107090 0x7f0c8c19a970 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 --2- 172.21.15.19:0/2195045501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c104730 0x7f0c8c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.009+0000 7f0c9355e640 1 -- 172.21.15.19:0/2195045501 >> 172.21.15.19:0/2195045501 conn(0x7f0c8c0fdd30 msgr2=0x7f0c8c106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.010+0000 7f0c9355e640 1 -- 172.21.15.19:0/2195045501 shutdown_connections 2024-05-09T01:33:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:23.010+0000 7f0c9355e640 1 -- 172.21.15.19:0/2195045501 wait complete. 2024-05-09T01:33:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 -- 172.21.15.19:0/585140909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc108740 msgr2=0x7f52cc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 --2- 172.21.15.19:0/585140909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc108740 0x7f52cc1100a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f52c400b0d0 tx=0x7f52c400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 -- 172.21.15.19:0/585140909 shutdown_connections 2024-05-09T01:33:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 --2- 172.21.15.19:0/585140909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc108740 0x7f52cc1100a0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 --2- 172.21.15.19:0/585140909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc1034c0 0x7f52cc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 --2- 172.21.15.19:0/585140909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc107df0 0x7f52cc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 -- 172.21.15.19:0/585140909 >> 172.21.15.19:0/585140909 conn(0x7f52cc0fedf0 msgr2=0x7f52cc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 -- 172.21.15.19:0/585140909 shutdown_connections 2024-05-09T01:33:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 -- 172.21.15.19:0/585140909 wait complete. 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.233+0000 7f52d2498700 1 Processor -- start 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52d2498700 1 -- start start 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52d2498700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc1034c0 0x7f52cc19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52d2498700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc107df0 0x7f52cc19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52d2498700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc108740 0x7f52cc199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52d2498700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc113390 con 0x7f52cc107df0 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52d2498700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc113210 con 0x7f52cc1034c0 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52d2498700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc113510 con 0x7f52cc108740 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc107df0 0x7f52cc19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc107df0 0x7f52cc19fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55128/0 (socket says 172.21.15.19:55128) 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52c3fff700 1 -- 172.21.15.19:0/3260709962 learned_addr learned my addr 172.21.15.19:0/3260709962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52d0a35700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc108740 0x7f52cc199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52cbfff700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc1034c0 0x7f52cc19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52c3fff700 1 -- 172.21.15.19:0/3260709962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc108740 msgr2=0x7f52cc199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52c3fff700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc108740 0x7f52cc199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52c3fff700 1 -- 172.21.15.19:0/3260709962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc1034c0 msgr2=0x7f52cc19f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52c3fff700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc1034c0 0x7f52cc19f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.234+0000 7f52c3fff700 1 -- 172.21.15.19:0/3260709962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc19a1b0 con 0x7f52cc107df0 2024-05-09T01:33:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.235+0000 7f52c3fff700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc107df0 0x7f52cc19fdf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f52b40078e0 tx=0x7f52b4007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.235+0000 7f52c9ffb700 1 -- 172.21.15.19:0/3260709962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4004020 con 0x7f52cc107df0 2024-05-09T01:33:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.235+0000 7f52d2498700 1 -- 172.21.15.19:0/3260709962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc19a4a0 con 0x7f52cc107df0 2024-05-09T01:33:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.235+0000 7f52c9ffb700 1 -- 172.21.15.19:0/3260709962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b4004830 con 0x7f52cc107df0 2024-05-09T01:33:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.235+0000 7f52c9ffb700 1 -- 172.21.15.19:0/3260709962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4005560 con 0x7f52cc107df0 2024-05-09T01:33:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.235+0000 7f52d2498700 1 -- 172.21.15.19:0/3260709962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc0759f0 con 0x7f52cc107df0 2024-05-09T01:33:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.238+0000 7f52c9ffb700 1 -- 172.21.15.19:0/3260709962 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f52b401f020 con 0x7f52cc107df0 2024-05-09T01:33:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.239+0000 7f52c9ffb700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52b80b2880 0x7f52b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.239+0000 7f52e0eaeb80 1 Processor -- start 2024-05-09T01:33:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.239+0000 7f52cbfff700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52b80b2880 0x7f52b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.239+0000 7f52c9ffb700 1 -- 172.21.15.19:0/3260709962 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f52b40e3160 con 0x7f52cc107df0 2024-05-09T01:33:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52e0eaeb80 1 -- start start 2024-05-09T01:33:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f9294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f929a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f923ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ef3f845ff0 con 0x55ef3f890160 2024-05-09T01:33:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ef3f7e47f0 con 0x55ef3f7a64a0 2024-05-09T01:33:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef3f841fb0 con 0x55ef3f9236a0 2024-05-09T01:33:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52cbfff700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52b80b2880 0x7f52b80b4d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f52cc19ac10 tx=0x7f52bc00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f9294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f9294f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55130/0 (socket says 172.21.15.19:55130) 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52cbfff700 1 -- 172.21.15.19:0/2372411335 learned_addr learned my addr 172.21.15.19:0/2372411335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52c3fff700 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f929a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52d0a35700 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f923ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52cbfff700 1 -- 172.21.15.19:0/2372411335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 msgr2=0x55ef3f923ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52cbfff700 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f923ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52cbfff700 1 -- 172.21.15.19:0/2372411335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 msgr2=0x55ef3f929a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52cbfff700 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f929a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.240+0000 7f52cbfff700 1 -- 172.21.15.19:0/2372411335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3f7dc370 con 0x55ef3f890160 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.241+0000 7f52d0a35700 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f923ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.241+0000 7f52c3fff700 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f929a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.241+0000 7f52cbfff700 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f9294f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f52bc003af0 tx=0x7f52bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbddf3d661f313fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.241+0000 7f52c27fc700 1 -- 172.21.15.19:0/2372411335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc011ad0 con 0x55ef3f890160 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.241+0000 7f52c27fc700 1 -- 172.21.15.19:0/2372411335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52bc011c70 con 0x55ef3f890160 2024-05-09T01:33:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.241+0000 7f52c27fc700 1 -- 172.21.15.19:0/2372411335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc01d410 con 0x55ef3f890160 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/2372411335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 msgr2=0x55ef3f9294f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f9294f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f52bc003af0 tx=0x7f52bc013000 comp rx=0 tx=0).stop 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/2372411335 shutdown_connections 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f923ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f9294f0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/2372411335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f929a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/2372411335 >> 172.21.15.19:0/2372411335 conn(0x55ef3f8066a0 msgr2=0x55ef3f91e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/2372411335 shutdown_connections 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/2372411335 wait complete. 2024-05-09T01:33:23.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.242+0000 7f52e0eaeb80 1 -- start start 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52e0eaeb80 1 Processor -- start 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 0x55ef3f92d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f9236a0 0x55ef3f92dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ef3f6c1ba0 con 0x55ef3f9236a0 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ef3f800150 con 0x55ef3f890160 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef3f9301d0 con 0x55ef3f7a64a0 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52e0eaeb80 10 client.0 _ll_register_callbacks cb 0x55ef3f6b76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52cbfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52d0a35700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f9236a0 0x55ef3f92dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52cbfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:60126/0 (socket says 172.21.15.19:60126) 2024-05-09T01:33:23.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52cbfff700 1 -- 172.21.15.19:0/3544665001 learned_addr learned my addr 172.21.15.19:0/3544665001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.243+0000 7f52c3fff700 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 0x55ef3f92d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52cbfff700 1 -- 172.21.15.19:0/3544665001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 msgr2=0x55ef3f92d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52cbfff700 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 0x55ef3f92d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52cbfff700 1 -- 172.21.15.19:0/3544665001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f9236a0 msgr2=0x55ef3f92dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52cbfff700 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f9236a0 0x55ef3f92dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52cbfff700 1 -- 172.21.15.19:0/3544665001 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3f6c1490 con 0x55ef3f7a64a0 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52d0a35700 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f9236a0 0x55ef3f92dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52cbfff700 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52bc01ff90 tx=0x7f52bc01ffc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a927eddc545ba16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52ab7fe700 1 -- 172.21.15.19:0/3544665001 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc035060 con 0x55ef3f7a64a0 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/3544665001 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52bc011c70 con 0x55ef3f7a64a0 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52ab7fe700 1 -- 172.21.15.19:0/3544665001 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f52bc01e420 con 0x55ef3f7a64a0 2024-05-09T01:33:23.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52ab7fe700 1 -- 172.21.15.19:0/3544665001 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52bc01e620 con 0x55ef3f7a64a0 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52e0eaeb80 10 client.26800 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.244+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/3544665001 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef3f93ac60 con 0x55ef3f7a64a0 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.245+0000 7f52ab7fe700 1 -- 172.21.15.19:0/3544665001 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52bc011c70 con 0x55ef3f7a64a0 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.245+0000 7f52ab7fe700 1 -- 172.21.15.19:0/3544665001 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f52bc026400 con 0x55ef3f7a64a0 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.245+0000 7f52e0eaeb80 10 client.26800 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.245+0000 7f52e0eaeb80 10 client.26800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.245+0000 7f52e0eaeb80 20 client.26800 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.245+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ef3f93d240 0x55ef3f93f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.245+0000 7f52e0eaeb80 4 client.26800 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.245+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/3544665001 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef3f8539d0 con 0x55ef3f93d240 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.245+0000 7f52c3fff700 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ef3f93d240 0x55ef3f93f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.246+0000 7f52c3fff700 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ef3f93d240 0x55ef3f93f660 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678e1b5d27163259 server_cookie=11b618c1cd4ec1e6 in_seq=0 out_seq=0 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.246+0000 7f52ab7fe700 10 client.26800 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.246+0000 7f52ab7fe700 1 -- 172.21.15.19:0/3544665001 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef3f8539d0 con 0x55ef3f93d240 2024-05-09T01:33:23.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52ab7fe700 10 client.26800 handle_command_reply: tid=0 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 client.26800 shutdown 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/3544665001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f7a64a0 msgr2=0x55ef3f92d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52bc01ff90 tx=0x7f52bc01ffc0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/3544665001 shutdown_connections 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ef3f93d240 0x55ef3f93f660 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f9236a0 0x55ef3f92dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d280 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52ab7fe700 0 client.26800 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/3544665001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 0x55ef3f92d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/3544665001 >> 172.21.15.19:0/3544665001 conn(0x55ef3f8066a0 msgr2=0x55ef3f91e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/3544665001 shutdown_connections 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/3544665001 wait complete. 2024-05-09T01:33:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.247+0000 7f52e0eaeb80 20 client.26800 trim_cache size 0 max 16384 2024-05-09T01:33:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.267+0000 7f52e0eaeb80 1 Processor -- start 2024-05-09T01:33:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.267+0000 7f52e0eaeb80 1 -- start start 2024-05-09T01:33:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f922690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f922bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1acb10 con 0x55ef3f890160 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1ae650 con 0x55ef3f7a64a0 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1ad1e0 con 0x55ef3f9236a0 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f922690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f922690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55142/0 (socket says 172.21.15.19:55142) 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c3fff700 1 -- 172.21.15.19:0/4020596516 learned_addr learned my addr 172.21.15.19:0/4020596516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52d0a35700 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f922bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52cbfff700 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c3fff700 1 -- 172.21.15.19:0/4020596516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 msgr2=0x55ef3f922bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c3fff700 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f922bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c3fff700 1 -- 172.21.15.19:0/4020596516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 msgr2=0x55ef3f92d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c3fff700 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c3fff700 1 -- 172.21.15.19:0/4020596516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3f939250 con 0x55ef3f890160 2024-05-09T01:33:23.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52cbfff700 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:23.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c3fff700 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f922690 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f52b40cd1d0 tx=0x7f52b4092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca488b999872336 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:23.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.268+0000 7f52c1ffb700 1 -- 172.21.15.19:0/4020596516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b401c040 con 0x55ef3f890160 2024-05-09T01:33:23.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52c1ffb700 1 -- 172.21.15.19:0/4020596516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b40aeb40 con 0x55ef3f890160 2024-05-09T01:33:23.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52c1ffb700 1 -- 172.21.15.19:0/4020596516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b40049e0 con 0x55ef3f890160 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4020596516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 msgr2=0x55ef3f922690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f922690 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f52b40cd1d0 tx=0x7f52b4092040 comp rx=0 tx=0).stop 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4020596516 shutdown_connections 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f922bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f7a64a0 0x55ef3f92d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4020596516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f890160 0x55ef3f922690 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4020596516 >> 172.21.15.19:0/4020596516 conn(0x55ef3f8066a0 msgr2=0x55ef3f92d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.269+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4020596516 shutdown_connections 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.270+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4020596516 wait complete. 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.270+0000 7f52e0eaeb80 1 -- start start 2024-05-09T01:33:23.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.270+0000 7f52e0eaeb80 1 Processor -- start 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f7a64a0 0x55ef3f9427b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 0x55ef3f942cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52e0eaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f92eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1adf80 con 0x55ef3f7a64a0 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1ad8b0 con 0x55ef3f890160 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52e0eaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1aaab0 con 0x55ef3f9236a0 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52e0eaeb80 10 client.0 _ll_register_callbacks cb 0x55ef3f6b76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f7a64a0 0x55ef3f9427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f7a64a0 0x55ef3f9427b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55146/0 (socket says 172.21.15.19:55146) 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52cbfff700 1 -- 172.21.15.19:0/4007176097 learned_addr learned my addr 172.21.15.19:0/4007176097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52c3fff700 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 0x55ef3f942cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52d0a35700 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f92eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52cbfff700 1 -- 172.21.15.19:0/4007176097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 msgr2=0x55ef3f92eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:23.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52cbfff700 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f92eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52cbfff700 1 -- 172.21.15.19:0/4007176097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 msgr2=0x55ef3f942cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52cbfff700 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 0x55ef3f942cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52cbfff700 1 -- 172.21.15.19:0/4007176097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3f92f300 con 0x55ef3f7a64a0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52cbfff700 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f7a64a0 0x55ef3f9427b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0267a0 tx=0x7f52bc026c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53139fc48e6f181d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.271+0000 7f52ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52ab7fe700 1 -- 172.21.15.19:0/4007176097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc035050 con 0x55ef3f7a64a0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52ab7fe700 1 -- 172.21.15.19:0/4007176097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f52bc01e420 con 0x55ef3f7a64a0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4007176097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b40aeb40 con 0x55ef3f7a64a0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52ab7fe700 1 -- 172.21.15.19:0/4007176097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52bc011c70 con 0x55ef3f7a64a0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52e0eaeb80 10 client.17472 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4007176097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef3f9ab670 con 0x55ef3f7a64a0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52ab7fe700 1 -- 172.21.15.19:0/4007176097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b40aeb40 con 0x55ef3f7a64a0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52ab7fe700 1 -- 172.21.15.19:0/4007176097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f52bc044470 con 0x55ef3f7a64a0 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52e0eaeb80 10 client.17472 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:23.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52e0eaeb80 10 client.17472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52e0eaeb80 20 client.17472 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ef3f9abc90 0x55ef3f9ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52e0eaeb80 4 client.17472 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.272+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4007176097 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef3f84da70 con 0x55ef3f9abc90 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.273+0000 7f52c3fff700 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ef3f9abc90 0x55ef3f9ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.273+0000 7f52ab7fe700 10 client.17472 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.273+0000 7f52c3fff700 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ef3f9abc90 0x55ef3f9ae0b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=611f049e7255db89 server_cookie=c6927b1e5b5ddd6 in_seq=0 out_seq=0 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.273+0000 7f52ab7fe700 1 -- 172.21.15.19:0/4007176097 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ef3f84da70 con 0x55ef3f9abc90 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.273+0000 7f52ab7fe700 10 client.17472 handle_command_reply: tid=0 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.273+0000 7f52e0eaeb80 1 client.17472 shutdown 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4007176097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f7a64a0 msgr2=0x55ef3f9427b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f7a64a0 0x55ef3f9427b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0267a0 tx=0x7f52bc026c40 comp rx=0 tx=0).stop 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4007176097 shutdown_connections 2024-05-09T01:33:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ef3f9abc90 0x55ef3f9ae0b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3f9236a0 0x55ef3f92eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ef3f7a64a0 0x55ef3f9427b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 --2- 172.21.15.19:0/4007176097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef3f890160 0x55ef3f942cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4007176097 >> 172.21.15.19:0/4007176097 conn(0x55ef3f8066a0 msgr2=0x55ef3f9389d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52ab7fe700 0 client.17472 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4007176097 shutdown_connections 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 1 -- 172.21.15.19:0/4007176097 wait complete. 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.274+0000 7f52e0eaeb80 20 client.17472 trim_cache size 0 max 16384 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:33:23.275 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:33:23.276 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:33:23.276 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:33:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 -- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52b80b2880 msgr2=0x7f52b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52b80b2880 0x7f52b80b4d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f52cc19ac10 tx=0x7f52bc00b3c0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 -- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc107df0 msgr2=0x7f52cc19fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc107df0 0x7f52cc19fdf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f52b40078e0 tx=0x7f52b4007bf0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 -- 172.21.15.19:0/3260709962 shutdown_connections 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc108740 0x7f52cc199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc1034c0 0x7f52cc19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52b80b2880 0x7f52b80b4d90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 --2- 172.21.15.19:0/3260709962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc107df0 0x7f52cc19fdf0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 -- 172.21.15.19:0/3260709962 >> 172.21.15.19:0/3260709962 conn(0x7f52cc0fedf0 msgr2=0x7f52cc107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 -- 172.21.15.19:0/3260709962 shutdown_connections 2024-05-09T01:33:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:23.275+0000 7f52d2498700 1 -- 172.21.15.19:0/3260709962 wait complete. 2024-05-09T01:33:24.080 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:33:24.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:33:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:24.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:24 smithi019 ceph-mon[30158]: pgmap v614: 97 pgs: 97 active+clean; 840 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:33:24.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:24 smithi196 ceph-mon[35041]: pgmap v614: 97 pgs: 97 active+clean; 840 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:33:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:24 smithi059 ceph-mon[34351]: pgmap v614: 97 pgs: 97 active+clean; 840 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:33:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.076+0000 7f7b03b0d700 1 -- 172.21.15.19:0/564107548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc103080 msgr2=0x7f7afc103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.076+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/564107548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc103080 0x7f7afc103490 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009990 tx=0x7f7aec009ca0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.076+0000 7f7b03b0d700 1 -- 172.21.15.19:0/564107548 shutdown_connections 2024-05-09T01:33:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.076+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/564107548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7afc110c30 0x7f7afc113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.076+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/564107548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc103080 0x7f7afc103490 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.076+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/564107548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7afc1039d0 0x7f7afc1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.076+0000 7f7b03b0d700 1 -- 172.21.15.19:0/564107548 >> 172.21.15.19:0/564107548 conn(0x7f7afc0fedf0 msgr2=0x7f7afc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:26.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.076+0000 7f7b03b0d700 1 -- 172.21.15.19:0/564107548 shutdown_connections 2024-05-09T01:33:26.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.076+0000 7f7b03b0d700 1 -- 172.21.15.19:0/564107548 wait complete. 2024-05-09T01:33:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b03b0d700 1 Processor -- start 2024-05-09T01:33:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b03b0d700 1 -- start start 2024-05-09T01:33:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b03b0d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7afc103080 0x7f7afc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b03b0d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc1039d0 0x7f7afc19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b03b0d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7afc110c30 0x7f7afc1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b03b0d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc1166a0 con 0x7f7afc1039d0 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b03b0d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc116520 con 0x7f7afc103080 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b03b0d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc116820 con 0x7f7afc110c30 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b010a8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc1039d0 0x7f7afc19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b010a8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc1039d0 0x7f7afc19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55178/0 (socket says 172.21.15.19:55178) 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.077+0000 7f7b010a8700 1 -- 172.21.15.19:0/54885908 learned_addr learned my addr 172.21.15.19:0/54885908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b018a9700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7afc103080 0x7f7afc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b020aa700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7afc110c30 0x7f7afc1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b010a8700 1 -- 172.21.15.19:0/54885908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7afc110c30 msgr2=0x7f7afc1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b010a8700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7afc110c30 0x7f7afc1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b010a8700 1 -- 172.21.15.19:0/54885908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7afc103080 msgr2=0x7f7afc19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b010a8700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7afc103080 0x7f7afc19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b010a8700 1 -- 172.21.15.19:0/54885908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7afc1a1ec0 con 0x7f7afc1039d0 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b018a9700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7afc103080 0x7f7afc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b010a8700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc1039d0 0x7f7afc19d590 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7af800c8c0 tx=0x7f7af800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7af2ffd700 1 -- 172.21.15.19:0/54885908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8004220 con 0x7f7afc1039d0 2024-05-09T01:33:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7afc1a21b0 con 0x7f7afc1039d0 2024-05-09T01:33:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7af2ffd700 1 -- 172.21.15.19:0/54885908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7af8007db0 con 0x7f7afc1039d0 2024-05-09T01:33:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7af2ffd700 1 -- 172.21.15.19:0/54885908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8005020 con 0x7f7afc1039d0 2024-05-09T01:33:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.078+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7afc1a9ad0 con 0x7f7afc1039d0 2024-05-09T01:33:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.079+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7afc0512c0 con 0x7f7afc1039d0 2024-05-09T01:33:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.082+0000 7f7af2ffd700 1 -- 172.21.15.19:0/54885908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7af8007900 con 0x7f7afc1039d0 2024-05-09T01:33:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.083+0000 7f7af2ffd700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7ae80b2880 0x7f7ae80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.083+0000 7f7b018a9700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7ae80b2880 0x7f7ae80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.083+0000 7f7af2ffd700 1 -- 172.21.15.19:0/54885908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7af80e22b0 con 0x7f7afc1039d0 2024-05-09T01:33:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.084+0000 7f7b018a9700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7ae80b2880 0x7f7ae80b4d90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7aec003800 tx=0x7f7aec01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.087+0000 7f7af2ffd700 1 -- 172.21.15.19:0/54885908 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7af8014030 con 0x7f7afc1039d0 2024-05-09T01:33:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.349+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7afc19e350 con 0x7f7afc1039d0 2024-05-09T01:33:26.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.350+0000 7f7af2ffd700 1 -- 172.21.15.19:0/54885908 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f7af80a9f50 con 0x7f7afc1039d0 2024-05-09T01:33:26.347 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:33:26.347 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:33:26.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7ae80b2880 msgr2=0x7f7ae80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:26.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7ae80b2880 0x7f7ae80b4d90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7aec003800 tx=0x7f7aec01a040 comp rx=0 tx=0).stop 2024-05-09T01:33:26.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc1039d0 msgr2=0x7f7afc19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:26.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc1039d0 0x7f7afc19d590 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7af800c8c0 tx=0x7f7af800cc80 comp rx=0 tx=0).stop 2024-05-09T01:33:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 shutdown_connections 2024-05-09T01:33:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7afc110c30 0x7f7afc1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7afc103080 0x7f7afc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7ae80b2880 0x7f7ae80b4d90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 --2- 172.21.15.19:0/54885908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7afc1039d0 0x7f7afc19d590 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 >> 172.21.15.19:0/54885908 conn(0x7f7afc0fedf0 msgr2=0x7f7afc1127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.352+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 shutdown_connections 2024-05-09T01:33:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:26.353+0000 7f7b03b0d700 1 -- 172.21.15.19:0/54885908 wait complete. 2024-05-09T01:33:26.350 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:33:26.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:26 smithi019 ceph-mon[30158]: pgmap v615: 97 pgs: 97 active+clean; 899 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-05-09T01:33:26.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:26 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/54885908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:33:26.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:26 smithi196 ceph-mon[35041]: pgmap v615: 97 pgs: 97 active+clean; 899 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-05-09T01:33:26.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:26 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/54885908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:33:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:26 smithi059 ceph-mon[34351]: pgmap v615: 97 pgs: 97 active+clean; 899 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-05-09T01:33:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:26 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/54885908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:33:27.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:33:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:28.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:28 smithi019 ceph-mon[30158]: pgmap v616: 97 pgs: 97 active+clean; 899 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:33:28.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:28 smithi196 ceph-mon[35041]: pgmap v616: 97 pgs: 97 active+clean; 899 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:33:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:28 smithi059 ceph-mon[34351]: pgmap v616: 97 pgs: 97 active+clean; 899 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:33:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.269+0000 7fb39e4ae700 1 -- 172.21.15.19:0/225311472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 msgr2=0x7fb398108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.269+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/225311472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 0x7fb398108200 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb3800099c0 tx=0x7fb380009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.269+0000 7fb39e4ae700 1 -- 172.21.15.19:0/225311472 shutdown_connections 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.269+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/225311472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb398108740 0x7fb3981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.269+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/225311472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3981034c0 0x7fb3981038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.269+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/225311472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 0x7fb398108200 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.269+0000 7fb39e4ae700 1 -- 172.21.15.19:0/225311472 >> 172.21.15.19:0/225311472 conn(0x7fb3980fedf0 msgr2=0x7fb398101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.269+0000 7fb39e4ae700 1 -- 172.21.15.19:0/225311472 shutdown_connections 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.269+0000 7fb39e4ae700 1 -- 172.21.15.19:0/225311472 wait complete. 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb39e4ae700 1 Processor -- start 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb39e4ae700 1 -- start start 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb39e4ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3981034c0 0x7fb3980ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb39e4ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 0x7fb3980ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb39e4ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb398108740 0x7fb3980ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb39e4ae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb398113350 con 0x7fb398107df0 2024-05-09T01:33:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb39e4ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981131d0 con 0x7fb398108740 2024-05-09T01:33:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb39e4ae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981134d0 con 0x7fb3981034c0 2024-05-09T01:33:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb38ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 0x7fb3980ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb397fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3981034c0 0x7fb3980ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb38ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 0x7fb3980ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55190/0 (socket says 172.21.15.19:55190) 2024-05-09T01:33:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb38ffff700 1 -- 172.21.15.19:0/1276355434 learned_addr learned my addr 172.21.15.19:0/1276355434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.270+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb398108740 0x7fb3980ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb38ffff700 1 -- 172.21.15.19:0/1276355434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3981034c0 msgr2=0x7fb3980ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb38ffff700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3981034c0 0x7fb3980ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb38ffff700 1 -- 172.21.15.19:0/1276355434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb398108740 msgr2=0x7fb3980ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:29.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb38ffff700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb398108740 0x7fb3980ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb38ffff700 1 -- 172.21.15.19:0/1276355434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3981a9240 con 0x7fb398107df0 2024-05-09T01:33:29.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb38ffff700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 0x7fb3980ff810 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb380006010 tx=0x7fb3800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:29.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb395ffb700 1 -- 172.21.15.19:0/1276355434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380027070 con 0x7fb398107df0 2024-05-09T01:33:29.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb395ffb700 1 -- 172.21.15.19:0/1276355434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb380003f20 con 0x7fb398107df0 2024-05-09T01:33:29.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb395ffb700 1 -- 172.21.15.19:0/1276355434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380022c60 con 0x7fb398107df0 2024-05-09T01:33:29.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb39e4ae700 1 -- 172.21.15.19:0/1276355434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3981a94d0 con 0x7fb398107df0 2024-05-09T01:33:29.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.271+0000 7fb39e4ae700 1 -- 172.21.15.19:0/1276355434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3981a9920 con 0x7fb398107df0 2024-05-09T01:33:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.274+0000 7fb395ffb700 1 -- 172.21.15.19:0/1276355434 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb38002a070 con 0x7fb398107df0 2024-05-09T01:33:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.275+0000 7fb395ffb700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3840b2880 0x7fb3840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.275+0000 7fb397fff700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3840b2880 0x7fb3840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.275+0000 7fb395ffb700 1 -- 172.21.15.19:0/1276355434 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb380022e00 con 0x7fb398107df0 2024-05-09T01:33:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.275+0000 7fb3acec4b80 1 Processor -- start 2024-05-09T01:33:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.275+0000 7fb3acec4b80 1 -- start start 2024-05-09T01:33:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.275+0000 7fb397fff700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3840b2880 0x7fb3840b4d90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb388005950 tx=0x7fb38800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.275+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0e0f160 0x561fa0ea84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.275+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 0x561fa0ea8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.275+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0ea2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561fa0dc4ff0 con 0x561fa0ea26a0 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561fa0d637f0 con 0x561fa0d254a0 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561fa0dc0fb0 con 0x561fa0e0f160 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb39ca4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0ea2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb39ca4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0ea2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55202/0 (socket says 172.21.15.19:55202) 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb39ca4b700 1 -- 172.21.15.19:0/805690737 learned_addr learned my addr 172.21.15.19:0/805690737 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb38ffff700 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 0x561fa0ea8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb397fff700 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0e0f160 0x561fa0ea84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb39ca4b700 1 -- 172.21.15.19:0/805690737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0e0f160 msgr2=0x561fa0ea84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0e0f160 0x561fa0ea84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb39ca4b700 1 -- 172.21.15.19:0/805690737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 msgr2=0x561fa0ea8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 0x561fa0ea8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb39ca4b700 1 -- 172.21.15.19:0/805690737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa0d5b370 con 0x561fa0ea26a0 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb38ffff700 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 0x561fa0ea8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0ea2ad0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb39000a730 tx=0x7fb3900078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801ce4ce5171dc80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb38e7fc700 1 -- 172.21.15.19:0/805690737 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390003a00 con 0x561fa0ea26a0 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb38e7fc700 1 -- 172.21.15.19:0/805690737 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb390003ba0 con 0x561fa0ea26a0 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.276+0000 7fb38e7fc700 1 -- 172.21.15.19:0/805690737 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390014660 con 0x561fa0ea26a0 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.277+0000 7fb3acec4b80 1 -- 172.21.15.19:0/805690737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 msgr2=0x561fa0ea2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.277+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0ea2ad0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb39000a730 tx=0x7fb3900078d0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.277+0000 7fb3acec4b80 1 -- 172.21.15.19:0/805690737 shutdown_connections 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.277+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0ea2ad0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.277+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0e0f160 0x561fa0ea84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.277+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/805690737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 0x561fa0ea8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.277+0000 7fb3acec4b80 1 -- 172.21.15.19:0/805690737 >> 172.21.15.19:0/805690737 conn(0x561fa0d856a0 msgr2=0x561fa0e9d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.277+0000 7fb3acec4b80 1 -- 172.21.15.19:0/805690737 shutdown_connections 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.277+0000 7fb3acec4b80 1 -- 172.21.15.19:0/805690737 wait complete. 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.278+0000 7fb3acec4b80 1 -- start start 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.278+0000 7fb3acec4b80 1 Processor -- start 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.278+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0d254a0 0x561fa0eac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.278+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 0x561fa0eac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.278+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0eacc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.278+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561fa0c40ba0 con 0x561fa0ea26a0 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.278+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561fa0d7f150 con 0x561fa0e0f160 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.278+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561fa0eaf1d0 con 0x561fa0d254a0 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb3acec4b80 10 client.0 _ll_register_callbacks cb 0x561fa0c366c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb39ca4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0eacc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb39ca4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0eacc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55214/0 (socket says 172.21.15.19:55214) 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb39ca4b700 1 -- 172.21.15.19:0/353977166 learned_addr learned my addr 172.21.15.19:0/353977166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb397fff700 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0d254a0 0x561fa0eac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb38ffff700 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 0x561fa0eac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb39ca4b700 1 -- 172.21.15.19:0/353977166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0d254a0 msgr2=0x561fa0eac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0d254a0 0x561fa0eac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb39ca4b700 1 -- 172.21.15.19:0/353977166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 msgr2=0x561fa0eac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 0x561fa0eac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb39ca4b700 1 -- 172.21.15.19:0/353977166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa0c40490 con 0x561fa0ea26a0 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb38ffff700 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 0x561fa0eac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0eacc30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb390007fa0 tx=0x7fb390004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46359073b49fd04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb3777fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb3777fe700 1 -- 172.21.15.19:0/353977166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390022070 con 0x561fa0ea26a0 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb3777fe700 1 -- 172.21.15.19:0/353977166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb390004ad0 con 0x561fa0ea26a0 2024-05-09T01:33:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.279+0000 7fb3acec4b80 1 -- 172.21.15.19:0/353977166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb390003ba0 con 0x561fa0ea26a0 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3777fe700 1 -- 172.21.15.19:0/353977166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb390003a00 con 0x561fa0ea26a0 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3acec4b80 10 client.17490 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3acec4b80 1 -- 172.21.15.19:0/353977166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fa0eb9b50 con 0x561fa0ea26a0 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3777fe700 1 -- 172.21.15.19:0/353977166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb390003ba0 con 0x561fa0ea26a0 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3777fe700 1 -- 172.21.15.19:0/353977166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb390014400 con 0x561fa0ea26a0 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3acec4b80 10 client.17490 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3acec4b80 10 client.17490 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3acec4b80 20 client.17490 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x561fa0ebc080 0x561fa0ebe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3acec4b80 4 client.17490 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb3acec4b80 1 -- 172.21.15.19:0/353977166 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fa0dd29d0 con 0x561fa0ebc080 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.280+0000 7fb397fff700 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x561fa0ebc080 0x561fa0ebe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.281+0000 7fb3777fe700 10 client.17490 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:29.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.281+0000 7fb397fff700 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x561fa0ebc080 0x561fa0ebe4a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1209e47846c99fa server_cookie=57e6a25c6dc1a500 in_seq=0 out_seq=0 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3777fe700 1 -- 172.21.15.19:0/353977166 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561fa0dd29d0 con 0x561fa0ebc080 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3777fe700 10 client.17490 handle_command_reply: tid=0 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 client.17490 shutdown 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 -- 172.21.15.19:0/353977166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 msgr2=0x561fa0eacc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0eacc30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb390007fa0 tx=0x7fb390004480 comp rx=0 tx=0).stop 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 -- 172.21.15.19:0/353977166 shutdown_connections 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x561fa0ebc080 0x561fa0ebe4a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0ea26a0 0x561fa0eacc30 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0d254a0 0x561fa0eac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/353977166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 0x561fa0eac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 -- 172.21.15.19:0/353977166 >> 172.21.15.19:0/353977166 conn(0x561fa0d856a0 msgr2=0x561fa0e9d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3777fe700 0 client.17490 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 -- 172.21.15.19:0/353977166 shutdown_connections 2024-05-09T01:33:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 1 -- 172.21.15.19:0/353977166 wait complete. 2024-05-09T01:33:29.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.282+0000 7fb3acec4b80 20 client.17490 trim_cache size 0 max 16384 2024-05-09T01:33:29.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb3acec4b80 1 Processor -- start 2024-05-09T01:33:29.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb3acec4b80 1 -- start start 2024-05-09T01:33:29.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0d254a0 0x561fa0ea67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 0x561fa0eb83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 0x561fa0ea20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981b0ee0 con 0x561fa0d254a0 2024-05-09T01:33:29.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981b2a20 con 0x561fa0e0f160 2024-05-09T01:33:29.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981b15b0 con 0x561fa0ea26a0 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb397fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0d254a0 0x561fa0ea67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb397fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0d254a0 0x561fa0ea67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55218/0 (socket says 172.21.15.19:55218) 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb397fff700 1 -- 172.21.15.19:0/2118793809 learned_addr learned my addr 172.21.15.19:0/2118793809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb38ffff700 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 0x561fa0eb83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.303+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 0x561fa0ea20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb397fff700 1 -- 172.21.15.19:0/2118793809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 msgr2=0x561fa0ea20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb397fff700 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 0x561fa0ea20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb397fff700 1 -- 172.21.15.19:0/2118793809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 msgr2=0x561fa0eb83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb397fff700 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 0x561fa0eb83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb397fff700 1 -- 172.21.15.19:0/2118793809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa0ea8050 con 0x561fa0d254a0 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 0x561fa0ea20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb397fff700 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0d254a0 0x561fa0ea67c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb38800ba80 tx=0x7fb388013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e342a50acbd781d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb38dffb700 1 -- 172.21.15.19:0/2118793809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388021b00 con 0x561fa0d254a0 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb38dffb700 1 -- 172.21.15.19:0/2118793809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb388004ce0 con 0x561fa0d254a0 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.304+0000 7fb38dffb700 1 -- 172.21.15.19:0/2118793809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388005410 con 0x561fa0d254a0 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2118793809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0d254a0 msgr2=0x561fa0ea67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0d254a0 0x561fa0ea67c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb38800ba80 tx=0x7fb388013000 comp rx=0 tx=0).stop 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2118793809 shutdown_connections 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 0x561fa0ea20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0d254a0 0x561fa0ea67c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2118793809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0e0f160 0x561fa0eb83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2118793809 >> 172.21.15.19:0/2118793809 conn(0x561fa0d856a0 msgr2=0x561fa0ea7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2118793809 shutdown_connections 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2118793809 wait complete. 2024-05-09T01:33:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.305+0000 7fb3acec4b80 1 -- start start 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb3acec4b80 1 Processor -- start 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 0x561fa0ec1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0e0f160 0x561fa0ec18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb3acec4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 0x561fa0ebc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981b2350 con 0x561fa0e0f160 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981b1c80 con 0x561fa0d254a0 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb3acec4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981aee80 con 0x561fa0ea26a0 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb3acec4b80 10 client.0 _ll_register_callbacks cb 0x561fa0c366c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb38ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0e0f160 0x561fa0ec18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb38ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0e0f160 0x561fa0ec18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55226/0 (socket says 172.21.15.19:55226) 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb38ffff700 1 -- 172.21.15.19:0/2333857118 learned_addr learned my addr 172.21.15.19:0/2333857118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb397fff700 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 0x561fa0ec1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb39ca4b700 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 0x561fa0ebc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb38ffff700 1 -- 172.21.15.19:0/2333857118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 msgr2=0x561fa0ebc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb38ffff700 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 0x561fa0ebc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb38ffff700 1 -- 172.21.15.19:0/2333857118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 msgr2=0x561fa0ec1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb38ffff700 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 0x561fa0ec1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.306+0000 7fb38ffff700 1 -- 172.21.15.19:0/2333857118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa0ebcfc0 con 0x561fa0e0f160 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb38ffff700 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0e0f160 0x561fa0ec18e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb3800e0cb0 tx=0x7fb380009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19db6f334355db90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3777fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3777fe700 1 -- 172.21.15.19:0/2333857118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3800e2070 con 0x561fa0e0f160 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3777fe700 1 -- 172.21.15.19:0/2333857118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb380022470 con 0x561fa0e0f160 2024-05-09T01:33:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2333857118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb388004ce0 con 0x561fa0e0f160 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3777fe700 1 -- 172.21.15.19:0/2333857118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3800aaf60 con 0x561fa0e0f160 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3acec4b80 10 client.17496 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2333857118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fa0f25f80 con 0x561fa0e0f160 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3777fe700 1 -- 172.21.15.19:0/2333857118 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb388004ce0 con 0x561fa0e0f160 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3777fe700 1 -- 172.21.15.19:0/2333857118 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb3800b31a0 con 0x561fa0e0f160 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3acec4b80 10 client.17496 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3acec4b80 10 client.17496 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3acec4b80 20 client.17496 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x561fa0f26760 0x561fa0f28b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3acec4b80 4 client.17496 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.307+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2333857118 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fa0dcca70 con 0x561fa0f26760 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.308+0000 7fb397fff700 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x561fa0f26760 0x561fa0f28b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.308+0000 7fb3777fe700 10 client.17496 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.308+0000 7fb397fff700 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x561fa0f26760 0x561fa0f28b80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c06357a072918a14 server_cookie=fb8fbc4944e9ddec in_seq=0 out_seq=0 2024-05-09T01:33:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.308+0000 7fb3777fe700 1 -- 172.21.15.19:0/2333857118 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fa0dcca70 con 0x561fa0f26760 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.308+0000 7fb3777fe700 10 client.17496 handle_command_reply: tid=0 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.308+0000 7fb3acec4b80 1 client.17496 shutdown 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2333857118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0e0f160 msgr2=0x561fa0ec18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0e0f160 0x561fa0ec18e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb3800e0cb0 tx=0x7fb380009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2333857118 shutdown_connections 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x561fa0f26760 0x561fa0f28b80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fa0ea26a0 0x561fa0ebc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fa0d254a0 0x561fa0ec1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 --2- 172.21.15.19:0/2333857118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561fa0e0f160 0x561fa0ec18e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2333857118 >> 172.21.15.19:0/2333857118 conn(0x561fa0d856a0 msgr2=0x561fa0ea5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3777fe700 0 client.17496 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2333857118 shutdown_connections 2024-05-09T01:33:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 1 -- 172.21.15.19:0/2333857118 wait complete. 2024-05-09T01:33:29.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb3acec4b80 20 client.17496 trim_cache size 0 max 16384 2024-05-09T01:33:29.313 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:29.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb39e4ae700 1 -- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3840b2880 msgr2=0x7fb3840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:29.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3840b2880 0x7fb3840b4d90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb388005950 tx=0x7fb38800b410 comp rx=0 tx=0).stop 2024-05-09T01:33:29.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb39e4ae700 1 -- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 msgr2=0x7fb3980ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.309+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 0x7fb3980ff810 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb380006010 tx=0x7fb3800047b0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.310+0000 7fb39e4ae700 1 -- 172.21.15.19:0/1276355434 shutdown_connections 2024-05-09T01:33:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.310+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb398108740 0x7fb3980ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.310+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3981034c0 0x7fb3980ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.310+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb3840b2880 0x7fb3840b4d90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.310+0000 7fb39e4ae700 1 --2- 172.21.15.19:0/1276355434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398107df0 0x7fb3980ff810 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.310+0000 7fb39e4ae700 1 -- 172.21.15.19:0/1276355434 >> 172.21.15.19:0/1276355434 conn(0x7fb3980fedf0 msgr2=0x7fb398075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.310+0000 7fb39e4ae700 1 -- 172.21.15.19:0/1276355434 shutdown_connections 2024-05-09T01:33:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:29.310+0000 7fb39e4ae700 1 -- 172.21.15.19:0/1276355434 wait complete. 2024-05-09T01:33:29.926 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:33:30.176 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:30.637 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:30 smithi019 ceph-mon[30158]: pgmap v617: 97 pgs: 97 active+clean; 952 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:33:30.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:30 smithi196 ceph-mon[35041]: pgmap v617: 97 pgs: 97 active+clean; 952 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:33:30.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:30 smithi059 ceph-mon[34351]: pgmap v617: 97 pgs: 97 active+clean; 952 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:33:31.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.961+0000 7fbd7ba75700 1 -- 172.21.15.19:0/1997586091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 msgr2=0x7fbd74077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:31.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.961+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/1997586091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 0x7fbd74077250 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009990 tx=0x7fbd70009ca0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.961+0000 7fbd7ba75700 1 -- 172.21.15.19:0/1997586091 shutdown_connections 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.961+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/1997586091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd74110b80 0x7fbd74112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.961+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/1997586091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 0x7fbd74077250 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.961+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/1997586091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd74077790 0x7fbd74110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.961+0000 7fbd7ba75700 1 -- 172.21.15.19:0/1997586091 >> 172.21.15.19:0/1997586091 conn(0x7fbd740ffea0 msgr2=0x7fbd741022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.961+0000 7fbd7ba75700 1 -- 172.21.15.19:0/1997586091 shutdown_connections 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.961+0000 7fbd7ba75700 1 -- 172.21.15.19:0/1997586091 wait complete. 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd7ba75700 1 Processor -- start 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd7ba75700 1 -- start start 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd7ba75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 0x7fbd7419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd7ba75700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd74077790 0x7fbd7419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd7ba75700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd74110b80 0x7fbd741a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd7ba75700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74116670 con 0x7fbd74076e40 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd7ba75700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741164f0 con 0x7fbd74077790 2024-05-09T01:33:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd7ba75700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741167f0 con 0x7fbd74110b80 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd79811700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 0x7fbd7419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd79811700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 0x7fbd7419d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37434/0 (socket says 172.21.15.19:37434) 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd79811700 1 -- 172.21.15.19:0/315178976 learned_addr learned my addr 172.21.15.19:0/315178976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.962+0000 7fbd79010700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd74077790 0x7fbd7419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd7a012700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd74110b80 0x7fbd741a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd79811700 1 -- 172.21.15.19:0/315178976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd74110b80 msgr2=0x7fbd741a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd79811700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd74110b80 0x7fbd741a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd79811700 1 -- 172.21.15.19:0/315178976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd74077790 msgr2=0x7fbd7419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd79811700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd74077790 0x7fbd7419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd79811700 1 -- 172.21.15.19:0/315178976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd741a1ed0 con 0x7fbd74076e40 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd79811700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 0x7fbd7419d000 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbd70000c00 tx=0x7fbd700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd6affd700 1 -- 172.21.15.19:0/315178976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70027070 con 0x7fbd74076e40 2024-05-09T01:33:31.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd7ba75700 1 -- 172.21.15.19:0/315178976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd741a2160 con 0x7fbd74076e40 2024-05-09T01:33:31.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd6affd700 1 -- 172.21.15.19:0/315178976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd70003f20 con 0x7fbd74076e40 2024-05-09T01:33:31.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd6affd700 1 -- 172.21.15.19:0/315178976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70022cf0 con 0x7fbd74076e40 2024-05-09T01:33:31.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.963+0000 7fbd7ba75700 1 -- 172.21.15.19:0/315178976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd741a9ae0 con 0x7fbd74076e40 2024-05-09T01:33:31.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.966+0000 7fbd6affd700 1 -- 172.21.15.19:0/315178976 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbd7002a070 con 0x7fbd74076e40 2024-05-09T01:33:31.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.967+0000 7fbd6affd700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbd600b2880 0x7fbd600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.967+0000 7fbd79010700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbd600b2880 0x7fbd600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.967+0000 7fbd6affd700 1 -- 172.21.15.19:0/315178976 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbd700e3030 con 0x7fbd74076e40 2024-05-09T01:33:31.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.967+0000 7fbd8a48bb80 1 Processor -- start 2024-05-09T01:33:31.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.967+0000 7fbd8a48bb80 1 -- start start 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 0x5570c7660240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbd600b2880 0x7fbd600b4d90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbd741033a0 tx=0x7fbd6400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c74dd4a0 0x5570c7660780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c765a400 0x5570c765a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570c757d0a0 con 0x5570c74dd4a0 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5570c75229a0 con 0x5570c765a400 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5570c7579060 con 0x5570c7557d00 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c74dd4a0 0x5570c7660780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c74dd4a0 0x5570c7660780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37436/0 (socket says 172.21.15.19:37436) 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 -- 172.21.15.19:0/3780622311 learned_addr learned my addr 172.21.15.19:0/3780622311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd7a012700 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c765a400 0x5570c765a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79811700 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 0x5570c7660240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 -- 172.21.15.19:0/3780622311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 msgr2=0x5570c7660240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:31.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 0x5570c7660240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 -- 172.21.15.19:0/3780622311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c765a400 msgr2=0x5570c765a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:31.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c765a400 0x5570c765a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 -- 172.21.15.19:0/3780622311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c7513500 con 0x5570c74dd4a0 2024-05-09T01:33:31.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd7a012700 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c765a400 0x5570c765a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:31.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd79010700 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c74dd4a0 0x5570c7660780 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbd64010f70 tx=0x7fbd64013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a09ac54f9fcb286 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:31.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.968+0000 7fbd5ffff700 1 -- 172.21.15.19:0/3780622311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400be60 con 0x5570c74dd4a0 2024-05-09T01:33:31.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.969+0000 7fbd5ffff700 1 -- 172.21.15.19:0/3780622311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd6401ead0 con 0x5570c74dd4a0 2024-05-09T01:33:31.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.969+0000 7fbd5ffff700 1 -- 172.21.15.19:0/3780622311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6401cbe0 con 0x5570c74dd4a0 2024-05-09T01:33:31.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.969+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3780622311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c74dd4a0 msgr2=0x5570c7660780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:31.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.969+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c74dd4a0 0x5570c7660780 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbd64010f70 tx=0x7fbd64013000 comp rx=0 tx=0).stop 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.969+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3780622311 shutdown_connections 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.969+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c765a400 0x5570c765a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.969+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 0x5570c7660240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.969+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3780622311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c74dd4a0 0x5570c7660780 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.969+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3780622311 >> 172.21.15.19:0/3780622311 conn(0x5570c74e0f70 msgr2=0x5570c7655220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.970+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3780622311 shutdown_connections 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.970+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3780622311 wait complete. 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.970+0000 7fbd8a48bb80 1 -- start start 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.970+0000 7fbd8a48bb80 1 Processor -- start 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c74dd4a0 0x5570c7663fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c7557d00 0x5570c7664500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570c765b160 con 0x5570c765a400 2024-05-09T01:33:31.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5570c7517b20 con 0x5570c7557d00 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5570c7667020 con 0x5570c74dd4a0 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd8a48bb80 10 client.0 _ll_register_callbacks cb 0x5570c73ee6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd7a012700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd79010700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c7557d00 0x5570c7664500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd7a012700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37446/0 (socket says 172.21.15.19:37446) 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd7a012700 1 -- 172.21.15.19:0/143465407 learned_addr learned my addr 172.21.15.19:0/143465407 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd79811700 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c74dd4a0 0x5570c7663fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd7a012700 1 -- 172.21.15.19:0/143465407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c74dd4a0 msgr2=0x5570c7663fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd7a012700 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c74dd4a0 0x5570c7663fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd7a012700 1 -- 172.21.15.19:0/143465407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c7557d00 msgr2=0x5570c7664500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd7a012700 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c7557d00 0x5570c7664500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd7a012700 1 -- 172.21.15.19:0/143465407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c73f8490 con 0x5570c765a400 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.971+0000 7fbd79010700 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c7557d00 0x5570c7664500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd7a012700 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664a40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c007ba0 tx=0x7fbd6c007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32569bd3dec73e37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/143465407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c01a020 con 0x5570c765a400 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/143465407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbd6c0118d0 con 0x5570c765a400 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/143465407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6401ead0 con 0x5570c765a400 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/143465407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbd6c025050 con 0x5570c765a400 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd8a48bb80 10 client.17508 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/143465407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570c76719f0 con 0x5570c765a400 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/143465407 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd6401ead0 con 0x5570c765a400 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.972+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/143465407 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbd6c016bf0 con 0x5570c765a400 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.973+0000 7fbd8a48bb80 10 client.17508 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.973+0000 7fbd8a48bb80 10 client.17508 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:33:31.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.973+0000 7fbd8a48bb80 20 client.17508 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.973+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5570c7674030 0x5570c7676480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.973+0000 7fbd8a48bb80 4 client.17508 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.973+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/143465407 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570c758aa80 con 0x5570c7674030 2024-05-09T01:33:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.973+0000 7fbd79811700 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5570c7674030 0x5570c7676480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.974+0000 7fbd5d7fa700 10 client.17508 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:31.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.974+0000 7fbd79811700 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5570c7674030 0x5570c7676480 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=47d9f49fcd2fa4e7 server_cookie=a38c461d0e6fd055 in_seq=0 out_seq=0 2024-05-09T01:33:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.974+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/143465407 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5570c758aa80 con 0x5570c7674030 2024-05-09T01:33:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.974+0000 7fbd5d7fa700 10 client.17508 handle_command_reply: tid=0 2024-05-09T01:33:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.974+0000 7fbd8a48bb80 1 client.17508 shutdown 2024-05-09T01:33:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/143465407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 msgr2=0x5570c7664a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664a40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c007ba0 tx=0x7fbd6c007eb0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/143465407 shutdown_connections 2024-05-09T01:33:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5570c7674030 0x5570c7676480 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664a40 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c74dd4a0 0x5570c7663fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd5d7fa700 0 client.17508 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/143465407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c7557d00 0x5570c7664500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/143465407 >> 172.21.15.19:0/143465407 conn(0x5570c74e0f70 msgr2=0x5570c764eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/143465407 shutdown_connections 2024-05-09T01:33:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/143465407 wait complete. 2024-05-09T01:33:31.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.975+0000 7fbd8a48bb80 20 client.17508 trim_cache size 0 max 16384 2024-05-09T01:33:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd8a48bb80 1 Processor -- start 2024-05-09T01:33:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd8a48bb80 1 -- start start 2024-05-09T01:33:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 0x5570c76632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 0x5570c7655330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741b0ea0 con 0x5570c765a400 2024-05-09T01:33:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741b29e0 con 0x5570c74dd4a0 2024-05-09T01:33:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741b1570 con 0x5570c7557d00 2024-05-09T01:33:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd7a012700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd7a012700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37454/0 (socket says 172.21.15.19:37454) 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.990+0000 7fbd7a012700 1 -- 172.21.15.19:0/3350276191 learned_addr learned my addr 172.21.15.19:0/3350276191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd79811700 1 --2- 172.21.15.19:0/3350276191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 0x5570c76632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd79010700 1 --2- 172.21.15.19:0/3350276191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 0x5570c7655330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd7a012700 1 -- 172.21.15.19:0/3350276191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 msgr2=0x5570c7655330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd7a012700 1 --2- 172.21.15.19:0/3350276191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 0x5570c7655330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd7a012700 1 -- 172.21.15.19:0/3350276191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 msgr2=0x5570c76632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd7a012700 1 --2- 172.21.15.19:0/3350276191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 0x5570c76632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd7a012700 1 -- 172.21.15.19:0/3350276191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c7665080 con 0x5570c765a400 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd7a012700 1 --2- 172.21.15.19:0/3350276191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664880 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c007b70 tx=0x7fbd6c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531e5de790d5f746 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd5f7fe700 1 -- 172.21.15.19:0/3350276191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c01a070 con 0x5570c765a400 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd5f7fe700 1 -- 172.21.15.19:0/3350276191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd6c005c60 con 0x5570c765a400 2024-05-09T01:33:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.991+0000 7fbd5f7fe700 1 -- 172.21.15.19:0/3350276191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c015950 con 0x5570c765a400 2024-05-09T01:33:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3350276191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 msgr2=0x5570c7664880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:31.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3350276191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664880 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c007b70 tx=0x7fbd6c0093d0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3350276191 shutdown_connections 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3350276191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c765a400 0x5570c7664880 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3350276191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 0x5570c76632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3350276191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c7557d00 0x5570c7655330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3350276191 >> 172.21.15.19:0/3350276191 conn(0x5570c74e0f70 msgr2=0x5570c764eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3350276191 shutdown_connections 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3350276191 wait complete. 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.992+0000 7fbd8a48bb80 1 -- start start 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd8a48bb80 1 Processor -- start 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 0x5570c766c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c7557d00 0x5570c766c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd8a48bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c765a400 0x5570c767af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741b2310 con 0x5570c7557d00 2024-05-09T01:33:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741b1c40 con 0x5570c74dd4a0 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd8a48bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741aee40 con 0x5570c765a400 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd8a48bb80 10 client.0 _ll_register_callbacks cb 0x5570c73ee6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd79010700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c7557d00 0x5570c766c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd79010700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c7557d00 0x5570c766c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37468/0 (socket says 172.21.15.19:37468) 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd79010700 1 -- 172.21.15.19:0/3694707810 learned_addr learned my addr 172.21.15.19:0/3694707810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd79811700 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 0x5570c766c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.993+0000 7fbd7a012700 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c765a400 0x5570c767af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd79010700 1 -- 172.21.15.19:0/3694707810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c765a400 msgr2=0x5570c767af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd79010700 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c765a400 0x5570c767af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd79010700 1 -- 172.21.15.19:0/3694707810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 msgr2=0x5570c766c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd79010700 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 0x5570c766c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd79010700 1 -- 172.21.15.19:0/3694707810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c767b6c0 con 0x5570c7557d00 2024-05-09T01:33:31.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd7a012700 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c765a400 0x5570c767af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd79010700 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c7557d00 0x5570c766c840 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fbd64010f10 tx=0x7fbd6400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64eaa328bd5292f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/3694707810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6401e830 con 0x5570c7557d00 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/3694707810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbd6401e9d0 con 0x5570c7557d00 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3694707810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6c005c60 con 0x5570c7557d00 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/3694707810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbd64004710 con 0x5570c7557d00 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd8a48bb80 10 client.17514 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3694707810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570c76de300 con 0x5570c7557d00 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.994+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/3694707810 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd6c005c60 con 0x5570c7557d00 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/3694707810 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbd640328d0 con 0x5570c7557d00 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd8a48bb80 10 client.17514 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd8a48bb80 10 client.17514 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd8a48bb80 20 client.17514 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5570c76de8a0 0x5570c76e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:31.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd8a48bb80 4 client.17514 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3694707810 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570c7584b20 con 0x5570c76de8a0 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd79811700 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5570c76de8a0 0x5570c76e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd79811700 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5570c76de8a0 0x5570c76e0cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f21d2baee9ccf812 server_cookie=1e3081f78c87da4f in_seq=0 out_seq=0 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.995+0000 7fbd5d7fa700 10 client.17514 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd5d7fa700 1 -- 172.21.15.19:0/3694707810 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570c7584b20 con 0x5570c76de8a0 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd5d7fa700 10 client.17514 handle_command_reply: tid=0 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 client.17514 shutdown 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3694707810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c7557d00 msgr2=0x5570c766c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c7557d00 0x5570c766c840 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fbd64010f10 tx=0x7fbd6400bea0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3694707810 shutdown_connections 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x5570c76de8a0 0x5570c76e0cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570c765a400 0x5570c767af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570c74dd4a0 0x5570c766c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 --2- 172.21.15.19:0/3694707810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c7557d00 0x5570c766c840 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd5d7fa700 0 client.17514 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:31.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3694707810 >> 172.21.15.19:0/3694707810 conn(0x5570c74e0f70 msgr2=0x5570c7663040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:31.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3694707810 shutdown_connections 2024-05-09T01:33:31.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 1 -- 172.21.15.19:0/3694707810 wait complete. 2024-05-09T01:33:31.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.996+0000 7fbd8a48bb80 20 client.17514 trim_cache size 0 max 16384 2024-05-09T01:33:31.998 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 -- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbd600b2880 msgr2=0x7fbd600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbd600b2880 0x7fbd600b4d90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbd741033a0 tx=0x7fbd6400b410 comp rx=0 tx=0).stop 2024-05-09T01:33:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 -- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 msgr2=0x7fbd7419d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 0x7fbd7419d000 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbd70000c00 tx=0x7fbd700047b0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 -- 172.21.15.19:0/315178976 shutdown_connections 2024-05-09T01:33:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd74110b80 0x7fbd741a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbd600b2880 0x7fbd600b4d90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd74076e40 0x7fbd7419d000 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 --2- 172.21.15.19:0/315178976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd74077790 0x7fbd7419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:32.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 -- 172.21.15.19:0/315178976 >> 172.21.15.19:0/315178976 conn(0x7fbd740ffea0 msgr2=0x7fbd741004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:32.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 -- 172.21.15.19:0/315178976 shutdown_connections 2024-05-09T01:33:32.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:31.997+0000 7fbd7ba75700 1 -- 172.21.15.19:0/315178976 wait complete. 2024-05-09T01:33:32.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:32 smithi019 ceph-mon[30158]: pgmap v618: 97 pgs: 97 active+clean; 952 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-05-09T01:33:32.672 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:33:32.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:32 smithi196 ceph-mon[35041]: pgmap v618: 97 pgs: 97 active+clean; 952 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-05-09T01:33:32.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:32 smithi059 ceph-mon[34351]: pgmap v618: 97 pgs: 97 active+clean; 952 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-05-09T01:33:32.928 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.670+0000 7f8e2c745700 1 -- 172.21.15.19:0/2540012536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24109860 msgr2=0x7f8e241111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.670+0000 7f8e2c745700 1 --2- 172.21.15.19:0/2540012536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24109860 0x7f8e241111c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00b0d0 tx=0x7f8e1c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:34 smithi019 ceph-mon[30158]: pgmap v619: 97 pgs: 97 active+clean; 952 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:34.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.670+0000 7f8e2c745700 1 -- 172.21.15.19:0/2540012536 shutdown_connections 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.670+0000 7f8e2c745700 1 --2- 172.21.15.19:0/2540012536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24109860 0x7f8e241111c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.670+0000 7f8e2c745700 1 --2- 172.21.15.19:0/2540012536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e241045e0 0x7f8e241049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.670+0000 7f8e2c745700 1 --2- 172.21.15.19:0/2540012536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e24108f10 0x7f8e24109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.670+0000 7f8e2c745700 1 -- 172.21.15.19:0/2540012536 >> 172.21.15.19:0/2540012536 conn(0x7f8e240ffef0 msgr2=0x7f8e24102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.670+0000 7f8e2c745700 1 -- 172.21.15.19:0/2540012536 shutdown_connections 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.670+0000 7f8e2c745700 1 -- 172.21.15.19:0/2540012536 wait complete. 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.671+0000 7f8e2c745700 1 Processor -- start 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.671+0000 7f8e2c745700 1 -- start start 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.671+0000 7f8e2c745700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241045e0 0x7f8e2419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.671+0000 7f8e2c745700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24108f10 0x7f8e2419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.671+0000 7f8e2c745700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e24109860 0x7f8e2419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.671+0000 7f8e2c745700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e24114490 con 0x7f8e24108f10 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.671+0000 7f8e2c745700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e24114310 con 0x7f8e24109860 2024-05-09T01:33:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.671+0000 7f8e2c745700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e24114610 con 0x7f8e241045e0 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e29ce0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24108f10 0x7f8e2419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e29ce0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24108f10 0x7f8e2419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37478/0 (socket says 172.21.15.19:37478) 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e29ce0700 1 -- 172.21.15.19:0/1570806013 learned_addr learned my addr 172.21.15.19:0/1570806013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241045e0 0x7f8e2419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e2ace2700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e24109860 0x7f8e2419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e29ce0700 1 -- 172.21.15.19:0/1570806013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241045e0 msgr2=0x7f8e2419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241045e0 0x7f8e2419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e29ce0700 1 -- 172.21.15.19:0/1570806013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e24109860 msgr2=0x7f8e2419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e24109860 0x7f8e2419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e29ce0700 1 -- 172.21.15.19:0/1570806013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2419fd30 con 0x7f8e24108f10 2024-05-09T01:33:34.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24108f10 0x7f8e2419b400 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400d980 tx=0x7f8e1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:34.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e1b7fe700 1 -- 172.21.15.19:0/1570806013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14017070 con 0x7f8e24108f10 2024-05-09T01:33:34.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.672+0000 7f8e1b7fe700 1 -- 172.21.15.19:0/1570806013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1400fd40 con 0x7f8e24108f10 2024-05-09T01:33:34.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.673+0000 7f8e1b7fe700 1 -- 172.21.15.19:0/1570806013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1400e3b0 con 0x7f8e24108f10 2024-05-09T01:33:34.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.673+0000 7f8e2c745700 1 -- 172.21.15.19:0/1570806013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e241a0020 con 0x7f8e24108f10 2024-05-09T01:33:34.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.673+0000 7f8e2c745700 1 -- 172.21.15.19:0/1570806013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e241a7940 con 0x7f8e24108f10 2024-05-09T01:33:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.675+0000 7f8e1b7fe700 1 -- 172.21.15.19:0/1570806013 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8e14010610 con 0x7f8e24108f10 2024-05-09T01:33:34.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.676+0000 7f8e1b7fe700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e100b2900 0x7f8e100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.676+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e100b2900 0x7f8e100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.676+0000 7f8e1b7fe700 1 -- 172.21.15.19:0/1570806013 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8e140e1bf0 con 0x7f8e24108f10 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e3b15bb80 1 Processor -- start 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e100b2900 0x7f8e100b4e10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009c60 tx=0x7f8e20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e3b15bb80 1 -- start start 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4772d00 0x5595f487b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4875400 0x5595f4875810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5595f47980a0 con 0x5595f4875400 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5595f473d9a0 con 0x5595f46f84a0 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595f4794060 con 0x5595f4772d00 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e2ace2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4875400 0x5595f4875810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e2ace2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4875400 0x5595f4875810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37490/0 (socket says 172.21.15.19:37490) 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e2ace2700 1 -- 172.21.15.19:0/1816030944 learned_addr learned my addr 172.21.15.19:0/1816030944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4772d00 0x5595f487b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.677+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e2ace2700 1 -- 172.21.15.19:0/1816030944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4772d00 msgr2=0x5595f487b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e2ace2700 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4772d00 0x5595f487b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e2ace2700 1 -- 172.21.15.19:0/1816030944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 msgr2=0x5595f487b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e2ace2700 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e2ace2700 1 -- 172.21.15.19:0/1816030944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f472e500 con 0x5595f4875400 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e2ace2700 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4875400 0x5595f4875810 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009c30 tx=0x7f8e1c009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1411c1aee33edfd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e18ff9700 1 -- 172.21.15.19:0/1816030944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0092e0 con 0x5595f4875400 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e18ff9700 1 -- 172.21.15.19:0/1816030944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1c020070 con 0x5595f4875400 2024-05-09T01:33:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.678+0000 7f8e18ff9700 1 -- 172.21.15.19:0/1816030944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c005490 con 0x5595f4875400 2024-05-09T01:33:34.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.679+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1816030944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4875400 msgr2=0x5595f4875810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:34.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.679+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4875400 0x5595f4875810 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009c30 tx=0x7f8e1c009c60 comp rx=0 tx=0).stop 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.679+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1816030944 shutdown_connections 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.679+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4875400 0x5595f4875810 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.679+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4772d00 0x5595f487b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.679+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/1816030944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.679+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1816030944 >> 172.21.15.19:0/1816030944 conn(0x5595f46fbf70 msgr2=0x5595f4870220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.679+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1816030944 shutdown_connections 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.679+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1816030944 wait complete. 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e3b15bb80 1 -- start start 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e3b15bb80 1 Processor -- start 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 0x5595f487fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5595f48760a0 con 0x5595f4772d00 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5595f4732b20 con 0x5595f46f84a0 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595f4876220 con 0x5595f4875400 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e3b15bb80 10 client.0 _ll_register_callbacks cb 0x5595f46096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e29ce0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e29ce0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37498/0 (socket says 172.21.15.19:37498) 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e29ce0700 1 -- 172.21.15.19:0/2626781623 learned_addr learned my addr 172.21.15.19:0/2626781623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e2ace2700 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 0x5595f487fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e29ce0700 1 -- 172.21.15.19:0/2626781623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 msgr2=0x5595f487fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 0x5595f487fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e29ce0700 1 -- 172.21.15.19:0/2626781623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 msgr2=0x5595f487f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.680+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e29ce0700 1 -- 172.21.15.19:0/2626781623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f4613490 con 0x5595f4772d00 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487f650 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400bfd0 tx=0x7f8e140a96d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8d75cfb82fec4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e0dffb700 1 -- 172.21.15.19:0/2626781623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14020030 con 0x5595f4772d00 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/2626781623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e1c020070 con 0x5595f4772d00 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e0dffb700 1 -- 172.21.15.19:0/2626781623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8e1400faa0 con 0x5595f4772d00 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e0dffb700 1 -- 172.21.15.19:0/2626781623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e140a8ca0 con 0x5595f4772d00 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e3b15bb80 10 client.17526 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/2626781623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595f488ca30 con 0x5595f4772d00 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e0dffb700 1 -- 172.21.15.19:0/2626781623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1c020070 con 0x5595f4772d00 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.681+0000 7f8e0dffb700 1 -- 172.21.15.19:0/2626781623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8e140ad120 con 0x5595f4772d00 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.682+0000 7f8e3b15bb80 10 client.17526 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.682+0000 7f8e3b15bb80 10 client.17526 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.682+0000 7f8e3b15bb80 20 client.17526 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.682+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5595f488efc0 0x5595f4891400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.682+0000 7f8e3b15bb80 4 client.17526 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.682+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/2626781623 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595f47a5a80 con 0x5595f488efc0 2024-05-09T01:33:34.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.682+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5595f488efc0 0x5595f4891400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.682+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5595f488efc0 0x5595f4891400 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7cb2edf39b834d server_cookie=2e343d1b965f6f24 in_seq=0 out_seq=0 2024-05-09T01:33:34.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.682+0000 7f8e0dffb700 10 client.17526 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e0dffb700 1 -- 172.21.15.19:0/2626781623 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5595f47a5a80 con 0x5595f488efc0 2024-05-09T01:33:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e0dffb700 10 client.17526 handle_command_reply: tid=0 2024-05-09T01:33:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e3b15bb80 1 client.17526 shutdown 2024-05-09T01:33:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/2626781623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 msgr2=0x5595f487f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487f650 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400bfd0 tx=0x7f8e140a96d0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/2626781623 shutdown_connections 2024-05-09T01:33:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 0x5595f487fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5595f488efc0 0x5595f4891400 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e0dffb700 0 client.17526 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.683+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/2626781623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487f650 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.684+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/2626781623 >> 172.21.15.19:0/2626781623 conn(0x5595f46fbf70 msgr2=0x5595f4869ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.684+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/2626781623 shutdown_connections 2024-05-09T01:33:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.684+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/2626781623 wait complete. 2024-05-09T01:33:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.684+0000 7f8e3b15bb80 20 client.17526 trim_cache size 0 max 16384 2024-05-09T01:33:34.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.699+0000 7f8e3b15bb80 1 Processor -- start 2024-05-09T01:33:34.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.699+0000 7f8e3b15bb80 1 -- start start 2024-05-09T01:33:34.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.699+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f46f84a0 0x5595f4869ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f486a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f4875400 0x5595f4878c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241ac8d0 con 0x5595f4772d00 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241ae410 con 0x5595f4875400 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241acfa0 con 0x5595f46f84a0 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e29ce0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f486a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e29ce0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f486a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37502/0 (socket says 172.21.15.19:37502) 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e29ce0700 1 -- 172.21.15.19:0/1203634560 learned_addr learned my addr 172.21.15.19:0/1203634560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f46f84a0 0x5595f4869ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e2ace2700 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f4875400 0x5595f4878c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e29ce0700 1 -- 172.21.15.19:0/1203634560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f46f84a0 msgr2=0x5595f4869ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f46f84a0 0x5595f4869ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e29ce0700 1 -- 172.21.15.19:0/1203634560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f4875400 msgr2=0x5595f4878c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f4875400 0x5595f4878c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e29ce0700 1 -- 172.21.15.19:0/1203634560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f486c9d0 con 0x5595f4772d00 2024-05-09T01:33:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e2ace2700 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f4875400 0x5595f4878c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f486a530 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8e140a8e40 tx=0x7f8e14010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc86979d9e81205e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e0ffff700 1 -- 172.21.15.19:0/1203634560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14020070 con 0x5595f4772d00 2024-05-09T01:33:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e0ffff700 1 -- 172.21.15.19:0/1203634560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e140a9890 con 0x5595f4772d00 2024-05-09T01:33:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.700+0000 7f8e0ffff700 1 -- 172.21.15.19:0/1203634560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e140ad120 con 0x5595f4772d00 2024-05-09T01:33:34.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.701+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1203634560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 msgr2=0x5595f486a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.701+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f486a530 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8e140a8e40 tx=0x7f8e14010000 comp rx=0 tx=0).stop 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.701+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1203634560 shutdown_connections 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.701+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f4875400 0x5595f4878c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.701+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f46f84a0 0x5595f4869ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.701+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/1203634560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f486a530 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.701+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1203634560 >> 172.21.15.19:0/1203634560 conn(0x5595f46fbf70 msgr2=0x5595f487da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.701+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1203634560 shutdown_connections 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.701+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/1203634560 wait complete. 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e3b15bb80 1 -- start start 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e3b15bb80 1 Processor -- start 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e3b15bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 0x5595f487ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241add40 con 0x5595f4772d00 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241ad670 con 0x5595f46f84a0 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e3b15bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241aa870 con 0x5595f4875400 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e3b15bb80 10 client.0 _ll_register_callbacks cb 0x5595f46096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e29ce0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e29ce0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37514/0 (socket says 172.21.15.19:37514) 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e29ce0700 1 -- 172.21.15.19:0/4204394667 learned_addr learned my addr 172.21.15.19:0/4204394667 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e2ace2700 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 0x5595f487ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.702+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e29ce0700 1 -- 172.21.15.19:0/4204394667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 msgr2=0x5595f487ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 0x5595f487ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e29ce0700 1 -- 172.21.15.19:0/4204394667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 msgr2=0x5595f487bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e29ce0700 1 -- 172.21.15.19:0/4204394667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f48f8770 con 0x5595f4772d00 2024-05-09T01:33:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e29ce0700 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487c4c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8e140ad700 tx=0x7f8e140a96d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563369ed3a12c7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e0dffb700 1 -- 172.21.15.19:0/4204394667 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14020070 con 0x5595f4772d00 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/4204394667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e140a9890 con 0x5595f4772d00 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e0dffb700 1 -- 172.21.15.19:0/4204394667 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f8e140dfd40 con 0x5595f4772d00 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e0dffb700 1 -- 172.21.15.19:0/4204394667 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e140a9190 con 0x5595f4772d00 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e3b15bb80 10 client.17532 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/4204394667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595f48f8770 con 0x5595f4772d00 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.703+0000 7f8e0dffb700 1 -- 172.21.15.19:0/4204394667 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e140a9890 con 0x5595f4772d00 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e0dffb700 1 -- 172.21.15.19:0/4204394667 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8e140ae6d0 con 0x5595f4772d00 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e3b15bb80 10 client.17532 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e3b15bb80 10 client.17532 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e3b15bb80 20 client.17532 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5595f488efc0 0x5595f48fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e3b15bb80 4 client.17532 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/4204394667 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595f479fb20 con 0x5595f488efc0 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5595f488efc0 0x5595f48fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e2a4e1700 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5595f488efc0 0x5595f48fbad0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b79c3bd205ea63e server_cookie=2b489aac2a23696a in_seq=0 out_seq=0 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.704+0000 7f8e0dffb700 10 client.17532 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e0dffb700 1 -- 172.21.15.19:0/4204394667 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595f479fb20 con 0x5595f488efc0 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e0dffb700 10 client.17532 handle_command_reply: tid=0 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 client.17532 shutdown 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/4204394667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 msgr2=0x5595f487c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487c4c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8e140ad700 tx=0x7f8e140a96d0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/4204394667 shutdown_connections 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f4875400 0x5595f487ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x5595f488efc0 0x5595f48fbad0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595f46f84a0 0x5595f487bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 --2- 172.21.15.19:0/4204394667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595f4772d00 0x5595f487c4c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/4204394667 >> 172.21.15.19:0/4204394667 conn(0x5595f46fbf70 msgr2=0x5595f4878c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:34.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e0dffb700 0 client.17532 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:34.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/4204394667 shutdown_connections 2024-05-09T01:33:34.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 1 -- 172.21.15.19:0/4204394667 wait complete. 2024-05-09T01:33:34.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e3b15bb80 20 client.17532 trim_cache size 0 max 16384 2024-05-09T01:33:34.707 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e2c745700 1 -- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e100b2900 msgr2=0x7f8e100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e2c745700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e100b2900 0x7f8e100b4e10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009c60 tx=0x7f8e20009260 comp rx=0 tx=0).stop 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e2c745700 1 -- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24108f10 msgr2=0x7f8e2419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.705+0000 7f8e2c745700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24108f10 0x7f8e2419b400 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400d980 tx=0x7f8e1400dc90 comp rx=0 tx=0).stop 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.706+0000 7f8e2c745700 1 -- 172.21.15.19:0/1570806013 shutdown_connections 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.706+0000 7f8e2c745700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e24109860 0x7f8e2419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.706+0000 7f8e2c745700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241045e0 0x7f8e2419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.706+0000 7f8e2c745700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f8e100b2900 0x7f8e100b4e10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.706+0000 7f8e2c745700 1 --2- 172.21.15.19:0/1570806013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e24108f10 0x7f8e2419b400 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.706+0000 7f8e2c745700 1 -- 172.21.15.19:0/1570806013 >> 172.21.15.19:0/1570806013 conn(0x7f8e240ffef0 msgr2=0x7f8e24108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.706+0000 7f8e2c745700 1 -- 172.21.15.19:0/1570806013 shutdown_connections 2024-05-09T01:33:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:34.706+0000 7f8e2c745700 1 -- 172.21.15.19:0/1570806013 wait complete. 2024-05-09T01:33:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:34 smithi059 ceph-mon[34351]: pgmap v619: 97 pgs: 97 active+clean; 952 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:34.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:34 smithi196 ceph-mon[35041]: pgmap v619: 97 pgs: 97 active+clean; 952 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:35.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:33:35.732 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 -- 172.21.15.19:0/1211412535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880103d90 msgr2=0x7f68801041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 --2- 172.21.15.19:0/1211412535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880103d90 0x7f68801041a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6874009900 tx=0x7f687402a410 comp rx=0 tx=0).stop 2024-05-09T01:33:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 -- 172.21.15.19:0/1211412535 shutdown_connections 2024-05-09T01:33:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 --2- 172.21.15.19:0/1211412535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880108a90 0x7f688010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 --2- 172.21.15.19:0/1211412535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880103d90 0x7f68801041a0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 --2- 172.21.15.19:0/1211412535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6880102b50 0x7f6880103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 -- 172.21.15.19:0/1211412535 >> 172.21.15.19:0/1211412535 conn(0x7f68800fdd40 msgr2=0x7f68801001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 -- 172.21.15.19:0/1211412535 shutdown_connections 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 -- 172.21.15.19:0/1211412535 wait complete. 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.116+0000 7f6888772640 1 Processor -- start 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6888772640 1 -- start start 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6888772640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6880102b50 0x7f688006af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6888772640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880103d90 0x7f688019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6888772640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880108a90 0x7f68801a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6888772640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f688010e680 con 0x7f6880108a90 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6888772640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f688010e500 con 0x7f6880102b50 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6888772640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f688010e800 con 0x7f6880103d90 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6886d0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880108a90 0x7f68801a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6886d0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880108a90 0x7f68801a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37534/0 (socket says 172.21.15.19:37534) 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6886d0f640 1 -- 172.21.15.19:0/89658210 learned_addr learned my addr 172.21.15.19:0/89658210 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f688650e640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6880102b50 0x7f688006af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6885d0d640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880103d90 0x7f688019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6886d0f640 1 -- 172.21.15.19:0/89658210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880103d90 msgr2=0x7f688019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6886d0f640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880103d90 0x7f688019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6886d0f640 1 -- 172.21.15.19:0/89658210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6880102b50 msgr2=0x7f688006af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6886d0f640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6880102b50 0x7f688006af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6886d0f640 1 -- 172.21.15.19:0/89658210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68801a1dc0 con 0x7f6880108a90 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.117+0000 7f6886d0f640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880108a90 0x7f68801a1630 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f687800e970 tx=0x7f687800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.118+0000 7f687f7fe640 1 -- 172.21.15.19:0/89658210 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687800ccc0 con 0x7f6880108a90 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.118+0000 7f687f7fe640 1 -- 172.21.15.19:0/89658210 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687800ce60 con 0x7f6880108a90 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.118+0000 7f6888772640 1 -- 172.21.15.19:0/89658210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68801a20b0 con 0x7f6880108a90 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.118+0000 7f687f7fe640 1 -- 172.21.15.19:0/89658210 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878010430 con 0x7f6880108a90 2024-05-09T01:33:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.118+0000 7f6888772640 1 -- 172.21.15.19:0/89658210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68801a2650 con 0x7f6880108a90 2024-05-09T01:33:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.120+0000 7f687f7fe640 1 -- 172.21.15.19:0/89658210 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6878004f90 con 0x7f6880108a90 2024-05-09T01:33:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f688f622740 1 Processor -- start 2024-05-09T01:33:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f688f622740 1 -- start start 2024-05-09T01:33:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f688f622740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c3034a1b0 0x555c306f2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f688f622740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 0x555c306f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f688f622740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f688f622740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c306527c0 con 0x555c303294d0 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f688f622740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c305ee0e0 con 0x555c303212f0 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f688f622740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555c3064e780 con 0x555c3034a1b0 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f6885d0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f6885d0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306f9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37542/0 (socket says 172.21.15.19:37542) 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f6885d0d640 1 -- 172.21.15.19:0/2731467439 learned_addr learned my addr 172.21.15.19:0/2731467439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f688650e640 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 0x555c306f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.121+0000 7f6885d0d640 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c3034a1b0 0x555c306f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f687f7fe640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68680b2860 0x7f68680b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f688650e640 1 -- 172.21.15.19:0/2731467439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c3034a1b0 msgr2=0x555c306f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f688650e640 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c3034a1b0 0x555c306f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f688650e640 1 -- 172.21.15.19:0/2731467439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 msgr2=0x555c306f9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f688650e640 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306f9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f688650e640 1 -- 172.21.15.19:0/2731467439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c306177c0 con 0x555c303212f0 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f6885d0d640 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306f9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f6886d0f640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68680b2860 0x7f68680b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f687f7fe640 1 -- 172.21.15.19:0/89658210 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f68780e2630 con 0x7f6880108a90 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f688650e640 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 0x555c306f2e90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6880102420 tx=0x7f687000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c37222c866cdbec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f687cff9640 1 -- 172.21.15.19:0/2731467439 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870003b70 con 0x555c303212f0 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f6886d0f640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68680b2860 0x7f68680b4d70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f687800e940 tx=0x7f687809e4e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f687cff9640 1 -- 172.21.15.19:0/2731467439 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6870003d10 con 0x555c303212f0 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.122+0000 7f687cff9640 1 -- 172.21.15.19:0/2731467439 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870010a40 con 0x555c303212f0 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.123+0000 7f688f622740 1 -- 172.21.15.19:0/2731467439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 msgr2=0x555c306f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.123+0000 7f688f622740 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 0x555c306f2e90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6880102420 tx=0x7f687000f790 comp rx=0 tx=0).stop 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.123+0000 7f688f622740 1 -- 172.21.15.19:0/2731467439 shutdown_connections 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.123+0000 7f688f622740 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306f9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.123+0000 7f688f622740 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 0x555c306f2e90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.123+0000 7f688f622740 1 --2- 172.21.15.19:0/2731467439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c3034a1b0 0x555c306f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.123+0000 7f688f622740 1 -- 172.21.15.19:0/2731467439 >> 172.21.15.19:0/2731467439 conn(0x555c3033e300 msgr2=0x555c306e9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.123+0000 7f688f622740 1 -- 172.21.15.19:0/2731467439 shutdown_connections 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.123+0000 7f688f622740 1 -- 172.21.15.19:0/2731467439 wait complete. 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688f622740 1 -- start start 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688f622740 1 Processor -- start 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688f622740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 0x555c306fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688f622740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688f622740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 0x555c30703b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688f622740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c306fa7e0 con 0x555c303294d0 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688f622740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c305d99c0 con 0x555c3034a1b0 2024-05-09T01:33:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688f622740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555c306fa960 con 0x555c303212f0 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688f622740 10 client.0 _ll_register_callbacks cb 0x555c30606020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688650e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688650e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306fcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37546/0 (socket says 172.21.15.19:37546) 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f688650e640 1 -- 172.21.15.19:0/94535025 learned_addr learned my addr 172.21.15.19:0/94535025 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f6885d0d640 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 0x555c30703b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.124+0000 7f6885d0d640 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 0x555c306fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f688650e640 1 -- 172.21.15.19:0/94535025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 msgr2=0x555c306fc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f688650e640 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 0x555c306fc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f688650e640 1 -- 172.21.15.19:0/94535025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 msgr2=0x555c30703b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f688650e640 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 0x555c30703b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f688650e640 1 -- 172.21.15.19:0/94535025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c306001f0 con 0x555c303294d0 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f688650e640 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306fcbb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f68700073d0 tx=0x7f6870007400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a67312a4fff0da6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f6865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f6865ffb640 1 -- 172.21.15.19:0/94535025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870026020 con 0x555c303294d0 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f688f622740 1 -- 172.21.15.19:0/94535025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6870003d10 con 0x555c303294d0 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f6865ffb640 1 -- 172.21.15.19:0/94535025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6870010b40 con 0x555c303294d0 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f6865ffb640 1 -- 172.21.15.19:0/94535025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6870031050 con 0x555c303294d0 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f688f622740 10 client.17544 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f688f622740 1 -- 172.21.15.19:0/94535025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c303b5790 con 0x555c303294d0 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.125+0000 7f6865ffb640 1 -- 172.21.15.19:0/94535025 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6870003d10 con 0x555c303294d0 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.126+0000 7f6865ffb640 1 -- 172.21.15.19:0/94535025 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6870022640 con 0x555c303294d0 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.126+0000 7f688f622740 10 client.17544 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.126+0000 7f688f622740 10 client.17544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.126+0000 7f688f622740 20 client.17544 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.126+0000 7f688f622740 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555c30308d80 0x555c30709c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.126+0000 7f688f622740 4 client.17544 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.126+0000 7f688f622740 1 -- 172.21.15.19:0/94535025 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c306603d0 con 0x555c30308d80 2024-05-09T01:33:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.126+0000 7f6885d0d640 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555c30308d80 0x555c30709c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.127+0000 7f6885d0d640 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555c30308d80 0x555c30709c90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b93b2bb2d5c5b6 server_cookie=85c8c12966e2259f in_seq=0 out_seq=0 2024-05-09T01:33:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.127+0000 7f6865ffb640 10 client.17544 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.127+0000 7f6865ffb640 1 -- 172.21.15.19:0/94535025 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c306603d0 con 0x555c30308d80 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.127+0000 7f6865ffb640 10 client.17544 handle_command_reply: tid=0 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.127+0000 7f688f622740 1 client.17544 shutdown 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 -- 172.21.15.19:0/94535025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 msgr2=0x555c306fcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306fcbb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f68700073d0 tx=0x7f6870007400 comp rx=0 tx=0).stop 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 -- 172.21.15.19:0/94535025 shutdown_connections 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555c30308d80 0x555c30709c90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 0x555c30703b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c306fcbb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f6865ffb640 0 client.17544 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 --2- 172.21.15.19:0/94535025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 0x555c306fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 -- 172.21.15.19:0/94535025 >> 172.21.15.19:0/94535025 conn(0x555c3033e300 msgr2=0x555c306e8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 -- 172.21.15.19:0/94535025 shutdown_connections 2024-05-09T01:33:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 1 -- 172.21.15.19:0/94535025 wait complete. 2024-05-09T01:33:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.128+0000 7f688f622740 20 client.17544 trim_cache size 0 max 16384 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688f622740 1 Processor -- start 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688f622740 1 -- start start 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688f622740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 0x555c3070e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688f622740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c30703a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688f622740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 0x555c30703f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688f622740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68801a9210 con 0x555c303294d0 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688f622740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f68801aad50 con 0x555c3034a1b0 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688f622740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68801a98e0 con 0x555c303212f0 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688650e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c30703a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688650e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c30703a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37554/0 (socket says 172.21.15.19:37554) 2024-05-09T01:33:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688650e640 1 -- 172.21.15.19:0/576030709 learned_addr learned my addr 172.21.15.19:0/576030709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f6885d0d640 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 0x555c3070e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f6885d0d640 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 0x555c30703f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.143+0000 7f688650e640 1 -- 172.21.15.19:0/576030709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 msgr2=0x555c3070e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.144+0000 7f688650e640 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 0x555c3070e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.144+0000 7f688650e640 1 -- 172.21.15.19:0/576030709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 msgr2=0x555c30703f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.144+0000 7f688650e640 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 0x555c30703f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.144+0000 7f688650e640 1 -- 172.21.15.19:0/576030709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3042c5f0 con 0x555c303294d0 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.144+0000 7f6885d0d640 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 0x555c30703f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.144+0000 7f688650e640 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c30703a30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6870007ff0 tx=0x7f6870007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fde26b0a4058d92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.144+0000 7f6867fff640 1 -- 172.21.15.19:0/576030709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687003a070 con 0x555c303294d0 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.144+0000 7f6867fff640 1 -- 172.21.15.19:0/576030709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68700049c0 con 0x555c303294d0 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.144+0000 7f6867fff640 1 -- 172.21.15.19:0/576030709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870026040 con 0x555c303294d0 2024-05-09T01:33:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 -- 172.21.15.19:0/576030709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 msgr2=0x555c30703a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c30703a30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6870007ff0 tx=0x7f6870007d10 comp rx=0 tx=0).stop 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 -- 172.21.15.19:0/576030709 shutdown_connections 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c3034a1b0 0x555c30703f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c303294d0 0x555c30703a30 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 --2- 172.21.15.19:0/576030709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303212f0 0x555c3070e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 -- 172.21.15.19:0/576030709 >> 172.21.15.19:0/576030709 conn(0x555c3033e300 msgr2=0x555c306f6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 -- 172.21.15.19:0/576030709 shutdown_connections 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 -- 172.21.15.19:0/576030709 wait complete. 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 -- start start 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 Processor -- start 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 0x555c30700f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303294d0 0x555c307014a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c3034a1b0 0x555c3076ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68801aa680 con 0x555c3034a1b0 2024-05-09T01:33:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f68801a9fb0 con 0x555c303212f0 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68801a71f0 con 0x555c303294d0 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f688f622740 10 client.0 _ll_register_callbacks cb 0x7f6878005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f6885d0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c3034a1b0 0x555c3076ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f6885d0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c3034a1b0 0x555c3076ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37566/0 (socket says 172.21.15.19:37566) 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.145+0000 7f6885d0d640 1 -- 172.21.15.19:0/2006923314 learned_addr learned my addr 172.21.15.19:0/2006923314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6885d0d640 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 0x555c30700f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f688650e640 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303294d0 0x555c307014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6885d0d640 1 -- 172.21.15.19:0/2006923314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303294d0 msgr2=0x555c307014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6885d0d640 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303294d0 0x555c307014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6885d0d640 1 -- 172.21.15.19:0/2006923314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 msgr2=0x555c30700f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6885d0d640 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 0x555c30700f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6885d0d640 1 -- 172.21.15.19:0/2006923314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c30616ed0 con 0x555c3034a1b0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6885d0d640 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c3034a1b0 0x555c3076ffe0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f68740096c0 tx=0x7f6870003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a3095a38ed913 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6865ffb640 1 -- 172.21.15.19:0/2006923314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874035040 con 0x555c3034a1b0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6865ffb640 1 -- 172.21.15.19:0/2006923314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f6874005020 con 0x555c3034a1b0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f688f622740 1 -- 172.21.15.19:0/2006923314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68700049c0 con 0x555c3034a1b0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f6865ffb640 1 -- 172.21.15.19:0/2006923314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6874061050 con 0x555c3034a1b0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f688f622740 10 client.17550 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.146+0000 7f688f622740 1 -- 172.21.15.19:0/2006923314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c30771290 con 0x555c3034a1b0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.147+0000 7f6865ffb640 1 -- 172.21.15.19:0/2006923314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68700049c0 con 0x555c3034a1b0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.147+0000 7f6865ffb640 1 -- 172.21.15.19:0/2006923314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f687403a040 con 0x555c3034a1b0 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.147+0000 7f688f622740 10 client.17550 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.147+0000 7f688f622740 10 client.17550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.147+0000 7f688f622740 20 client.17550 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.147+0000 7f688f622740 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555c302de1c0 0x555c30773c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.147+0000 7f688f622740 4 client.17550 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:33:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.147+0000 7f688f622740 1 -- 172.21.15.19:0/2006923314 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c3065a240 con 0x555c302de1c0 2024-05-09T01:33:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.147+0000 7f688650e640 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555c302de1c0 0x555c30773c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.148+0000 7f6865ffb640 10 client.17550 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.148+0000 7f688650e640 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555c302de1c0 0x555c30773c00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b788a7cffd3ca2 server_cookie=34dbaf45b23235c1 in_seq=0 out_seq=0 2024-05-09T01:33:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f6865ffb640 1 -- 172.21.15.19:0/2006923314 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555c3065a240 con 0x555c302de1c0 2024-05-09T01:33:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f6865ffb640 10 client.17550 handle_command_reply: tid=0 2024-05-09T01:33:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 client.17550 shutdown 2024-05-09T01:33:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 -- 172.21.15.19:0/2006923314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c3034a1b0 msgr2=0x555c3076ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c3034a1b0 0x555c3076ffe0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f68740096c0 tx=0x7f6870003c80 comp rx=0 tx=0).stop 2024-05-09T01:33:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 -- 172.21.15.19:0/2006923314 shutdown_connections 2024-05-09T01:33:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555c302de1c0 0x555c30773c00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c3034a1b0 0x555c3076ffe0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c303294d0 0x555c307014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f6865ffb640 0 client.17550 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 --2- 172.21.15.19:0/2006923314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c303212f0 0x555c30700f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 -- 172.21.15.19:0/2006923314 >> 172.21.15.19:0/2006923314 conn(0x555c3033e300 msgr2=0x555c306ecc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.161+0000 7f688f622740 1 -- 172.21.15.19:0/2006923314 shutdown_connections 2024-05-09T01:33:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f688f622740 1 -- 172.21.15.19:0/2006923314 wait complete. 2024-05-09T01:33:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f688f622740 20 client.17550 trim_cache size 0 max 16384 2024-05-09T01:33:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:33:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:33:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:33:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 -- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68680b2860 msgr2=0x7f68680b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68680b2860 0x7f68680b4d70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f687800e940 tx=0x7f687809e4e0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 -- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880108a90 msgr2=0x7f68801a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880108a90 0x7f68801a1630 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f687800e970 tx=0x7f687800ed70 comp rx=0 tx=0).stop 2024-05-09T01:33:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 -- 172.21.15.19:0/89658210 shutdown_connections 2024-05-09T01:33:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f68680b2860 0x7f68680b4d70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6880108a90 0x7f68801a1630 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880103d90 0x7f688019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 --2- 172.21.15.19:0/89658210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6880102b50 0x7f688006af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 -- 172.21.15.19:0/89658210 >> 172.21.15.19:0/89658210 conn(0x7f68800fdd40 msgr2=0x7f6880104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 -- 172.21.15.19:0/89658210 shutdown_connections 2024-05-09T01:33:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:36.162+0000 7f6888772640 1 -- 172.21.15.19:0/89658210 wait complete. 2024-05-09T01:33:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:36 smithi059 ceph-mon[34351]: pgmap v620: 97 pgs: 97 active+clean; 1013 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:33:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:36 smithi019 ceph-mon[30158]: pgmap v620: 97 pgs: 97 active+clean; 1013 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:33:36.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:36 smithi196 ceph-mon[35041]: pgmap v620: 97 pgs: 97 active+clean; 1013 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.467+0000 7fc9a76ae700 1 -- 172.21.15.19:0/4098226736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a01034c0 msgr2=0x7fc9a01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.467+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/4098226736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a01034c0 0x7fc9a01038d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc99000b0d0 tx=0x7fc99000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.467+0000 7fc9a76ae700 1 -- 172.21.15.19:0/4098226736 shutdown_connections 2024-05-09T01:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.467+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/4098226736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9a0108740 0x7fc9a01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.467+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/4098226736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a01034c0 0x7fc9a01038d0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.467+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/4098226736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a0107df0 0x7fc9a0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.467+0000 7fc9a76ae700 1 -- 172.21.15.19:0/4098226736 >> 172.21.15.19:0/4098226736 conn(0x7fc9a00fedc0 msgr2=0x7fc9a01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:37.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.468+0000 7fc9a76ae700 1 -- 172.21.15.19:0/4098226736 shutdown_connections 2024-05-09T01:33:37.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.468+0000 7fc9a76ae700 1 -- 172.21.15.19:0/4098226736 wait complete. 2024-05-09T01:33:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.468+0000 7fc9a76ae700 1 Processor -- start 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a76ae700 1 -- start start 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a76ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a01034c0 0x7fc9a019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a76ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9a0107df0 0x7fc9a019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a76ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a0108740 0x7fc9a019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a76ae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a0114520 con 0x7fc9a0108740 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a76ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01143a0 con 0x7fc9a01034c0 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a76ae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01146a0 con 0x7fc9a0107df0 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a5c4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a0108740 0x7fc9a019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a5c4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a0108740 0x7fc9a019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37586/0 (socket says 172.21.15.19:37586) 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/1514201174 learned_addr learned my addr 172.21.15.19:0/1514201174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9a0107df0 0x7fc9a019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a544a700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a01034c0 0x7fc9a019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/1514201174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9a0107df0 msgr2=0x7fc9a019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9a0107df0 0x7fc9a019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/1514201174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a01034c0 msgr2=0x7fc9a019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a01034c0 0x7fc9a019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.469+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/1514201174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a019fcb0 con 0x7fc9a0108740 2024-05-09T01:33:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.470+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a0108740 0x7fc9a019f6c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc99800c900 tx=0x7fc99800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.470+0000 7fc99e7fc700 1 -- 172.21.15.19:0/1514201174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9980078d0 con 0x7fc9a0108740 2024-05-09T01:33:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.470+0000 7fc99e7fc700 1 -- 172.21.15.19:0/1514201174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc99800f450 con 0x7fc9a0108740 2024-05-09T01:33:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.470+0000 7fc99e7fc700 1 -- 172.21.15.19:0/1514201174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998018620 con 0x7fc9a0108740 2024-05-09T01:33:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.470+0000 7fc9a76ae700 1 -- 172.21.15.19:0/1514201174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a019ffa0 con 0x7fc9a0108740 2024-05-09T01:33:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.470+0000 7fc9a76ae700 1 -- 172.21.15.19:0/1514201174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a01a0490 con 0x7fc9a0108740 2024-05-09T01:33:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.473+0000 7fc99e7fc700 1 -- 172.21.15.19:0/1514201174 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc998007a70 con 0x7fc9a0108740 2024-05-09T01:33:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.474+0000 7fc99e7fc700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc9880b2880 0x7fc9880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.474+0000 7fc99e7fc700 1 -- 172.21.15.19:0/1514201174 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc9980e2090 con 0x7fc9a0108740 2024-05-09T01:33:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.474+0000 7fc9a544a700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc9880b2880 0x7fc9880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.474+0000 7fc9b60c4b80 1 Processor -- start 2024-05-09T01:33:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.474+0000 7fc9b60c4b80 1 -- start start 2024-05-09T01:33:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.474+0000 7fc9a544a700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc9880b2880 0x7fc9880b4d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc990009900 tx=0x7fc99000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.474+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff530c6160 0x55ff5315f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.474+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 0x55ff5315fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.474+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff53159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff5307bff0 con 0x55ff531596a0 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ff5301a7f0 con 0x55ff530c6160 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff53077fb0 con 0x55ff52fdc4a0 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a5c4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff53159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a5c4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff53159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37598/0 (socket says 172.21.15.19:37598) 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/1388937463 learned_addr learned my addr 172.21.15.19:0/1388937463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a544a700 1 --2- 172.21.15.19:0/1388937463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff530c6160 0x55ff5315f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/1388937463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 0x55ff5315fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/1388937463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 msgr2=0x55ff5315fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/1388937463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 0x55ff5315fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/1388937463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff530c6160 msgr2=0x55ff5315f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:37.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/1388937463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff530c6160 0x55ff5315f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/1388937463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff53012370 con 0x55ff531596a0 2024-05-09T01:33:37.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/1388937463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff53159ad0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc9980a56b0 tx=0x7fc9980a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aefde826875d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:37.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9877fe700 1 -- 172.21.15.19:0/1388937463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9980a9630 con 0x55ff531596a0 2024-05-09T01:33:37.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9877fe700 1 -- 172.21.15.19:0/1388937463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9980ad9e0 con 0x55ff531596a0 2024-05-09T01:33:37.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.475+0000 7fc9877fe700 1 -- 172.21.15.19:0/1388937463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998016300 con 0x55ff531596a0 2024-05-09T01:33:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.476+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/1388937463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 msgr2=0x55ff53159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.476+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/1388937463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff53159ad0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc9980a56b0 tx=0x7fc9980a5620 comp rx=0 tx=0).stop 2024-05-09T01:33:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.476+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/1388937463 shutdown_connections 2024-05-09T01:33:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.476+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/1388937463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff53159ad0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.476+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/1388937463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff530c6160 0x55ff5315f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.476+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/1388937463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 0x55ff5315fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.476+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/1388937463 >> 172.21.15.19:0/1388937463 conn(0x55ff5303c6a0 msgr2=0x55ff53154400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.477+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/1388937463 shutdown_connections 2024-05-09T01:33:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.477+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/1388937463 wait complete. 2024-05-09T01:33:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.477+0000 7fc9b60c4b80 1 -- start start 2024-05-09T01:33:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.477+0000 7fc9b60c4b80 1 Processor -- start 2024-05-09T01:33:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 0x55ff531631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff531636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff531596a0 0x55ff53163c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ff52ef7ba0 con 0x55ff530c6160 2024-05-09T01:33:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ff53036150 con 0x55ff531596a0 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff531661d0 con 0x55ff52fdc4a0 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9b60c4b80 10 client.0 _ll_register_callbacks cb 0x55ff52eed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a4c49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff531636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a4c49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff531636e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37602/0 (socket says 172.21.15.19:37602) 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a4c49700 1 -- 172.21.15.19:0/3710113872 learned_addr learned my addr 172.21.15.19:0/3710113872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a544a700 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 0x55ff531631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff531596a0 0x55ff53163c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a4c49700 1 -- 172.21.15.19:0/3710113872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 msgr2=0x55ff531631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 0x55ff531631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a4c49700 1 -- 172.21.15.19:0/3710113872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff531596a0 msgr2=0x55ff53163c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff531596a0 0x55ff53163c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a4c49700 1 -- 172.21.15.19:0/3710113872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff52ef7490 con 0x55ff530c6160 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff531596a0 0x55ff53163c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff531636e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc994007e60 tx=0x7fc994012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7670e33580e7cfb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.478+0000 7fc984ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc984ff9700 1 -- 172.21.15.19:0/3710113872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994004520 con 0x55ff530c6160 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc984ff9700 1 -- 172.21.15.19:0/3710113872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc994003680 con 0x55ff530c6160 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/3710113872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9980ad9e0 con 0x55ff530c6160 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc984ff9700 1 -- 172.21.15.19:0/3710113872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc994003880 con 0x55ff530c6160 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc9b60c4b80 10 client.17562 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/3710113872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff53170c60 con 0x55ff530c6160 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc984ff9700 1 -- 172.21.15.19:0/3710113872 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9980ad9e0 con 0x55ff530c6160 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc984ff9700 1 -- 172.21.15.19:0/3710113872 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc994019020 con 0x55ff530c6160 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc9b60c4b80 10 client.17562 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc9b60c4b80 10 client.17562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc9b60c4b80 20 client.17562 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55ff531732f0 0x55ff53175710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc9b60c4b80 4 client.17562 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.479+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/3710113872 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff530899d0 con 0x55ff531732f0 2024-05-09T01:33:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.480+0000 7fc9a544a700 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55ff531732f0 0x55ff53175710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.480+0000 7fc9a544a700 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55ff531732f0 0x55ff53175710 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a96d88b2869f9614 server_cookie=337d63d00c53023d in_seq=0 out_seq=0 2024-05-09T01:33:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.480+0000 7fc984ff9700 10 client.17562 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.481+0000 7fc984ff9700 1 -- 172.21.15.19:0/3710113872 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff530899d0 con 0x55ff531732f0 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.481+0000 7fc984ff9700 10 client.17562 handle_command_reply: tid=0 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.481+0000 7fc9b60c4b80 1 client.17562 shutdown 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/3710113872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 msgr2=0x55ff531636e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff531636e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc994007e60 tx=0x7fc994012730 comp rx=0 tx=0).stop 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/3710113872 shutdown_connections 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55ff531732f0 0x55ff53175710 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc984ff9700 0 client.17562 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff531596a0 0x55ff53163c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff52fdc4a0 0x55ff531631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/3710113872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff531636e0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/3710113872 >> 172.21.15.19:0/3710113872 conn(0x55ff5303c6a0 msgr2=0x55ff531541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/3710113872 shutdown_connections 2024-05-09T01:33:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/3710113872 wait complete. 2024-05-09T01:33:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.482+0000 7fc9b60c4b80 20 client.17562 trim_cache size 0 max 16384 2024-05-09T01:33:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9b60c4b80 1 Processor -- start 2024-05-09T01:33:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9b60c4b80 1 -- start start 2024-05-09T01:33:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 0x55ff53163a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff5315d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff531596a0 0x55ff5316f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01aece0 con 0x55ff530c6160 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01b0820 con 0x55ff52fdc4a0 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01af3b0 con 0x55ff531596a0 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9a4c49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff5315d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9a4c49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff5315d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37610/0 (socket says 172.21.15.19:37610) 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9a4c49700 1 -- 172.21.15.19:0/689693846 learned_addr learned my addr 172.21.15.19:0/689693846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.497+0000 7fc9a544a700 1 --2- 172.21.15.19:0/689693846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 0x55ff53163a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.498+0000 7fc9a4c49700 1 -- 172.21.15.19:0/689693846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff531596a0 msgr2=0x55ff5316f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.498+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/689693846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff531596a0 0x55ff5316f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.498+0000 7fc9a4c49700 1 -- 172.21.15.19:0/689693846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 msgr2=0x55ff53163a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.498+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/689693846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 0x55ff53163a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.498+0000 7fc9a4c49700 1 -- 172.21.15.19:0/689693846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff53161710 con 0x55ff530c6160 2024-05-09T01:33:37.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.498+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/689693846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff5315d510 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc99401bc00 tx=0x7fc994012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7e316c99c67702 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:37.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.498+0000 7fc986ffd700 1 -- 172.21.15.19:0/689693846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994033070 con 0x55ff530c6160 2024-05-09T01:33:37.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.498+0000 7fc986ffd700 1 -- 172.21.15.19:0/689693846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc994003680 con 0x55ff530c6160 2024-05-09T01:33:37.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.498+0000 7fc986ffd700 1 -- 172.21.15.19:0/689693846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99401b710 con 0x55ff530c6160 2024-05-09T01:33:37.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/689693846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 msgr2=0x55ff5315d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:37.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/689693846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff5315d510 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc99401bc00 tx=0x7fc994012a80 comp rx=0 tx=0).stop 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/689693846 shutdown_connections 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/689693846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff531596a0 0x55ff5316f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/689693846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 0x55ff53163a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/689693846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff530c6160 0x55ff5315d510 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/689693846 >> 172.21.15.19:0/689693846 conn(0x55ff5303c6a0 msgr2=0x55ff5315ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/689693846 shutdown_connections 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/689693846 wait complete. 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 -- start start 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.499+0000 7fc9b60c4b80 1 Processor -- start 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 0x55ff5316b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff530c6160 0x55ff53179fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9b60c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff5317a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01b0150 con 0x55ff531596a0 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01afa80 con 0x55ff52fdc4a0 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9b60c4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01acc80 con 0x55ff530c6160 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9b60c4b80 10 client.0 _ll_register_callbacks cb 0x55ff52eed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a5c4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff5317a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a5c4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff5317a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37614/0 (socket says 172.21.15.19:37614) 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/406152934 learned_addr learned my addr 172.21.15.19:0/406152934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff530c6160 0x55ff53179fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a544a700 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 0x55ff5316b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/406152934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff530c6160 msgr2=0x55ff53179fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff530c6160 0x55ff53179fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/406152934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 msgr2=0x55ff5316b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 0x55ff5316b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a5c4b700 1 -- 172.21.15.19:0/406152934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff5317aa40 con 0x55ff531596a0 2024-05-09T01:33:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.500+0000 7fc9a4c49700 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff530c6160 0x55ff53179fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc9a5c4b700 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff5317a500 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc9980a8f10 tx=0x7fc9980b2500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e26c9b850ac7da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc984ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc984ff9700 1 -- 172.21.15.19:0/406152934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9980ad9e0 con 0x55ff531596a0 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc984ff9700 1 -- 172.21.15.19:0/406152934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc9980a9890 con 0x55ff531596a0 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/406152934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc994003680 con 0x55ff531596a0 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc984ff9700 1 -- 172.21.15.19:0/406152934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc9980c8050 con 0x55ff531596a0 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc9b60c4b80 10 client.17568 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/406152934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff531dd5a0 con 0x55ff531596a0 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc984ff9700 1 -- 172.21.15.19:0/406152934 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc994003680 con 0x55ff531596a0 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc984ff9700 1 -- 172.21.15.19:0/406152934 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc99808fba0 con 0x55ff531596a0 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc9b60c4b80 10 client.17568 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc9b60c4b80 10 client.17568 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.501+0000 7fc9b60c4b80 20 client.17568 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.502+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55ff531732f0 0x55ff531df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.502+0000 7fc9b60c4b80 4 client.17568 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.502+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/406152934 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff53083a70 con 0x55ff531732f0 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.502+0000 7fc9a544a700 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55ff531732f0 0x55ff531df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.502+0000 7fc9a544a700 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55ff531732f0 0x55ff531df890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=735e4032941a24ae server_cookie=c5a8f5cc18a1cc35 in_seq=0 out_seq=0 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.502+0000 7fc984ff9700 10 client.17568 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc984ff9700 1 -- 172.21.15.19:0/406152934 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff53083a70 con 0x55ff531732f0 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc984ff9700 10 client.17568 handle_command_reply: tid=0 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 client.17568 shutdown 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/406152934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 msgr2=0x55ff5317a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff5317a500 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc9980a8f10 tx=0x7fc9980b2500 comp rx=0 tx=0).stop 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/406152934 shutdown_connections 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55ff531732f0 0x55ff531df890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff531596a0 0x55ff5317a500 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff52fdc4a0 0x55ff5316b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 --2- 172.21.15.19:0/406152934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff530c6160 0x55ff53179fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc984ff9700 0 client.17568 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:33:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/406152934 >> 172.21.15.19:0/406152934 conn(0x55ff5303c6a0 msgr2=0x55ff5315ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/406152934 shutdown_connections 2024-05-09T01:33:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 1 -- 172.21.15.19:0/406152934 wait complete. 2024-05-09T01:33:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.503+0000 7fc9b60c4b80 20 client.17568 trim_cache size 0 max 16384 2024-05-09T01:33:37.505 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 -- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc9880b2880 msgr2=0x7fc9880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc9880b2880 0x7fc9880b4d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc990009900 tx=0x7fc99000b040 comp rx=0 tx=0).stop 2024-05-09T01:33:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 -- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a0108740 msgr2=0x7fc9a019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a0108740 0x7fc9a019f6c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc99800c900 tx=0x7fc99800cc10 comp rx=0 tx=0).stop 2024-05-09T01:33:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 -- 172.21.15.19:0/1514201174 shutdown_connections 2024-05-09T01:33:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc9880b2880 0x7fc9880b4d90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a0108740 0x7fc9a019f6c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a01034c0 0x7fc9a019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 --2- 172.21.15.19:0/1514201174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9a0107df0 0x7fc9a019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 -- 172.21.15.19:0/1514201174 >> 172.21.15.19:0/1514201174 conn(0x7fc9a00fedc0 msgr2=0x7fc9a01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 -- 172.21.15.19:0/1514201174 shutdown_connections 2024-05-09T01:33:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:37.504+0000 7fc9a76ae700 1 -- 172.21.15.19:0/1514201174 wait complete. 2024-05-09T01:33:38.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:33:38.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:38 smithi019 ceph-mon[30158]: pgmap v621: 97 pgs: 97 active+clean; 1013 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:38.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:38 smithi059 ceph-mon[34351]: pgmap v621: 97 pgs: 97 active+clean; 1013 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:38.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:38 smithi196 ceph-mon[35041]: pgmap v621: 97 pgs: 97 active+clean; 1013 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.487+0000 7f7f62d55700 1 -- 172.21.15.19:0/3985117316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 msgr2=0x7f7f5c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.487+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3985117316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 0x7f7f5c109310 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7f500099c0 tx=0x7f7f50009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.487+0000 7f7f62d55700 1 -- 172.21.15.19:0/3985117316 shutdown_connections 2024-05-09T01:33:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.487+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3985117316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f5c109850 0x7f7f5c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.487+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3985117316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f5c1045d0 0x7f7f5c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.487+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3985117316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 0x7f7f5c109310 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.487+0000 7f7f62d55700 1 -- 172.21.15.19:0/3985117316 >> 172.21.15.19:0/3985117316 conn(0x7f7f5c0ffea0 msgr2=0x7f7f5c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.487+0000 7f7f62d55700 1 -- 172.21.15.19:0/3985117316 shutdown_connections 2024-05-09T01:33:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.487+0000 7f7f62d55700 1 -- 172.21.15.19:0/3985117316 wait complete. 2024-05-09T01:33:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.488+0000 7f7f62d55700 1 Processor -- start 2024-05-09T01:33:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.488+0000 7f7f62d55700 1 -- start start 2024-05-09T01:33:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.488+0000 7f7f62d55700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f5c1045d0 0x7f7f5c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.488+0000 7f7f62d55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 0x7f7f5c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.488+0000 7f7f62d55700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f5c109850 0x7f7f5c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f62d55700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c114460 con 0x7f7f5c108f00 2024-05-09T01:33:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f62d55700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1142e0 con 0x7f7f5c109850 2024-05-09T01:33:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f62d55700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1145e0 con 0x7f7f5c1045d0 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 0x7f7f5c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 0x7f7f5c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37914/0 (socket says 172.21.15.19:37914) 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f5bfff700 1 -- 172.21.15.19:0/3213373124 learned_addr learned my addr 172.21.15.19:0/3213373124 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f60af1700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f5c1045d0 0x7f7f5c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f612f2700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f5c109850 0x7f7f5c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f5bfff700 1 -- 172.21.15.19:0/3213373124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f5c1045d0 msgr2=0x7f7f5c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f5c1045d0 0x7f7f5c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f5bfff700 1 -- 172.21.15.19:0/3213373124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f5c109850 msgr2=0x7f7f5c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f5c109850 0x7f7f5c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f5bfff700 1 -- 172.21.15.19:0/3213373124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c19fce0 con 0x7f7f5c108f00 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 0x7f7f5c19b3b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7f50006010 tx=0x7f7f500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f59ffb700 1 -- 172.21.15.19:0/3213373124 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50027070 con 0x7f7f5c108f00 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f62d55700 1 -- 172.21.15.19:0/3213373124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5c19ff70 con 0x7f7f5c108f00 2024-05-09T01:33:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.489+0000 7f7f59ffb700 1 -- 172.21.15.19:0/3213373124 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f50003f20 con 0x7f7f5c108f00 2024-05-09T01:33:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.490+0000 7f7f59ffb700 1 -- 172.21.15.19:0/3213373124 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50022c60 con 0x7f7f5c108f00 2024-05-09T01:33:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.490+0000 7f7f62d55700 1 -- 172.21.15.19:0/3213373124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c1a78f0 con 0x7f7f5c108f00 2024-05-09T01:33:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.492+0000 7f7f59ffb700 1 -- 172.21.15.19:0/3213373124 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7f5002a070 con 0x7f7f5c108f00 2024-05-09T01:33:40.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.493+0000 7f7f59ffb700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7f440b2880 0x7f7f440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.493+0000 7f7f60af1700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7f440b2880 0x7f7f440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.493+0000 7f7f59ffb700 1 -- 172.21.15.19:0/3213373124 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7f50022e00 con 0x7f7f5c108f00 2024-05-09T01:33:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.493+0000 7f7f7176bb80 1 Processor -- start 2024-05-09T01:33:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f7176bb80 1 -- start start 2024-05-09T01:33:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f60af1700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7f440b2880 0x7f7f440b4d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c005950 tx=0x7f7f4c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be0d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be07810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ed5bd2a0a0 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed5bccf9a0 con 0x55ed5bd04d00 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed5bd26060 con 0x55ed5be07400 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37916/0 (socket says 172.21.15.19:37916) 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f5bfff700 1 -- 172.21.15.19:0/4138860149 learned_addr learned my addr 172.21.15.19:0/4138860149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f612f2700 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be07810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f60af1700 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be0d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f5bfff700 1 -- 172.21.15.19:0/4138860149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 msgr2=0x55ed5be07810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be07810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f5bfff700 1 -- 172.21.15.19:0/4138860149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 msgr2=0x55ed5be0d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be0d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f5bfff700 1 -- 172.21.15.19:0/4138860149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed5bcc0500 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.494+0000 7f7f612f2700 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be07810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.495+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0d780 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7f500a7920 tx=0x7f7f50009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e64c1cb587f553 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.495+0000 7f7f42ffd700 1 -- 172.21.15.19:0/4138860149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500040d0 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.495+0000 7f7f42ffd700 1 -- 172.21.15.19:0/4138860149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f500aadb0 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.495+0000 7f7f42ffd700 1 -- 172.21.15.19:0/4138860149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500b3250 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 -- 172.21.15.19:0/4138860149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 msgr2=0x55ed5be0d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0d780 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7f500a7920 tx=0x7f7f50009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 -- 172.21.15.19:0/4138860149 shutdown_connections 2024-05-09T01:33:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be07810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be0d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/4138860149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0d780 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 -- 172.21.15.19:0/4138860149 >> 172.21.15.19:0/4138860149 conn(0x55ed5bc8df70 msgr2=0x55ed5be02220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 -- 172.21.15.19:0/4138860149 shutdown_connections 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 -- 172.21.15.19:0/4138860149 wait complete. 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 -- start start 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.496+0000 7f7f7176bb80 1 Processor -- start 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be16a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be16f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be10e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ed5bd2a0a0 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed5bcc4b20 con 0x55ed5bd04d00 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed5be07fb0 con 0x55ed5be07400 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f60af1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be16a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f7176bb80 10 client.0 _ll_register_callbacks cb 0x55ed5bb9b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f60af1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be16a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37932/0 (socket says 172.21.15.19:37932) 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f60af1700 1 -- 172.21.15.19:0/2348637771 learned_addr learned my addr 172.21.15.19:0/2348637771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be16f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f612f2700 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be10e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f60af1700 1 -- 172.21.15.19:0/2348637771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 msgr2=0x55ed5be10e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f60af1700 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be10e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f60af1700 1 -- 172.21.15.19:0/2348637771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 msgr2=0x55ed5be16f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f60af1700 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be16f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f60af1700 1 -- 172.21.15.19:0/2348637771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed5bba5490 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.497+0000 7f7f612f2700 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be10e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f60af1700 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be16a00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0116c0 tx=0x7f7f4c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7bc1c2f9ee83ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2348637771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c004a30 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2348637771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7f4c004bd0 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2348637771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f500aadb0 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2348637771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f4c004dd0 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f7176bb80 10 client.17580 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2348637771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed5be1e680 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2348637771 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f500aadb0 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.498+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2348637771 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7f4c01f940 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.499+0000 7f7f7176bb80 10 client.17580 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.499+0000 7f7f7176bb80 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.499+0000 7f7f7176bb80 20 client.17580 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.499+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ed5be20cc0 0x55ed5be23150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.499+0000 7f7f7176bb80 4 client.17580 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.499+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2348637771 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed5bd37a80 con 0x55ed5be20cc0 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.499+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ed5be20cc0 0x55ed5be23150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.499+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ed5be20cc0 0x55ed5be23150 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37be31b5706afb71 server_cookie=3bace41838b33918 in_seq=0 out_seq=0 2024-05-09T01:33:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.499+0000 7f7f3bfff700 10 client.17580 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.500+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2348637771 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed5bd37a80 con 0x55ed5be20cc0 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.500+0000 7f7f3bfff700 10 client.17580 handle_command_reply: tid=0 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.500+0000 7f7f7176bb80 1 client.17580 shutdown 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.500+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2348637771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 msgr2=0x55ed5be16a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.500+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be16a00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0116c0 tx=0x7f7f4c0047c0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.500+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2348637771 shutdown_connections 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.500+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ed5be20cc0 0x55ed5be23150 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.501+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5be07400 0x55ed5be10e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.501+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be16a00 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.501+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2348637771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be16f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.501+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2348637771 >> 172.21.15.19:0/2348637771 conn(0x55ed5bc8df70 msgr2=0x55ed5bdfbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.501+0000 7f7f3bfff700 0 client.17580 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.501+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2348637771 shutdown_connections 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.501+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2348637771 wait complete. 2024-05-09T01:33:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.501+0000 7f7f7176bb80 20 client.17580 trim_cache size 0 max 16384 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f7176bb80 1 Processor -- start 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f7176bb80 1 -- start start 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bd04d00 0x55ed5be28960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be28ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1aeb80 con 0x55ed5be07400 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1b06c0 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1af250 con 0x55ed5bd04d00 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f612f2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be28ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f612f2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be28ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37940/0 (socket says 172.21.15.19:37940) 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f612f2700 1 -- 172.21.15.19:0/2452958329 learned_addr learned my addr 172.21.15.19:0/2452958329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f60af1700 1 --2- 172.21.15.19:0/2452958329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.516+0000 7f7f5bfff700 1 --2- 172.21.15.19:0/2452958329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bd04d00 0x55ed5be28960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.517+0000 7f7f612f2700 1 -- 172.21.15.19:0/2452958329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bd04d00 msgr2=0x55ed5be28960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.517+0000 7f7f612f2700 1 --2- 172.21.15.19:0/2452958329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bd04d00 0x55ed5be28960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.517+0000 7f7f612f2700 1 -- 172.21.15.19:0/2452958329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bc8a4a0 msgr2=0x55ed5be0b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.517+0000 7f7f612f2700 1 --2- 172.21.15.19:0/2452958329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.517+0000 7f7f612f2700 1 -- 172.21.15.19:0/2452958329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed5be02f70 con 0x55ed5be07400 2024-05-09T01:33:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.517+0000 7f7f612f2700 1 --2- 172.21.15.19:0/2452958329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be28ea0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009a40 tx=0x7f7f54003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06bbd42210143d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.517+0000 7f7f427fc700 1 -- 172.21.15.19:0/2452958329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5401a070 con 0x55ed5be07400 2024-05-09T01:33:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.517+0000 7f7f427fc700 1 -- 172.21.15.19:0/2452958329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f54004db0 con 0x55ed5be07400 2024-05-09T01:33:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.517+0000 7f7f427fc700 1 -- 172.21.15.19:0/2452958329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54007d20 con 0x55ed5be07400 2024-05-09T01:33:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2452958329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 msgr2=0x55ed5be28ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2452958329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be28ea0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009a40 tx=0x7f7f54003f90 comp rx=0 tx=0).stop 2024-05-09T01:33:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2452958329 shutdown_connections 2024-05-09T01:33:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2452958329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be28ea0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2452958329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be0b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2452958329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bd04d00 0x55ed5be28960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2452958329 >> 172.21.15.19:0/2452958329 conn(0x55ed5bc8df70 msgr2=0x55ed5be0ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2452958329 shutdown_connections 2024-05-09T01:33:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2452958329 wait complete. 2024-05-09T01:33:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 -- start start 2024-05-09T01:33:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.518+0000 7f7f7176bb80 1 Processor -- start 2024-05-09T01:33:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be875a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be87ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f7176bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be819a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1afff0 con 0x55ed5be07400 2024-05-09T01:33:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1af920 con 0x55ed5bd04d00 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f7176bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1acb40 con 0x55ed5bc8a4a0 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f7176bb80 10 client.0 _ll_register_callbacks cb 0x55ed5bb9b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f612f2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f612f2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be819a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37948/0 (socket says 172.21.15.19:37948) 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f5bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be87ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f612f2700 1 -- 172.21.15.19:0/2155697071 learned_addr learned my addr 172.21.15.19:0/2155697071 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f60af1700 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be875a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f612f2700 1 -- 172.21.15.19:0/2155697071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bc8a4a0 msgr2=0x55ed5be875a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f612f2700 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be875a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f612f2700 1 -- 172.21.15.19:0/2155697071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 msgr2=0x55ed5be87ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f612f2700 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be87ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.519+0000 7f7f612f2700 1 -- 172.21.15.19:0/2155697071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed5be82010 con 0x55ed5be07400 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f612f2700 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be819a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7f54006070 tx=0x7f7f54007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ac343d6a83e472 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2155697071 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5401a040 con 0x55ed5be07400 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2155697071 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7f54004310 con 0x55ed5be07400 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2155697071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f54004db0 con 0x55ed5be07400 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2155697071 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f54004510 con 0x55ed5be07400 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f7176bb80 10 client.17586 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2155697071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed5be8ec70 con 0x55ed5be07400 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2155697071 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f54004db0 con 0x55ed5be07400 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2155697071 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7f54033420 con 0x55ed5be07400 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f7176bb80 10 client.17586 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f7176bb80 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f7176bb80 20 client.17586 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ed5be8f260 0x55ed5be91680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f7176bb80 4 client.17586 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.520+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2155697071 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed5bd31b20 con 0x55ed5be8f260 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.521+0000 7f7f60af1700 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ed5be8f260 0x55ed5be91680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.521+0000 7f7f60af1700 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ed5be8f260 0x55ed5be91680 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a07733d562231d1 server_cookie=ca20486df268999e in_seq=0 out_seq=0 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.521+0000 7f7f3bfff700 10 client.17586 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.521+0000 7f7f3bfff700 1 -- 172.21.15.19:0/2155697071 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed5bd31b20 con 0x55ed5be8f260 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.521+0000 7f7f3bfff700 10 client.17586 handle_command_reply: tid=0 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.521+0000 7f7f7176bb80 1 client.17586 shutdown 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2155697071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 msgr2=0x55ed5be819a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be819a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7f54006070 tx=0x7f7f54007bd0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2155697071 shutdown_connections 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ed5be8f260 0x55ed5be91680 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed5be07400 0x55ed5be819a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed5bc8a4a0 0x55ed5be875a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 --2- 172.21.15.19:0/2155697071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed5bd04d00 0x55ed5be87ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2155697071 >> 172.21.15.19:0/2155697071 conn(0x55ed5bc8df70 msgr2=0x55ed5be0bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:40.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f3bfff700 0 client.17586 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:40.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2155697071 shutdown_connections 2024-05-09T01:33:40.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 1 -- 172.21.15.19:0/2155697071 wait complete. 2024-05-09T01:33:40.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.522+0000 7f7f7176bb80 20 client.17586 trim_cache size 0 max 16384 2024-05-09T01:33:40.524 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 -- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7f440b2880 msgr2=0x7f7f440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7f440b2880 0x7f7f440b4d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c005950 tx=0x7f7f4c00b410 comp rx=0 tx=0).stop 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 -- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 msgr2=0x7f7f5c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 0x7f7f5c19b3b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7f50006010 tx=0x7f7f500047b0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 -- 172.21.15.19:0/3213373124 shutdown_connections 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f5c109850 0x7f7f5c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f5c1045d0 0x7f7f5c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7f440b2880 0x7f7f440b4d90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 --2- 172.21.15.19:0/3213373124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c108f00 0x7f7f5c19b3b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 -- 172.21.15.19:0/3213373124 >> 172.21.15.19:0/3213373124 conn(0x7f7f5c0ffea0 msgr2=0x7f7f5c1086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 -- 172.21.15.19:0/3213373124 shutdown_connections 2024-05-09T01:33:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:40.523+0000 7f7f62d55700 1 -- 172.21.15.19:0/3213373124 wait complete. 2024-05-09T01:33:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:40 smithi059 ceph-mon[34351]: pgmap v622: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-05-09T01:33:40.892 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:40 smithi019 ceph-mon[30158]: pgmap v622: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-05-09T01:33:40.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:40 smithi196 ceph-mon[35041]: pgmap v622: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-05-09T01:33:42.448 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:33:42.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:33:42.715 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:42.837 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:42 smithi019 ceph-mon[30158]: pgmap v623: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:33:42.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:42 smithi059 ceph-mon[34351]: pgmap v623: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:33:42.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:42 smithi196 ceph-mon[35041]: pgmap v623: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:33:44.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.430+0000 7f38a44ec700 1 -- 172.21.15.19:0/3859713746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c108740 msgr2=0x7f389c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:44.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.430+0000 7f38a44ec700 1 --2- 172.21.15.19:0/3859713746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c108740 0x7f389c1100a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f38940099c0 tx=0x7f3894009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.430+0000 7f38a44ec700 1 -- 172.21.15.19:0/3859713746 shutdown_connections 2024-05-09T01:33:44.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.430+0000 7f38a44ec700 1 --2- 172.21.15.19:0/3859713746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c108740 0x7f389c1100a0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.430+0000 7f38a44ec700 1 --2- 172.21.15.19:0/3859713746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f389c1034c0 0x7f389c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.430+0000 7f38a44ec700 1 --2- 172.21.15.19:0/3859713746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f389c107df0 0x7f389c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.430+0000 7f38a44ec700 1 -- 172.21.15.19:0/3859713746 >> 172.21.15.19:0/3859713746 conn(0x7f389c0fedc0 msgr2=0x7f389c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:44.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.430+0000 7f38a44ec700 1 -- 172.21.15.19:0/3859713746 shutdown_connections 2024-05-09T01:33:44.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.430+0000 7f38a44ec700 1 -- 172.21.15.19:0/3859713746 wait complete. 2024-05-09T01:33:44.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a44ec700 1 Processor -- start 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a44ec700 1 -- start start 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a44ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c1034c0 0x7f389c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a44ec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f389c107df0 0x7f389c19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a44ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f389c108740 0x7f389c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a44ec700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f389c113370 con 0x7f389c1034c0 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a44ec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f389c1131f0 con 0x7f389c108740 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a44ec700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f389c1134f0 con 0x7f389c107df0 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a2288700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c1034c0 0x7f389c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a2288700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c1034c0 0x7f389c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37960/0 (socket says 172.21.15.19:37960) 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a2288700 1 -- 172.21.15.19:0/2872670380 learned_addr learned my addr 172.21.15.19:0/2872670380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a2a89700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f389c108740 0x7f389c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.431+0000 7f38a1a87700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f389c107df0 0x7f389c19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f38a2288700 1 -- 172.21.15.19:0/2872670380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f389c107df0 msgr2=0x7f389c19c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f38a2288700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f389c107df0 0x7f389c19c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f38a2288700 1 -- 172.21.15.19:0/2872670380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f389c108740 msgr2=0x7f389c19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f38a2288700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f389c108740 0x7f389c19c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f38a2288700 1 -- 172.21.15.19:0/2872670380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c1a6f00 con 0x7f389c1034c0 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f38a2288700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c1034c0 0x7f389c19bb90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f388c0060b0 tx=0x7f388c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f389b7fe700 1 -- 172.21.15.19:0/2872670380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c01a070 con 0x7f389c1034c0 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f389b7fe700 1 -- 172.21.15.19:0/2872670380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c00dd20 con 0x7f389c1034c0 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f389b7fe700 1 -- 172.21.15.19:0/2872670380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c014600 con 0x7f389c1034c0 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f38a44ec700 1 -- 172.21.15.19:0/2872670380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c1a71f0 con 0x7f389c1034c0 2024-05-09T01:33:44.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.432+0000 7f38a44ec700 1 -- 172.21.15.19:0/2872670380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c1a76e0 con 0x7f389c1034c0 2024-05-09T01:33:44.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.435+0000 7f389b7fe700 1 -- 172.21.15.19:0/2872670380 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f388c01d070 con 0x7f389c1034c0 2024-05-09T01:33:44.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.436+0000 7f38b2f02b80 1 Processor -- start 2024-05-09T01:33:44.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.436+0000 7f389b7fe700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f38840b2900 0x7f38840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.436+0000 7f38a2a89700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f38840b2900 0x7f38840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.436+0000 7f38b2f02b80 1 -- start start 2024-05-09T01:33:44.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.436+0000 7f389b7fe700 1 -- 172.21.15.19:0/2872670380 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f388c0e1a50 con 0x7f389c1034c0 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 0x55df59d56b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d5b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df59c75120 con 0x55df59d5b260 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df59c1aa20 con 0x55df59c4fd80 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df59c710e0 con 0x55df59bd5520 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d5b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d5b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37964/0 (socket says 172.21.15.19:37964) 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 -- 172.21.15.19:0/1346763274 learned_addr learned my addr 172.21.15.19:0/1346763274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f38840b2900 0x7f38840b4e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f389c06c6f0 tx=0x7f3894005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2288700 1 --2- 172.21.15.19:0/1346763274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 0x55df59d56b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a1a87700 1 --2- 172.21.15.19:0/1346763274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 -- 172.21.15.19:0/1346763274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 msgr2=0x55df59d570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:44.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 --2- 172.21.15.19:0/1346763274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 -- 172.21.15.19:0/1346763274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 msgr2=0x55df59d56b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:44.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 --2- 172.21.15.19:0/1346763274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 0x55df59d56b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 -- 172.21.15.19:0/1346763274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df59c0b580 con 0x55df59d5b260 2024-05-09T01:33:44.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.437+0000 7f38a2a89700 1 --2- 172.21.15.19:0/1346763274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d5b670 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f38940099c0 tx=0x7f389401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a926fb1657c3910e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:44.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.438+0000 7f3898d47700 1 -- 172.21.15.19:0/1346763274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894003e90 con 0x55df59d5b260 2024-05-09T01:33:44.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.438+0000 7f3898d47700 1 -- 172.21.15.19:0/1346763274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f389400bc40 con 0x55df59d5b260 2024-05-09T01:33:44.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.438+0000 7f3898d47700 1 -- 172.21.15.19:0/1346763274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389400f970 con 0x55df59d5b260 2024-05-09T01:33:44.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.438+0000 7f38b2f02b80 1 -- 172.21.15.19:0/1346763274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 msgr2=0x55df59d5b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:44.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.438+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/1346763274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d5b670 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f38940099c0 tx=0x7f389401e000 comp rx=0 tx=0).stop 2024-05-09T01:33:44.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.439+0000 7f38b2f02b80 1 -- 172.21.15.19:0/1346763274 shutdown_connections 2024-05-09T01:33:44.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.439+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/1346763274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d5b670 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.439+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/1346763274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 0x55df59d56b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.439+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/1346763274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.439+0000 7f38b2f02b80 1 -- 172.21.15.19:0/1346763274 >> 172.21.15.19:0/1346763274 conn(0x55df59bd8ff0 msgr2=0x55df59d4d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:44.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.439+0000 7f38b2f02b80 1 -- 172.21.15.19:0/1346763274 shutdown_connections 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.439+0000 7f38b2f02b80 1 -- 172.21.15.19:0/1346763274 wait complete. 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.439+0000 7f38b2f02b80 1 -- start start 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.439+0000 7f38b2f02b80 1 Processor -- start 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 0x55df59d60a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d60f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df59d5bf20 con 0x55df59d5b260 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df59c0fba0 con 0x55df59c4fd80 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df59d5c0a0 con 0x55df59bd5520 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38b2f02b80 10 client.0 _ll_register_callbacks cb 0x55df59ae6740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a2a89700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d60f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a2a89700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d60f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37980/0 (socket says 172.21.15.19:37980) 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a2a89700 1 -- 172.21.15.19:0/868401739 learned_addr learned my addr 172.21.15.19:0/868401739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a2288700 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a1a87700 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 0x55df59d60a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a2a89700 1 -- 172.21.15.19:0/868401739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 msgr2=0x55df59d604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a2a89700 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a2a89700 1 -- 172.21.15.19:0/868401739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 msgr2=0x55df59d60a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a2a89700 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 0x55df59d60a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a2a89700 1 -- 172.21.15.19:0/868401739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df59af0510 con 0x55df59d5b260 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.440+0000 7f38a1a87700 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 0x55df59d60a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f38a2a89700 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d60f40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f389400b580 tx=0x7f3894005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ede62ce78f5345 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f3881ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f3881ffb700 1 -- 172.21.15.19:0/868401739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894033030 con 0x55df59d5b260 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f3881ffb700 1 -- 172.21.15.19:0/868401739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f389402b050 con 0x55df59d5b260 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f38b2f02b80 1 -- 172.21.15.19:0/868401739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389400bc40 con 0x55df59d5b260 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f3881ffb700 1 -- 172.21.15.19:0/868401739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f389400b9a0 con 0x55df59d5b260 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f38b2f02b80 10 client.17598 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f38b2f02b80 1 -- 172.21.15.19:0/868401739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df59d6ddd0 con 0x55df59d5b260 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f3881ffb700 1 -- 172.21.15.19:0/868401739 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f389400bc40 con 0x55df59d5b260 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f3881ffb700 1 -- 172.21.15.19:0/868401739 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3894022070 con 0x55df59d5b260 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f38b2f02b80 10 client.17598 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.441+0000 7f38b2f02b80 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.442+0000 7f38b2f02b80 20 client.17598 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.442+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df59d702a0 0x55df59d726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.442+0000 7f38b2f02b80 4 client.17598 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.442+0000 7f38b2f02b80 1 -- 172.21.15.19:0/868401739 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df59c82b00 con 0x55df59d702a0 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.442+0000 7f38a2288700 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df59d702a0 0x55df59d726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.442+0000 7f38a2288700 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df59d702a0 0x55df59d726c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a8b648400baf709 server_cookie=4de1a8047c363549 in_seq=0 out_seq=0 2024-05-09T01:33:44.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.442+0000 7f3881ffb700 10 client.17598 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:44.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f3881ffb700 1 -- 172.21.15.19:0/868401739 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df59c82b00 con 0x55df59d702a0 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f3881ffb700 10 client.17598 handle_command_reply: tid=0 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 client.17598 shutdown 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 -- 172.21.15.19:0/868401739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 msgr2=0x55df59d60f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d60f40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f389400b580 tx=0x7f3894005710 comp rx=0 tx=0).stop 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 -- 172.21.15.19:0/868401739 shutdown_connections 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df59d702a0 0x55df59d726c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59d5b260 0x55df59d60f40 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/868401739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59c4fd80 0x55df59d60a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f3881ffb700 0 client.17598 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 -- 172.21.15.19:0/868401739 >> 172.21.15.19:0/868401739 conn(0x55df59bd8ff0 msgr2=0x55df59d47070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.443+0000 7f38b2f02b80 1 -- 172.21.15.19:0/868401739 shutdown_connections 2024-05-09T01:33:44.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.444+0000 7f38b2f02b80 1 -- 172.21.15.19:0/868401739 wait complete. 2024-05-09T01:33:44.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.444+0000 7f38b2f02b80 20 client.17598 trim_cache size 0 max 16384 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.464+0000 7f38b2f02b80 1 Processor -- start 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.464+0000 7f38b2f02b80 1 -- start start 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d51ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59c4fd80 0x55df59d6c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 0x55df59d60d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f389c1ac9c0 con 0x55df59c4fd80 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f389c1ae500 con 0x55df59d5b260 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f389c1ad090 con 0x55df59bd5520 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a1a87700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59c4fd80 0x55df59d6c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a1a87700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59c4fd80 0x55df59d6c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37994/0 (socket says 172.21.15.19:37994) 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a1a87700 1 -- 172.21.15.19:0/2177109351 learned_addr learned my addr 172.21.15.19:0/2177109351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a1a87700 1 -- 172.21.15.19:0/2177109351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 msgr2=0x55df59d51ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:44.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a1a87700 1 --2- 172.21.15.19:0/2177109351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d51ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a1a87700 1 -- 172.21.15.19:0/2177109351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 msgr2=0x55df59d60d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:44.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a2a89700 1 --2- 172.21.15.19:0/2177109351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 0x55df59d60d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a1a87700 1 --2- 172.21.15.19:0/2177109351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 0x55df59d60d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a1a87700 1 -- 172.21.15.19:0/2177109351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df59d49590 con 0x55df59c4fd80 2024-05-09T01:33:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f38a1a87700 1 --2- 172.21.15.19:0/2177109351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59c4fd80 0x55df59d6c5d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f389c076ab0 tx=0x7f38900046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a5b4f2eb7f022d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.465+0000 7f3883fff700 1 -- 172.21.15.19:0/2177109351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890004830 con 0x55df59c4fd80 2024-05-09T01:33:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f3883fff700 1 -- 172.21.15.19:0/2177109351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3890005020 con 0x55df59c4fd80 2024-05-09T01:33:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f3883fff700 1 -- 172.21.15.19:0/2177109351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38900039a0 con 0x55df59c4fd80 2024-05-09T01:33:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f38b2f02b80 1 -- 172.21.15.19:0/2177109351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59c4fd80 msgr2=0x55df59d6c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/2177109351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59c4fd80 0x55df59d6c5d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f389c076ab0 tx=0x7f38900046a0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f38b2f02b80 1 -- 172.21.15.19:0/2177109351 shutdown_connections 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/2177109351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 0x55df59d60d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/2177109351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59bd5520 0x55df59d51ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/2177109351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59c4fd80 0x55df59d6c5d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f38b2f02b80 1 -- 172.21.15.19:0/2177109351 >> 172.21.15.19:0/2177109351 conn(0x55df59bd8ff0 msgr2=0x55df59d476e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f38b2f02b80 1 -- 172.21.15.19:0/2177109351 shutdown_connections 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.466+0000 7f38b2f02b80 1 -- 172.21.15.19:0/2177109351 wait complete. 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38b2f02b80 1 -- start start 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38b2f02b80 1 Processor -- start 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59bd5520 0x55df59d73080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59c4fd80 0x55df59d735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38b2f02b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 0x55df59d73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f389c1ade30 con 0x55df59bd5520 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f389c1ad760 con 0x55df59d5b260 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38b2f02b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f389c1aa960 con 0x55df59c4fd80 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38b2f02b80 10 client.0 _ll_register_callbacks cb 0x55df59ae6740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38a2288700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59bd5520 0x55df59d73080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38a2288700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59bd5520 0x55df59d73080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38002/0 (socket says 172.21.15.19:38002) 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.467+0000 7f38a2288700 1 -- 172.21.15.19:0/601010302 learned_addr learned my addr 172.21.15.19:0/601010302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38a2a89700 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 0x55df59d73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38a1a87700 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59c4fd80 0x55df59d735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38a2288700 1 -- 172.21.15.19:0/601010302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59c4fd80 msgr2=0x55df59d735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38a2288700 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59c4fd80 0x55df59d735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38a2288700 1 -- 172.21.15.19:0/601010302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 msgr2=0x55df59d73b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38a2288700 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 0x55df59d73b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38a2288700 1 -- 172.21.15.19:0/601010302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df59dd9960 con 0x55df59bd5520 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38a2288700 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59bd5520 0x55df59d73080 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f388c0a8d50 tx=0x7f388c0a8f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6a4663d6bfe38f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f3881ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f3881ffb700 1 -- 172.21.15.19:0/601010302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0c8060 con 0x55df59bd5520 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f3881ffb700 1 -- 172.21.15.19:0/601010302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f388c0ad0d0 con 0x55df59bd5520 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38b2f02b80 1 -- 172.21.15.19:0/601010302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3890005020 con 0x55df59bd5520 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f3881ffb700 1 -- 172.21.15.19:0/601010302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f388c0b1eb0 con 0x55df59bd5520 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38b2f02b80 10 client.17604 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.468+0000 7f38b2f02b80 1 -- 172.21.15.19:0/601010302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df59ddabe0 con 0x55df59bd5520 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.469+0000 7f3881ffb700 1 -- 172.21.15.19:0/601010302 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3890005020 con 0x55df59bd5520 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.469+0000 7f3881ffb700 1 -- 172.21.15.19:0/601010302 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f388c015400 con 0x55df59bd5520 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.469+0000 7f38b2f02b80 10 client.17604 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.469+0000 7f38b2f02b80 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.469+0000 7f38b2f02b80 20 client.17604 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.469+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df59ddb2f0 0x55df59ddd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.469+0000 7f38b2f02b80 4 client.17604 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:33:44.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.469+0000 7f38b2f02b80 1 -- 172.21.15.19:0/601010302 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55df59c7cba0 con 0x55df59ddb2f0 2024-05-09T01:33:44.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.469+0000 7f38a1a87700 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df59ddb2f0 0x55df59ddd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:44.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.470+0000 7f38a1a87700 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df59ddb2f0 0x55df59ddd710 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351a14e8323dbe7a server_cookie=68aa4b4de5c46227 in_seq=0 out_seq=0 2024-05-09T01:33:44.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.470+0000 7f3881ffb700 10 client.17604 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.471+0000 7f3881ffb700 1 -- 172.21.15.19:0/601010302 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55df59c7cba0 con 0x55df59ddb2f0 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.471+0000 7f3881ffb700 10 client.17604 handle_command_reply: tid=0 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.471+0000 7f38b2f02b80 1 client.17604 shutdown 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.471+0000 7f38b2f02b80 1 -- 172.21.15.19:0/601010302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59bd5520 msgr2=0x55df59d73080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.471+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59bd5520 0x55df59d73080 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f388c0a8d50 tx=0x7f388c0a8f30 comp rx=0 tx=0).stop 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f38b2f02b80 1 -- 172.21.15.19:0/601010302 shutdown_connections 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df59ddb2f0 0x55df59ddd710 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df59d5b260 0x55df59d73b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df59bd5520 0x55df59d73080 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f38b2f02b80 1 --2- 172.21.15.19:0/601010302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df59c4fd80 0x55df59d735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f38b2f02b80 1 -- 172.21.15.19:0/601010302 >> 172.21.15.19:0/601010302 conn(0x55df59bd8ff0 msgr2=0x55df59d5ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f3881ffb700 0 client.17604 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f38b2f02b80 1 -- 172.21.15.19:0/601010302 shutdown_connections 2024-05-09T01:33:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f38b2f02b80 1 -- 172.21.15.19:0/601010302 wait complete. 2024-05-09T01:33:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.472+0000 7f38b2f02b80 20 client.17604 trim_cache size 0 max 16384 2024-05-09T01:33:44.472 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:33:44.472 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:33:44.472 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "58d063ae-5139-4296-92e8-a93032e279a6", 2024-05-09T01:33:44.472 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:33:44.472 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 -- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f38840b2900 msgr2=0x7f38840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f38840b2900 0x7f38840b4e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f389c06c6f0 tx=0x7f3894005fb0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 -- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c1034c0 msgr2=0x7f389c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c1034c0 0x7f389c19bb90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f388c0060b0 tx=0x7f388c00d3b0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 -- 172.21.15.19:0/2872670380 shutdown_connections 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f389c108740 0x7f389c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f38840b2900 0x7f38840b4e10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389c1034c0 0x7f389c19bb90 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 --2- 172.21.15.19:0/2872670380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f389c107df0 0x7f389c19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 -- 172.21.15.19:0/2872670380 >> 172.21.15.19:0/2872670380 conn(0x7f389c0fedc0 msgr2=0x7f389c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 -- 172.21.15.19:0/2872670380 shutdown_connections 2024-05-09T01:33:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:44.473+0000 7f38a44ec700 1 -- 172.21.15.19:0/2872670380 wait complete. 2024-05-09T01:33:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:44 smithi059 ceph-mon[34351]: pgmap v624: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:33:44.848 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:44 smithi019 ceph-mon[30158]: pgmap v624: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:33:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:44 smithi196 ceph-mon[35041]: pgmap v624: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:33:45.278 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:33:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:45.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:45 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:33:45.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:45 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:45.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:45 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:33:45.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:45 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:45.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:45 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:33:45.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:45 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:45.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:45 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:33:45.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:45 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:45.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:45 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:33:45.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:45 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:45.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:45 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:33:45.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:45 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:33:46.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:46 smithi019 ceph-mon[30158]: pgmap v625: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:33:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:46 smithi059 ceph-mon[34351]: pgmap v625: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:33:46.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:46 smithi196 ceph-mon[35041]: pgmap v625: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:33:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.254+0000 7f1a05d38700 1 -- 172.21.15.19:0/4085078127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a00107df0 msgr2=0x7f1a00108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.254+0000 7f1a05d38700 1 --2- 172.21.15.19:0/4085078127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a00107df0 0x7f1a00108200 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f19f40099c0 tx=0x7f19f4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.254+0000 7f1a05d38700 1 -- 172.21.15.19:0/4085078127 shutdown_connections 2024-05-09T01:33:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.254+0000 7f1a05d38700 1 --2- 172.21.15.19:0/4085078127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a00108740 0x7f1a001100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.254+0000 7f1a05d38700 1 --2- 172.21.15.19:0/4085078127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a001034c0 0x7f1a001038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.254+0000 7f1a05d38700 1 --2- 172.21.15.19:0/4085078127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a00107df0 0x7f1a00108200 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.254+0000 7f1a05d38700 1 -- 172.21.15.19:0/4085078127 >> 172.21.15.19:0/4085078127 conn(0x7f1a000fedc0 msgr2=0x7f1a001011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.254+0000 7f1a05d38700 1 -- 172.21.15.19:0/4085078127 shutdown_connections 2024-05-09T01:33:47.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.254+0000 7f1a05d38700 1 -- 172.21.15.19:0/4085078127 wait complete. 2024-05-09T01:33:47.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.255+0000 7f1a05d38700 1 Processor -- start 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f1a05d38700 1 -- start start 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f1a05d38700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a001034c0 0x7f1a0019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f1a05d38700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00107df0 0x7f1a0019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f1a05d38700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a00108740 0x7f1a001a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f1a05d38700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00113350 con 0x7f1a001034c0 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f1a05d38700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001131d0 con 0x7f1a00108740 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f1a05d38700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001134d0 con 0x7f1a00107df0 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a001034c0 0x7f1a0019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a001034c0 0x7f1a0019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38012/0 (socket says 172.21.15.19:38012) 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19ff7fe700 1 -- 172.21.15.19:0/20697529 learned_addr learned my addr 172.21.15.19:0/20697529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19fffff700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a00108740 0x7f1a001a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19feffd700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00107df0 0x7f1a0019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19ff7fe700 1 -- 172.21.15.19:0/20697529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00107df0 msgr2=0x7f1a0019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00107df0 0x7f1a0019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19ff7fe700 1 -- 172.21.15.19:0/20697529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a00108740 msgr2=0x7f1a001a6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a00108740 0x7f1a001a6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19ff7fe700 1 -- 172.21.15.19:0/20697529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0019ca00 con 0x7f1a001034c0 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19feffd700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00107df0 0x7f1a0019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.256+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a001034c0 0x7f1a0019bc10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d950 tx=0x7f19f000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.257+0000 7f19fcff9700 1 -- 172.21.15.19:0/20697529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0016070 con 0x7f1a001034c0 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.257+0000 7f1a05d38700 1 -- 172.21.15.19:0/20697529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a001a72f0 con 0x7f1a001034c0 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.257+0000 7f19fcff9700 1 -- 172.21.15.19:0/20697529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f0010460 con 0x7f1a001034c0 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.257+0000 7f19fcff9700 1 -- 172.21.15.19:0/20697529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f000f620 con 0x7f1a001034c0 2024-05-09T01:33:47.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.257+0000 7f1a05d38700 1 -- 172.21.15.19:0/20697529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a001a77e0 con 0x7f1a001034c0 2024-05-09T01:33:47.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.260+0000 7f19fcff9700 1 -- 172.21.15.19:0/20697529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f19f0019070 con 0x7f1a001034c0 2024-05-09T01:33:47.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.260+0000 7f19fcff9700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19e80b2880 0x7f19e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f19feffd700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19e80b2880 0x7f19e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f19fcff9700 1 -- 172.21.15.19:0/20697529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f19f00e1ef0 con 0x7f1a001034c0 2024-05-09T01:33:47.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f1a1474eb80 1 Processor -- start 2024-05-09T01:33:47.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f1a1474eb80 1 -- start start 2024-05-09T01:33:47.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4276ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4277030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a427b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f19feffd700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19e80b2880 0x7f19e80b4d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006010 tx=0x7f19f4005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a4a41950a0 con 0x55a4a416fd00 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a4a413a9a0 con 0x55a4a40f54a0 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a4a4191060 con 0x55a4a427b1e0 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4276ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4276ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38018/0 (socket says 172.21.15.19:38018) 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f19ff7fe700 1 -- 172.21.15.19:0/2614051178 learned_addr learned my addr 172.21.15.19:0/2614051178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f19fffff700 1 --2- 172.21.15.19:0/2614051178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a427b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.261+0000 7f19feffd700 1 --2- 172.21.15.19:0/2614051178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4277030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.262+0000 7f19ff7fe700 1 -- 172.21.15.19:0/2614051178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 msgr2=0x55a4a427b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.262+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/2614051178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a427b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.262+0000 7f19ff7fe700 1 -- 172.21.15.19:0/2614051178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 msgr2=0x55a4a4277030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.262+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/2614051178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4277030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.262+0000 7f19ff7fe700 1 -- 172.21.15.19:0/2614051178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a412b500 con 0x55a4a416fd00 2024-05-09T01:33:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.262+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/2614051178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4276ad0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f19f00a5570 tx=0x7f19f00a8f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e30baa46999eabc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.262+0000 7f19e5ffb700 1 -- 172.21.15.19:0/2614051178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0020070 con 0x55a4a416fd00 2024-05-09T01:33:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.262+0000 7f19e5ffb700 1 -- 172.21.15.19:0/2614051178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f00ad8a0 con 0x55a4a416fd00 2024-05-09T01:33:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.262+0000 7f19e5ffb700 1 -- 172.21.15.19:0/2614051178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f00b1ec0 con 0x55a4a416fd00 2024-05-09T01:33:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.263+0000 7f1a1474eb80 1 -- 172.21.15.19:0/2614051178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 msgr2=0x55a4a4276ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.263+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/2614051178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4276ad0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f19f00a5570 tx=0x7f19f00a8f70 comp rx=0 tx=0).stop 2024-05-09T01:33:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.263+0000 7f1a1474eb80 1 -- 172.21.15.19:0/2614051178 shutdown_connections 2024-05-09T01:33:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.263+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/2614051178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a427b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.263+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/2614051178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4276ad0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.263+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/2614051178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4277030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.263+0000 7f1a1474eb80 1 -- 172.21.15.19:0/2614051178 >> 172.21.15.19:0/2614051178 conn(0x55a4a40f8f70 msgr2=0x55a4a426d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.263+0000 7f1a1474eb80 1 -- 172.21.15.19:0/2614051178 shutdown_connections 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.263+0000 7f1a1474eb80 1 -- 172.21.15.19:0/2614051178 wait complete. 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f1a1474eb80 1 -- start start 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f1a1474eb80 1 Processor -- start 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4280440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4280980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a4280ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a4a427bea0 con 0x55a4a416fd00 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a4a412fb20 con 0x55a4a40f54a0 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a4a427c020 con 0x55a4a427b1e0 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f1a1474eb80 10 client.0 _ll_register_callbacks cb 0x55a4a40066c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.264+0000 7f19feffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4280980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19feffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4280980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38026/0 (socket says 172.21.15.19:38026) 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19feffd700 1 -- 172.21.15.19:0/3362053289 learned_addr learned my addr 172.21.15.19:0/3362053289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4280440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19fffff700 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a4280ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19feffd700 1 -- 172.21.15.19:0/3362053289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 msgr2=0x55a4a4280ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19feffd700 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a4280ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19feffd700 1 -- 172.21.15.19:0/3362053289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 msgr2=0x55a4a4280440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19feffd700 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4280440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19feffd700 1 -- 172.21.15.19:0/3362053289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a4010490 con 0x55a4a416fd00 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19feffd700 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4280980 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f19f4003800 tx=0x7f19f4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fa4f54a6f59048 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19deffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19deffd700 1 -- 172.21.15.19:0/3362053289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f400b9a0 con 0x55a4a416fd00 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f1a1474eb80 1 -- 172.21.15.19:0/3362053289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f00ad8a0 con 0x55a4a416fd00 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.265+0000 7f19deffd700 1 -- 172.21.15.19:0/3362053289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f19f401e030 con 0x55a4a416fd00 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f19deffd700 1 -- 172.21.15.19:0/3362053289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19f400bb40 con 0x55a4a416fd00 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f1a1474eb80 10 client.17616 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f1a1474eb80 1 -- 172.21.15.19:0/3362053289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4a428ddf0 con 0x55a4a416fd00 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f19deffd700 1 -- 172.21.15.19:0/3362053289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f00ad8a0 con 0x55a4a416fd00 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f19deffd700 1 -- 172.21.15.19:0/3362053289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f19f400fcc0 con 0x55a4a416fd00 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f1a1474eb80 10 client.17616 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f1a1474eb80 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f1a1474eb80 20 client.17616 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a4a4290380 0x55a4a42927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f1a1474eb80 4 client.17616 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f1a1474eb80 1 -- 172.21.15.19:0/3362053289 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4a41a2a80 con 0x55a4a4290380 2024-05-09T01:33:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.266+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a4a4290380 0x55a4a42927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.267+0000 7f19deffd700 10 client.17616 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.267+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a4a4290380 0x55a4a42927a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148f7344d973f7b0 server_cookie=f895567fc672521f in_seq=0 out_seq=0 2024-05-09T01:33:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f19deffd700 1 -- 172.21.15.19:0/3362053289 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a4a41a2a80 con 0x55a4a4290380 2024-05-09T01:33:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f19deffd700 10 client.17616 handle_command_reply: tid=0 2024-05-09T01:33:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 client.17616 shutdown 2024-05-09T01:33:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 -- 172.21.15.19:0/3362053289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 msgr2=0x55a4a4280980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4280980 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f19f4003800 tx=0x7f19f4009310 comp rx=0 tx=0).stop 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 -- 172.21.15.19:0/3362053289 shutdown_connections 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a4a4290380 0x55a4a42927a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a4280ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4280440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/3362053289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a416fd00 0x55a4a4280980 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 -- 172.21.15.19:0/3362053289 >> 172.21.15.19:0/3362053289 conn(0x55a4a40f8f70 msgr2=0x55a4a42713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f19deffd700 0 client.17616 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 -- 172.21.15.19:0/3362053289 shutdown_connections 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 1 -- 172.21.15.19:0/3362053289 wait complete. 2024-05-09T01:33:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.268+0000 7f1a1474eb80 20 client.17616 trim_cache size 0 max 16384 2024-05-09T01:33:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f1a1474eb80 1 Processor -- start 2024-05-09T01:33:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f1a1474eb80 1 -- start start 2024-05-09T01:33:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a40f54a0 0x55a4a427f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a416fd00 0x55a4a4267a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a4280d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001acb80 con 0x55a4a40f54a0 2024-05-09T01:33:47.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ae6c0 con 0x55a4a416fd00 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ad250 con 0x55a4a427b1e0 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a40f54a0 0x55a4a427f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a40f54a0 0x55a4a427f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38030/0 (socket says 172.21.15.19:38030) 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f19ff7fe700 1 -- 172.21.15.19:0/1394350133 learned_addr learned my addr 172.21.15.19:0/1394350133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f19feffd700 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a416fd00 0x55a4a4267a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.289+0000 7f19fffff700 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a4280d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19ff7fe700 1 -- 172.21.15.19:0/1394350133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 msgr2=0x55a4a4280d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a4280d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19ff7fe700 1 -- 172.21.15.19:0/1394350133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a416fd00 msgr2=0x55a4a4267a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a416fd00 0x55a4a4267a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19ff7fe700 1 -- 172.21.15.19:0/1394350133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a42699d0 con 0x55a4a40f54a0 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19feffd700 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a416fd00 0x55a4a4267a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a40f54a0 0x55a4a427f400 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f19f00dfa20 tx=0x7f19f00a54e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5cb85e7126f9b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:47.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19e57fa700 1 -- 172.21.15.19:0/1394350133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f00c2070 con 0x55a4a40f54a0 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19e57fa700 1 -- 172.21.15.19:0/1394350133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f0010930 con 0x55a4a40f54a0 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.290+0000 7f19e57fa700 1 -- 172.21.15.19:0/1394350133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0020040 con 0x55a4a40f54a0 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1394350133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a40f54a0 msgr2=0x55a4a427f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a40f54a0 0x55a4a427f400 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f19f00dfa20 tx=0x7f19f00a54e0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1394350133 shutdown_connections 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a427b1e0 0x55a4a4280d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a40f54a0 0x55a4a427f400 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1394350133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a416fd00 0x55a4a4267a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1394350133 >> 172.21.15.19:0/1394350133 conn(0x55a4a40f8f70 msgr2=0x55a4a427aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1394350133 shutdown_connections 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1394350133 wait complete. 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 -- start start 2024-05-09T01:33:47.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.291+0000 7f1a1474eb80 1 Processor -- start 2024-05-09T01:33:47.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4288910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a416fd00 0x55a4a4278700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f1a1474eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a427b1e0 0x55a4a4278c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001adff0 con 0x55a4a427b1e0 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ad920 con 0x55a4a40f54a0 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f1a1474eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001aab20 con 0x55a4a416fd00 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f1a1474eb80 10 client.0 _ll_register_callbacks cb 0x55a4a40066c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a427b1e0 0x55a4a4278c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19fffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a427b1e0 0x55a4a4278c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38038/0 (socket says 172.21.15.19:38038) 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19fffff700 1 -- 172.21.15.19:0/1450159728 learned_addr learned my addr 172.21.15.19:0/1450159728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:47.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19feffd700 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a416fd00 0x55a4a4278700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4288910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19fffff700 1 -- 172.21.15.19:0/1450159728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a416fd00 msgr2=0x55a4a4278700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19fffff700 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a416fd00 0x55a4a4278700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19fffff700 1 -- 172.21.15.19:0/1450159728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 msgr2=0x55a4a4288910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19fffff700 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4288910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19fffff700 1 -- 172.21.15.19:0/1450159728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a42791d0 con 0x55a4a427b1e0 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19fffff700 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a427b1e0 0x55a4a4278c40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007cb0 tx=0x7f19f8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8dc45c9879787f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.292+0000 7f19deffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f19deffd700 1 -- 172.21.15.19:0/1450159728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8017a20 con 0x55a4a427b1e0 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f19deffd700 1 -- 172.21.15.19:0/1450159728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f19f8017bc0 con 0x55a4a427b1e0 2024-05-09T01:33:47.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1450159728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f0010930 con 0x55a4a427b1e0 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f19deffd700 1 -- 172.21.15.19:0/1450159728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19f8017dc0 con 0x55a4a427b1e0 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f1a1474eb80 10 client.17622 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1450159728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4a4288440 con 0x55a4a427b1e0 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f19deffd700 1 -- 172.21.15.19:0/1450159728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f0010930 con 0x55a4a427b1e0 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f19deffd700 1 -- 172.21.15.19:0/1450159728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f19f802b5e0 con 0x55a4a427b1e0 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f1a1474eb80 10 client.17622 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f1a1474eb80 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f1a1474eb80 20 client.17622 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a4a42fafc0 0x55a4a42fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f1a1474eb80 4 client.17622 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.293+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1450159728 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a4a419cb20 con 0x55a4a42fafc0 2024-05-09T01:33:47.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a4a42fafc0 0x55a4a42fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f19ff7fe700 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a4a42fafc0 0x55a4a42fd3e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def940888015f629 server_cookie=f1fb3f3783fd76e4 in_seq=0 out_seq=0 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f19deffd700 10 client.17622 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f19deffd700 1 -- 172.21.15.19:0/1450159728 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a4a419cb20 con 0x55a4a42fafc0 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f19deffd700 10 client.17622 handle_command_reply: tid=0 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f1a1474eb80 1 client.17622 shutdown 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1450159728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a427b1e0 msgr2=0x55a4a4278c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a427b1e0 0x55a4a4278c40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007cb0 tx=0x7f19f8007d90 comp rx=0 tx=0).stop 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1450159728 shutdown_connections 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a4a42fafc0 0x55a4a42fd3e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a427b1e0 0x55a4a4278c40 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4a40f54a0 0x55a4a4288910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f1a1474eb80 1 --2- 172.21.15.19:0/1450159728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4a416fd00 0x55a4a4278700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.294+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1450159728 >> 172.21.15.19:0/1450159728 conn(0x55a4a40f8f70 msgr2=0x55a4a427aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:47.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f19deffd700 0 client.17622 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:47.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1450159728 shutdown_connections 2024-05-09T01:33:47.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a1474eb80 1 -- 172.21.15.19:0/1450159728 wait complete. 2024-05-09T01:33:47.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a1474eb80 20 client.17622 trim_cache size 0 max 16384 2024-05-09T01:33:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:33:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:33:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:33:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:33:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 -- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19e80b2880 msgr2=0x7f19e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19e80b2880 0x7f19e80b4d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006010 tx=0x7f19f4005e80 comp rx=0 tx=0).stop 2024-05-09T01:33:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 -- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a001034c0 msgr2=0x7f1a0019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a001034c0 0x7f1a0019bc10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d950 tx=0x7f19f000dd10 comp rx=0 tx=0).stop 2024-05-09T01:33:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 -- 172.21.15.19:0/20697529 shutdown_connections 2024-05-09T01:33:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a00108740 0x7f1a001a6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f19e80b2880 0x7f19e80b4d90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a001034c0 0x7f1a0019bc10 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 --2- 172.21.15.19:0/20697529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00107df0 0x7f1a0019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 -- 172.21.15.19:0/20697529 >> 172.21.15.19:0/20697529 conn(0x7f1a000fedc0 msgr2=0x7f1a001011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 -- 172.21.15.19:0/20697529 shutdown_connections 2024-05-09T01:33:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:47.295+0000 7f1a05d38700 1 -- 172.21.15.19:0/20697529 wait complete. 2024-05-09T01:33:47.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:47 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:33:47.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:47 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:33:47.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:47 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:33:48.161 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:33:48.161 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:33:48.418 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:48.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:48 smithi019 ceph-mon[30158]: pgmap v626: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:33:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:48 smithi059 ceph-mon[34351]: pgmap v626: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:33:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:48 smithi196 ceph-mon[35041]: pgmap v626: 97 pgs: 97 active+clean; 1.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:33:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 -- 172.21.15.19:0/1316989860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 msgr2=0x7f9be410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 --2- 172.21.15.19:0/1316989860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 0x7f9be410bde0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009900 tx=0x7f9bd402a410 comp rx=0 tx=0).stop 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 -- 172.21.15.19:0/1316989860 shutdown_connections 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 --2- 172.21.15.19:0/1316989860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 0x7f9be410bde0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 --2- 172.21.15.19:0/1316989860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be4106d60 0x7f9be4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 --2- 172.21.15.19:0/1316989860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9be4104370 0x7f9be4106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 -- 172.21.15.19:0/1316989860 >> 172.21.15.19:0/1316989860 conn(0x7f9be40fdd30 msgr2=0x7f9be41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 -- 172.21.15.19:0/1316989860 shutdown_connections 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 -- 172.21.15.19:0/1316989860 wait complete. 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 Processor -- start 2024-05-09T01:33:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.267+0000 7f9bebf34640 1 -- start start 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bebf34640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9be4104370 0x7f9be4103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bebf34640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be4106d60 0x7f9be4102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bebf34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 0x7f9be41027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bebf34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9be410f740 con 0x7f9be41098d0 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bebf34640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9be410f5c0 con 0x7f9be4104370 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bebf34640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9be410f8c0 con 0x7f9be4106d60 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bea4d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 0x7f9be41027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bea4d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 0x7f9be41027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38048/0 (socket says 172.21.15.19:38048) 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bea4d1640 1 -- 172.21.15.19:0/4111179616 learned_addr learned my addr 172.21.15.19:0/4111179616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9be4104370 0x7f9be4103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9be94cf640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be4106d60 0x7f9be4102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bea4d1640 1 -- 172.21.15.19:0/4111179616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be4106d60 msgr2=0x7f9be4102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bea4d1640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be4106d60 0x7f9be4102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bea4d1640 1 -- 172.21.15.19:0/4111179616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9be4104370 msgr2=0x7f9be4103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bea4d1640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9be4104370 0x7f9be4103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bea4d1640 1 -- 172.21.15.19:0/4111179616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be4102ef0 con 0x7f9be41098d0 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9be94cf640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be4106d60 0x7f9be4102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.268+0000 7f9bea4d1640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 0x7f9be41027c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009230 tx=0x7f9bd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.269+0000 7f9be2ffd640 1 -- 172.21.15.19:0/4111179616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd403f030 con 0x7f9be41098d0 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.269+0000 7f9be2ffd640 1 -- 172.21.15.19:0/4111179616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd4031910 con 0x7f9be41098d0 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.269+0000 7f9bebf34640 1 -- 172.21.15.19:0/4111179616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be41a43a0 con 0x7f9be41098d0 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.269+0000 7f9be2ffd640 1 -- 172.21.15.19:0/4111179616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4035070 con 0x7f9be41098d0 2024-05-09T01:33:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.269+0000 7f9bebf34640 1 -- 172.21.15.19:0/4111179616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be41a4800 con 0x7f9be41098d0 2024-05-09T01:33:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.271+0000 7f9be2ffd640 1 -- 172.21.15.19:0/4111179616 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9bd4042030 con 0x7f9be41098d0 2024-05-09T01:33:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.271+0000 7f9bf2de5740 1 Processor -- start 2024-05-09T01:33:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9bf2de5740 1 -- start start 2024-05-09T01:33:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae53500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 0x55f89ae53a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f89adb77c0 con 0x55f89aaaf1b0 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f89ad530e0 con 0x55f89aa862f0 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f89adb3780 con 0x55f89aa8e4d0 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae53500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae53500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38058/0 (socket says 172.21.15.19:38058) 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 -- 172.21.15.19:0/3739701371 learned_addr learned my addr 172.21.15.19:0/3739701371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be2ffd640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9bcc0b2860 0x7f9bcc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/3739701371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 0x55f89ae53a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 --2- 172.21.15.19:0/3739701371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 -- 172.21.15.19:0/3739701371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa8e4d0 msgr2=0x55f89ae5a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 --2- 172.21.15.19:0/3739701371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 -- 172.21.15.19:0/3739701371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 msgr2=0x55f89ae53a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 --2- 172.21.15.19:0/3739701371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 0x55f89ae53a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 -- 172.21.15.19:0/3739701371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89ad7c7c0 con 0x55f89aaaf1b0 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9bea4d1640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9bcc0b2860 0x7f9bcc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be2ffd640 1 -- 172.21.15.19:0/4111179616 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9bd4102b10 con 0x7f9be41098d0 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.272+0000 7f9be94cf640 1 --2- 172.21.15.19:0/3739701371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae53500 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80114e0 tx=0x7f9bd8008860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db92a638037625e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.273+0000 7f9bcbfff640 1 -- 172.21.15.19:0/3739701371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8012920 con 0x55f89aaaf1b0 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.273+0000 7f9bcbfff640 1 -- 172.21.15.19:0/3739701371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd801a400 con 0x55f89aaaf1b0 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.273+0000 7f9bcbfff640 1 -- 172.21.15.19:0/3739701371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800f620 con 0x55f89aaaf1b0 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3739701371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 msgr2=0x55f89ae53500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bea4d1640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9bcc0b2860 0x7f9bcc0b4d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9bd41005f0 tx=0x7f9bd408f650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3739701371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae53500 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80114e0 tx=0x7f9bd8008860 comp rx=0 tx=0).stop 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3739701371 shutdown_connections 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3739701371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3739701371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 0x55f89ae53a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3739701371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae53500 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3739701371 >> 172.21.15.19:0/3739701371 conn(0x55f89aaa3300 msgr2=0x55f89ae4ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3739701371 shutdown_connections 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3739701371 wait complete. 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 -- start start 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.274+0000 7f9bf2de5740 1 Processor -- start 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 0x55f89ae5d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f89ae5b3d0 con 0x55f89aa8e4d0 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f89ad3e9c0 con 0x55f89aa862f0 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f89ae5b550 con 0x55f89aaaf1b0 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bf2de5740 10 client.0 _ll_register_callbacks cb 0x55f89ad6b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be9cd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be9cd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38062/0 (socket says 172.21.15.19:38062) 2024-05-09T01:33:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be9cd0640 1 -- 172.21.15.19:0/2821218248 learned_addr learned my addr 172.21.15.19:0/2821218248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be94cf640 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 0x55f89ae5d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be94cf640 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be9cd0640 1 -- 172.21.15.19:0/2821218248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aaaf1b0 msgr2=0x55f89ae64740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be9cd0640 1 -- 172.21.15.19:0/2821218248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 msgr2=0x55f89ae5d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 0x55f89ae5d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be9cd0640 1 -- 172.21.15.19:0/2821218248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89ad651f0 con 0x55f89aa8e4d0 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be94cf640 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 0x55f89ae5d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5d7e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc013710 tx=0x7f9bdc013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd3ddc25fbb6031 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bc97fa640 1 -- 172.21.15.19:0/2821218248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc011dd0 con 0x55f89aa8e4d0 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bc97fa640 1 -- 172.21.15.19:0/2821218248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9bdc004030 con 0x55f89aa8e4d0 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.275+0000 7f9bf2de5740 1 -- 172.21.15.19:0/2821218248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bd801a400 con 0x55f89aa8e4d0 2024-05-09T01:33:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bc97fa640 1 -- 172.21.15.19:0/2821218248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bdc004270 con 0x55f89aa8e4d0 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bf2de5740 10 client.17634 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bf2de5740 1 -- 172.21.15.19:0/2821218248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f89ab1a790 con 0x55f89aa8e4d0 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bc97fa640 1 -- 172.21.15.19:0/2821218248 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd801a400 con 0x55f89aa8e4d0 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bc97fa640 1 -- 172.21.15.19:0/2821218248 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9bdc01a070 con 0x55f89aa8e4d0 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bf2de5740 10 client.17634 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bf2de5740 10 client.17634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bf2de5740 20 client.17634 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f89aa6dd80 0x55f89ae6ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bf2de5740 4 client.17634 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9bf2de5740 1 -- 172.21.15.19:0/2821218248 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f89adc53d0 con 0x55f89aa6dd80 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.276+0000 7f9be94cf640 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f89aa6dd80 0x55f89ae6ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.277+0000 7f9be94cf640 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f89aa6dd80 0x55f89ae6ad20 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4283f68cc15074f0 server_cookie=37e8421b02f7a973 in_seq=0 out_seq=0 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.277+0000 7f9bc97fa640 10 client.17634 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bc97fa640 1 -- 172.21.15.19:0/2821218248 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f89adc53d0 con 0x55f89aa6dd80 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bc97fa640 10 client.17634 handle_command_reply: tid=0 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 client.17634 shutdown 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 -- 172.21.15.19:0/2821218248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aa8e4d0 msgr2=0x55f89ae5d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5d7e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc013710 tx=0x7f9bdc013bd0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 -- 172.21.15.19:0/2821218248 shutdown_connections 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f89aa6dd80 0x55f89ae6ad20 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aa8e4d0 0x55f89ae5d7e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/2821218248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa862f0 0x55f89ae5d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bc97fa640 0 client.17634 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 -- 172.21.15.19:0/2821218248 >> 172.21.15.19:0/2821218248 conn(0x55f89aaa3300 msgr2=0x55f89ae4d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 -- 172.21.15.19:0/2821218248 shutdown_connections 2024-05-09T01:33:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 1 -- 172.21.15.19:0/2821218248 wait complete. 2024-05-09T01:33:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.278+0000 7f9bf2de5740 20 client.17634 trim_cache size 0 max 16384 2024-05-09T01:33:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.293+0000 7f9bf2de5740 1 Processor -- start 2024-05-09T01:33:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.293+0000 7f9bf2de5740 1 -- start start 2024-05-09T01:33:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.293+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 0x55f89ae56e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.293+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae64660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.293+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.293+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41ad620 con 0x55f89aaaf1b0 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.293+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41af160 con 0x55f89aa8e4d0 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.293+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41adcf0 con 0x55f89aa862f0 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38064/0 (socket says 172.21.15.19:38064) 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 -- 172.21.15.19:0/1993868402 learned_addr learned my addr 172.21.15.19:0/1993868402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae64660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 0x55f89ae56e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 -- 172.21.15.19:0/1993868402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 msgr2=0x55f89ae56e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 0x55f89ae56e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 -- 172.21.15.19:0/1993868402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 msgr2=0x55f89ae64660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae64660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 -- 172.21.15.19:0/1993868402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89ab915f0 con 0x55f89aaaf1b0 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae64660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9be94cf640 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64ba0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8005c40 tx=0x7f9bd8008e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f566236e33b3cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9bcb7fe640 1 -- 172.21.15.19:0/1993868402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd801a600 con 0x55f89aaaf1b0 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9bcb7fe640 1 -- 172.21.15.19:0/1993868402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd801a7a0 con 0x55f89aaaf1b0 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.294+0000 7f9bcb7fe640 1 -- 172.21.15.19:0/1993868402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8012a10 con 0x55f89aaaf1b0 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.295+0000 7f9bf2de5740 1 -- 172.21.15.19:0/1993868402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 msgr2=0x55f89ae64ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.295+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64ba0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8005c40 tx=0x7f9bd8008e60 comp rx=0 tx=0).stop 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.295+0000 7f9bf2de5740 1 -- 172.21.15.19:0/1993868402 shutdown_connections 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.295+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae64ba0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.295+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae64660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.295+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/1993868402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 0x55f89ae56e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.295+0000 7f9bf2de5740 1 -- 172.21.15.19:0/1993868402 >> 172.21.15.19:0/1993868402 conn(0x55f89aaa3300 msgr2=0x55f89ae56710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.295+0000 7f9bf2de5740 1 -- 172.21.15.19:0/1993868402 shutdown_connections 2024-05-09T01:33:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.295+0000 7f9bf2de5740 1 -- 172.21.15.19:0/1993868402 wait complete. 2024-05-09T01:33:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9bf2de5740 1 -- start start 2024-05-09T01:33:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9bf2de5740 1 Processor -- start 2024-05-09T01:33:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 0x55f89ae627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae62cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9bf2de5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae63c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41aea90 con 0x55f89aaaf1b0 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41ae3c0 con 0x55f89aa8e4d0 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9bf2de5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41ab600 con 0x55f89aa862f0 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9bf2de5740 10 client.0 _ll_register_callbacks cb 0x7f9bdc009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9be94cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae63c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9be94cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae63c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38074/0 (socket says 172.21.15.19:38074) 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9be94cf640 1 -- 172.21.15.19:0/3761645782 learned_addr learned my addr 172.21.15.19:0/3761645782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9be9cd0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae62cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.296+0000 7f9be94cf640 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 0x55f89ae627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9be94cf640 1 -- 172.21.15.19:0/3761645782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 msgr2=0x55f89ae627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9be94cf640 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 0x55f89ae627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9be94cf640 1 -- 172.21.15.19:0/3761645782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 msgr2=0x55f89ae62cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9be94cf640 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae62cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9be94cf640 1 -- 172.21.15.19:0/3761645782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89ad7bed0 con 0x55f89aaaf1b0 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae62cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9be94cf640 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae63c20 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8008d80 tx=0x7f9bd8008b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd64295fa05745f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9bc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9bc97fa640 1 -- 172.21.15.19:0/3761645782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8037030 con 0x55f89aaaf1b0 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9bc97fa640 1 -- 172.21.15.19:0/3761645782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9bd8011e30 con 0x55f89aaaf1b0 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3761645782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bd801a7a0 con 0x55f89aaaf1b0 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9bc97fa640 1 -- 172.21.15.19:0/3761645782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bd801a400 con 0x55f89aaaf1b0 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9bf2de5740 10 client.17640 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3761645782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f89aedbab0 con 0x55f89aaaf1b0 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.297+0000 7f9bc97fa640 1 -- 172.21.15.19:0/3761645782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd801a7a0 con 0x55f89aaaf1b0 2024-05-09T01:33:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.298+0000 7f9bc97fa640 1 -- 172.21.15.19:0/3761645782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9bd8012c50 con 0x55f89aaaf1b0 2024-05-09T01:33:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.298+0000 7f9bf2de5740 10 client.17640 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.298+0000 7f9bf2de5740 10 client.17640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:33:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.298+0000 7f9bf2de5740 20 client.17640 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.298+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f89aa431c0 0x55f89aede280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.298+0000 7f9bf2de5740 4 client.17640 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:33:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.298+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3761645782 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f89adbf240 con 0x55f89aa431c0 2024-05-09T01:33:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.298+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f89aa431c0 0x55f89aede280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.298+0000 7f9be9cd0640 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f89aa431c0 0x55f89aede280 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39502734153baf1e server_cookie=1c8d44de0b13e11c in_seq=0 out_seq=0 2024-05-09T01:33:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.299+0000 7f9bc97fa640 10 client.17640 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.313+0000 7f9bc97fa640 1 -- 172.21.15.19:0/3761645782 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f89adbf240 con 0x55f89aa431c0 2024-05-09T01:33:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.313+0000 7f9bc97fa640 10 client.17640 handle_command_reply: tid=0 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.313+0000 7f9bf2de5740 1 client.17640 shutdown 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3761645782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 msgr2=0x55f89ae63c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae63c20 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8008d80 tx=0x7f9bd8008b80 comp rx=0 tx=0).stop 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3761645782 shutdown_connections 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f89aa431c0 0x55f89aede280 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f89aaaf1b0 0x55f89ae63c20 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bc97fa640 0 client.17640 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f89aa8e4d0 0x55f89ae62cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 --2- 172.21.15.19:0/3761645782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f89aa862f0 0x55f89ae627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3761645782 >> 172.21.15.19:0/3761645782 conn(0x55f89aaa3300 msgr2=0x55f89ae4e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3761645782 shutdown_connections 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 1 -- 172.21.15.19:0/3761645782 wait complete. 2024-05-09T01:33:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.314+0000 7f9bf2de5740 20 client.17640 trim_cache size 0 max 16384 2024-05-09T01:33:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:33:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:33:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:33:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 -- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9bcc0b2860 msgr2=0x7f9bcc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9bcc0b2860 0x7f9bcc0b4d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9bd41005f0 tx=0x7f9bd408f650 comp rx=0 tx=0).stop 2024-05-09T01:33:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 -- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 msgr2=0x7f9be41027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 0x7f9be41027c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009230 tx=0x7f9bd40042f0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 -- 172.21.15.19:0/4111179616 shutdown_connections 2024-05-09T01:33:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9bcc0b2860 0x7f9bcc0b4d70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be41098d0 0x7f9be41027c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be4106d60 0x7f9be4102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 --2- 172.21.15.19:0/4111179616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9be4104370 0x7f9be4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 -- 172.21.15.19:0/4111179616 >> 172.21.15.19:0/4111179616 conn(0x7f9be40fdd30 msgr2=0x7f9be4104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 -- 172.21.15.19:0/4111179616 shutdown_connections 2024-05-09T01:33:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:33:49.315+0000 7f9bebf34640 1 -- 172.21.15.19:0/4111179616 wait complete. 2024-05-09T01:33:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.174+0000 7effdf0e8700 1 -- 172.21.15.19:0/545203227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 msgr2=0x7effd8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.174+0000 7effdf0e8700 1 --2- 172.21.15.19:0/545203227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 0x7effd8108200 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7effc80099c0 tx=0x7effc8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.175+0000 7effdf0e8700 1 -- 172.21.15.19:0/545203227 shutdown_connections 2024-05-09T01:33:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.175+0000 7effdf0e8700 1 --2- 172.21.15.19:0/545203227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effd8108740 0x7effd81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.175+0000 7effdf0e8700 1 --2- 172.21.15.19:0/545203227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd81034c0 0x7effd81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.175+0000 7effdf0e8700 1 --2- 172.21.15.19:0/545203227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 0x7effd8108200 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.175+0000 7effdf0e8700 1 -- 172.21.15.19:0/545203227 >> 172.21.15.19:0/545203227 conn(0x7effd80fedc0 msgr2=0x7effd81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.175+0000 7effdf0e8700 1 -- 172.21.15.19:0/545203227 shutdown_connections 2024-05-09T01:33:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.175+0000 7effdf0e8700 1 -- 172.21.15.19:0/545203227 wait complete. 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.175+0000 7effdf0e8700 1 Processor -- start 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.175+0000 7effdf0e8700 1 -- start start 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effdf0e8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effd81034c0 0x7effd819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effdf0e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 0x7effd819c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effdf0e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd8108740 0x7effd819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effdf0e8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7effd81133c0 con 0x7effd8107df0 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effdf0e8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7effd8113240 con 0x7effd8108740 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effdf0e8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effd8113540 con 0x7effd81034c0 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 0x7effd819c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 0x7effd819c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60350/0 (socket says 172.21.15.19:60350) 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effd7fff700 1 -- 172.21.15.19:0/2753255825 learned_addr learned my addr 172.21.15.19:0/2753255825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effdd685700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd8108740 0x7effd819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effdce84700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effd81034c0 0x7effd819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effd7fff700 1 -- 172.21.15.19:0/2753255825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effd81034c0 msgr2=0x7effd819bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effd7fff700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effd81034c0 0x7effd819bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effd7fff700 1 -- 172.21.15.19:0/2753255825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd8108740 msgr2=0x7effd819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effd7fff700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd8108740 0x7effd819c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effd7fff700 1 -- 172.21.15.19:0/2753255825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd81a7010 con 0x7effd8107df0 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.176+0000 7effd7fff700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 0x7effd819c200 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7effc8006010 tx=0x7effc80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.177+0000 7effd5ffb700 1 -- 172.21.15.19:0/2753255825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8027070 con 0x7effd8107df0 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.177+0000 7effd5ffb700 1 -- 172.21.15.19:0/2753255825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc8003f20 con 0x7effd8107df0 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.177+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd81a72a0 con 0x7effd8107df0 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.177+0000 7effd5ffb700 1 -- 172.21.15.19:0/2753255825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8022cd0 con 0x7effd8107df0 2024-05-09T01:33:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.177+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81a7750 con 0x7effd8107df0 2024-05-09T01:33:50.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.178+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd80512c0 con 0x7effd8107df0 2024-05-09T01:33:50.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.179+0000 7effd5ffb700 1 -- 172.21.15.19:0/2753255825 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7effc802a070 con 0x7effd8107df0 2024-05-09T01:33:50.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.180+0000 7effd5ffb700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7effc40b2900 0x7effc40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:50.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.180+0000 7effd5ffb700 1 -- 172.21.15.19:0/2753255825 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7effc80e3410 con 0x7effd8107df0 2024-05-09T01:33:50.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.180+0000 7effdce84700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7effc40b2900 0x7effc40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.181+0000 7effdce84700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7effc40b2900 0x7effc40b4e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7effc0005950 tx=0x7effc00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.184+0000 7effd5ffb700 1 -- 172.21.15.19:0/2753255825 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7effc8031080 con 0x7effd8107df0 2024-05-09T01:33:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.446+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effd802e5f0 con 0x7effd8107df0 2024-05-09T01:33:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.447+0000 7effd5ffb700 1 -- 172.21.15.19:0/2753255825 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7effc8013070 con 0x7effd8107df0 2024-05-09T01:33:50.444 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:33:50.444 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:33:50.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7effc40b2900 msgr2=0x7effc40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:50.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7effc40b2900 0x7effc40b4e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7effc0005950 tx=0x7effc00058e0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 msgr2=0x7effd819c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:50.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 0x7effd819c200 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7effc8006010 tx=0x7effc80047b0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 shutdown_connections 2024-05-09T01:33:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd8108740 0x7effd819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effd81034c0 0x7effd819bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7effc40b2900 0x7effc40b4e10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 --2- 172.21.15.19:0/2753255825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd8107df0 0x7effd819c200 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 >> 172.21.15.19:0/2753255825 conn(0x7effd80fedc0 msgr2=0x7effd81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 shutdown_connections 2024-05-09T01:33:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:50.449+0000 7effdf0e8700 1 -- 172.21.15.19:0/2753255825 wait complete. 2024-05-09T01:33:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:33:50.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:50 smithi019 ceph-mon[30158]: pgmap v627: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:33:50.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:50 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2753255825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:33:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:50 smithi059 ceph-mon[34351]: pgmap v627: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:33:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:50 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2753255825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:33:50.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:50 smithi196 ceph-mon[35041]: pgmap v627: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:33:50.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:50 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2753255825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:33:51.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:33:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:52.790 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:52 smithi019 ceph-mon[30158]: pgmap v628: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-05-09T01:33:52.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:52 smithi059 ceph-mon[34351]: pgmap v628: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-05-09T01:33:52.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:52 smithi196 ceph-mon[35041]: pgmap v628: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-05-09T01:33:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.174+0000 7f55469be700 1 -- 172.21.15.19:0/2554559398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55401034c0 msgr2=0x7f55401038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.174+0000 7f55469be700 1 --2- 172.21.15.19:0/2554559398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55401034c0 0x7f55401038d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f55300099c0 tx=0x7f5530009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.174+0000 7f55469be700 1 -- 172.21.15.19:0/2554559398 shutdown_connections 2024-05-09T01:33:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.174+0000 7f55469be700 1 --2- 172.21.15.19:0/2554559398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5540108740 0x7f55401100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.174+0000 7f55469be700 1 --2- 172.21.15.19:0/2554559398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55401034c0 0x7f55401038d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.174+0000 7f55469be700 1 --2- 172.21.15.19:0/2554559398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5540107df0 0x7f5540108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.174+0000 7f55469be700 1 -- 172.21.15.19:0/2554559398 >> 172.21.15.19:0/2554559398 conn(0x7f55400fedf0 msgr2=0x7f5540101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.174+0000 7f55469be700 1 -- 172.21.15.19:0/2554559398 shutdown_connections 2024-05-09T01:33:53.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.174+0000 7f55469be700 1 -- 172.21.15.19:0/2554559398 wait complete. 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.175+0000 7f55469be700 1 Processor -- start 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.175+0000 7f55469be700 1 -- start start 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.175+0000 7f55469be700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55401034c0 0x7f554019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.175+0000 7f55469be700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5540107df0 0x7f554019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.175+0000 7f55469be700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5540108740 0x7f554019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.175+0000 7f55469be700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55401144e0 con 0x7f5540107df0 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.175+0000 7f55469be700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5540114360 con 0x7f55401034c0 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.175+0000 7f55469be700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5540114660 con 0x7f5540108740 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5537fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5540107df0 0x7f554019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5537fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5540107df0 0x7f554019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60370/0 (socket says 172.21.15.19:60370) 2024-05-09T01:33:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5537fff700 1 -- 172.21.15.19:0/3606596516 learned_addr learned my addr 172.21.15.19:0/3606596516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5544f5b700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5540108740 0x7f554019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f553ffff700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55401034c0 0x7f554019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5537fff700 1 -- 172.21.15.19:0/3606596516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5540108740 msgr2=0x7f554019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5537fff700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5540108740 0x7f554019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5537fff700 1 -- 172.21.15.19:0/3606596516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55401034c0 msgr2=0x7f554019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5537fff700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55401034c0 0x7f554019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5537fff700 1 -- 172.21.15.19:0/3606596516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554019fd60 con 0x7f5540107df0 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f5537fff700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5540107df0 0x7f554019b430 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f552800e970 tx=0x7f552800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f553dffb700 1 -- 172.21.15.19:0/3606596516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552800cbd0 con 0x7f5540107df0 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f55469be700 1 -- 172.21.15.19:0/3606596516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55401a0050 con 0x7f5540107df0 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f553dffb700 1 -- 172.21.15.19:0/3606596516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f552800cd70 con 0x7f5540107df0 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f553dffb700 1 -- 172.21.15.19:0/3606596516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528005680 con 0x7f5540107df0 2024-05-09T01:33:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.176+0000 7f55469be700 1 -- 172.21.15.19:0/3606596516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55401a7970 con 0x7f5540107df0 2024-05-09T01:33:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.179+0000 7f553dffb700 1 -- 172.21.15.19:0/3606596516 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5528004750 con 0x7f5540107df0 2024-05-09T01:33:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.180+0000 7f553dffb700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f552c0b2880 0x7f552c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.180+0000 7f553ffff700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f552c0b2880 0x7f552c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.180+0000 7f55553d4b80 1 Processor -- start 2024-05-09T01:33:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.180+0000 7f553dffb700 1 -- 172.21.15.19:0/3606596516 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f55280e24a0 con 0x7f5540107df0 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.180+0000 7f55553d4b80 1 -- start start 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03df6d00 0x55ae03eff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03d7c4a0 0x55ae03eff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 0x55ae03ef9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f55553d4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ae03e1c0a0 con 0x55ae03d7c4a0 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f55553d4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ae03dc19a0 con 0x55ae03ef9400 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f55553d4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae03e18060 con 0x55ae03df6d00 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f553ffff700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f552c0b2880 0x7f552c0b4d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5530003760 tx=0x7f553001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5537fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03d7c4a0 0x55ae03eff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5537fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03d7c4a0 0x55ae03eff780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60378/0 (socket says 172.21.15.19:60378) 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5537fff700 1 -- 172.21.15.19:0/524258446 learned_addr learned my addr 172.21.15.19:0/524258446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f553ffff700 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03df6d00 0x55ae03eff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5544f5b700 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 0x55ae03ef9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5537fff700 1 -- 172.21.15.19:0/524258446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03df6d00 msgr2=0x55ae03eff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5537fff700 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03df6d00 0x55ae03eff240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5537fff700 1 -- 172.21.15.19:0/524258446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 msgr2=0x55ae03ef9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5537fff700 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 0x55ae03ef9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5537fff700 1 -- 172.21.15.19:0/524258446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae03db2500 con 0x55ae03d7c4a0 2024-05-09T01:33:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5544f5b700 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 0x55ae03ef9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.181+0000 7f5537fff700 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03d7c4a0 0x55ae03eff780 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f55280a9a30 tx=0x7f55280a9470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b051522894c037 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.182+0000 7f55367fc700 1 -- 172.21.15.19:0/524258446 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55280c3070 con 0x55ae03d7c4a0 2024-05-09T01:33:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.182+0000 7f55367fc700 1 -- 172.21.15.19:0/524258446 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55280b2250 con 0x55ae03d7c4a0 2024-05-09T01:33:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.182+0000 7f55367fc700 1 -- 172.21.15.19:0/524258446 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55280b2900 con 0x55ae03d7c4a0 2024-05-09T01:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.182+0000 7f55553d4b80 1 -- 172.21.15.19:0/524258446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03d7c4a0 msgr2=0x55ae03eff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.182+0000 7f55553d4b80 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03d7c4a0 0x55ae03eff780 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f55280a9a30 tx=0x7f55280a9470 comp rx=0 tx=0).stop 2024-05-09T01:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.182+0000 7f55553d4b80 1 -- 172.21.15.19:0/524258446 shutdown_connections 2024-05-09T01:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.182+0000 7f55553d4b80 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 0x55ae03ef9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.182+0000 7f55553d4b80 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03df6d00 0x55ae03eff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.182+0000 7f55553d4b80 1 --2- 172.21.15.19:0/524258446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03d7c4a0 0x55ae03eff780 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.183+0000 7f55553d4b80 1 -- 172.21.15.19:0/524258446 >> 172.21.15.19:0/524258446 conn(0x55ae03d7ff70 msgr2=0x55ae03ef4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.183+0000 7f55553d4b80 1 -- 172.21.15.19:0/524258446 shutdown_connections 2024-05-09T01:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.183+0000 7f55553d4b80 1 -- 172.21.15.19:0/524258446 wait complete. 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.183+0000 7f55553d4b80 1 -- start start 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.183+0000 7f55553d4b80 1 Processor -- start 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 0x55ae03f03040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03df6d00 0x55ae03f03580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03ef9400 0x55ae03f03ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f55553d4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ae03efa160 con 0x55ae03ef9400 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f55553d4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ae03db6b20 con 0x55ae03df6d00 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f55553d4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae03f06020 con 0x55ae03d7c4a0 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f55553d4b80 10 client.0 _ll_register_callbacks cb 0x55ae03c8d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5544f5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03ef9400 0x55ae03f03ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5544f5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03ef9400 0x55ae03f03ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60382/0 (socket says 172.21.15.19:60382) 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5544f5b700 1 -- 172.21.15.19:0/3107933274 learned_addr learned my addr 172.21.15.19:0/3107933274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5537fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03df6d00 0x55ae03f03580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f553ffff700 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 0x55ae03f03040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5544f5b700 1 -- 172.21.15.19:0/3107933274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 msgr2=0x55ae03f03040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5544f5b700 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 0x55ae03f03040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5544f5b700 1 -- 172.21.15.19:0/3107933274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03df6d00 msgr2=0x55ae03f03580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5544f5b700 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03df6d00 0x55ae03f03580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5544f5b700 1 -- 172.21.15.19:0/3107933274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae03c97490 con 0x55ae03ef9400 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f553ffff700 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 0x55ae03f03040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f5544f5b700 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03ef9400 0x55ae03f03ac0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f553800aee0 tx=0x7f553800af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c7ffad37a91e7b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.184+0000 7f551f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f551f7fe700 1 -- 172.21.15.19:0/3107933274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538007ab0 con 0x55ae03ef9400 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f551f7fe700 1 -- 172.21.15.19:0/3107933274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5538007c50 con 0x55ae03ef9400 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f55553d4b80 1 -- 172.21.15.19:0/3107933274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55280b2250 con 0x55ae03ef9400 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f551f7fe700 1 -- 172.21.15.19:0/3107933274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5538007e50 con 0x55ae03ef9400 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f55553d4b80 10 client.17658 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f55553d4b80 1 -- 172.21.15.19:0/3107933274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae03f109e0 con 0x55ae03ef9400 2024-05-09T01:33:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f551f7fe700 1 -- 172.21.15.19:0/3107933274 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55280b2250 con 0x55ae03ef9400 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f551f7fe700 1 -- 172.21.15.19:0/3107933274 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5538015480 con 0x55ae03ef9400 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f55553d4b80 10 client.17658 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f55553d4b80 10 client.17658 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f55553d4b80 20 client.17658 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55ae03f13070 0x55ae03f154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f55553d4b80 4 client.17658 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.185+0000 7f55553d4b80 1 -- 172.21.15.19:0/3107933274 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae03e29a80 con 0x55ae03f13070 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.186+0000 7f553ffff700 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55ae03f13070 0x55ae03f154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.186+0000 7f551f7fe700 10 client.17658 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.186+0000 7f553ffff700 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55ae03f13070 0x55ae03f154d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba136fb9bbcd45d3 server_cookie=89e97b6257a32955 in_seq=0 out_seq=0 2024-05-09T01:33:53.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f551f7fe700 1 -- 172.21.15.19:0/3107933274 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ae03e29a80 con 0x55ae03f13070 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f551f7fe700 10 client.17658 handle_command_reply: tid=0 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 client.17658 shutdown 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 -- 172.21.15.19:0/3107933274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03ef9400 msgr2=0x55ae03f03ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03ef9400 0x55ae03f03ac0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f553800aee0 tx=0x7f553800af10 comp rx=0 tx=0).stop 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 -- 172.21.15.19:0/3107933274 shutdown_connections 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55ae03f13070 0x55ae03f154d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03ef9400 0x55ae03f03ac0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 0x55ae03f03040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f551f7fe700 0 client.17658 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3107933274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03df6d00 0x55ae03f03580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 -- 172.21.15.19:0/3107933274 >> 172.21.15.19:0/3107933274 conn(0x55ae03d7ff70 msgr2=0x55ae03eedff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 -- 172.21.15.19:0/3107933274 shutdown_connections 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 1 -- 172.21.15.19:0/3107933274 wait complete. 2024-05-09T01:33:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.187+0000 7f55553d4b80 20 client.17658 trim_cache size 0 max 16384 2024-05-09T01:33:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.202+0000 7f55553d4b80 1 Processor -- start 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f55553d4b80 1 -- start start 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03d7c4a0 0x55ae03eeea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f02940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03ef9400 0x55ae03f03000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f55553d4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55401aee60 con 0x55ae03df6d00 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f55553d4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55401b09a0 con 0x55ae03d7c4a0 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f55553d4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55401af530 con 0x55ae03ef9400 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5537fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f02940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5537fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f02940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60392/0 (socket says 172.21.15.19:60392) 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5537fff700 1 -- 172.21.15.19:0/3376267853 learned_addr learned my addr 172.21.15.19:0/3376267853 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f553ffff700 1 --2- 172.21.15.19:0/3376267853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03d7c4a0 0x55ae03eeea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5544f5b700 1 --2- 172.21.15.19:0/3376267853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03ef9400 0x55ae03f03000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5537fff700 1 -- 172.21.15.19:0/3376267853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03ef9400 msgr2=0x55ae03f03000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:53.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5537fff700 1 --2- 172.21.15.19:0/3376267853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03ef9400 0x55ae03f03000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5537fff700 1 -- 172.21.15.19:0/3376267853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03d7c4a0 msgr2=0x55ae03eeea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:53.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5537fff700 1 --2- 172.21.15.19:0/3376267853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03d7c4a0 0x55ae03eeea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5537fff700 1 -- 172.21.15.19:0/3376267853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae03ef09d0 con 0x55ae03df6d00 2024-05-09T01:33:53.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.203+0000 7f5537fff700 1 --2- 172.21.15.19:0/3376267853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f02940 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f552800c5b0 tx=0x7f5528005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bee19eb4a181369 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:53.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f5535ffb700 1 -- 172.21.15.19:0/3376267853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55280c3040 con 0x55ae03df6d00 2024-05-09T01:33:53.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f5535ffb700 1 -- 172.21.15.19:0/3376267853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55280aa1a0 con 0x55ae03df6d00 2024-05-09T01:33:53.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f5535ffb700 1 -- 172.21.15.19:0/3376267853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55280b2050 con 0x55ae03df6d00 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f55553d4b80 1 -- 172.21.15.19:0/3376267853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 msgr2=0x55ae03f02940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3376267853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f02940 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f552800c5b0 tx=0x7f5528005910 comp rx=0 tx=0).stop 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f55553d4b80 1 -- 172.21.15.19:0/3376267853 shutdown_connections 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3376267853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03ef9400 0x55ae03f03000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3376267853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03d7c4a0 0x55ae03eeea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f55553d4b80 1 --2- 172.21.15.19:0/3376267853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f02940 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.204+0000 7f55553d4b80 1 -- 172.21.15.19:0/3376267853 >> 172.21.15.19:0/3376267853 conn(0x55ae03d7ff70 msgr2=0x55ae03efe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 -- 172.21.15.19:0/3376267853 shutdown_connections 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 -- 172.21.15.19:0/3376267853 wait complete. 2024-05-09T01:33:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 -- start start 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 Processor -- start 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 0x55ae03f0b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f0b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 0x55ae03f19c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55401b02d0 con 0x55ae03df6d00 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55401afc00 con 0x55ae03ef9400 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.205+0000 7f55553d4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55401ace00 con 0x55ae03d7c4a0 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f55553d4b80 10 client.0 _ll_register_callbacks cb 0x55ae03c8d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5537fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f0b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5537fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f0b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60408/0 (socket says 172.21.15.19:60408) 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5537fff700 1 -- 172.21.15.19:0/2801013542 learned_addr learned my addr 172.21.15.19:0/2801013542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5544f5b700 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 0x55ae03f19c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f553ffff700 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 0x55ae03f0b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5537fff700 1 -- 172.21.15.19:0/2801013542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 msgr2=0x55ae03f0b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5537fff700 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 0x55ae03f0b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5537fff700 1 -- 172.21.15.19:0/2801013542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 msgr2=0x55ae03f19c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5537fff700 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 0x55ae03f19c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5537fff700 1 -- 172.21.15.19:0/2801013542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae03effaf0 con 0x55ae03df6d00 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f5537fff700 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f0b650 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f55280a6d10 tx=0x7f55280a96b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4639c23bdcaf62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f551f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f551f7fe700 1 -- 172.21.15.19:0/2801013542 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55280c3050 con 0x55ae03df6d00 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f551f7fe700 1 -- 172.21.15.19:0/2801013542 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f55280a6090 con 0x55ae03df6d00 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f55553d4b80 1 -- 172.21.15.19:0/2801013542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55280aa1a0 con 0x55ae03df6d00 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f551f7fe700 1 -- 172.21.15.19:0/2801013542 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55280b2240 con 0x55ae03df6d00 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f55553d4b80 10 client.17664 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.206+0000 7f55553d4b80 1 -- 172.21.15.19:0/2801013542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae03f7cbe0 con 0x55ae03df6d00 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.207+0000 7f551f7fe700 1 -- 172.21.15.19:0/2801013542 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55280aa1a0 con 0x55ae03df6d00 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.207+0000 7f551f7fe700 1 -- 172.21.15.19:0/2801013542 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5528087020 con 0x55ae03df6d00 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.207+0000 7f55553d4b80 10 client.17664 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.207+0000 7f55553d4b80 10 client.17664 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.207+0000 7f55553d4b80 20 client.17664 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.207+0000 7f55553d4b80 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55ae03f7cdb0 0x55ae03f7f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.207+0000 7f55553d4b80 4 client.17664 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.207+0000 7f55553d4b80 1 -- 172.21.15.19:0/2801013542 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae03e23b20 con 0x55ae03f7cdb0 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.207+0000 7f553ffff700 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55ae03f7cdb0 0x55ae03f7f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f551f7fe700 10 client.17664 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f553ffff700 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55ae03f7cdb0 0x55ae03f7f1d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db3c152a9e3690cd server_cookie=33954d9ecfe1852c in_seq=0 out_seq=0 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f551f7fe700 1 -- 172.21.15.19:0/2801013542 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae03e23b20 con 0x55ae03f7cdb0 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f551f7fe700 10 client.17664 handle_command_reply: tid=0 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 client.17664 shutdown 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 -- 172.21.15.19:0/2801013542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 msgr2=0x55ae03f0b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f0b650 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f55280a6d10 tx=0x7f55280a96b0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 -- 172.21.15.19:0/2801013542 shutdown_connections 2024-05-09T01:33:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x55ae03f7cdb0 0x55ae03f7f1d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae03ef9400 0x55ae03f19c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae03d7c4a0 0x55ae03f0b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 --2- 172.21.15.19:0/2801013542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ae03df6d00 0x55ae03f0b650 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 -- 172.21.15.19:0/2801013542 >> 172.21.15.19:0/2801013542 conn(0x55ae03d7ff70 msgr2=0x55ae03efe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:53.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f551f7fe700 0 client.17664 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:33:53.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 -- 172.21.15.19:0/2801013542 shutdown_connections 2024-05-09T01:33:53.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 1 -- 172.21.15.19:0/2801013542 wait complete. 2024-05-09T01:33:53.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.208+0000 7f55553d4b80 20 client.17664 trim_cache size 0 max 16384 2024-05-09T01:33:53.210 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 -- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f552c0b2880 msgr2=0x7f552c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f552c0b2880 0x7f552c0b4d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5530003760 tx=0x7f553001a040 comp rx=0 tx=0).stop 2024-05-09T01:33:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 -- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5540107df0 msgr2=0x7f554019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5540107df0 0x7f554019b430 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f552800e970 tx=0x7f552800ec80 comp rx=0 tx=0).stop 2024-05-09T01:33:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 -- 172.21.15.19:0/3606596516 shutdown_connections 2024-05-09T01:33:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5540108740 0x7f554019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55401034c0 0x7f554019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f552c0b2880 0x7f552c0b4d90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 --2- 172.21.15.19:0/3606596516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5540107df0 0x7f554019b430 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 -- 172.21.15.19:0/3606596516 >> 172.21.15.19:0/3606596516 conn(0x7f55400fedf0 msgr2=0x7f55401096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 -- 172.21.15.19:0/3606596516 shutdown_connections 2024-05-09T01:33:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:53.209+0000 7f55469be700 1 -- 172.21.15.19:0/3606596516 wait complete. 2024-05-09T01:33:53.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:33:54.170 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:54.640 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:54 smithi019 ceph-mon[30158]: pgmap v629: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:54 smithi059 ceph-mon[34351]: pgmap v629: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:54.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:54 smithi196 ceph-mon[35041]: pgmap v629: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:33:55.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:55 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:33:55.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:55 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:33:55.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.879+0000 7f22aee12700 1 -- 172.21.15.19:0/1317616852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 msgr2=0x7f22a8104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.879+0000 7f22aee12700 1 --2- 172.21.15.19:0/1317616852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 0x7f22a8104a00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f229c0099c0 tx=0x7f229c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.879+0000 7f22aee12700 1 -- 172.21.15.19:0/1317616852 shutdown_connections 2024-05-09T01:33:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.879+0000 7f22aee12700 1 --2- 172.21.15.19:0/1317616852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a8109870 0x7f22a81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.879+0000 7f22aee12700 1 --2- 172.21.15.19:0/1317616852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 0x7f22a8104a00 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.879+0000 7f22aee12700 1 --2- 172.21.15.19:0/1317616852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a8108f20 0x7f22a8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.879+0000 7f22aee12700 1 -- 172.21.15.19:0/1317616852 >> 172.21.15.19:0/1317616852 conn(0x7f22a8100040 msgr2=0x7f22a8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.879+0000 7f22aee12700 1 -- 172.21.15.19:0/1317616852 shutdown_connections 2024-05-09T01:33:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.879+0000 7f22aee12700 1 -- 172.21.15.19:0/1317616852 wait complete. 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.880+0000 7f22aee12700 1 Processor -- start 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.880+0000 7f22aee12700 1 -- start start 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.880+0000 7f22aee12700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 0x7f22a8075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.880+0000 7f22aee12700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a8108f20 0x7f22a8076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.880+0000 7f22aee12700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a8109870 0x7f22a8076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22aee12700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81144a0 con 0x7f22a81045f0 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22aee12700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8114320 con 0x7f22a8109870 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22aee12700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8114620 con 0x7f22a8108f20 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22acbae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 0x7f22a8075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22acbae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 0x7f22a8075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60434/0 (socket says 172.21.15.19:60434) 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22acbae700 1 -- 172.21.15.19:0/1397980893 learned_addr learned my addr 172.21.15.19:0/1397980893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22ad3af700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a8109870 0x7f22a8076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22a7fff700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a8108f20 0x7f22a8076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22acbae700 1 -- 172.21.15.19:0/1397980893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a8108f20 msgr2=0x7f22a8076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22acbae700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a8108f20 0x7f22a8076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22acbae700 1 -- 172.21.15.19:0/1397980893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a8109870 msgr2=0x7f22a8076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22acbae700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a8109870 0x7f22a8076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22acbae700 1 -- 172.21.15.19:0/1397980893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a81a9210 con 0x7f22a81045f0 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22ad3af700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a8109870 0x7f22a8076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.881+0000 7f22acbae700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 0x7f22a8075ac0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f229c006010 tx=0x7f229c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.882+0000 7f22a5ffb700 1 -- 172.21.15.19:0/1397980893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c026070 con 0x7f22a81045f0 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.882+0000 7f22a5ffb700 1 -- 172.21.15.19:0/1397980893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c003c70 con 0x7f22a81045f0 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.882+0000 7f22aee12700 1 -- 172.21.15.19:0/1397980893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a81a94a0 con 0x7f22a81045f0 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.882+0000 7f22a5ffb700 1 -- 172.21.15.19:0/1397980893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c00fbe0 con 0x7f22a81045f0 2024-05-09T01:33:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.882+0000 7f22aee12700 1 -- 172.21.15.19:0/1397980893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a81a98f0 con 0x7f22a81045f0 2024-05-09T01:33:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.885+0000 7f22a5ffb700 1 -- 172.21.15.19:0/1397980893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f229c029070 con 0x7f22a81045f0 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.886+0000 7f22a5ffb700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22980b2880 0x7f22980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.886+0000 7f22bd828b80 1 Processor -- start 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.886+0000 7f22bd828b80 1 -- start start 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.886+0000 7f22a5ffb700 1 -- 172.21.15.19:0/1397980893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f229c0e2ab0 con 0x7f22a81045f0 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22a7fff700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22980b2880 0x7f22980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc87240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbc87780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc81810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22bd828b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbba40a0 con 0x55c5dbb7ed00 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22bd828b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbb499a0 con 0x55c5dbc81400 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22bd828b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbba0060 con 0x55c5dbb044a0 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22acbae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc87240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22ad3af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc81810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22acbae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc87240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60436/0 (socket says 172.21.15.19:60436) 2024-05-09T01:33:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22acbae700 1 -- 172.21.15.19:0/3067653270 learned_addr learned my addr 172.21.15.19:0/3067653270 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22ad3af700 1 -- 172.21.15.19:0/3067653270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 msgr2=0x55c5dbc87780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T01:33:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22ad3af700 1 --2- 172.21.15.19:0/3067653270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbc87780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22ad3af700 1 -- 172.21.15.19:0/3067653270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 msgr2=0x55c5dbc87240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.887+0000 7f22ad3af700 1 --2- 172.21.15.19:0/3067653270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc87240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.888+0000 7f22ad3af700 1 -- 172.21.15.19:0/3067653270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5dbb3a500 con 0x55c5dbc81400 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.888+0000 7f22acbae700 1 --2- 172.21.15.19:0/3067653270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc87240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.888+0000 7f22ad3af700 1 --2- 172.21.15.19:0/3067653270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc81810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f22a0003db0 tx=0x7f22a0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49161bd870a527a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.888+0000 7f22a7fff700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22980b2880 0x7f22980b4d90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55c5dbc7da00 tx=0x7f2294009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.888+0000 7f2292ffd700 1 -- 172.21.15.19:0/3067653270 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0019030 con 0x55c5dbc81400 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.888+0000 7f2292ffd700 1 -- 172.21.15.19:0/3067653270 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a0004bb0 con 0x55c5dbc81400 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.888+0000 7f2292ffd700 1 -- 172.21.15.19:0/3067653270 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0007d30 con 0x55c5dbc81400 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.889+0000 7f22bd828b80 1 -- 172.21.15.19:0/3067653270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 msgr2=0x55c5dbc81810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.889+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3067653270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc81810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f22a0003db0 tx=0x7f22a0009920 comp rx=0 tx=0).stop 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.889+0000 7f22bd828b80 1 -- 172.21.15.19:0/3067653270 shutdown_connections 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.889+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3067653270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc81810 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.889+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3067653270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc87240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.889+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3067653270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbc87780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.889+0000 7f22bd828b80 1 -- 172.21.15.19:0/3067653270 >> 172.21.15.19:0/3067653270 conn(0x55c5dbb07f70 msgr2=0x55c5dbc7c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:55.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.889+0000 7f22bd828b80 1 -- 172.21.15.19:0/3067653270 shutdown_connections 2024-05-09T01:33:55.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.889+0000 7f22bd828b80 1 -- 172.21.15.19:0/3067653270 wait complete. 2024-05-09T01:33:55.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.890+0000 7f22bd828b80 1 -- start start 2024-05-09T01:33:55.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.890+0000 7f22bd828b80 1 Processor -- start 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.890+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb044a0 0x55c5dbc8b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.890+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc8b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.890+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc8ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.890+0000 7f22bd828b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbc82160 con 0x55c5dbb044a0 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.890+0000 7f22bd828b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbb3eb20 con 0x55c5dbc81400 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.890+0000 7f22bd828b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbc8e020 con 0x55c5dbb7ed00 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.890+0000 7f22acbae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb044a0 0x55c5dbc8b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22a7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc8b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22acbae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb044a0 0x55c5dbc8b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60450/0 (socket says 172.21.15.19:60450) 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22acbae700 1 -- 172.21.15.19:0/3085828944 learned_addr learned my addr 172.21.15.19:0/3085828944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22ad3af700 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc8ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22bd828b80 10 client.0 _ll_register_callbacks cb 0x55c5dba156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22ad3af700 1 -- 172.21.15.19:0/3085828944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb7ed00 msgr2=0x55c5dbc8b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22ad3af700 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc8b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22ad3af700 1 -- 172.21.15.19:0/3085828944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb044a0 msgr2=0x55c5dbc8b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22ad3af700 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb044a0 0x55c5dbc8b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22ad3af700 1 -- 172.21.15.19:0/3085828944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5dba1f490 con 0x55c5dbc81400 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22ad3af700 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc8ba80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22a00094f0 tx=0x7f22a0007730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca1d93ba8f288338 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f2287fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f2287fff700 1 -- 172.21.15.19:0/3085828944 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0023070 con 0x55c5dbc81400 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f2287fff700 1 -- 172.21.15.19:0/3085828944 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f22a0007d60 con 0x55c5dbc81400 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.891+0000 7f22bd828b80 1 -- 172.21.15.19:0/3085828944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a0004bb0 con 0x55c5dbc81400 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f2287fff700 1 -- 172.21.15.19:0/3085828944 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22a002e070 con 0x55c5dbc81400 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f22bd828b80 10 client.27065 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f22bd828b80 1 -- 172.21.15.19:0/3085828944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5dbc989c0 con 0x55c5dbc81400 2024-05-09T01:33:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f2287fff700 1 -- 172.21.15.19:0/3085828944 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a0004bb0 con 0x55c5dbc81400 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f2287fff700 1 -- 172.21.15.19:0/3085828944 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f22a0028580 con 0x55c5dbc81400 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f22bd828b80 10 client.27065 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f22bd828b80 10 client.27065 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f22bd828b80 20 client.27065 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55c5dbc9af40 0x55c5dbc9d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f22bd828b80 4 client.27065 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f22bd828b80 1 -- 172.21.15.19:0/3085828944 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5dbbb1a80 con 0x55c5dbc9af40 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.892+0000 7f22acbae700 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55c5dbc9af40 0x55c5dbc9d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.893+0000 7f2287fff700 10 client.27065 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.893+0000 7f22acbae700 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55c5dbc9af40 0x55c5dbc9d360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca33f3a108b32f70 server_cookie=9de455387b9ef955 in_seq=0 out_seq=0 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.893+0000 7f2287fff700 1 -- 172.21.15.19:0/3085828944 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c5dbbb1a80 con 0x55c5dbc9af40 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.893+0000 7f2287fff700 10 client.27065 handle_command_reply: tid=0 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.893+0000 7f22bd828b80 1 client.27065 shutdown 2024-05-09T01:33:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 -- 172.21.15.19:0/3085828944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 msgr2=0x55c5dbc8ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc8ba80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22a00094f0 tx=0x7f22a0007730 comp rx=0 tx=0).stop 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 -- 172.21.15.19:0/3085828944 shutdown_connections 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55c5dbc9af40 0x55c5dbc9d360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc8ba80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb044a0 0x55c5dbc8b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 --2- 172.21.15.19:0/3085828944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc8b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f2287fff700 0 client.27065 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 -- 172.21.15.19:0/3085828944 >> 172.21.15.19:0/3085828944 conn(0x55c5dbb07f70 msgr2=0x55c5dbc75ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 -- 172.21.15.19:0/3085828944 shutdown_connections 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 1 -- 172.21.15.19:0/3085828944 wait complete. 2024-05-09T01:33:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.894+0000 7f22bd828b80 20 client.27065 trim_cache size 0 max 16384 2024-05-09T01:33:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22bd828b80 1 Processor -- start 2024-05-09T01:33:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22bd828b80 1 -- start start 2024-05-09T01:33:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbc97160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbb7ed00 0x55c5dbca0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbc81400 0x55c5dbca08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22bd828b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b0f10 con 0x55c5dbc81400 2024-05-09T01:33:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22bd828b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b2a50 con 0x55c5dbb7ed00 2024-05-09T01:33:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22bd828b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b15e0 con 0x55c5dbb044a0 2024-05-09T01:33:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22ad3af700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbc81400 0x55c5dbca08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.909+0000 7f22a7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbb7ed00 0x55c5dbca0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22ad3af700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbc81400 0x55c5dbca08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60458/0 (socket says 172.21.15.19:60458) 2024-05-09T01:33:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22ad3af700 1 -- 172.21.15.19:0/2439110457 learned_addr learned my addr 172.21.15.19:0/2439110457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22a7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbb7ed00 0x55c5dbca0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:34422/0 (socket says 172.21.15.19:34422) 2024-05-09T01:33:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22acbae700 1 --2- 172.21.15.19:0/2439110457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbc97160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22a7fff700 1 -- 172.21.15.19:0/2439110457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 msgr2=0x55c5dbc97160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22a7fff700 1 --2- 172.21.15.19:0/2439110457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbc97160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22a7fff700 1 -- 172.21.15.19:0/2439110457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbc81400 msgr2=0x55c5dbca08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22a7fff700 1 --2- 172.21.15.19:0/2439110457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbc81400 0x55c5dbca08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22a7fff700 1 -- 172.21.15.19:0/2439110457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5dbc789d0 con 0x55c5dbb7ed00 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22ad3af700 1 --2- 172.21.15.19:0/2439110457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbc81400 0x55c5dbca08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22a7fff700 1 --2- 172.21.15.19:0/2439110457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbb7ed00 0x55c5dbca0380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22940097d0 tx=0x7f2294014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a960c195791defa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22927fc700 1 -- 172.21.15.19:0/2439110457 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294004220 con 0x55c5dbb7ed00 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22927fc700 1 -- 172.21.15.19:0/2439110457 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2294012d60 con 0x55c5dbb7ed00 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.910+0000 7f22927fc700 1 -- 172.21.15.19:0/2439110457 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294003d70 con 0x55c5dbb7ed00 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.911+0000 7f22bd828b80 1 -- 172.21.15.19:0/2439110457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbb7ed00 msgr2=0x55c5dbca0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.911+0000 7f22bd828b80 1 --2- 172.21.15.19:0/2439110457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbb7ed00 0x55c5dbca0380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22940097d0 tx=0x7f2294014000 comp rx=0 tx=0).stop 2024-05-09T01:33:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.911+0000 7f22bd828b80 1 -- 172.21.15.19:0/2439110457 shutdown_connections 2024-05-09T01:33:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.911+0000 7f22bd828b80 1 --2- 172.21.15.19:0/2439110457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbc81400 0x55c5dbca08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.911+0000 7f22bd828b80 1 --2- 172.21.15.19:0/2439110457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbc97160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.911+0000 7f22bd828b80 1 --2- 172.21.15.19:0/2439110457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbb7ed00 0x55c5dbca0380 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.911+0000 7f22bd828b80 1 -- 172.21.15.19:0/2439110457 >> 172.21.15.19:0/2439110457 conn(0x55c5dbb07f70 msgr2=0x55c5dbc89910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.911+0000 7f22bd828b80 1 -- 172.21.15.19:0/2439110457 shutdown_connections 2024-05-09T01:33:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.911+0000 7f22bd828b80 1 -- 172.21.15.19:0/2439110457 wait complete. 2024-05-09T01:33:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.912+0000 7f22bd828b80 1 -- start start 2024-05-09T01:33:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.912+0000 7f22bd828b80 1 Processor -- start 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.912+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbca0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc9de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22bd828b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc9e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22bd828b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b2380 con 0x55c5dbb7ed00 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22bd828b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b1cb0 con 0x55c5dbc81400 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22bd828b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81aeeb0 con 0x55c5dbb044a0 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc9de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22ad3af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc9e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22bd828b80 10 client.0 _ll_register_callbacks cb 0x55c5dba156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc9de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60466/0 (socket says 172.21.15.19:60466) 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22a7fff700 1 -- 172.21.15.19:0/407276518 learned_addr learned my addr 172.21.15.19:0/407276518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22acbae700 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbca0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22ad3af700 1 -- 172.21.15.19:0/407276518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 msgr2=0x55c5dbca0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22ad3af700 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbca0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22ad3af700 1 -- 172.21.15.19:0/407276518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 msgr2=0x55c5dbc9de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22ad3af700 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc9de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22ad3af700 1 -- 172.21.15.19:0/407276518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5dbc9ec20 con 0x55c5dbc81400 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22a7fff700 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc9de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f22ad3af700 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc9e4a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f22a0000f80 tx=0x7f22a0028df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0f031a60ed90eab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.913+0000 7f2287fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f2287fff700 1 -- 172.21.15.19:0/407276518 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0023070 con 0x55c5dbc81400 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f22bd828b80 1 -- 172.21.15.19:0/407276518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2294012d60 con 0x55c5dbc81400 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f2287fff700 1 -- 172.21.15.19:0/407276518 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f22a0007d70 con 0x55c5dbc81400 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f2287fff700 1 -- 172.21.15.19:0/407276518 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22a0019040 con 0x55c5dbc81400 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f22bd828b80 10 client.27071 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f22bd828b80 1 -- 172.21.15.19:0/407276518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5dbd05520 con 0x55c5dbc81400 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f2287fff700 1 -- 172.21.15.19:0/407276518 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a0028460 con 0x55c5dbc81400 2024-05-09T01:33:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f2287fff700 1 -- 172.21.15.19:0/407276518 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f22a000ec10 con 0x55c5dbc81400 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f22bd828b80 10 client.27071 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f22bd828b80 10 client.27071 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f22bd828b80 20 client.27071 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f22bd828b80 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55c5dbd05b10 0x55c5dbd07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f22bd828b80 4 client.27071 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.914+0000 7f22bd828b80 1 -- 172.21.15.19:0/407276518 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2294012d60 con 0x55c5dbd05b10 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.915+0000 7f22acbae700 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55c5dbd05b10 0x55c5dbd07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.915+0000 7f2287fff700 10 client.27071 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.915+0000 7f22acbae700 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55c5dbd05b10 0x55c5dbd07f30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab52cfcc6648738a server_cookie=399db5bddb978152 in_seq=0 out_seq=0 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.915+0000 7f2287fff700 1 -- 172.21.15.19:0/407276518 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2294012d60 con 0x55c5dbd05b10 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.915+0000 7f2287fff700 10 client.27071 handle_command_reply: tid=0 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.915+0000 7f22bd828b80 1 client.27071 shutdown 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 -- 172.21.15.19:0/407276518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 msgr2=0x55c5dbc9e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc9e4a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f22a0000f80 tx=0x7f22a0028df0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 -- 172.21.15.19:0/407276518 shutdown_connections 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55c5dbd05b10 0x55c5dbd07f30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c5dbc81400 0x55c5dbc9e4a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5dbb044a0 0x55c5dbca0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 --2- 172.21.15.19:0/407276518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c5dbb7ed00 0x55c5dbc9de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f2287fff700 0 client.27071 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 -- 172.21.15.19:0/407276518 >> 172.21.15.19:0/407276518 conn(0x55c5dbb07f70 msgr2=0x55c5dbc84cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 -- 172.21.15.19:0/407276518 shutdown_connections 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 1 -- 172.21.15.19:0/407276518 wait complete. 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.916+0000 7f22bd828b80 20 client.27071 trim_cache size 0 max 16384 2024-05-09T01:33:55.916 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 -- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22980b2880 msgr2=0x7f22980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22980b2880 0x7f22980b4d90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55c5dbc7da00 tx=0x7f2294009260 comp rx=0 tx=0).stop 2024-05-09T01:33:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 -- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 msgr2=0x7f22a8075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 0x7f22a8075ac0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f229c006010 tx=0x7f229c004270 comp rx=0 tx=0).stop 2024-05-09T01:33:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 -- 172.21.15.19:0/1397980893 shutdown_connections 2024-05-09T01:33:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a8109870 0x7f22a8076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f22980b2880 0x7f22980b4d90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22a81045f0 0x7f22a8075ac0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 --2- 172.21.15.19:0/1397980893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a8108f20 0x7f22a8076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 -- 172.21.15.19:0/1397980893 >> 172.21.15.19:0/1397980893 conn(0x7f22a8100040 msgr2=0x7f22a8108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 -- 172.21.15.19:0/1397980893 shutdown_connections 2024-05-09T01:33:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:55.917+0000 7f22aee12700 1 -- 172.21.15.19:0/1397980893 wait complete. 2024-05-09T01:33:55.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:55 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:33:56.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:56 smithi019 ceph-mon[30158]: pgmap v630: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:33:56.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:33:56.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:33:56.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:33:56.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:33:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:56 smithi059 ceph-mon[34351]: pgmap v630: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:33:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:33:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:33:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:33:56.868 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:33:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:56 smithi196 ceph-mon[35041]: pgmap v630: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:33:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:33:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:33:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:33:56.991 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.991 INFO:tasks.workunit.client.0.smithi019.stdout:313/600: open fd = 3 2024-05-09T01:33:56.991 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.991 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.991 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.991 INFO:tasks.workunit.client.0.smithi019.stdout:314/600: open fd = 3 2024-05-09T01:33:56.991 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:315/600: open fd = 3 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:316/600: open fd = 3 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:317/600: open fd = 3 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.992 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:318/600: open fd = 3 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:319/600: open fd = 3 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:320/600: open fd = 3 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:321/600: open fd = 3 2024-05-09T01:33:56.993 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:322/600: open fd = 3 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:323/600: open fd = 3 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:324/600: open fd = 3 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.994 INFO:tasks.workunit.client.0.smithi019.stdout:325/600: open fd = 3 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:326/600: open fd = 3 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:327/600: open fd = 3 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:328/600: open fd = 3 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.995 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:329/600: open fd = 3 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:330/600: open fd = 3 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:331/600: open fd = 3 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:332/600: open fd = 3 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.996 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:333/600: open fd = 3 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:334/600: open fd = 3 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:335/600: open fd = 3 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:336/600: open fd = 3 2024-05-09T01:33:56.997 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:337/600: open fd = 3 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:338/600: open fd = 3 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:339/600: open fd = 3 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.998 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:340/600: open fd = 3 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:341/600: open fd = 3 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:342/600: open fd = 3 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:343/600: open fd = 3 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:56.999 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:344/600: open fd = 3 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:345/600: open fd = 3 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:346/600: open fd = 3 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.000 INFO:tasks.workunit.client.0.smithi019.stdout:347/600: open fd = 3 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:348/600: open fd = 3 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:349/600: open fd = 3 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:350/600: open fd = 3 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.001 INFO:tasks.workunit.client.0.smithi019.stdout:351/600: open fd = 3 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:352/600: open fd = 3 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:353/600: open fd = 3 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:354/600: open fd = 3 2024-05-09T01:33:57.002 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:355/600: open fd = 3 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:356/600: open fd = 3 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.003 INFO:tasks.workunit.client.0.smithi019.stdout:357/600: open fd = 3 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:358/600: open fd = 3 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:359/600: open fd = 3 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:360/600: open fd = 3 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.004 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:361/600: open fd = 3 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:362/600: open fd = 3 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:363/600: open fd = 3 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:364/600: open fd = 3 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.005 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:365/600: open fd = 3 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:366/600: open fd = 3 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:367/600: open fd = 3 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.006 INFO:tasks.workunit.client.0.smithi019.stdout:368/600: open fd = 3 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:369/600: open fd = 3 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:370/600: open fd = 3 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:371/600: open fd = 3 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.007 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:372/600: open fd = 3 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:373/600: open fd = 3 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:374/600: open fd = 3 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:33:57.008 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:33:58.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:33:58 smithi019 ceph-mon[30158]: pgmap v631: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.651+0000 7f87d282e700 1 -- 172.21.15.19:0/1882987849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 msgr2=0x7f87cc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.651+0000 7f87d282e700 1 --2- 172.21.15.19:0/1882987849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 0x7f87cc102ef0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0099c0 tx=0x7f87bc009cd0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.651+0000 7f87d282e700 1 -- 172.21.15.19:0/1882987849 shutdown_connections 2024-05-09T01:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.651+0000 7f87d282e700 1 --2- 172.21.15.19:0/1882987849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87cc109850 0x7f87cc1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.651+0000 7f87d282e700 1 --2- 172.21.15.19:0/1882987849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 0x7f87cc102ef0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.651+0000 7f87d282e700 1 --2- 172.21.15.19:0/1882987849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc108f00 0x7f87cc109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.651+0000 7f87d282e700 1 -- 172.21.15.19:0/1882987849 >> 172.21.15.19:0/1882987849 conn(0x7f87cc0fe540 msgr2=0x7f87cc100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.651+0000 7f87d282e700 1 -- 172.21.15.19:0/1882987849 shutdown_connections 2024-05-09T01:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.651+0000 7f87d282e700 1 -- 172.21.15.19:0/1882987849 wait complete. 2024-05-09T01:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.652+0000 7f87d282e700 1 Processor -- start 2024-05-09T01:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.652+0000 7f87d282e700 1 -- start start 2024-05-09T01:33:58.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.652+0000 7f87d282e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 0x7f87cc103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87d282e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87cc108f00 0x7f87cc103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87d282e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc109850 0x7f87cc1044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87d282e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1144d0 con 0x7f87cc102ae0 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87d282e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc114350 con 0x7f87cc108f00 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87d282e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc114650 con 0x7f87cc109850 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 0x7f87cc103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 0x7f87cc103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60484/0 (socket says 172.21.15.19:60484) 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87cbfff700 1 -- 172.21.15.19:0/3864065344 learned_addr learned my addr 172.21.15.19:0/3864065344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87c3fff700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87cc108f00 0x7f87cc103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc109850 0x7f87cc1044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87cbfff700 1 -- 172.21.15.19:0/3864065344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc109850 msgr2=0x7f87cc1044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87cbfff700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc109850 0x7f87cc1044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87cbfff700 1 -- 172.21.15.19:0/3864065344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87cc108f00 msgr2=0x7f87cc103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87cbfff700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87cc108f00 0x7f87cc103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87cbfff700 1 -- 172.21.15.19:0/3864065344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc1a91d0 con 0x7f87cc102ae0 2024-05-09T01:33:58.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87cbfff700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 0x7f87cc103a30 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009990 tx=0x7f87bc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:58.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.653+0000 7f87c9ffb700 1 -- 172.21.15.19:0/3864065344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc026070 con 0x7f87cc102ae0 2024-05-09T01:33:58.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.654+0000 7f87c9ffb700 1 -- 172.21.15.19:0/3864065344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87bc003f20 con 0x7f87cc102ae0 2024-05-09T01:33:58.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.654+0000 7f87d282e700 1 -- 172.21.15.19:0/3864065344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc1a9460 con 0x7f87cc102ae0 2024-05-09T01:33:58.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.654+0000 7f87c9ffb700 1 -- 172.21.15.19:0/3864065344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc00fc00 con 0x7f87cc102ae0 2024-05-09T01:33:58.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.654+0000 7f87d282e700 1 -- 172.21.15.19:0/3864065344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc1a9910 con 0x7f87cc102ae0 2024-05-09T01:33:58.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.656+0000 7f87c9ffb700 1 -- 172.21.15.19:0/3864065344 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f87bc029070 con 0x7f87cc102ae0 2024-05-09T01:33:58.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.657+0000 7f87c9ffb700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f87b80b2900 0x7f87b80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.657+0000 7f87c3fff700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f87b80b2900 0x7f87b80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.657+0000 7f87c9ffb700 1 -- 172.21.15.19:0/3864065344 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f87bc0e2be0 con 0x7f87cc102ae0 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.657+0000 7f87e1244b80 1 Processor -- start 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.657+0000 7f87e1244b80 1 -- start start 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87c3fff700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f87b80b2900 0x7f87b80b4e10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f87cc101c10 tx=0x7f87b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac05a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5abff6a0 0x557c5abffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87e1244b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c5ab21ff0 con 0x557c5aa824a0 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87e1244b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c5aac07f0 con 0x557c5abff6a0 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87e1244b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557c5ab1dfb0 con 0x557c5ab6c160 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac05a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87c3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac05a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60490/0 (socket says 172.21.15.19:60490) 2024-05-09T01:33:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87c3fff700 1 -- 172.21.15.19:0/370175681 learned_addr learned my addr 172.21.15.19:0/370175681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87cbfff700 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5abff6a0 0x557c5abffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87c3fff700 1 -- 172.21.15.19:0/370175681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 msgr2=0x557c5ac054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87c3fff700 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87c3fff700 1 -- 172.21.15.19:0/370175681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5abff6a0 msgr2=0x557c5abffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87c3fff700 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5abff6a0 0x557c5abffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87c3fff700 1 -- 172.21.15.19:0/370175681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5aab8370 con 0x557c5aa824a0 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5abff6a0 0x557c5abffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.658+0000 7f87cbfff700 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.659+0000 7f87c3fff700 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac05a30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f87b4010f40 tx=0x7f87b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3acc33dd7dcf602 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.659+0000 7f87c27fc700 1 -- 172.21.15.19:0/370175681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b400be60 con 0x557c5aa824a0 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.659+0000 7f87c27fc700 1 -- 172.21.15.19:0/370175681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b401ead0 con 0x557c5aa824a0 2024-05-09T01:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.659+0000 7f87c27fc700 1 -- 172.21.15.19:0/370175681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b401cad0 con 0x557c5aa824a0 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 -- 172.21.15.19:0/370175681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 msgr2=0x557c5ac05a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac05a30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f87b4010f40 tx=0x7f87b4013000 comp rx=0 tx=0).stop 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 -- 172.21.15.19:0/370175681 shutdown_connections 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5abff6a0 0x557c5abffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 --2- 172.21.15.19:0/370175681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac05a30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 -- 172.21.15.19:0/370175681 >> 172.21.15.19:0/370175681 conn(0x557c5aae26a0 msgr2=0x557c5abfa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 -- 172.21.15.19:0/370175681 shutdown_connections 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 -- 172.21.15.19:0/370175681 wait complete. 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.660+0000 7f87e1244b80 1 -- start start 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87e1244b80 1 Processor -- start 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac09230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5ab6c160 0x557c5ac09770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5abff6a0 0x557c5ac09cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87e1244b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c5a99dba0 con 0x557c5aa824a0 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87e1244b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c5aadc150 con 0x557c5ab6c160 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87e1244b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557c5ac0c1d0 con 0x557c5abff6a0 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87e1244b80 10 client.0 _ll_register_callbacks cb 0x557c5a9936c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac09230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac09230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60504/0 (socket says 172.21.15.19:60504) 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87cbfff700 1 -- 172.21.15.19:0/2479737203 learned_addr learned my addr 172.21.15.19:0/2479737203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5abff6a0 0x557c5ac09cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87c3fff700 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5ab6c160 0x557c5ac09770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87cbfff700 1 -- 172.21.15.19:0/2479737203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5abff6a0 msgr2=0x557c5ac09cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87cbfff700 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5abff6a0 0x557c5ac09cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87cbfff700 1 -- 172.21.15.19:0/2479737203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5ab6c160 msgr2=0x557c5ac09770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87cbfff700 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5ab6c160 0x557c5ac09770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87cbfff700 1 -- 172.21.15.19:0/2479737203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5a99d490 con 0x557c5aa824a0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.661+0000 7f87c3fff700 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5ab6c160 0x557c5ac09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87cbfff700 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac09230 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f87bc003800 tx=0x7f87bc0aa7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996fdaa19738401d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87ab7fe700 1 -- 172.21.15.19:0/2479737203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0aa270 con 0x557c5aa824a0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87ab7fe700 1 -- 172.21.15.19:0/2479737203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f87bc0ae170 con 0x557c5aa824a0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87e1244b80 1 -- 172.21.15.19:0/2479737203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87b401ead0 con 0x557c5aa824a0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87ab7fe700 1 -- 172.21.15.19:0/2479737203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87bc0ae620 con 0x557c5aa824a0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87e1244b80 10 client.17694 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87e1244b80 1 -- 172.21.15.19:0/2479737203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c5ac16c10 con 0x557c5aa824a0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87ab7fe700 1 -- 172.21.15.19:0/2479737203 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b401ead0 con 0x557c5aa824a0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87ab7fe700 1 -- 172.21.15.19:0/2479737203 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f87bc0e0b80 con 0x557c5aa824a0 2024-05-09T01:33:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87e1244b80 10 client.17694 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87e1244b80 10 client.17694 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87e1244b80 20 client.17694 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.662+0000 7f87e1244b80 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557c5ac191d0 0x557c5ac1b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.663+0000 7f87e1244b80 4 client.17694 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.663+0000 7f87e1244b80 1 -- 172.21.15.19:0/2479737203 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c5ab2f9d0 con 0x557c5ac191d0 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.663+0000 7f87c3fff700 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557c5ac191d0 0x557c5ac1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.663+0000 7f87c3fff700 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557c5ac191d0 0x557c5ac1b630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be029ab241ecdf31 server_cookie=4ad6e8e49b4d92ea in_seq=0 out_seq=0 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.663+0000 7f87ab7fe700 10 client.17694 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.664+0000 7f87ab7fe700 1 -- 172.21.15.19:0/2479737203 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c5ab2f9d0 con 0x557c5ac191d0 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.664+0000 7f87ab7fe700 10 client.17694 handle_command_reply: tid=0 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.664+0000 7f87e1244b80 1 client.17694 shutdown 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.664+0000 7f87e1244b80 1 -- 172.21.15.19:0/2479737203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 msgr2=0x557c5ac09230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.664+0000 7f87e1244b80 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac09230 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f87bc003800 tx=0x7f87bc0aa7b0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.664+0000 7f87e1244b80 1 -- 172.21.15.19:0/2479737203 shutdown_connections 2024-05-09T01:33:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.664+0000 7f87e1244b80 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5abff6a0 0x557c5ac09cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.664+0000 7f87e1244b80 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5aa824a0 0x557c5ac09230 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.664+0000 7f87e1244b80 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557c5ac191d0 0x557c5ac1b630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.665+0000 7f87ab7fe700 0 client.17694 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.665+0000 7f87e1244b80 1 --2- 172.21.15.19:0/2479737203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5ab6c160 0x557c5ac09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.665+0000 7f87e1244b80 1 -- 172.21.15.19:0/2479737203 >> 172.21.15.19:0/2479737203 conn(0x557c5aae26a0 msgr2=0x557c5abfa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.665+0000 7f87e1244b80 1 -- 172.21.15.19:0/2479737203 shutdown_connections 2024-05-09T01:33:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.665+0000 7f87e1244b80 1 -- 172.21.15.19:0/2479737203 wait complete. 2024-05-09T01:33:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.665+0000 7f87e1244b80 20 client.17694 trim_cache size 0 max 16384 2024-05-09T01:33:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87e1244b80 1 Processor -- start 2024-05-09T01:33:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87e1244b80 1 -- start start 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 0x557c5ac08190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac03510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87e1244b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1b0df0 con 0x557c5abff6a0 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87e1244b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1b2930 con 0x557c5aa824a0 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87e1244b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1b14c0 con 0x557c5ab6c160 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87d0dcb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87d0dcb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60506/0 (socket says 172.21.15.19:60506) 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87d0dcb700 1 -- 172.21.15.19:0/1547508664 learned_addr learned my addr 172.21.15.19:0/1547508664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87cbfff700 1 --2- 172.21.15.19:0/1547508664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 0x557c5ac08190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87d0dcb700 1 -- 172.21.15.19:0/1547508664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 msgr2=0x557c5ac03510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/1547508664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac03510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87d0dcb700 1 -- 172.21.15.19:0/1547508664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 msgr2=0x557c5ac08190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:58.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.680+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/1547508664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 0x557c5ac08190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.681+0000 7f87d0dcb700 1 -- 172.21.15.19:0/1547508664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5ac0c030 con 0x557c5abff6a0 2024-05-09T01:33:58.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.681+0000 7f87cbfff700 1 --2- 172.21.15.19:0/1547508664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 0x557c5ac08190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:33:58.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.681+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/1547508664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac091f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f87c40135f0 tx=0x7f87c4007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d32adce7fa366b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:58.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.681+0000 7f87c1ffb700 1 -- 172.21.15.19:0/1547508664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c401a070 con 0x557c5abff6a0 2024-05-09T01:33:58.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.681+0000 7f87c1ffb700 1 -- 172.21.15.19:0/1547508664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c4003c10 con 0x557c5abff6a0 2024-05-09T01:33:58.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.681+0000 7f87c1ffb700 1 -- 172.21.15.19:0/1547508664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c40167d0 con 0x557c5abff6a0 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 -- 172.21.15.19:0/1547508664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 msgr2=0x557c5ac091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1547508664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac091f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f87c40135f0 tx=0x7f87c4007950 comp rx=0 tx=0).stop 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 -- 172.21.15.19:0/1547508664 shutdown_connections 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1547508664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac091f0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1547508664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 0x557c5ac08190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1547508664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac03510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 -- 172.21.15.19:0/1547508664 >> 172.21.15.19:0/1547508664 conn(0x557c5aae26a0 msgr2=0x557c5ac02e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 -- 172.21.15.19:0/1547508664 shutdown_connections 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 -- 172.21.15.19:0/1547508664 wait complete. 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 -- start start 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.682+0000 7f87e1244b80 1 Processor -- start 2024-05-09T01:33:58.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 0x557c5ac11660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac0a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87e1244b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac0aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87e1244b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1b2260 con 0x557c5abff6a0 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87e1244b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1b1b90 con 0x557c5aa824a0 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87e1244b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1aed90 con 0x557c5ab6c160 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87e1244b80 10 client.0 _ll_register_callbacks cb 0x557c5a9936c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87d0dcb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac0aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87d0dcb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac0aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60518/0 (socket says 172.21.15.19:60518) 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87d0dcb700 1 -- 172.21.15.19:0/1800571041 learned_addr learned my addr 172.21.15.19:0/1800571041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87cbfff700 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 0x557c5ac11660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87c3fff700 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac0a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87d0dcb700 1 -- 172.21.15.19:0/1800571041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 msgr2=0x557c5ac0a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac0a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87d0dcb700 1 -- 172.21.15.19:0/1800571041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 msgr2=0x557c5ac11660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 0x557c5ac11660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.683+0000 7f87d0dcb700 1 -- 172.21.15.19:0/1800571041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5ac0b030 con 0x557c5abff6a0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87d0dcb700 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac0aac0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007e10 tx=0x7f87c4007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cafaa686efc83d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87ab7fe700 1 -- 172.21.15.19:0/1800571041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c401a040 con 0x557c5abff6a0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87ab7fe700 1 -- 172.21.15.19:0/1800571041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f87c4029680 con 0x557c5abff6a0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87e1244b80 1 -- 172.21.15.19:0/1800571041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c4003c10 con 0x557c5abff6a0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87ab7fe700 1 -- 172.21.15.19:0/1800571041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87c4029880 con 0x557c5abff6a0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87e1244b80 10 client.17700 fetch_fsmap learned FSMap version 28 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87e1244b80 1 -- 172.21.15.19:0/1800571041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c5ac835f0 con 0x557c5abff6a0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87ab7fe700 1 -- 172.21.15.19:0/1800571041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c4003c10 con 0x557c5abff6a0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87ab7fe700 1 -- 172.21.15.19:0/1800571041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f87c401fcd0 con 0x557c5abff6a0 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87e1244b80 10 client.17700 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87e1244b80 10 client.17700 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:33:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87e1244b80 20 client.17700 populate_metadata read hostname 'smithi019' 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557c5ac83b70 0x557c5ac85f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87e1244b80 4 client.17700 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.684+0000 7f87e1244b80 1 -- 172.21.15.19:0/1800571041 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c5ab29a70 con 0x557c5ac83b70 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.685+0000 7f87cbfff700 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557c5ac83b70 0x557c5ac85f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.685+0000 7f87ab7fe700 10 client.17700 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.685+0000 7f87cbfff700 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557c5ac83b70 0x557c5ac85f90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d76d71ae46d29568 server_cookie=39f85c5ed9100f39 in_seq=0 out_seq=0 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.685+0000 7f87ab7fe700 1 -- 172.21.15.19:0/1800571041 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c5ab29a70 con 0x557c5ac83b70 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.685+0000 7f87ab7fe700 10 client.17700 handle_command_reply: tid=0 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.685+0000 7f87e1244b80 1 client.17700 shutdown 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 -- 172.21.15.19:0/1800571041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 msgr2=0x557c5ac0aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac0aac0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007e10 tx=0x7f87c4007e40 comp rx=0 tx=0).stop 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 -- 172.21.15.19:0/1800571041 shutdown_connections 2024-05-09T01:33:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c5abff6a0 0x557c5ac0aac0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557c5ac83b70 0x557c5ac85f90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c5aa824a0 0x557c5ac11660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 --2- 172.21.15.19:0/1800571041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c5ab6c160 0x557c5ac0a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 -- 172.21.15.19:0/1800571041 >> 172.21.15.19:0/1800571041 conn(0x557c5aae26a0 msgr2=0x557c5abfe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87ab7fe700 0 client.17700 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:33:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 -- 172.21.15.19:0/1800571041 shutdown_connections 2024-05-09T01:33:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 1 -- 172.21.15.19:0/1800571041 wait complete. 2024-05-09T01:33:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87e1244b80 20 client.17700 trim_cache size 0 max 16384 2024-05-09T01:33:58.687 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:33:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87d282e700 1 -- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f87b80b2900 msgr2=0x7f87b80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87d282e700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f87b80b2900 0x7f87b80b4e10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f87cc101c10 tx=0x7f87b400b410 comp rx=0 tx=0).stop 2024-05-09T01:33:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87d282e700 1 -- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 msgr2=0x7f87cc103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:33:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87d282e700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 0x7f87cc103a30 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009990 tx=0x7f87bc0047b0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.686+0000 7f87d282e700 1 -- 172.21.15.19:0/3864065344 shutdown_connections 2024-05-09T01:33:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.687+0000 7f87d282e700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc109850 0x7f87cc1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.687+0000 7f87d282e700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f87b80b2900 0x7f87b80b4e10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.687+0000 7f87d282e700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87cc102ae0 0x7f87cc103a30 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.687+0000 7f87d282e700 1 --2- 172.21.15.19:0/3864065344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87cc108f00 0x7f87cc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:33:58.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.687+0000 7f87d282e700 1 -- 172.21.15.19:0/3864065344 >> 172.21.15.19:0/3864065344 conn(0x7f87cc0fe540 msgr2=0x7f87cc107120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:33:58.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.687+0000 7f87d282e700 1 -- 172.21.15.19:0/3864065344 shutdown_connections 2024-05-09T01:33:58.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:33:58.687+0000 7f87d282e700 1 -- 172.21.15.19:0/3864065344 wait complete. 2024-05-09T01:33:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:33:58 smithi059 ceph-mon[34351]: pgmap v631: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:33:58.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:33:58 smithi196 ceph-mon[35041]: pgmap v631: 97 pgs: 97 active+clean; 1.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:33:59.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:33:59.868 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:00 smithi059 ceph-mon[34351]: pgmap v632: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:00.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:00 smithi019 ceph-mon[30158]: pgmap v632: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:00.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:00 smithi196 ceph-mon[35041]: pgmap v632: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.671+0000 7f018bc8c700 1 -- 172.21.15.19:0/2228637372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184108740 msgr2=0x7f01841100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.671+0000 7f018bc8c700 1 --2- 172.21.15.19:0/2228637372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184108740 0x7f01841100a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f017c00b0d0 tx=0x7f017c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.671+0000 7f018bc8c700 1 -- 172.21.15.19:0/2228637372 shutdown_connections 2024-05-09T01:34:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.671+0000 7f018bc8c700 1 --2- 172.21.15.19:0/2228637372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184108740 0x7f01841100a0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.671+0000 7f018bc8c700 1 --2- 172.21.15.19:0/2228637372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01841034c0 0x7f01841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.671+0000 7f018bc8c700 1 --2- 172.21.15.19:0/2228637372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0184107df0 0x7f0184108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.671+0000 7f018bc8c700 1 -- 172.21.15.19:0/2228637372 >> 172.21.15.19:0/2228637372 conn(0x7f01840fedf0 msgr2=0x7f0184101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.671+0000 7f018bc8c700 1 -- 172.21.15.19:0/2228637372 shutdown_connections 2024-05-09T01:34:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.671+0000 7f018bc8c700 1 -- 172.21.15.19:0/2228637372 wait complete. 2024-05-09T01:34:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.672+0000 7f018bc8c700 1 Processor -- start 2024-05-09T01:34:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.672+0000 7f018bc8c700 1 -- start start 2024-05-09T01:34:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.672+0000 7f018bc8c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01841034c0 0x7f018419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.672+0000 7f018bc8c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184107df0 0x7f018419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.672+0000 7f018bc8c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0184108740 0x7f018419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.672+0000 7f018bc8c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0184113370 con 0x7f0184107df0 2024-05-09T01:34:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.672+0000 7f018bc8c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01841131f0 con 0x7f0184108740 2024-05-09T01:34:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.672+0000 7f018bc8c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01841134f0 con 0x7f01841034c0 2024-05-09T01:34:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184107df0 0x7f018419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184107df0 0x7f018419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54866/0 (socket says 172.21.15.19:54866) 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189227700 1 -- 172.21.15.19:0/516453414 learned_addr learned my addr 172.21.15.19:0/516453414 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189a28700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01841034c0 0x7f018419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f018a229700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0184108740 0x7f018419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189227700 1 -- 172.21.15.19:0/516453414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01841034c0 msgr2=0x7f018419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189227700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01841034c0 0x7f018419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189227700 1 -- 172.21.15.19:0/516453414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0184108740 msgr2=0x7f018419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189227700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0184108740 0x7f018419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189227700 1 -- 172.21.15.19:0/516453414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018419fd00 con 0x7f0184107df0 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f0189227700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184107df0 0x7f018419b3d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f017400d980 tx=0x7f017400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f017affd700 1 -- 172.21.15.19:0/516453414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174017070 con 0x7f0184107df0 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f017affd700 1 -- 172.21.15.19:0/516453414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f017400fd40 con 0x7f0184107df0 2024-05-09T01:34:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f018bc8c700 1 -- 172.21.15.19:0/516453414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018419fff0 con 0x7f0184107df0 2024-05-09T01:34:01.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f017affd700 1 -- 172.21.15.19:0/516453414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017400e3b0 con 0x7f0184107df0 2024-05-09T01:34:01.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.673+0000 7f018bc8c700 1 -- 172.21.15.19:0/516453414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01841a7910 con 0x7f0184107df0 2024-05-09T01:34:01.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.676+0000 7f017affd700 1 -- 172.21.15.19:0/516453414 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f017400e550 con 0x7f0184107df0 2024-05-09T01:34:01.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.677+0000 7f017affd700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01700b2880 0x7f01700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.677+0000 7f019a6a2b80 1 Processor -- start 2024-05-09T01:34:01.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.677+0000 7f0189a28700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01700b2880 0x7f01700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.677+0000 7f017affd700 1 -- 172.21.15.19:0/516453414 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f01740e1c80 con 0x7f0184107df0 2024-05-09T01:34:01.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f019a6a2b80 1 -- start start 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860ea160 0x55f1861834f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f186183a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f18617dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f18609fff0 con 0x55f1860004a0 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f18603e7f0 con 0x55f18617d6a0 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f18609bfb0 con 0x55f1860ea160 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189a28700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01700b2880 0x7f01700b4d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0180005950 tx=0x7f018000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f186183a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f186183a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54878/0 (socket says 172.21.15.19:54878) 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189227700 1 -- 172.21.15.19:0/1655433478 learned_addr learned my addr 172.21.15.19:0/1655433478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189a28700 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860ea160 0x55f1861834f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f018a229700 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f18617dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189227700 1 -- 172.21.15.19:0/1655433478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860ea160 msgr2=0x55f1861834f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189227700 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860ea160 0x55f1861834f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189227700 1 -- 172.21.15.19:0/1655433478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 msgr2=0x55f18617dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189227700 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f18617dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f0189227700 1 -- 172.21.15.19:0/1655433478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f186036370 con 0x55f1860004a0 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.678+0000 7f018a229700 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f18617dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.679+0000 7f0189227700 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f186183a30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f01740a5300 tx=0x7f01740a8ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858d2835566312da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.679+0000 7f016ffff700 1 -- 172.21.15.19:0/1655433478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174020070 con 0x55f1860004a0 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.679+0000 7f016ffff700 1 -- 172.21.15.19:0/1655433478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01740ad630 con 0x55f1860004a0 2024-05-09T01:34:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.679+0000 7f016ffff700 1 -- 172.21.15.19:0/1655433478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740b1e70 con 0x55f1860004a0 2024-05-09T01:34:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1655433478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 msgr2=0x55f186183a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f186183a30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f01740a5300 tx=0x7f01740a8ac0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1655433478 shutdown_connections 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f18617dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860ea160 0x55f1861834f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1655433478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f186183a30 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1655433478 >> 172.21.15.19:0/1655433478 conn(0x55f1860606a0 msgr2=0x55f186178400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1655433478 shutdown_connections 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1655433478 wait complete. 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 -- start start 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.680+0000 7f019a6a2b80 1 Processor -- start 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 0x55f1861871f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f186187c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f185f1bba0 con 0x55f1860ea160 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f18605a150 con 0x55f18617d6a0 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f18618a1d0 con 0x55f1860004a0 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f019a6a2b80 10 client.0 _ll_register_callbacks cb 0x55f185f116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f0189227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f0189227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54882/0 (socket says 172.21.15.19:54882) 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f0189227700 1 -- 172.21.15.19:0/1634297620 learned_addr learned my addr 172.21.15.19:0/1634297620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f0189a28700 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 0x55f1861871f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f018a229700 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f186187c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f0189227700 1 -- 172.21.15.19:0/1634297620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 msgr2=0x55f1861871f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f0189227700 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 0x55f1861871f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f0189227700 1 -- 172.21.15.19:0/1634297620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 msgr2=0x55f186187c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f0189227700 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f186187c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f0189227700 1 -- 172.21.15.19:0/1634297620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f185f1b490 con 0x55f1860ea160 2024-05-09T01:34:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.681+0000 7f018a229700 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f186187c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f0189227700 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187730 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f01740a64f0 tx=0x7f01740a9710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ac348820ab1f0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f016d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f016d7fa700 1 -- 172.21.15.19:0/1634297620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740c2070 con 0x55f1860ea160 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f016d7fa700 1 -- 172.21.15.19:0/1634297620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f01740b1e60 con 0x55f1860ea160 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1634297620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01740ad630 con 0x55f1860ea160 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f016d7fa700 1 -- 172.21.15.19:0/1634297620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0174020030 con 0x55f1860ea160 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f019a6a2b80 10 client.17712 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1634297620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f186194af0 con 0x55f1860ea160 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f016d7fa700 1 -- 172.21.15.19:0/1634297620 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01740ad630 con 0x55f1860ea160 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f016d7fa700 1 -- 172.21.15.19:0/1634297620 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f01740a84e0 con 0x55f1860ea160 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.682+0000 7f019a6a2b80 10 client.17712 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.683+0000 7f019a6a2b80 10 client.17712 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.683+0000 7f019a6a2b80 20 client.17712 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.683+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f186196fc0 0x55f1861993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.683+0000 7f019a6a2b80 4 client.17712 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.683+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1634297620 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1860ad9d0 con 0x55f186196fc0 2024-05-09T01:34:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.683+0000 7f0189a28700 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f186196fc0 0x55f1861993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.683+0000 7f016d7fa700 10 client.17712 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:34:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.683+0000 7f0189a28700 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f186196fc0 0x55f1861993e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=407cc3357bf5d510 server_cookie=3f2224feb8cdf2c5 in_seq=0 out_seq=0 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.684+0000 7f016d7fa700 1 -- 172.21.15.19:0/1634297620 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1860ad9d0 con 0x55f186196fc0 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.684+0000 7f016d7fa700 10 client.17712 handle_command_reply: tid=0 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.684+0000 7f019a6a2b80 1 client.17712 shutdown 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.684+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1634297620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 msgr2=0x55f186187730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.684+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187730 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f01740a64f0 tx=0x7f01740a9710 comp rx=0 tx=0).stop 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1634297620 shutdown_connections 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f186196fc0 0x55f1861993e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f186187c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 0x55f1861871f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f016d7fa700 0 client.17712 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:34:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/1634297620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187730 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1634297620 >> 172.21.15.19:0/1634297620 conn(0x55f1860606a0 msgr2=0x55f1861781c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1634297620 shutdown_connections 2024-05-09T01:34:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f019a6a2b80 1 -- 172.21.15.19:0/1634297620 wait complete. 2024-05-09T01:34:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.685+0000 7f019a6a2b80 20 client.17712 trim_cache size 0 max 16384 2024-05-09T01:34:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.700+0000 7f019a6a2b80 1 Processor -- start 2024-05-09T01:34:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.700+0000 7f019a6a2b80 1 -- start start 2024-05-09T01:34:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.700+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 0x55f186186150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.700+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.700+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f186193230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.700+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01841aede0 con 0x55f1860ea160 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.700+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01841b0920 con 0x55f18617d6a0 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.700+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01841af4b0 con 0x55f1860004a0 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.700+0000 7f0189227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189227700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54896/0 (socket says 172.21.15.19:54896) 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189227700 1 -- 172.21.15.19:0/2457153950 learned_addr learned my addr 172.21.15.19:0/2457153950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189a28700 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 0x55f186186150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189227700 1 -- 172.21.15.19:0/2457153950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 msgr2=0x55f186186150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f018a229700 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f186193230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189227700 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 0x55f186186150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189227700 1 -- 172.21.15.19:0/2457153950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 msgr2=0x55f186193230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189227700 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f186193230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189227700 1 -- 172.21.15.19:0/2457153950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f186189a90 con 0x55f1860ea160 2024-05-09T01:34:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189a28700 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 0x55f186186150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f0189227700 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187ab0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f01740a9210 tx=0x7f01740a8ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c679d521b74b1c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f016f7fe700 1 -- 172.21.15.19:0/2457153950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740c2070 con 0x55f1860ea160 2024-05-09T01:34:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f016f7fe700 1 -- 172.21.15.19:0/2457153950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01740ad630 con 0x55f1860ea160 2024-05-09T01:34:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.701+0000 7f016f7fe700 1 -- 172.21.15.19:0/2457153950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740c3dd0 con 0x55f1860ea160 2024-05-09T01:34:01.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.702+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2457153950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 msgr2=0x55f186187ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:01.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.702+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187ab0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f01740a9210 tx=0x7f01740a8ac0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.702+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2457153950 shutdown_connections 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.702+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f18617d6a0 0x55f186193230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.702+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1860004a0 0x55f186186150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.702+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2457153950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860ea160 0x55f186187ab0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.702+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2457153950 >> 172.21.15.19:0/2457153950 conn(0x55f1860606a0 msgr2=0x55f186180e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.702+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2457153950 shutdown_connections 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.702+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2457153950 wait complete. 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f019a6a2b80 1 -- start start 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f019a6a2b80 1 Processor -- start 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f1861993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1860ea160 0x55f186199920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f019a6a2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f18617d6a0 0x55f1861ff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01841b0250 con 0x55f1860004a0 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01841afb80 con 0x55f1860ea160 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f019a6a2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01841acd80 con 0x55f18617d6a0 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f019a6a2b80 10 client.0 _ll_register_callbacks cb 0x55f185f116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f0189a28700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f1861993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f0189a28700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f1861993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54908/0 (socket says 172.21.15.19:54908) 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f0189a28700 1 -- 172.21.15.19:0/2536826089 learned_addr learned my addr 172.21.15.19:0/2536826089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.703+0000 7f018a229700 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f18617d6a0 0x55f1861ff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f0189227700 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1860ea160 0x55f186199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f0189a28700 1 -- 172.21.15.19:0/2536826089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f18617d6a0 msgr2=0x55f1861ff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f0189a28700 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f18617d6a0 0x55f1861ff500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f0189a28700 1 -- 172.21.15.19:0/2536826089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1860ea160 msgr2=0x55f186199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f0189a28700 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1860ea160 0x55f186199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f0189a28700 1 -- 172.21.15.19:0/2536826089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18619a160 con 0x55f1860004a0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f0189227700 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1860ea160 0x55f186199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f0189a28700 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f1861993e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x55f18618a1b0 tx=0x7f01800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e17be60d95dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f016d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f016d7fa700 1 -- 172.21.15.19:0/2536826089 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180016070 con 0x55f1860004a0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f016d7fa700 1 -- 172.21.15.19:0/2536826089 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f01800054d0 con 0x55f1860004a0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2536826089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01740ad630 con 0x55f1860004a0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f016d7fa700 1 -- 172.21.15.19:0/2536826089 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f01800056d0 con 0x55f1860004a0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f019a6a2b80 10 client.17718 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2536826089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1862009e0 con 0x55f1860004a0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.704+0000 7f016d7fa700 1 -- 172.21.15.19:0/2536826089 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01740ad630 con 0x55f1860004a0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.705+0000 7f016d7fa700 1 -- 172.21.15.19:0/2536826089 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f01800348b0 con 0x55f1860004a0 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.705+0000 7f019a6a2b80 10 client.17718 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.705+0000 7f019a6a2b80 10 client.17718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.705+0000 7f019a6a2b80 20 client.17718 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.705+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f186201080 0x55f1862034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.705+0000 7f019a6a2b80 4 client.17718 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.705+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2536826089 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1860a7a70 con 0x55f186201080 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.705+0000 7f0189227700 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f186201080 0x55f1862034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f016d7fa700 10 client.17718 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f0189227700 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f186201080 0x55f1862034a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcf939ada8918554 server_cookie=978190cd988f6ada in_seq=0 out_seq=0 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f016d7fa700 1 -- 172.21.15.19:0/2536826089 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1860a7a70 con 0x55f186201080 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f016d7fa700 10 client.17718 handle_command_reply: tid=0 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 client.17718 shutdown 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2536826089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 msgr2=0x55f1861993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f1861993e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x55f18618a1b0 tx=0x7f01800047c0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2536826089 shutdown_connections 2024-05-09T01:34:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55f186201080 0x55f1862034a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f18617d6a0 0x55f1861ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1860004a0 0x55f1861993e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 --2- 172.21.15.19:0/2536826089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1860ea160 0x55f186199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f016d7fa700 0 client.17718 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:34:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2536826089 >> 172.21.15.19:0/2536826089 conn(0x55f1860606a0 msgr2=0x55f186180e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2536826089 shutdown_connections 2024-05-09T01:34:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 1 -- 172.21.15.19:0/2536826089 wait complete. 2024-05-09T01:34:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.706+0000 7f019a6a2b80 20 client.17718 trim_cache size 0 max 16384 2024-05-09T01:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:34:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 -- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01700b2880 msgr2=0x7f01700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01700b2880 0x7f01700b4d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0180005950 tx=0x7f018000a300 comp rx=0 tx=0).stop 2024-05-09T01:34:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 -- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184107df0 msgr2=0x7f018419b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184107df0 0x7f018419b3d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f017400d980 tx=0x7f017400dc90 comp rx=0 tx=0).stop 2024-05-09T01:34:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 -- 172.21.15.19:0/516453414 shutdown_connections 2024-05-09T01:34:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0184108740 0x7f018419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01841034c0 0x7f018419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01700b2880 0x7f01700b4d90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 --2- 172.21.15.19:0/516453414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0184107df0 0x7f018419b3d0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 -- 172.21.15.19:0/516453414 >> 172.21.15.19:0/516453414 conn(0x7f01840fedf0 msgr2=0x7f0184075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 -- 172.21.15.19:0/516453414 shutdown_connections 2024-05-09T01:34:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:01.707+0000 7f018bc8c700 1 -- 172.21.15.19:0/516453414 wait complete. 2024-05-09T01:34:02.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:34:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.424+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/1265754255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb28106d70 msgr2=0x7fdb28109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.424+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/1265754255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb28106d70 0x7fdb28109250 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ca0 tx=0x7fdb1802a8c0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.425+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/1265754255 shutdown_connections 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.425+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/1265754255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb281098e0 0x7fdb2810bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.425+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/1265754255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb28106d70 0x7fdb28109250 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.425+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/1265754255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb28104380 0x7fdb281067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.425+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/1265754255 >> 172.21.15.19:0/1265754255 conn(0x7fdb280fdd40 msgr2=0x7fdb281001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.425+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/1265754255 shutdown_connections 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.425+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/1265754255 wait complete. 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.425+0000 7fdb2ea8b640 1 Processor -- start 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.425+0000 7fdb2ea8b640 1 -- start start 2024-05-09T01:34:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2ea8b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28104380 0x7fdb2819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2ea8b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb28106d70 0x7fdb2819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2ea8b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb281098e0 0x7fdb281a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2ea8b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2810f750 con 0x7fdb281098e0 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2ea8b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2810f5d0 con 0x7fdb28104380 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2ea8b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2810f8d0 con 0x7fdb28106d70 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2d028640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb281098e0 0x7fdb281a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2d028640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb281098e0 0x7fdb281a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54920/0 (socket says 172.21.15.19:54920) 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2d028640 1 -- 172.21.15.19:0/194421879 learned_addr learned my addr 172.21.15.19:0/194421879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2c827640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28104380 0x7fdb2819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb27fff640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb28106d70 0x7fdb2819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2d028640 1 -- 172.21.15.19:0/194421879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb28106d70 msgr2=0x7fdb2819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2d028640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb28106d70 0x7fdb2819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2d028640 1 -- 172.21.15.19:0/194421879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28104380 msgr2=0x7fdb2819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2d028640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28104380 0x7fdb2819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.426+0000 7fdb2d028640 1 -- 172.21.15.19:0/194421879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb281a1ec0 con 0x7fdb281098e0 2024-05-09T01:34:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.427+0000 7fdb2d028640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb281098e0 0x7fdb281a1850 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000e970 tx=0x7fdb1000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.427+0000 7fdb25ffb640 1 -- 172.21.15.19:0/194421879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000ccc0 con 0x7fdb281098e0 2024-05-09T01:34:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.427+0000 7fdb25ffb640 1 -- 172.21.15.19:0/194421879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb1000ce60 con 0x7fdb281098e0 2024-05-09T01:34:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.427+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/194421879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb281a21b0 con 0x7fdb281098e0 2024-05-09T01:34:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.427+0000 7fdb25ffb640 1 -- 172.21.15.19:0/194421879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10010430 con 0x7fdb281098e0 2024-05-09T01:34:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.427+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/194421879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb281a26b0 con 0x7fdb281098e0 2024-05-09T01:34:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.429+0000 7fdb35a15740 1 Processor -- start 2024-05-09T01:34:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb25ffb640 1 -- 172.21.15.19:0/194421879 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdb10004ff0 con 0x7fdb281098e0 2024-05-09T01:34:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb35a15740 1 -- start start 2024-05-09T01:34:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8f11b0 0x56185dc95500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc95a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8d04d0 0x56185dc9c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb35a15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56185dbf97c0 con 0x56185d8d04d0 2024-05-09T01:34:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb35a15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56185db950e0 con 0x56185d8f11b0 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb35a15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56185dbf5780 con 0x56185d8c82f0 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8d04d0 0x56185dc9c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8d04d0 0x56185dc9c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54922/0 (socket says 172.21.15.19:54922) 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.430+0000 7fdb27fff640 1 -- 172.21.15.19:0/2382365375 learned_addr learned my addr 172.21.15.19:0/2382365375 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2382365375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc95a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb27fff640 1 --2- 172.21.15.19:0/2382365375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8f11b0 0x56185dc95500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb2c827640 1 -- 172.21.15.19:0/2382365375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8f11b0 msgr2=0x56185dc95500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2382365375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8f11b0 0x56185dc95500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb2c827640 1 -- 172.21.15.19:0/2382365375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8d04d0 msgr2=0x56185dc9c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2382365375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8d04d0 0x56185dc9c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb2c827640 1 -- 172.21.15.19:0/2382365375 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185dbbe7c0 con 0x56185d8c82f0 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb25ffb640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb140b27e0 0x7fdb140b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb27fff640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb140b27e0 0x7fdb140b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2382365375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc95a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009bd0 tx=0x7fdb20011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9197e65e50cd53b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb25ffb640 1 -- 172.21.15.19:0/194421879 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdb100e2610 con 0x7fdb281098e0 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.431+0000 7fdb1effd640 1 -- 172.21.15.19:0/2382365375 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2001e070 con 0x56185d8c82f0 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb1effd640 1 -- 172.21.15.19:0/2382365375 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb20009d80 con 0x56185d8c82f0 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb1effd640 1 -- 172.21.15.19:0/2382365375 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20019450 con 0x56185d8c82f0 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb35a15740 1 -- 172.21.15.19:0/2382365375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 msgr2=0x56185dc95a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2382365375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc95a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009bd0 tx=0x7fdb20011810 comp rx=0 tx=0).stop 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb27fff640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb140b27e0 0x7fdb140b4cf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56185dc966d0 tx=0x7fdb1802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb35a15740 1 -- 172.21.15.19:0/2382365375 shutdown_connections 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2382365375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8d04d0 0x56185dc9c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2382365375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc95a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2382365375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8f11b0 0x56185dc95500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.432+0000 7fdb35a15740 1 -- 172.21.15.19:0/2382365375 >> 172.21.15.19:0/2382365375 conn(0x56185d8e5300 msgr2=0x56185dc90e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 -- 172.21.15.19:0/2382365375 shutdown_connections 2024-05-09T01:34:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 -- 172.21.15.19:0/2382365375 wait complete. 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 -- start start 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 Processor -- start 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8d04d0 0x56185dc9f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8f11b0 0x56185dca66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56185dc9d3d0 con 0x56185d8f11b0 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56185db809c0 con 0x56185d8d04d0 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.433+0000 7fdb35a15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56185dc9d550 con 0x56185d8c82f0 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb35a15740 10 client.0 _ll_register_callbacks cb 0x56185dbad020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2d028640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8f11b0 0x56185dca66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2d028640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8f11b0 0x56185dca66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54928/0 (socket says 172.21.15.19:54928) 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2d028640 1 -- 172.21.15.19:0/2688554002 learned_addr learned my addr 172.21.15.19:0/2688554002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb27fff640 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8d04d0 0x56185dc9f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2d028640 1 -- 172.21.15.19:0/2688554002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 msgr2=0x56185dc9f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2d028640 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc9f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2d028640 1 -- 172.21.15.19:0/2688554002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8d04d0 msgr2=0x56185dc9f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2d028640 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8d04d0 0x56185dc9f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2d028640 1 -- 172.21.15.19:0/2688554002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56185dba71f0 con 0x56185d8f11b0 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb2d028640 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8f11b0 0x56185dca66d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdb100a9910 tx=0x7fdb100055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f7a11dc90a027a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb07fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb07fff640 1 -- 172.21.15.19:0/2688554002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb100a9720 con 0x56185d8f11b0 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb07fff640 1 -- 172.21.15.19:0/2688554002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdb100b25e0 con 0x56185d8f11b0 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.434+0000 7fdb35a15740 1 -- 172.21.15.19:0/2688554002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb20009d80 con 0x56185d8f11b0 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb07fff640 1 -- 172.21.15.19:0/2688554002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdb100aa310 con 0x56185d8f11b0 2024-05-09T01:34:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb35a15740 10 client.17730 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb35a15740 1 -- 172.21.15.19:0/2688554002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56185d95c790 con 0x56185d8f11b0 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb07fff640 1 -- 172.21.15.19:0/2688554002 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb20009d80 con 0x56185d8f11b0 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb07fff640 1 -- 172.21.15.19:0/2688554002 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdb100a5580 con 0x56185d8f11b0 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb35a15740 10 client.17730 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb35a15740 10 client.17730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb35a15740 20 client.17730 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56185d8afd80 0x56185dcacb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb35a15740 4 client.17730 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb35a15740 1 -- 172.21.15.19:0/2688554002 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56185dc073d0 con 0x56185d8afd80 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.435+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56185d8afd80 0x56185dcacb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.436+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56185d8afd80 0x56185dcacb90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=169cf9a1dd1e67ad server_cookie=9ea54ac3710c1549 in_seq=0 out_seq=0 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.436+0000 7fdb07fff640 10 client.17730 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb07fff640 1 -- 172.21.15.19:0/2688554002 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56185dc073d0 con 0x56185d8afd80 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb07fff640 10 client.17730 handle_command_reply: tid=0 2024-05-09T01:34:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 client.17730 shutdown 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 -- 172.21.15.19:0/2688554002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8f11b0 msgr2=0x56185dca66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8f11b0 0x56185dca66d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdb100a9910 tx=0x7fdb100055f0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 -- 172.21.15.19:0/2688554002 shutdown_connections 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56185d8afd80 0x56185dcacb90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8f11b0 0x56185dca66d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8d04d0 0x56185dc9f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2688554002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8c82f0 0x56185dc9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb07fff640 0 client.17730 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 -- 172.21.15.19:0/2688554002 >> 172.21.15.19:0/2688554002 conn(0x56185d8e5300 msgr2=0x56185dc8f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 -- 172.21.15.19:0/2688554002 shutdown_connections 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 1 -- 172.21.15.19:0/2688554002 wait complete. 2024-05-09T01:34:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.437+0000 7fdb35a15740 20 client.17730 trim_cache size 0 max 16384 2024-05-09T01:34:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.452+0000 7fdb35a15740 1 Processor -- start 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.452+0000 7fdb35a15740 1 -- start start 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 0x56185dc99660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 0x56185dc99ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb35a15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281ab410 con 0x56185d8c82f0 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb35a15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281acf50 con 0x56185d8afd80 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb35a15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281abae0 con 0x56185d8d04d0 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54930/0 (socket says 172.21.15.19:54930) 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb27fff640 1 -- 172.21.15.19:0/3453792510 learned_addr learned my addr 172.21.15.19:0/3453792510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb2c827640 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 0x56185dc99660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb2d028640 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 0x56185dc99ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb27fff640 1 -- 172.21.15.19:0/3453792510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 msgr2=0x56185dc99ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb27fff640 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 0x56185dc99ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb27fff640 1 -- 172.21.15.19:0/3453792510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 msgr2=0x56185dc99660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb27fff640 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 0x56185dc99660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb27fff640 1 -- 172.21.15.19:0/3453792510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185d9d35f0 con 0x56185d8c82f0 2024-05-09T01:34:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb2d028640 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 0x56185dc99ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.453+0000 7fdb27fff640 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca65f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ca0 tx=0x7fdb18042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54effce55d6fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.454+0000 7fdb1e7fc640 1 -- 172.21.15.19:0/3453792510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180316c0 con 0x56185d8c82f0 2024-05-09T01:34:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.454+0000 7fdb1e7fc640 1 -- 172.21.15.19:0/3453792510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb180044f0 con 0x56185d8c82f0 2024-05-09T01:34:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.454+0000 7fdb1e7fc640 1 -- 172.21.15.19:0/3453792510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1802f9c0 con 0x56185d8c82f0 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.454+0000 7fdb35a15740 1 -- 172.21.15.19:0/3453792510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 msgr2=0x56185dca65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.454+0000 7fdb35a15740 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca65f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ca0 tx=0x7fdb18042000 comp rx=0 tx=0).stop 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 -- 172.21.15.19:0/3453792510 shutdown_connections 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 0x56185dc99ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca65f0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 --2- 172.21.15.19:0/3453792510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 0x56185dc99660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 -- 172.21.15.19:0/3453792510 >> 172.21.15.19:0/3453792510 conn(0x56185d8e5300 msgr2=0x56185dc9d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 -- 172.21.15.19:0/3453792510 shutdown_connections 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 -- 172.21.15.19:0/3453792510 wait complete. 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 -- start start 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 Processor -- start 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 0x56185dca3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 0x56185dd12f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281ac880 con 0x56185d8c82f0 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281ac1b0 con 0x56185d8afd80 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.455+0000 7fdb35a15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281a93f0 con 0x56185d8d04d0 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb35a15740 10 client.0 _ll_register_callbacks cb 0x7fdb100a5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54938/0 (socket says 172.21.15.19:54938) 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb27fff640 1 -- 172.21.15.19:0/2724082895 learned_addr learned my addr 172.21.15.19:0/2724082895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 0x56185dca3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb2d028640 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 0x56185dd12f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb27fff640 1 -- 172.21.15.19:0/2724082895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 msgr2=0x56185dd12f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb27fff640 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 0x56185dd12f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb27fff640 1 -- 172.21.15.19:0/2724082895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 msgr2=0x56185dca3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb27fff640 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 0x56185dca3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb27fff640 1 -- 172.21.15.19:0/2724082895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56185dbbded0 con 0x56185d8c82f0 2024-05-09T01:34:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb27fff640 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca39a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdb18006010 tx=0x7fdb18031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed1e9fe936ef83b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb07fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb07fff640 1 -- 172.21.15.19:0/2724082895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1805b020 con 0x56185d8c82f0 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb07fff640 1 -- 172.21.15.19:0/2724082895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdb18046040 con 0x56185d8c82f0 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.456+0000 7fdb35a15740 1 -- 172.21.15.19:0/2724082895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb180044f0 con 0x56185d8c82f0 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb07fff640 1 -- 172.21.15.19:0/2724082895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdb18066050 con 0x56185d8c82f0 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb35a15740 10 client.17736 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb35a15740 1 -- 172.21.15.19:0/2724082895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56185dd143a0 con 0x56185d8c82f0 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb07fff640 1 -- 172.21.15.19:0/2724082895 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb180044f0 con 0x56185d8c82f0 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb07fff640 1 -- 172.21.15.19:0/2724082895 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdb18049050 con 0x56185d8c82f0 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb35a15740 10 client.17736 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb35a15740 10 client.17736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb35a15740 20 client.17736 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56185d8f11b0 0x56185dd16df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb35a15740 4 client.17736 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:34:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb35a15740 1 -- 172.21.15.19:0/2724082895 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56185dc01240 con 0x56185d8f11b0 2024-05-09T01:34:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.457+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56185d8f11b0 0x56185dd16df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.458+0000 7fdb07fff640 10 client.17736 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.458+0000 7fdb2c827640 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56185d8f11b0 0x56185dd16df0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8861915359caf4e server_cookie=547ab54e3640f172 in_seq=0 out_seq=0 2024-05-09T01:34:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb07fff640 1 -- 172.21.15.19:0/2724082895 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56185dc01240 con 0x56185d8f11b0 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb07fff640 10 client.17736 handle_command_reply: tid=0 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb35a15740 1 client.17736 shutdown 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb35a15740 1 -- 172.21.15.19:0/2724082895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 msgr2=0x56185dca39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca39a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdb18006010 tx=0x7fdb18031830 comp rx=0 tx=0).stop 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb35a15740 1 -- 172.21.15.19:0/2724082895 shutdown_connections 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56185d8f11b0 0x56185dd16df0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56185d8d04d0 0x56185dd12f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56185d8c82f0 0x56185dca39a0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb07fff640 0 client.17736 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb35a15740 1 --2- 172.21.15.19:0/2724082895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56185d8afd80 0x56185dca3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.473+0000 7fdb35a15740 1 -- 172.21.15.19:0/2724082895 >> 172.21.15.19:0/2724082895 conn(0x56185d8e5300 msgr2=0x56185dc9f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb35a15740 1 -- 172.21.15.19:0/2724082895 shutdown_connections 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb35a15740 1 -- 172.21.15.19:0/2724082895 wait complete. 2024-05-09T01:34:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb35a15740 20 client.17736 trim_cache size 0 max 16384 2024-05-09T01:34:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:34:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:34:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb140b27e0 msgr2=0x7fdb140b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb140b27e0 0x7fdb140b4cf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56185dc966d0 tx=0x7fdb1802a8c0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb281098e0 msgr2=0x7fdb281a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb281098e0 0x7fdb281a1850 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000e970 tx=0x7fdb1000ed70 comp rx=0 tx=0).stop 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/194421879 shutdown_connections 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb140b27e0 0x7fdb140b4cf0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb281098e0 0x7fdb281a1850 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb28106d70 0x7fdb2819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 --2- 172.21.15.19:0/194421879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28104380 0x7fdb2819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/194421879 >> 172.21.15.19:0/194421879 conn(0x7fdb280fdd40 msgr2=0x7fdb28106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.474+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/194421879 shutdown_connections 2024-05-09T01:34:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:02.475+0000 7fdb2ea8b640 1 -- 172.21.15.19:0/194421879 wait complete. 2024-05-09T01:34:02.622 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:02.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:02 smithi019 ceph-mon[30158]: pgmap v633: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:02 smithi059 ceph-mon[34351]: pgmap v633: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:02.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:02 smithi196 ceph-mon[35041]: pgmap v633: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:04.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.341+0000 7ff783428700 1 -- 172.21.15.19:0/4286125521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 msgr2=0x7ff77c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:04.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.341+0000 7ff783428700 1 --2- 172.21.15.19:0/4286125521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 0x7ff77c1038f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff7780099c0 tx=0x7ff778009cd0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.341+0000 7ff783428700 1 -- 172.21.15.19:0/4286125521 shutdown_connections 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.341+0000 7ff783428700 1 --2- 172.21.15.19:0/4286125521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff77c108760 0x7ff77c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.341+0000 7ff783428700 1 --2- 172.21.15.19:0/4286125521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 0x7ff77c1038f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.341+0000 7ff783428700 1 --2- 172.21.15.19:0/4286125521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77c107e10 0x7ff77c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.341+0000 7ff783428700 1 -- 172.21.15.19:0/4286125521 >> 172.21.15.19:0/4286125521 conn(0x7ff77c0fef30 msgr2=0x7ff77c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.341+0000 7ff783428700 1 -- 172.21.15.19:0/4286125521 shutdown_connections 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.341+0000 7ff783428700 1 -- 172.21.15.19:0/4286125521 wait complete. 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff783428700 1 Processor -- start 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff783428700 1 -- start start 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff783428700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 0x7ff77c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff783428700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff77c107e10 0x7ff77c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff783428700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77c108760 0x7ff77c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff783428700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c113390 con 0x7ff77c1034e0 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff783428700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c113210 con 0x7ff77c108760 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff783428700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c113510 con 0x7ff77c107e10 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff7811c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 0x7ff77c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff7811c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 0x7ff77c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54946/0 (socket says 172.21.15.19:54946) 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.342+0000 7ff7811c4700 1 -- 172.21.15.19:0/373392654 learned_addr learned my addr 172.21.15.19:0/373392654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7819c5700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77c108760 0x7ff77c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7809c3700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff77c107e10 0x7ff77c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7811c4700 1 -- 172.21.15.19:0/373392654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff77c107e10 msgr2=0x7ff77c19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7811c4700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff77c107e10 0x7ff77c19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7811c4700 1 -- 172.21.15.19:0/373392654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77c108760 msgr2=0x7ff77c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7811c4700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77c108760 0x7ff77c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7811c4700 1 -- 172.21.15.19:0/373392654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c1a7020 con 0x7ff77c1034e0 2024-05-09T01:34:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7819c5700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77c108760 0x7ff77c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:04.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7809c3700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff77c107e10 0x7ff77c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:04.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7811c4700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 0x7ff77c19bcb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff77800b580 tx=0x7ff7780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:04.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7727fc700 1 -- 172.21.15.19:0/373392654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778027070 con 0x7ff77c1034e0 2024-05-09T01:34:04.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7727fc700 1 -- 172.21.15.19:0/373392654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff778004020 con 0x7ff77c1034e0 2024-05-09T01:34:04.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff783428700 1 -- 172.21.15.19:0/373392654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c1a72b0 con 0x7ff77c1034e0 2024-05-09T01:34:04.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff7727fc700 1 -- 172.21.15.19:0/373392654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778022e30 con 0x7ff77c1034e0 2024-05-09T01:34:04.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.343+0000 7ff783428700 1 -- 172.21.15.19:0/373392654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c1a7700 con 0x7ff77c1034e0 2024-05-09T01:34:04.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.346+0000 7ff7727fc700 1 -- 172.21.15.19:0/373392654 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff778022590 con 0x7ff77c1034e0 2024-05-09T01:34:04.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.347+0000 7ff7727fc700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7680b2880 0x7ff7680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.347+0000 7ff7809c3700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7680b2880 0x7ff7680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.347+0000 7ff7727fc700 1 -- 172.21.15.19:0/373392654 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff7780e3060 con 0x7ff77c1034e0 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff7809c3700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7680b2880 0x7ff7680b4d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff76c005d90 tx=0x7ff76c005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff791e3eb80 1 Processor -- start 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff791e3eb80 1 -- start start 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e02dd00 0x561d2e136240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2dfb34a0 0x561d2e136780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e130400 0x561d2e130810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d2e0530a0 con 0x561d2dfb34a0 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d2dff89a0 con 0x561d2e02dd00 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d2e04f060 con 0x561d2e130400 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff7809c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2dfb34a0 0x561d2e136780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff7811c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e02dd00 0x561d2e136240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff7811c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e02dd00 0x561d2e136240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:51784/0 (socket says 172.21.15.19:51784) 2024-05-09T01:34:04.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff7809c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2dfb34a0 0x561d2e136780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54962/0 (socket says 172.21.15.19:54962) 2024-05-09T01:34:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.348+0000 7ff7811c4700 1 -- 172.21.15.19:0/1980070077 learned_addr learned my addr 172.21.15.19:0/1980070077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7819c5700 1 --2- 172.21.15.19:0/1980070077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e130400 0x561d2e130810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7809c3700 1 -- 172.21.15.19:0/1980070077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e130400 msgr2=0x561d2e130810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7809c3700 1 --2- 172.21.15.19:0/1980070077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e130400 0x561d2e130810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7809c3700 1 -- 172.21.15.19:0/1980070077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e02dd00 msgr2=0x561d2e136240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7809c3700 1 --2- 172.21.15.19:0/1980070077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e02dd00 0x561d2e136240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7809c3700 1 -- 172.21.15.19:0/1980070077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2dfe9500 con 0x561d2dfb34a0 2024-05-09T01:34:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7811c4700 1 --2- 172.21.15.19:0/1980070077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e02dd00 0x561d2e136240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:04.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7809c3700 1 --2- 172.21.15.19:0/1980070077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2dfb34a0 0x561d2e136780 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff77c076ab0 tx=0x7ff76c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5de456f2fd3902 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:04.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7677fe700 1 -- 172.21.15.19:0/1980070077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c011b00 con 0x561d2dfb34a0 2024-05-09T01:34:04.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7677fe700 1 -- 172.21.15.19:0/1980070077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff76c023070 con 0x561d2dfb34a0 2024-05-09T01:34:04.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.349+0000 7ff7677fe700 1 -- 172.21.15.19:0/1980070077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c010610 con 0x561d2dfb34a0 2024-05-09T01:34:04.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.350+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1980070077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2dfb34a0 msgr2=0x561d2e136780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:04.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.350+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/1980070077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2dfb34a0 0x561d2e136780 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff77c076ab0 tx=0x7ff76c009f90 comp rx=0 tx=0).stop 2024-05-09T01:34:04.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.350+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1980070077 shutdown_connections 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.350+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/1980070077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e130400 0x561d2e130810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.350+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/1980070077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e02dd00 0x561d2e136240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.350+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/1980070077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2dfb34a0 0x561d2e136780 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.350+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1980070077 >> 172.21.15.19:0/1980070077 conn(0x561d2dfb6f70 msgr2=0x561d2e12b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.350+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1980070077 shutdown_connections 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.350+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1980070077 wait complete. 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.351+0000 7ff791e3eb80 1 -- start start 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.351+0000 7ff791e3eb80 1 Processor -- start 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.351+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 0x561d2e139fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.351+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 0x561d2e13a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.351+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e13aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.351+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d2e131160 con 0x561d2e130400 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.351+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d2dfedb20 con 0x561d2dfb34a0 2024-05-09T01:34:04.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.351+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d2e13d020 con 0x561d2e02dd00 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff791e3eb80 10 client.0 _ll_register_callbacks cb 0x561d2dec46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7819c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e13aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7819c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e13aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54964/0 (socket says 172.21.15.19:54964) 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7819c5700 1 -- 172.21.15.19:0/357017913 learned_addr learned my addr 172.21.15.19:0/357017913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7811c4700 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 0x561d2e139fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7809c3700 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 0x561d2e13a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7819c5700 1 -- 172.21.15.19:0/357017913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 msgr2=0x561d2e13a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7819c5700 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 0x561d2e13a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7819c5700 1 -- 172.21.15.19:0/357017913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 msgr2=0x561d2e139fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7819c5700 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 0x561d2e139fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7819c5700 1 -- 172.21.15.19:0/357017913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2dece490 con 0x561d2e130400 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff7819c5700 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e13aa40 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff774008f50 tx=0x7ff774010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0026df2bf30d43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff764ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:04.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff764ff9700 1 -- 172.21.15.19:0/357017913 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774010dd0 con 0x561d2e130400 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff764ff9700 1 -- 172.21.15.19:0/357017913 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff774016470 con 0x561d2e130400 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.352+0000 7ff791e3eb80 1 -- 172.21.15.19:0/357017913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff76c023070 con 0x561d2e130400 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff764ff9700 1 -- 172.21.15.19:0/357017913 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff7740166b0 con 0x561d2e130400 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff791e3eb80 10 client.17748 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff791e3eb80 1 -- 172.21.15.19:0/357017913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d2e147a30 con 0x561d2e130400 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff764ff9700 1 -- 172.21.15.19:0/357017913 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff76c023070 con 0x561d2e130400 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff764ff9700 1 -- 172.21.15.19:0/357017913 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff77401f780 con 0x561d2e130400 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff791e3eb80 10 client.17748 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff791e3eb80 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff791e3eb80 20 client.17748 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d2e14a0c0 0x561d2e14c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff791e3eb80 4 client.17748 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff791e3eb80 1 -- 172.21.15.19:0/357017913 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d2e060a80 con 0x561d2e14a0c0 2024-05-09T01:34:04.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.353+0000 7ff7811c4700 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d2e14a0c0 0x561d2e14c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.354+0000 7ff764ff9700 10 client.17748 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.354+0000 7ff7811c4700 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d2e14a0c0 0x561d2e14c4e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=716a32892090732e server_cookie=e1571ec3b7f0a84a in_seq=0 out_seq=0 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff764ff9700 1 -- 172.21.15.19:0/357017913 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d2e060a80 con 0x561d2e14a0c0 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff764ff9700 10 client.17748 handle_command_reply: tid=0 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 client.17748 shutdown 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 -- 172.21.15.19:0/357017913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 msgr2=0x561d2e13aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e13aa40 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff774008f50 tx=0x7ff774010430 comp rx=0 tx=0).stop 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 -- 172.21.15.19:0/357017913 shutdown_connections 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d2e14a0c0 0x561d2e14c4e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e13aa40 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 0x561d2e139fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff764ff9700 0 client.17748 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:04.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/357017913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 0x561d2e13a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 -- 172.21.15.19:0/357017913 >> 172.21.15.19:0/357017913 conn(0x561d2dfb6f70 msgr2=0x561d2e124ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:04.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 -- 172.21.15.19:0/357017913 shutdown_connections 2024-05-09T01:34:04.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 1 -- 172.21.15.19:0/357017913 wait complete. 2024-05-09T01:34:04.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.355+0000 7ff791e3eb80 20 client.17748 trim_cache size 0 max 16384 2024-05-09T01:34:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.370+0000 7ff791e3eb80 1 Processor -- start 2024-05-09T01:34:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.370+0000 7ff791e3eb80 1 -- start start 2024-05-09T01:34:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 0x561d2e125700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 0x561d2e145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e12f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c1aece0 con 0x561d2e130400 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c1b0820 con 0x561d2dfb34a0 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c1af3b0 con 0x561d2e02dd00 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7819c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e12f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7819c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e12f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54968/0 (socket says 172.21.15.19:54968) 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7819c5700 1 -- 172.21.15.19:0/1391851612 learned_addr learned my addr 172.21.15.19:0/1391851612 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7809c3700 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 0x561d2e145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7811c4700 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 0x561d2e125700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7819c5700 1 -- 172.21.15.19:0/1391851612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 msgr2=0x561d2e145dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7819c5700 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 0x561d2e145dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7819c5700 1 -- 172.21.15.19:0/1391851612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 msgr2=0x561d2e125700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7819c5700 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 0x561d2e125700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7819c5700 1 -- 172.21.15.19:0/1391851612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2e1279d0 con 0x561d2e130400 2024-05-09T01:34:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.371+0000 7ff7811c4700 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 0x561d2e125700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.372+0000 7ff7819c5700 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e12f3f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff774008f20 tx=0x7ff774015450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c3c80957f81e15 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.372+0000 7ff766ffd700 1 -- 172.21.15.19:0/1391851612 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77402f070 con 0x561d2e130400 2024-05-09T01:34:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.372+0000 7ff766ffd700 1 -- 172.21.15.19:0/1391851612 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff77401b070 con 0x561d2e130400 2024-05-09T01:34:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.372+0000 7ff766ffd700 1 -- 172.21.15.19:0/1391851612 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77401f6f0 con 0x561d2e130400 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1391851612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 msgr2=0x561d2e12f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e12f3f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff774008f20 tx=0x7ff774015450 comp rx=0 tx=0).stop 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1391851612 shutdown_connections 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e130400 0x561d2e12f3f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2dfb34a0 0x561d2e125700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/1391851612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2e02dd00 0x561d2e145dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1391851612 >> 172.21.15.19:0/1391851612 conn(0x561d2dfb6f70 msgr2=0x561d2e124ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1391851612 shutdown_connections 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 -- 172.21.15.19:0/1391851612 wait complete. 2024-05-09T01:34:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 -- start start 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.373+0000 7ff791e3eb80 1 Processor -- start 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2dfb34a0 0x561d2e142150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e02dd00 0x561d2e142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff791e3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e130400 0x561d2e13f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c1b0150 con 0x561d2e02dd00 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c1afa80 con 0x561d2e130400 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff791e3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c1acc80 con 0x561d2dfb34a0 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff791e3eb80 10 client.0 _ll_register_callbacks cb 0x561d2dec46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7809c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e02dd00 0x561d2e142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7811c4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2dfb34a0 0x561d2e142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7809c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e02dd00 0x561d2e142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54980/0 (socket says 172.21.15.19:54980) 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7809c3700 1 -- 172.21.15.19:0/756933941 learned_addr learned my addr 172.21.15.19:0/756933941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7819c5700 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e130400 0x561d2e13f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7809c3700 1 -- 172.21.15.19:0/756933941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2dfb34a0 msgr2=0x561d2e142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7809c3700 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2dfb34a0 0x561d2e142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7809c3700 1 -- 172.21.15.19:0/756933941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e130400 msgr2=0x561d2e13f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7809c3700 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e130400 0x561d2e13f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.374+0000 7ff7809c3700 1 -- 172.21.15.19:0/756933941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2e136af0 con 0x561d2e02dd00 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.375+0000 7ff7809c3700 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e02dd00 0x561d2e142690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00ecc0 tx=0x7ff76c011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8617d0d0f325f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.375+0000 7ff764ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.375+0000 7ff764ff9700 1 -- 172.21.15.19:0/756933941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c004a80 con 0x561d2e02dd00 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.375+0000 7ff764ff9700 1 -- 172.21.15.19:0/756933941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff76c004c20 con 0x561d2e02dd00 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.375+0000 7ff791e3eb80 1 -- 172.21.15.19:0/756933941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff77401b070 con 0x561d2e02dd00 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.375+0000 7ff764ff9700 1 -- 172.21.15.19:0/756933941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff76c004e20 con 0x561d2e02dd00 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.375+0000 7ff791e3eb80 10 client.17754 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.375+0000 7ff791e3eb80 1 -- 172.21.15.19:0/756933941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d2e1b9130 con 0x561d2e02dd00 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.375+0000 7ff764ff9700 1 -- 172.21.15.19:0/756933941 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff77401b070 con 0x561d2e02dd00 2024-05-09T01:34:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff764ff9700 1 -- 172.21.15.19:0/756933941 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff76c00f040 con 0x561d2e02dd00 2024-05-09T01:34:04.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff791e3eb80 10 client.17754 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:04.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff791e3eb80 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:04.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff791e3eb80 20 client.17754 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:04.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d2e1b9630 0x561d2e1bba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:04.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff791e3eb80 4 client.17754 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:34:04.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff791e3eb80 1 -- 172.21.15.19:0/756933941 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d2e05ab20 con 0x561d2e1b9630 2024-05-09T01:34:04.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff7811c4700 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d2e1b9630 0x561d2e1bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:04.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff7811c4700 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d2e1b9630 0x561d2e1bba50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3ecd8b3155cf214 server_cookie=177bd651287801cd in_seq=0 out_seq=0 2024-05-09T01:34:04.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.376+0000 7ff764ff9700 10 client.17754 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:04.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff764ff9700 1 -- 172.21.15.19:0/756933941 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d2e05ab20 con 0x561d2e1b9630 2024-05-09T01:34:04.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff764ff9700 10 client.17754 handle_command_reply: tid=0 2024-05-09T01:34:04.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 client.17754 shutdown 2024-05-09T01:34:04.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 -- 172.21.15.19:0/756933941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e02dd00 msgr2=0x561d2e142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:04.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e02dd00 0x561d2e142690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00ecc0 tx=0x7ff76c011f20 comp rx=0 tx=0).stop 2024-05-09T01:34:04.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 -- 172.21.15.19:0/756933941 shutdown_connections 2024-05-09T01:34:04.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d2e1b9630 0x561d2e1bba50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d2e130400 0x561d2e13f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d2dfb34a0 0x561d2e142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 --2- 172.21.15.19:0/756933941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d2e02dd00 0x561d2e142690 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 -- 172.21.15.19:0/756933941 >> 172.21.15.19:0/756933941 conn(0x561d2dfb6f70 msgr2=0x561d2e139f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:04.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff764ff9700 0 client.17754 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:04.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 -- 172.21.15.19:0/756933941 shutdown_connections 2024-05-09T01:34:04.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 1 -- 172.21.15.19:0/756933941 wait complete. 2024-05-09T01:34:04.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.377+0000 7ff791e3eb80 20 client.17754 trim_cache size 0 max 16384 2024-05-09T01:34:04.378 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:34:04.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 -- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7680b2880 msgr2=0x7ff7680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:04.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7680b2880 0x7ff7680b4d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff76c005d90 tx=0x7ff76c005d00 comp rx=0 tx=0).stop 2024-05-09T01:34:04.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 -- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 msgr2=0x7ff77c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:04.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 0x7ff77c19bcb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff77800b580 tx=0x7ff7780047b0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 -- 172.21.15.19:0/373392654 shutdown_connections 2024-05-09T01:34:04.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77c108760 0x7ff77c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7680b2880 0x7ff7680b4d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff77c1034e0 0x7ff77c19bcb0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 --2- 172.21.15.19:0/373392654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff77c107e10 0x7ff77c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:04.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 -- 172.21.15.19:0/373392654 >> 172.21.15.19:0/373392654 conn(0x7ff77c0fef30 msgr2=0x7ff77c1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:04.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 -- 172.21.15.19:0/373392654 shutdown_connections 2024-05-09T01:34:04.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:04.378+0000 7ff783428700 1 -- 172.21.15.19:0/373392654 wait complete. 2024-05-09T01:34:04.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:04 smithi019 ceph-mon[30158]: pgmap v634: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-05-09T01:34:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:04 smithi059 ceph-mon[34351]: pgmap v634: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-05-09T01:34:04.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:04 smithi196 ceph-mon[35041]: pgmap v634: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-05-09T01:34:06.172 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:34:06.172 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:34:06.437 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:06.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:06 smithi019 ceph-mon[30158]: pgmap v635: 97 pgs: 97 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:34:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:06 smithi059 ceph-mon[34351]: pgmap v635: 97 pgs: 97 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:34:06.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:06 smithi196 ceph-mon[35041]: pgmap v635: 97 pgs: 97 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:34:08.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.154+0000 7f99cdcb6700 1 -- 172.21.15.19:0/2564055370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c81034c0 msgr2=0x7f99c81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:08.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.154+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/2564055370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c81034c0 0x7f99c81038d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f99b80099c0 tx=0x7f99b8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.154+0000 7f99cdcb6700 1 -- 172.21.15.19:0/2564055370 shutdown_connections 2024-05-09T01:34:08.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.154+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/2564055370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c8108740 0x7f99c81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.154+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/2564055370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c81034c0 0x7f99c81038d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.154+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/2564055370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c8107df0 0x7f99c8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.154+0000 7f99cdcb6700 1 -- 172.21.15.19:0/2564055370 >> 172.21.15.19:0/2564055370 conn(0x7f99c80fedf0 msgr2=0x7f99c8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:08.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.154+0000 7f99cdcb6700 1 -- 172.21.15.19:0/2564055370 shutdown_connections 2024-05-09T01:34:08.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.154+0000 7f99cdcb6700 1 -- 172.21.15.19:0/2564055370 wait complete. 2024-05-09T01:34:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99cdcb6700 1 Processor -- start 2024-05-09T01:34:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99cdcb6700 1 -- start start 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99cdcb6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c81034c0 0x7f99c819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99cdcb6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c8107df0 0x7f99c819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99cdcb6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c8108740 0x7f99c819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99cdcb6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81133c0 con 0x7f99c8108740 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99cdcb6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8113240 con 0x7f99c81034c0 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99cdcb6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8113540 con 0x7f99c8107df0 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99c7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c8108740 0x7f99c819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99c77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c81034c0 0x7f99c819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99c7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c8108740 0x7f99c819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54990/0 (socket says 172.21.15.19:54990) 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.155+0000 7f99c7fff700 1 -- 172.21.15.19:0/947304866 learned_addr learned my addr 172.21.15.19:0/947304866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99beffd700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c8107df0 0x7f99c819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99c7fff700 1 -- 172.21.15.19:0/947304866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c8107df0 msgr2=0x7f99c819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99c7fff700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c8107df0 0x7f99c819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99c7fff700 1 -- 172.21.15.19:0/947304866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c81034c0 msgr2=0x7f99c819ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99c7fff700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c81034c0 0x7f99c819ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99c7fff700 1 -- 172.21.15.19:0/947304866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c819fd50 con 0x7f99c8108740 2024-05-09T01:34:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99c7fff700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c8108740 0x7f99c819f6b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f99c000ef30 tx=0x7f99c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99c57fa700 1 -- 172.21.15.19:0/947304866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00092e0 con 0x7f99c8108740 2024-05-09T01:34:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99c57fa700 1 -- 172.21.15.19:0/947304866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99c0004bb0 con 0x7f99c8108740 2024-05-09T01:34:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99cdcb6700 1 -- 172.21.15.19:0/947304866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c819ffe0 con 0x7f99c8108740 2024-05-09T01:34:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99c57fa700 1 -- 172.21.15.19:0/947304866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0004280 con 0x7f99c8108740 2024-05-09T01:34:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.156+0000 7f99cdcb6700 1 -- 172.21.15.19:0/947304866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c81a04d0 con 0x7f99c8108740 2024-05-09T01:34:08.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.159+0000 7f99c57fa700 1 -- 172.21.15.19:0/947304866 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f99c001b030 con 0x7f99c8108740 2024-05-09T01:34:08.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.159+0000 7f99c57fa700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f99b40b2880 0x7f99b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99c57fa700 1 -- 172.21.15.19:0/947304866 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f99c00e1d60 con 0x7f99c8108740 2024-05-09T01:34:08.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99c77fe700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f99b40b2880 0x7f99b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99dc6ccb80 1 Processor -- start 2024-05-09T01:34:08.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99dc6ccb80 1 -- start start 2024-05-09T01:34:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99c77fe700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f99b40b2880 0x7f99b40b4d90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f99b8000c00 tx=0x7f99b801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 0x55824ec62570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec62ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec5cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55824eb7f070 con 0x55824eadf520 2024-05-09T01:34:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55824eb1d870 con 0x55824ec5c720 2024-05-09T01:34:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55824eb7b030 con 0x55824ebc91e0 2024-05-09T01:34:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99beffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec62ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99beffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec62ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54994/0 (socket says 172.21.15.19:54994) 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.160+0000 7f99beffd700 1 -- 172.21.15.19:0/1965556475 learned_addr learned my addr 172.21.15.19:0/1965556475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99c77fe700 1 --2- 172.21.15.19:0/1965556475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 0x55824ec62570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99c7fff700 1 --2- 172.21.15.19:0/1965556475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec5cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99beffd700 1 -- 172.21.15.19:0/1965556475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 msgr2=0x55824ec62570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99beffd700 1 --2- 172.21.15.19:0/1965556475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 0x55824ec62570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99beffd700 1 -- 172.21.15.19:0/1965556475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 msgr2=0x55824ec5cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99beffd700 1 --2- 172.21.15.19:0/1965556475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec5cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99beffd700 1 -- 172.21.15.19:0/1965556475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55824eb153f0 con 0x55824eadf520 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99beffd700 1 --2- 172.21.15.19:0/1965556475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec62ab0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f99b000bce0 tx=0x7f99b000f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a7116c1436099a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99bdffb700 1 -- 172.21.15.19:0/1965556475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0015070 con 0x55824eadf520 2024-05-09T01:34:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99bdffb700 1 -- 172.21.15.19:0/1965556475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b000fbe0 con 0x55824eadf520 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.161+0000 7f99bdffb700 1 -- 172.21.15.19:0/1965556475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0010630 con 0x55824eadf520 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.162+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/1965556475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 msgr2=0x55824ec62ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.162+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/1965556475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec62ab0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f99b000bce0 tx=0x7f99b000f360 comp rx=0 tx=0).stop 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.162+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/1965556475 shutdown_connections 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.162+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/1965556475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec5cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.162+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/1965556475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 0x55824ec62570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.162+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/1965556475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec62ab0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.162+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/1965556475 >> 172.21.15.19:0/1965556475 conn(0x55824eb3f720 msgr2=0x55824ec57480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.162+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/1965556475 shutdown_connections 2024-05-09T01:34:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.162+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/1965556475 wait complete. 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99dc6ccb80 1 -- start start 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99dc6ccb80 1 Processor -- start 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824eadf520 0x55824ec66280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec667c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec66d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55824e9fac20 con 0x55824ebc91e0 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55824eb391d0 con 0x55824ec5c720 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55824ec69250 con 0x55824eadf520 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99dc6ccb80 10 client.0 _ll_register_callbacks cb 0x55824e9f0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99beffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99beffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54998/0 (socket says 172.21.15.19:54998) 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99c7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec66d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99beffd700 1 -- 172.21.15.19:0/586800410 learned_addr learned my addr 172.21.15.19:0/586800410 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.163+0000 7f99c77fe700 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824eadf520 0x55824ec66280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99beffd700 1 -- 172.21.15.19:0/586800410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824eadf520 msgr2=0x55824ec66280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99beffd700 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824eadf520 0x55824ec66280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99beffd700 1 -- 172.21.15.19:0/586800410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 msgr2=0x55824ec66d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99beffd700 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec66d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99beffd700 1 -- 172.21.15.19:0/586800410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55824e9fa510 con 0x55824ebc91e0 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99c7fff700 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec66d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99beffd700 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec667c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f99b000f9e0 tx=0x7f99b000fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6019f5e91058fb2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99a6ffd700 1 -- 172.21.15.19:0/586800410 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0028060 con 0x55824ebc91e0 2024-05-09T01:34:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99a6ffd700 1 -- 172.21.15.19:0/586800410 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f99b0011850 con 0x55824ebc91e0 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/586800410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b000fbe0 con 0x55824ebc91e0 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99a6ffd700 1 -- 172.21.15.19:0/586800410 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99b0015050 con 0x55824ebc91e0 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99dc6ccb80 10 client.17766 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.164+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/586800410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55824ec73c10 con 0x55824ebc91e0 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.165+0000 7f99a6ffd700 1 -- 172.21.15.19:0/586800410 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b000fbe0 con 0x55824ebc91e0 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.165+0000 7f99a6ffd700 1 -- 172.21.15.19:0/586800410 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f99b0004c40 con 0x55824ebc91e0 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.165+0000 7f99dc6ccb80 10 client.17766 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.165+0000 7f99dc6ccb80 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.165+0000 7f99dc6ccb80 20 client.17766 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.165+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55824ec76190 0x55824ec785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.165+0000 7f99dc6ccb80 4 client.17766 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.165+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/586800410 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55824eb8ca50 con 0x55824ec76190 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.165+0000 7f99c77fe700 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55824ec76190 0x55824ec785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.166+0000 7f99c77fe700 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55824ec76190 0x55824ec785b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f46ef423725975 server_cookie=4d4c5c8feb71c783 in_seq=0 out_seq=0 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.166+0000 7f99a6ffd700 10 client.17766 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.166+0000 7f99a6ffd700 1 -- 172.21.15.19:0/586800410 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55824eb8ca50 con 0x55824ec76190 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.166+0000 7f99a6ffd700 10 client.17766 handle_command_reply: tid=0 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 client.17766 shutdown 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/586800410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 msgr2=0x55824ec667c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec667c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f99b000f9e0 tx=0x7f99b000fa10 comp rx=0 tx=0).stop 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/586800410 shutdown_connections 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55824ec76190 0x55824ec785b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99a6ffd700 0 client.17766 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec66d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824eadf520 0x55824ec66280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/586800410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec667c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/586800410 >> 172.21.15.19:0/586800410 conn(0x55824eb3f720 msgr2=0x55824ec57240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:08.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/586800410 shutdown_connections 2024-05-09T01:34:08.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/586800410 wait complete. 2024-05-09T01:34:08.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.167+0000 7f99dc6ccb80 20 client.17766 trim_cache size 0 max 16384 2024-05-09T01:34:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99dc6ccb80 1 Processor -- start 2024-05-09T01:34:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99dc6ccb80 1 -- start start 2024-05-09T01:34:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824eadf520 0x55824ec608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ec5c720 0x55824ec66b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81aed50 con 0x55824ebc91e0 2024-05-09T01:34:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81b0890 con 0x55824eadf520 2024-05-09T01:34:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81af420 con 0x55824ec5c720 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99beffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99beffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55004/0 (socket says 172.21.15.19:55004) 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.188+0000 7f99beffd700 1 -- 172.21.15.19:0/906354391 learned_addr learned my addr 172.21.15.19:0/906354391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99c77fe700 1 --2- 172.21.15.19:0/906354391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824eadf520 0x55824ec608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99beffd700 1 -- 172.21.15.19:0/906354391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ec5c720 msgr2=0x55824ec66b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99beffd700 1 --2- 172.21.15.19:0/906354391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ec5c720 0x55824ec66b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99beffd700 1 -- 172.21.15.19:0/906354391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824eadf520 msgr2=0x55824ec608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99beffd700 1 --2- 172.21.15.19:0/906354391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824eadf520 0x55824ec608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99beffd700 1 -- 172.21.15.19:0/906354391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55824ec68ad0 con 0x55824ebc91e0 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99c77fe700 1 --2- 172.21.15.19:0/906354391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824eadf520 0x55824ec608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99beffd700 1 --2- 172.21.15.19:0/906354391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec723f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005d90 tx=0x7f99b000b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca427e3afdd40e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99bd7fa700 1 -- 172.21.15.19:0/906354391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0028040 con 0x55824ebc91e0 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99bd7fa700 1 -- 172.21.15.19:0/906354391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b000fb40 con 0x55824ebc91e0 2024-05-09T01:34:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.189+0000 7f99bd7fa700 1 -- 172.21.15.19:0/906354391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0004290 con 0x55824ebc91e0 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/906354391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 msgr2=0x55824ec723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/906354391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec723f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005d90 tx=0x7f99b000b3c0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/906354391 shutdown_connections 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/906354391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ec5c720 0x55824ec66b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/906354391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824eadf520 0x55824ec608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/906354391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824ebc91e0 0x55824ec723f0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/906354391 >> 172.21.15.19:0/906354391 conn(0x55824eb3f720 msgr2=0x55824ec5bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/906354391 shutdown_connections 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/906354391 wait complete. 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.190+0000 7f99dc6ccb80 1 -- start start 2024-05-09T01:34:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99dc6ccb80 1 Processor -- start 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 0x55824ec79810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99dc6ccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec79d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81b01c0 con 0x55824eadf520 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81afaf0 con 0x55824ec5c720 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99dc6ccb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81accf0 con 0x55824ebc91e0 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99dc6ccb80 10 client.0 _ll_register_callbacks cb 0x55824e9f0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99c77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99c77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55020/0 (socket says 172.21.15.19:55020) 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99c77fe700 1 -- 172.21.15.19:0/789462585 learned_addr learned my addr 172.21.15.19:0/789462585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99c7fff700 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec79d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99c77fe700 1 -- 172.21.15.19:0/789462585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 msgr2=0x55824ec79810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99beffd700 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 0x55824ec79810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99c77fe700 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 0x55824ec79810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99c77fe700 1 -- 172.21.15.19:0/789462585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 msgr2=0x55824ec79d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99c77fe700 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec79d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.191+0000 7f99c77fe700 1 -- 172.21.15.19:0/789462585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55824ecdf730 con 0x55824eadf520 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99c77fe700 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec792d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f99b80098e0 tx=0x7f99b8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ed9c3a75a47807 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99a6ffd700 1 -- 172.21.15.19:0/789462585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b800b790 con 0x55824eadf520 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99a6ffd700 1 -- 172.21.15.19:0/789462585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f99b800b930 con 0x55824eadf520 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/789462585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b000fb40 con 0x55824eadf520 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99a6ffd700 1 -- 172.21.15.19:0/789462585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99b800bb30 con 0x55824eadf520 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99dc6ccb80 10 client.17772 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/789462585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55824ece0970 con 0x55824eadf520 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99a6ffd700 1 -- 172.21.15.19:0/789462585 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b000fb40 con 0x55824eadf520 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99a6ffd700 1 -- 172.21.15.19:0/789462585 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f99b8024410 con 0x55824eadf520 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99dc6ccb80 10 client.17772 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.192+0000 7f99dc6ccb80 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:08.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.193+0000 7f99dc6ccb80 20 client.17772 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.193+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55824ece10b0 0x55824ece34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.193+0000 7f99dc6ccb80 4 client.17772 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:34:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.193+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/789462585 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55824eb86af0 con 0x55824ece10b0 2024-05-09T01:34:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.193+0000 7f99beffd700 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55824ece10b0 0x55824ece34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.193+0000 7f99beffd700 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55824ece10b0 0x55824ece34d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c41f1c7a9f5d6d server_cookie=bab2c0b8704ceb87 in_seq=0 out_seq=0 2024-05-09T01:34:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.193+0000 7f99a6ffd700 10 client.17772 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99a6ffd700 1 -- 172.21.15.19:0/789462585 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55824eb86af0 con 0x55824ece10b0 2024-05-09T01:34:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99a6ffd700 10 client.17772 handle_command_reply: tid=0 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 client.17772 shutdown 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/789462585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 msgr2=0x55824ec792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec792d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f99b80098e0 tx=0x7f99b8005040 comp rx=0 tx=0).stop 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/789462585 shutdown_connections 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55824ece10b0 0x55824ece34d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55824ec5c720 0x55824ec79d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55824eadf520 0x55824ec792d0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 --2- 172.21.15.19:0/789462585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55824ebc91e0 0x55824ec79810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99a6ffd700 0 client.17772 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/789462585 >> 172.21.15.19:0/789462585 conn(0x55824eb3f720 msgr2=0x55824ec56960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/789462585 shutdown_connections 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 1 -- 172.21.15.19:0/789462585 wait complete. 2024-05-09T01:34:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.195+0000 7f99dc6ccb80 20 client.17772 trim_cache size 0 max 16384 2024-05-09T01:34:08.197 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:34:08.197 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:34:08.197 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "91316d95-9a46-41ae-9d7f-6ea74da41931", 2024-05-09T01:34:08.197 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:34:08.197 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 -- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f99b40b2880 msgr2=0x7f99b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f99b40b2880 0x7f99b40b4d90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f99b8000c00 tx=0x7f99b801a040 comp rx=0 tx=0).stop 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 -- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c8108740 msgr2=0x7f99c819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c8108740 0x7f99c819f6b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f99c000ef30 tx=0x7f99c000c3a0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 -- 172.21.15.19:0/947304866 shutdown_connections 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f99b40b2880 0x7f99b40b4d90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99c8108740 0x7f99c819f6b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c81034c0 0x7f99c819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 --2- 172.21.15.19:0/947304866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c8107df0 0x7f99c819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 -- 172.21.15.19:0/947304866 >> 172.21.15.19:0/947304866 conn(0x7f99c80fedf0 msgr2=0x7f99c8075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 -- 172.21.15.19:0/947304866 shutdown_connections 2024-05-09T01:34:08.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:08.196+0000 7f99cdcb6700 1 -- 172.21.15.19:0/947304866 wait complete. 2024-05-09T01:34:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:08 smithi059 ceph-mon[34351]: pgmap v636: 97 pgs: 97 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:08.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:08 smithi019 ceph-mon[30158]: pgmap v636: 97 pgs: 97 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:08.939 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:34:08.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:08 smithi196 ceph-mon[35041]: pgmap v636: 97 pgs: 97 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:09.193 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:09.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:09 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:34:09.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:09 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:34:09.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:09 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:34:09.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:09 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:34:09.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:09 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:34:09.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:09 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:34:09.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:09 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:34:09.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:09 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:34:09.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:09 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:34:09.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:09 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:34:09.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:09 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:34:09.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:09 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:34:10.725 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:10 smithi019 ceph-mon[30158]: pgmap v637: 97 pgs: 97 active+clean; 1.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:10.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:10 smithi059 ceph-mon[34351]: pgmap v637: 97 pgs: 97 active+clean; 1.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:10.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:10 smithi196 ceph-mon[35041]: pgmap v637: 97 pgs: 97 active+clean; 1.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.953+0000 7f7a2b73f700 1 -- 172.21.15.19:0/492898451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a24108740 msgr2=0x7f7a241100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.953+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/492898451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a24108740 0x7f7a241100a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00b0d0 tx=0x7f7a1c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.953+0000 7f7a2b73f700 1 -- 172.21.15.19:0/492898451 shutdown_connections 2024-05-09T01:34:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.953+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/492898451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a24108740 0x7f7a241100a0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.953+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/492898451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a241034c0 0x7f7a241038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.953+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/492898451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24107df0 0x7f7a24108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.953+0000 7f7a2b73f700 1 -- 172.21.15.19:0/492898451 >> 172.21.15.19:0/492898451 conn(0x7f7a240fedf0 msgr2=0x7f7a24101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.953+0000 7f7a2b73f700 1 -- 172.21.15.19:0/492898451 shutdown_connections 2024-05-09T01:34:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.953+0000 7f7a2b73f700 1 -- 172.21.15.19:0/492898451 wait complete. 2024-05-09T01:34:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a2b73f700 1 Processor -- start 2024-05-09T01:34:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a2b73f700 1 -- start start 2024-05-09T01:34:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a2b73f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a241034c0 0x7f7a2419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a2b73f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24107df0 0x7f7a2419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a2b73f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a24108740 0x7f7a2419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a2b73f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a24113370 con 0x7f7a241034c0 2024-05-09T01:34:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a2b73f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241131f0 con 0x7f7a24108740 2024-05-09T01:34:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a2b73f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241134f0 con 0x7f7a24107df0 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a294db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a241034c0 0x7f7a2419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a294db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a241034c0 0x7f7a2419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58642/0 (socket says 172.21.15.19:58642) 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a294db700 1 -- 172.21.15.19:0/1862735987 learned_addr learned my addr 172.21.15.19:0/1862735987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a28cda700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24107df0 0x7f7a2419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.954+0000 7f7a29cdc700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a24108740 0x7f7a2419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a294db700 1 -- 172.21.15.19:0/1862735987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24107df0 msgr2=0x7f7a2419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a294db700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24107df0 0x7f7a2419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a294db700 1 -- 172.21.15.19:0/1862735987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a24108740 msgr2=0x7f7a2419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a294db700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a24108740 0x7f7a2419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a294db700 1 -- 172.21.15.19:0/1862735987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2419fc60 con 0x7f7a241034c0 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a294db700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a241034c0 0x7f7a2419ad90 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000d950 tx=0x7f7a2000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a1a7fc700 1 -- 172.21.15.19:0/1862735987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20016070 con 0x7f7a241034c0 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a1a7fc700 1 -- 172.21.15.19:0/1862735987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a20009df0 con 0x7f7a241034c0 2024-05-09T01:34:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a2b73f700 1 -- 172.21.15.19:0/1862735987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2419ff50 con 0x7f7a241034c0 2024-05-09T01:34:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a1a7fc700 1 -- 172.21.15.19:0/1862735987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000f3c0 con 0x7f7a241034c0 2024-05-09T01:34:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.955+0000 7f7a2b73f700 1 -- 172.21.15.19:0/1862735987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a241a7870 con 0x7f7a241034c0 2024-05-09T01:34:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.958+0000 7f7a1a7fc700 1 -- 172.21.15.19:0/1862735987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7a20019070 con 0x7f7a241034c0 2024-05-09T01:34:10.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.959+0000 7f7a1a7fc700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a100b2880 0x7f7a100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.959+0000 7f7a28cda700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a100b2880 0x7f7a100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.959+0000 7f7a1a7fc700 1 -- 172.21.15.19:0/1862735987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7a200e1eb0 con 0x7f7a241034c0 2024-05-09T01:34:10.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.959+0000 7f7a28cda700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a100b2880 0x7f7a100b4d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7a241098a0 tx=0x7f7a1400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.959+0000 7f7a3a155b80 1 Processor -- start 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a3a155b80 1 -- start start 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c8cd00 0x560d36d93ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c124a0 0x560d36d94030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36d981e0 0x560d36d985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560d36cb20a0 con 0x560d36c8cd00 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560d36c579a0 con 0x560d36d981e0 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d36cae060 con 0x560d36c124a0 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a294db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c8cd00 0x560d36d93ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a29cdc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36d981e0 0x560d36d985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a294db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c8cd00 0x560d36d93ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58654/0 (socket says 172.21.15.19:58654) 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a294db700 1 -- 172.21.15.19:0/3851248290 learned_addr learned my addr 172.21.15.19:0/3851248290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:10.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a28cda700 1 --2- 172.21.15.19:0/3851248290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c124a0 0x560d36d94030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a294db700 1 -- 172.21.15.19:0/3851248290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c124a0 msgr2=0x560d36d94030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a294db700 1 --2- 172.21.15.19:0/3851248290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c124a0 0x560d36d94030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a294db700 1 -- 172.21.15.19:0/3851248290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36d981e0 msgr2=0x560d36d985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a294db700 1 --2- 172.21.15.19:0/3851248290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36d981e0 0x560d36d985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a294db700 1 -- 172.21.15.19:0/3851248290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d36c48500 con 0x560d36c8cd00 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.960+0000 7f7a29cdc700 1 --2- 172.21.15.19:0/3851248290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36d981e0 0x560d36d985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.961+0000 7f7a294db700 1 --2- 172.21.15.19:0/3851248290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c8cd00 0x560d36d93ad0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7a200a9150 tx=0x7f7a200a9bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55ef3fd0450cbf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.961+0000 7f7a0f7fe700 1 -- 172.21.15.19:0/3851248290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20020070 con 0x560d36c8cd00 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.961+0000 7f7a0f7fe700 1 -- 172.21.15.19:0/3851248290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a200a9450 con 0x560d36c8cd00 2024-05-09T01:34:10.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.961+0000 7f7a0f7fe700 1 -- 172.21.15.19:0/3851248290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a200b1e20 con 0x560d36c8cd00 2024-05-09T01:34:10.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3851248290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c8cd00 msgr2=0x560d36d93ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:10.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3851248290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c8cd00 0x560d36d93ad0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7a200a9150 tx=0x7f7a200a9bd0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3851248290 shutdown_connections 2024-05-09T01:34:10.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3851248290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36d981e0 0x560d36d985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3851248290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c8cd00 0x560d36d93ad0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3851248290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c124a0 0x560d36d94030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3851248290 >> 172.21.15.19:0/3851248290 conn(0x560d36c15f70 msgr2=0x560d36d8a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3851248290 shutdown_connections 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3851248290 wait complete. 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 -- start start 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.962+0000 7f7a3a155b80 1 Processor -- start 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d9d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d9d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d9def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560d36d98ea0 con 0x560d36c124a0 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560d36c4cb20 con 0x560d36c8cd00 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d36d99020 con 0x560d36d981e0 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a3a155b80 10 client.0 _ll_register_callbacks cb 0x560d36b236c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a294db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d9d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a294db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d9d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58656/0 (socket says 172.21.15.19:58656) 2024-05-09T01:34:10.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a294db700 1 -- 172.21.15.19:0/3121021797 learned_addr learned my addr 172.21.15.19:0/3121021797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a28cda700 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d9d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a29cdc700 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d9def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a294db700 1 -- 172.21.15.19:0/3121021797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 msgr2=0x560d36d9def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a294db700 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d9def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a294db700 1 -- 172.21.15.19:0/3121021797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 msgr2=0x560d36d9d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a294db700 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d9d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.963+0000 7f7a294db700 1 -- 172.21.15.19:0/3121021797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d36b2d490 con 0x560d36c124a0 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a294db700 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d9d470 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7a200a66c0 tx=0x7f7a200b22b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c1ac291d170f7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a0cff9700 1 -- 172.21.15.19:0/3121021797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20020070 con 0x560d36c124a0 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a0cff9700 1 -- 172.21.15.19:0/3121021797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7a200b1e20 con 0x560d36c124a0 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3121021797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a200a9450 con 0x560d36c124a0 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a0cff9700 1 -- 172.21.15.19:0/3121021797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a2006e070 con 0x560d36c124a0 2024-05-09T01:34:10.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a3a155b80 10 client.17784 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:10.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3121021797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d36daadd0 con 0x560d36c124a0 2024-05-09T01:34:10.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a0cff9700 1 -- 172.21.15.19:0/3121021797 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a200a9450 con 0x560d36c124a0 2024-05-09T01:34:10.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.964+0000 7f7a0cff9700 1 -- 172.21.15.19:0/3121021797 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7a200ca300 con 0x560d36c124a0 2024-05-09T01:34:10.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.965+0000 7f7a3a155b80 10 client.17784 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:10.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.965+0000 7f7a3a155b80 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:10.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.965+0000 7f7a3a155b80 20 client.17784 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:10.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.965+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560d36dad2a0 0x560d36daf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.965+0000 7f7a3a155b80 4 client.17784 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:10.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.965+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3121021797 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d36cbfa80 con 0x560d36dad2a0 2024-05-09T01:34:10.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.965+0000 7f7a28cda700 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560d36dad2a0 0x560d36daf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.965+0000 7f7a0cff9700 10 client.17784 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:10.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.965+0000 7f7a28cda700 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560d36dad2a0 0x560d36daf6c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c073b566d8ae72 server_cookie=1ed57e0229321581 in_seq=0 out_seq=0 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a0cff9700 1 -- 172.21.15.19:0/3121021797 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d36cbfa80 con 0x560d36dad2a0 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a0cff9700 10 client.17784 handle_command_reply: tid=0 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a3a155b80 1 client.17784 shutdown 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3121021797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 msgr2=0x560d36d9d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d9d470 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7a200a66c0 tx=0x7f7a200b22b0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3121021797 shutdown_connections 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560d36dad2a0 0x560d36daf6c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d9def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d9d470 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/3121021797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d9d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3121021797 >> 172.21.15.19:0/3121021797 conn(0x560d36c15f70 msgr2=0x560d36d83ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:10.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.966+0000 7f7a0cff9700 0 client.17784 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:10.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.967+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3121021797 shutdown_connections 2024-05-09T01:34:10.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.967+0000 7f7a3a155b80 1 -- 172.21.15.19:0/3121021797 wait complete. 2024-05-09T01:34:10.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.967+0000 7f7a3a155b80 20 client.17784 trim_cache size 0 max 16384 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.987+0000 7f7a3a155b80 1 Processor -- start 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.987+0000 7f7a3a155b80 1 -- start start 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.987+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c124a0 0x560d36da9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.987+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c8cd00 0x560d36d9dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.987+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36d981e0 0x560d36d84700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.987+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241acb00 con 0x560d36d981e0 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.987+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241ae640 con 0x560d36c124a0 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.987+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241ad1d0 con 0x560d36c8cd00 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a294db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c124a0 0x560d36da9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a29cdc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36d981e0 0x560d36d84700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a294db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c124a0 0x560d36da9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:39538/0 (socket says 172.21.15.19:39538) 2024-05-09T01:34:10.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a294db700 1 -- 172.21.15.19:0/4174393320 learned_addr learned my addr 172.21.15.19:0/4174393320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:10.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a28cda700 1 --2- 172.21.15.19:0/4174393320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c8cd00 0x560d36d9dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a294db700 1 -- 172.21.15.19:0/4174393320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c8cd00 msgr2=0x560d36d9dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:10.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a294db700 1 --2- 172.21.15.19:0/4174393320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c8cd00 0x560d36d9dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a294db700 1 -- 172.21.15.19:0/4174393320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36d981e0 msgr2=0x560d36d84700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:10.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a294db700 1 --2- 172.21.15.19:0/4174393320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36d981e0 0x560d36d84700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a294db700 1 -- 172.21.15.19:0/4174393320 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d36d86510 con 0x560d36c124a0 2024-05-09T01:34:10.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a29cdc700 1 --2- 172.21.15.19:0/4174393320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36d981e0 0x560d36d84700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.988+0000 7f7a294db700 1 --2- 172.21.15.19:0/4174393320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c124a0 0x560d36da9550 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000b2f0 tx=0x7f7a200a3b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f60b845f04374d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.989+0000 7f7a0effd700 1 -- 172.21.15.19:0/4174393320 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20020070 con 0x560d36c124a0 2024-05-09T01:34:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.989+0000 7f7a0effd700 1 -- 172.21.15.19:0/4174393320 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a200a9450 con 0x560d36c124a0 2024-05-09T01:34:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.989+0000 7f7a0effd700 1 -- 172.21.15.19:0/4174393320 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a200a86b0 con 0x560d36c124a0 2024-05-09T01:34:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.989+0000 7f7a3a155b80 1 -- 172.21.15.19:0/4174393320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c124a0 msgr2=0x560d36da9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.989+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/4174393320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c124a0 0x560d36da9550 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000b2f0 tx=0x7f7a200a3b70 comp rx=0 tx=0).stop 2024-05-09T01:34:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.989+0000 7f7a3a155b80 1 -- 172.21.15.19:0/4174393320 shutdown_connections 2024-05-09T01:34:10.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.989+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/4174393320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36d981e0 0x560d36d84700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.989+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/4174393320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c124a0 0x560d36da9550 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.989+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/4174393320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36c8cd00 0x560d36d9dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.990+0000 7f7a3a155b80 1 -- 172.21.15.19:0/4174393320 >> 172.21.15.19:0/4174393320 conn(0x560d36c15f70 msgr2=0x560d36d83ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.990+0000 7f7a3a155b80 1 -- 172.21.15.19:0/4174393320 shutdown_connections 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.990+0000 7f7a3a155b80 1 -- 172.21.15.19:0/4174393320 wait complete. 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.990+0000 7f7a3a155b80 1 -- start start 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.990+0000 7f7a3a155b80 1 Processor -- start 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d95c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d96190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a3a155b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241adf70 con 0x560d36c124a0 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241ad8a0 con 0x560d36c8cd00 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a3a155b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241aaaa0 con 0x560d36d981e0 2024-05-09T01:34:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a3a155b80 10 client.0 _ll_register_callbacks cb 0x560d36b236c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a294db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d95c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a28cda700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d96190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a29cdc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a28cda700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d96190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:39552/0 (socket says 172.21.15.19:39552) 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a294db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d95c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58684/0 (socket says 172.21.15.19:58684) 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a28cda700 1 -- 172.21.15.19:0/2134088740 learned_addr learned my addr 172.21.15.19:0/2134088740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a29cdc700 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:47348/0 (socket says 172.21.15.19:47348) 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a28cda700 1 -- 172.21.15.19:0/2134088740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 msgr2=0x560d36d966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a28cda700 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a28cda700 1 -- 172.21.15.19:0/2134088740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 msgr2=0x560d36d95c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a28cda700 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d95c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a28cda700 1 -- 172.21.15.19:0/2134088740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d36e167e0 con 0x560d36c8cd00 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a294db700 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d95c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:10.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.991+0000 7f7a29cdc700 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a28cda700 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d96190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a14005950 tx=0x7f7a140047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97faf0f657da3231 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a0cff9700 1 -- 172.21.15.19:0/2134088740 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14021bc0 con 0x560d36c8cd00 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a3a155b80 1 -- 172.21.15.19:0/2134088740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a200a9450 con 0x560d36c8cd00 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a0cff9700 1 -- 172.21.15.19:0/2134088740 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7a14021d60 con 0x560d36c8cd00 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a0cff9700 1 -- 172.21.15.19:0/2134088740 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a14004830 con 0x560d36c8cd00 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a3a155b80 10 client.27161 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a3a155b80 1 -- 172.21.15.19:0/2134088740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d36e17970 con 0x560d36c8cd00 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a0cff9700 1 -- 172.21.15.19:0/2134088740 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a200a9450 con 0x560d36c8cd00 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.992+0000 7f7a0cff9700 1 -- 172.21.15.19:0/2134088740 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7a1402a380 con 0x560d36c8cd00 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.993+0000 7f7a3a155b80 10 client.27161 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.993+0000 7f7a3a155b80 10 client.27161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.993+0000 7f7a3a155b80 20 client.27161 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.993+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560d36e180e0 0x560d36e1a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.993+0000 7f7a3a155b80 4 client.27161 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:34:10.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.993+0000 7f7a3a155b80 1 -- 172.21.15.19:0/2134088740 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d36cb9b20 con 0x560d36e180e0 2024-05-09T01:34:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.993+0000 7f7a294db700 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560d36e180e0 0x560d36e1a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.993+0000 7f7a0cff9700 10 client.27161 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.993+0000 7f7a294db700 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560d36e180e0 0x560d36e1a500 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b9c66550393718b server_cookie=df1331e3ce178d37 in_seq=0 out_seq=0 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a0cff9700 1 -- 172.21.15.19:0/2134088740 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560d36cb9b20 con 0x560d36e180e0 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a0cff9700 10 client.27161 handle_command_reply: tid=0 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 client.27161 shutdown 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 -- 172.21.15.19:0/2134088740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 msgr2=0x560d36d96190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d96190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a14005950 tx=0x7f7a140047c0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 -- 172.21.15.19:0/2134088740 shutdown_connections 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560d36e180e0 0x560d36e1a500 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d36d981e0 0x560d36d966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d36c124a0 0x560d36d95c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a0cff9700 0 client.27161 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 --2- 172.21.15.19:0/2134088740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d36c8cd00 0x560d36d96190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 -- 172.21.15.19:0/2134088740 >> 172.21.15.19:0/2134088740 conn(0x560d36c15f70 msgr2=0x560d36d8e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 -- 172.21.15.19:0/2134088740 shutdown_connections 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 1 -- 172.21.15.19:0/2134088740 wait complete. 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.994+0000 7f7a3a155b80 20 client.27161 trim_cache size 0 max 16384 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stdout: "91316d95-9a46-41ae-9d7f-6ea74da41931": { 2024-05-09T01:34:10.996 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T01:34:10.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "91316d95-9a46-41ae-9d7f-6ea74da41931", 2024-05-09T01:34:10.997 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T01:34:10.997 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:34:10.997 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:34:10.997 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:34:10.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 -- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a100b2880 msgr2=0x7f7a100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a100b2880 0x7f7a100b4d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7a241098a0 tx=0x7f7a1400b410 comp rx=0 tx=0).stop 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 -- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a241034c0 msgr2=0x7f7a2419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a241034c0 0x7f7a2419ad90 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000d950 tx=0x7f7a2000dc60 comp rx=0 tx=0).stop 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 -- 172.21.15.19:0/1862735987 shutdown_connections 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a24108740 0x7f7a2419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f7a100b2880 0x7f7a100b4d90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a241034c0 0x7f7a2419ad90 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 --2- 172.21.15.19:0/1862735987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24107df0 0x7f7a2419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 -- 172.21.15.19:0/1862735987 >> 172.21.15.19:0/1862735987 conn(0x7f7a240fedf0 msgr2=0x7f7a24075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 -- 172.21.15.19:0/1862735987 shutdown_connections 2024-05-09T01:34:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:10.995+0000 7f7a2b73f700 1 -- 172.21.15.19:0/1862735987 wait complete. 2024-05-09T01:34:11.772 INFO:tasks.cephfs.filesystem:scrub status for tag:91316d95-9a46-41ae-9d7f-6ea74da41931 - {'path': '/', 'tag': '91316d95-9a46-41ae-9d7f-6ea74da41931', 'options': 'recursive,force'} 2024-05-09T01:34:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:12 smithi059 ceph-mon[34351]: pgmap v638: 97 pgs: 97 active+clean; 1.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:34:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:12 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:34:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:12 smithi019 ceph-mon[30158]: pgmap v638: 97 pgs: 97 active+clean; 1.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:34:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:12 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:34:12.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:12 smithi196 ceph-mon[35041]: pgmap v638: 97 pgs: 97 active+clean; 1.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:34:12.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:12 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:34:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:14 smithi059 ceph-mon[34351]: pgmap v639: 97 pgs: 97 active+clean; 1.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:14 smithi019 ceph-mon[30158]: pgmap v639: 97 pgs: 97 active+clean; 1.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:14.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:14 smithi196 ceph-mon[35041]: pgmap v639: 97 pgs: 97 active+clean; 1.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.589+0000 7f451e5ea640 1 -- 172.21.15.19:0/3342694981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 msgr2=0x7f451810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.589+0000 7f451e5ea640 1 --2- 172.21.15.19:0/3342694981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 0x7f451810bfd0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4510009ca0 tx=0x7f451002a8c0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.589+0000 7f451e5ea640 1 -- 172.21.15.19:0/3342694981 shutdown_connections 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.589+0000 7f451e5ea640 1 --2- 172.21.15.19:0/3342694981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 0x7f451810bfd0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.589+0000 7f451e5ea640 1 --2- 172.21.15.19:0/3342694981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4518104eb0 0x7f45181052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.589+0000 7f451e5ea640 1 --2- 172.21.15.19:0/3342694981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518103c70 0x7f4518104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.589+0000 7f451e5ea640 1 -- 172.21.15.19:0/3342694981 >> 172.21.15.19:0/3342694981 conn(0x7f45180fee60 msgr2=0x7f4518101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.589+0000 7f451e5ea640 1 -- 172.21.15.19:0/3342694981 shutdown_connections 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.589+0000 7f451e5ea640 1 -- 172.21.15.19:0/3342694981 wait complete. 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.590+0000 7f451e5ea640 1 Processor -- start 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.590+0000 7f451e5ea640 1 -- start start 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.590+0000 7f451e5ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518103c70 0x7f4518077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.590+0000 7f451e5ea640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4518104eb0 0x7f4518076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.590+0000 7f451e5ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 0x7f4518076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.590+0000 7f451e5ea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f451810f780 con 0x7f4518109bb0 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.590+0000 7f451e5ea640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f451810f600 con 0x7f4518104eb0 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.590+0000 7f451e5ea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f451810f900 con 0x7f4518103c70 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451cb87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 0x7f4518076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451cb87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 0x7f4518076670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58700/0 (socket says 172.21.15.19:58700) 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451cb87640 1 -- 172.21.15.19:0/4167389098 learned_addr learned my addr 172.21.15.19:0/4167389098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f4517fff640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518103c70 0x7f4518077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f450ffff640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4518104eb0 0x7f4518076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451cb87640 1 -- 172.21.15.19:0/4167389098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518103c70 msgr2=0x7f4518077a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451cb87640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518103c70 0x7f4518077a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451cb87640 1 -- 172.21.15.19:0/4167389098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4518104eb0 msgr2=0x7f4518076130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451cb87640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4518104eb0 0x7f4518076130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451cb87640 1 -- 172.21.15.19:0/4167389098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4518076da0 con 0x7f4518109bb0 2024-05-09T01:34:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f450ffff640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4518104eb0 0x7f4518076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451cb87640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 0x7f4518076670 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f45100097b0 tx=0x7f4510031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f4515ffb640 1 -- 172.21.15.19:0/4167389098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510003bf0 con 0x7f4518109bb0 2024-05-09T01:34:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f4515ffb640 1 -- 172.21.15.19:0/4167389098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4510003d90 con 0x7f4518109bb0 2024-05-09T01:34:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f451e5ea640 1 -- 172.21.15.19:0/4167389098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45181a43b0 con 0x7f4518109bb0 2024-05-09T01:34:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.591+0000 7f4515ffb640 1 -- 172.21.15.19:0/4167389098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451002fc30 con 0x7f4518109bb0 2024-05-09T01:34:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.592+0000 7f451e5ea640 1 -- 172.21.15.19:0/4167389098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45181a4810 con 0x7f4518109bb0 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.594+0000 7f4525578740 1 Processor -- start 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.594+0000 7f4515ffb640 1 -- 172.21.15.19:0/4167389098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4510004040 con 0x7f4518109bb0 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f4525578740 1 -- start start 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f4525578740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b451500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f4525578740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b451a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f4525578740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b458980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f4525578740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e65b3b57c0 con 0x55e65b0ad1b0 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f4525578740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e65b3510e0 con 0x55e65b08c4d0 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f4525578740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e65b3b1780 con 0x55e65b0842f0 2024-05-09T01:34:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f450ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b451500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f450ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b451500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58706/0 (socket says 172.21.15.19:58706) 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f450ffff640 1 -- 172.21.15.19:0/2333088548 learned_addr learned my addr 172.21.15.19:0/2333088548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f4517fff640 1 --2- 172.21.15.19:0/2333088548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b451a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f450ffff640 1 --2- 172.21.15.19:0/2333088548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b458980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f450ffff640 1 -- 172.21.15.19:0/2333088548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 msgr2=0x55e65b451a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f450ffff640 1 --2- 172.21.15.19:0/2333088548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b451a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f450ffff640 1 -- 172.21.15.19:0/2333088548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 msgr2=0x55e65b458980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f450ffff640 1 --2- 172.21.15.19:0/2333088548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b458980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.595+0000 7f450ffff640 1 -- 172.21.15.19:0/2333088548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65b37a7c0 con 0x55e65b0ad1b0 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.596+0000 7f4515ffb640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f45040b27e0 0x7f45040b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.596+0000 7f450ffff640 1 --2- 172.21.15.19:0/2333088548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b451500 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f45000135d0 tx=0x7f4500009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b465db4c482e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.596+0000 7f4517fff640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f45040b27e0 0x7f45040b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.596+0000 7f4515ffb640 1 -- 172.21.15.19:0/4167389098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4510103920 con 0x7f4518109bb0 2024-05-09T01:34:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.596+0000 7f450e7fc640 1 -- 172.21.15.19:0/2333088548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45000114c0 con 0x55e65b0ad1b0 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.596+0000 7f450e7fc640 1 -- 172.21.15.19:0/2333088548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4500011b10 con 0x55e65b0ad1b0 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.596+0000 7f450e7fc640 1 -- 172.21.15.19:0/2333088548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450000fb00 con 0x55e65b0ad1b0 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 -- 172.21.15.19:0/2333088548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 msgr2=0x55e65b451500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 --2- 172.21.15.19:0/2333088548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b451500 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f45000135d0 tx=0x7f4500009790 comp rx=0 tx=0).stop 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4517fff640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f45040b27e0 0x7f45040b4cf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55e65b4528e0 tx=0x7f450800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 -- 172.21.15.19:0/2333088548 shutdown_connections 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 --2- 172.21.15.19:0/2333088548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b458980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 --2- 172.21.15.19:0/2333088548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b451a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 --2- 172.21.15.19:0/2333088548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b451500 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 -- 172.21.15.19:0/2333088548 >> 172.21.15.19:0/2333088548 conn(0x55e65b0a1300 msgr2=0x55e65b44ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 -- 172.21.15.19:0/2333088548 shutdown_connections 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 -- 172.21.15.19:0/2333088548 wait complete. 2024-05-09T01:34:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.597+0000 7f4525578740 1 -- start start 2024-05-09T01:34:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4525578740 1 Processor -- start 2024-05-09T01:34:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4525578740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0842f0 0x55e65b45b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4525578740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b45b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4525578740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0ad1b0 0x55e65b462790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4525578740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e65b4593d0 con 0x55e65b0842f0 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4525578740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e65b33c9c0 con 0x55e65b08c4d0 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4525578740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e65b459550 con 0x55e65b0ad1b0 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4525578740 10 client.0 _ll_register_callbacks cb 0x55e65b369020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4517fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0842f0 0x55e65b45b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4517fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0842f0 0x55e65b45b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58720/0 (socket says 172.21.15.19:58720) 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4517fff640 1 -- 172.21.15.19:0/2563964149 learned_addr learned my addr 172.21.15.19:0/2563964149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f451cb87640 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0ad1b0 0x55e65b462790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f450ffff640 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b45b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4517fff640 1 -- 172.21.15.19:0/2563964149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0ad1b0 msgr2=0x55e65b462790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4517fff640 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0ad1b0 0x55e65b462790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4517fff640 1 -- 172.21.15.19:0/2563964149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 msgr2=0x55e65b45b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4517fff640 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b45b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f4517fff640 1 -- 172.21.15.19:0/2563964149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65b3631f0 con 0x55e65b0842f0 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f450ffff640 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b45b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.598+0000 7f451cb87640 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0ad1b0 0x55e65b462790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f4517fff640 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0842f0 0x55e65b45b2f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4508003d00 tx=0x7f4508004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6841bf93c8338f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f44f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f44f77fe640 1 -- 172.21.15.19:0/2563964149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450802c060 con 0x55e65b0842f0 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f44f77fe640 1 -- 172.21.15.19:0/2563964149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4508017030 con 0x55e65b0842f0 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f4525578740 1 -- 172.21.15.19:0/2563964149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4500011b10 con 0x55e65b0842f0 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f44f77fe640 1 -- 172.21.15.19:0/2563964149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4508027410 con 0x55e65b0842f0 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f4525578740 10 client.17802 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f4525578740 1 -- 172.21.15.19:0/2563964149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e65b118790 con 0x55e65b0842f0 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f44f77fe640 1 -- 172.21.15.19:0/2563964149 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4500011b10 con 0x55e65b0842f0 2024-05-09T01:34:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f44f77fe640 1 -- 172.21.15.19:0/2563964149 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4508011440 con 0x55e65b0842f0 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f4525578740 10 client.17802 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f4525578740 10 client.17802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.599+0000 7f4525578740 20 client.17802 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.600+0000 7f4525578740 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e65b06bd80 0x55e65b468d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.600+0000 7f4525578740 4 client.17802 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.600+0000 7f4525578740 1 -- 172.21.15.19:0/2563964149 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e65b3c33d0 con 0x55e65b06bd80 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.600+0000 7f450ffff640 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e65b06bd80 0x55e65b468d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.600+0000 7f450ffff640 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e65b06bd80 0x55e65b468d30 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59136d3ab0bd140 server_cookie=32eb5a9d27673fe9 in_seq=0 out_seq=0 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.600+0000 7f44f77fe640 10 client.17802 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f44f77fe640 1 -- 172.21.15.19:0/2563964149 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e65b3c33d0 con 0x55e65b06bd80 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f44f77fe640 10 client.17802 handle_command_reply: tid=0 2024-05-09T01:34:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f4525578740 1 client.17802 shutdown 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f4525578740 1 -- 172.21.15.19:0/2563964149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0842f0 msgr2=0x55e65b45b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f4525578740 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0842f0 0x55e65b45b2f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4508003d00 tx=0x7f4508004730 comp rx=0 tx=0).stop 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f4525578740 1 -- 172.21.15.19:0/2563964149 shutdown_connections 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f4525578740 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e65b06bd80 0x55e65b468d30 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f4525578740 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0ad1b0 0x55e65b462790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f4525578740 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b45b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f44f77fe640 0 client.17802 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f4525578740 1 --2- 172.21.15.19:0/2563964149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0842f0 0x55e65b45b2f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.601+0000 7f4525578740 1 -- 172.21.15.19:0/2563964149 >> 172.21.15.19:0/2563964149 conn(0x55e65b0a1300 msgr2=0x55e65b44b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.602+0000 7f4525578740 1 -- 172.21.15.19:0/2563964149 shutdown_connections 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.602+0000 7f4525578740 1 -- 172.21.15.19:0/2563964149 wait complete. 2024-05-09T01:34:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.602+0000 7f4525578740 20 client.17802 trim_cache size 0 max 16384 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f4525578740 1 Processor -- start 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f4525578740 1 -- start start 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f4525578740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b44be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f4525578740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b462720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f4525578740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b462c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f4525578740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45181ad5e0 con 0x55e65b0ad1b0 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f4525578740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45181af120 con 0x55e65b08c4d0 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f4525578740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45181adcb0 con 0x55e65b0842f0 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f451cb87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b462c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f451cb87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b462c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58728/0 (socket says 172.21.15.19:58728) 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.617+0000 7f451cb87640 1 -- 172.21.15.19:0/1315166078 learned_addr learned my addr 172.21.15.19:0/1315166078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f450ffff640 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b462720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f4517fff640 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b44be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f451cb87640 1 -- 172.21.15.19:0/1315166078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 msgr2=0x55e65b44be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f451cb87640 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b44be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f451cb87640 1 -- 172.21.15.19:0/1315166078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 msgr2=0x55e65b462720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f451cb87640 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b462720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f451cb87640 1 -- 172.21.15.19:0/1315166078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65b18f5f0 con 0x55e65b0ad1b0 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f450ffff640 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b462720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f4517fff640 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b44be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f451cb87640 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b462c60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4510101400 tx=0x7f451002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9815cf951fb5818 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f450dffb640 1 -- 172.21.15.19:0/1315166078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100caa30 con 0x55e65b0ad1b0 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f450dffb640 1 -- 172.21.15.19:0/1315166078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4510036030 con 0x55e65b0ad1b0 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.618+0000 7f450dffb640 1 -- 172.21.15.19:0/1315166078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100046a0 con 0x55e65b0ad1b0 2024-05-09T01:34:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 -- 172.21.15.19:0/1315166078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 msgr2=0x55e65b462c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b462c60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4510101400 tx=0x7f451002a8c0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 -- 172.21.15.19:0/1315166078 shutdown_connections 2024-05-09T01:34:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b0ad1b0 0x55e65b462c60 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b08c4d0 0x55e65b462720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 --2- 172.21.15.19:0/1315166078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b44be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 -- 172.21.15.19:0/1315166078 >> 172.21.15.19:0/1315166078 conn(0x55e65b0a1300 msgr2=0x55e65b44b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 -- 172.21.15.19:0/1315166078 shutdown_connections 2024-05-09T01:34:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 -- 172.21.15.19:0/1315166078 wait complete. 2024-05-09T01:34:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.619+0000 7f4525578740 1 -- start start 2024-05-09T01:34:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4525578740 1 Processor -- start 2024-05-09T01:34:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4525578740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b45ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4525578740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b08c4d0 0x55e65b460470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4525578740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b0ad1b0 0x55e65b4613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4525578740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45181aea50 con 0x55e65b08c4d0 2024-05-09T01:34:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4525578740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45181ae380 con 0x55e65b0ad1b0 2024-05-09T01:34:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4525578740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45181ab5c0 con 0x55e65b0842f0 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4525578740 10 client.0 _ll_register_callbacks cb 0x7f450800bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f450ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b08c4d0 0x55e65b460470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f450ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b08c4d0 0x55e65b460470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58738/0 (socket says 172.21.15.19:58738) 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f450ffff640 1 -- 172.21.15.19:0/2833287553 learned_addr learned my addr 172.21.15.19:0/2833287553 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4517fff640 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b45ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f450ffff640 1 -- 172.21.15.19:0/2833287553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 msgr2=0x55e65b45ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f450ffff640 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b45ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f450ffff640 1 -- 172.21.15.19:0/2833287553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b0ad1b0 msgr2=0x55e65b4613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f451cb87640 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b0ad1b0 0x55e65b4613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f450ffff640 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b0ad1b0 0x55e65b4613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f450ffff640 1 -- 172.21.15.19:0/2833287553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65b379ed0 con 0x55e65b08c4d0 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f4517fff640 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b45ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f451cb87640 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b0ad1b0 0x55e65b4613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f450ffff640 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b08c4d0 0x55e65b460470 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4500013f20 tx=0x7f45000134d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f764d954223ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.620+0000 7f44f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f44f77fe640 1 -- 172.21.15.19:0/2833287553 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450002c030 con 0x55e65b08c4d0 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f44f77fe640 1 -- 172.21.15.19:0/2833287553 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4500005040 con 0x55e65b08c4d0 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f4525578740 1 -- 172.21.15.19:0/2833287553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4510036030 con 0x55e65b08c4d0 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f44f77fe640 1 -- 172.21.15.19:0/2833287553 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4500005240 con 0x55e65b08c4d0 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f4525578740 10 client.17808 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f4525578740 1 -- 172.21.15.19:0/2833287553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e65b4629c0 con 0x55e65b08c4d0 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f44f77fe640 1 -- 172.21.15.19:0/2833287553 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4510036030 con 0x55e65b08c4d0 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f44f77fe640 1 -- 172.21.15.19:0/2833287553 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f45000035f0 con 0x55e65b08c4d0 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f4525578740 10 client.17808 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f4525578740 10 client.17808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f4525578740 20 client.17808 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f4525578740 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e65b0411c0 0x55e65b4d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f4525578740 4 client.17808 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:34:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.621+0000 7f4525578740 1 -- 172.21.15.19:0/2833287553 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e65b3bd240 con 0x55e65b0411c0 2024-05-09T01:34:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.622+0000 7f4517fff640 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e65b0411c0 0x55e65b4d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.622+0000 7f4517fff640 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e65b0411c0 0x55e65b4d3160 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73bed9e81ee8aa4 server_cookie=f7cd8d1b50a3acc5 in_seq=0 out_seq=0 2024-05-09T01:34:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.622+0000 7f44f77fe640 10 client.17808 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f44f77fe640 1 -- 172.21.15.19:0/2833287553 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e65b3bd240 con 0x55e65b0411c0 2024-05-09T01:34:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f44f77fe640 10 client.17808 handle_command_reply: tid=0 2024-05-09T01:34:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 client.17808 shutdown 2024-05-09T01:34:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 -- 172.21.15.19:0/2833287553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b08c4d0 msgr2=0x55e65b460470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b08c4d0 0x55e65b460470 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4500013f20 tx=0x7f45000134d0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 -- 172.21.15.19:0/2833287553 shutdown_connections 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e65b0411c0 0x55e65b4d3160 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e65b0ad1b0 0x55e65b4613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e65b08c4d0 0x55e65b460470 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 --2- 172.21.15.19:0/2833287553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e65b0842f0 0x55e65b45ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f44f77fe640 0 client.17808 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 -- 172.21.15.19:0/2833287553 >> 172.21.15.19:0/2833287553 conn(0x55e65b0a1300 msgr2=0x55e65b44c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 -- 172.21.15.19:0/2833287553 shutdown_connections 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 1 -- 172.21.15.19:0/2833287553 wait complete. 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.638+0000 7f4525578740 20 client.17808 trim_cache size 0 max 16384 2024-05-09T01:34:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:34:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:34:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:34:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 -- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f45040b27e0 msgr2=0x7f45040b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f45040b27e0 0x7f45040b4cf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55e65b4528e0 tx=0x7f450800b410 comp rx=0 tx=0).stop 2024-05-09T01:34:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 -- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 msgr2=0x7f4518076670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 0x7f4518076670 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f45100097b0 tx=0x7f4510031840 comp rx=0 tx=0).stop 2024-05-09T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 -- 172.21.15.19:0/4167389098 shutdown_connections 2024-05-09T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f45040b27e0 0x7f45040b4cf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4518109bb0 0x7f4518076670 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4518104eb0 0x7f4518076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 --2- 172.21.15.19:0/4167389098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518103c70 0x7f4518077a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 -- 172.21.15.19:0/4167389098 >> 172.21.15.19:0/4167389098 conn(0x7f45180fee60 msgr2=0x7f4518105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 -- 172.21.15.19:0/4167389098 shutdown_connections 2024-05-09T01:34:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:15.639+0000 7f451e5ea640 1 -- 172.21.15.19:0/4167389098 wait complete. 2024-05-09T01:34:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:16 smithi059 ceph-mon[34351]: pgmap v640: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:34:16.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:16 smithi019 ceph-mon[30158]: pgmap v640: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:34:16.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:16 smithi196 ceph-mon[35041]: pgmap v640: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:34:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:18 smithi059 ceph-mon[34351]: pgmap v641: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:34:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:18 smithi019 ceph-mon[30158]: pgmap v641: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:34:18.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:18 smithi196 ceph-mon[35041]: pgmap v641: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:34:20.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:19 smithi059 ceph-mon[34351]: pgmap v642: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:20.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:19 smithi019 ceph-mon[30158]: pgmap v642: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:20.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:19 smithi196 ceph-mon[35041]: pgmap v642: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:22.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:22 smithi019 ceph-mon[30158]: pgmap v643: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:34:22.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:22 smithi196 ceph-mon[35041]: pgmap v643: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:34:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:22 smithi059 ceph-mon[34351]: pgmap v643: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:34:24.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:24 smithi019 ceph-mon[30158]: pgmap v644: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:34:24.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:24 smithi196 ceph-mon[35041]: pgmap v644: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:34:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:24 smithi059 ceph-mon[34351]: pgmap v644: 97 pgs: 97 active+clean; 1.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:34:26.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:26 smithi019 ceph-mon[30158]: pgmap v645: 97 pgs: 97 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:26.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:26 smithi196 ceph-mon[35041]: pgmap v645: 97 pgs: 97 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:26 smithi059 ceph-mon[34351]: pgmap v645: 97 pgs: 97 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:28.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:28 smithi019 ceph-mon[30158]: pgmap v646: 97 pgs: 97 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:28.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:28 smithi196 ceph-mon[35041]: pgmap v646: 97 pgs: 97 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.753+0000 7fb43fb8c640 1 -- 172.21.15.19:0/409033611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4381021f0 msgr2=0x7fb438102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.753+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/409033611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4381021f0 0x7fb438102fa0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb434009900 tx=0x7fb43402a410 comp rx=0 tx=0).stop 2024-05-09T01:34:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.753+0000 7fb43fb8c640 1 -- 172.21.15.19:0/409033611 shutdown_connections 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.753+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/409033611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb438108270 0x7fb43810a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.753+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/409033611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4381026b0 0x7fb4381035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.753+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/409033611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4381021f0 0x7fb438102fa0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.753+0000 7fb43fb8c640 1 -- 172.21.15.19:0/409033611 >> 172.21.15.19:0/409033611 conn(0x7fb4380fd4f0 msgr2=0x7fb4380ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.753+0000 7fb43fb8c640 1 -- 172.21.15.19:0/409033611 shutdown_connections 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.753+0000 7fb43fb8c640 1 -- 172.21.15.19:0/409033611 wait complete. 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43fb8c640 1 Processor -- start 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43fb8c640 1 -- start start 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43fb8c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4381021f0 0x7fb438199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43fb8c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb438108270 0x7fb43819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43fb8c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43819f440 0x7fb4381a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43fb8c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb43810d600 con 0x7fb438108270 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43fb8c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb43810d480 con 0x7fb4381021f0 2024-05-09T01:34:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43fb8c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb43810d780 con 0x7fb43819f440 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43d127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb438108270 0x7fb43819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43d127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb438108270 0x7fb43819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49706/0 (socket says 172.21.15.19:49706) 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43d127640 1 -- 172.21.15.19:0/3543833957 learned_addr learned my addr 172.21.15.19:0/3543833957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43e129640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43819f440 0x7fb4381a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43d928640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4381021f0 0x7fb438199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43d127640 1 -- 172.21.15.19:0/3543833957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43819f440 msgr2=0x7fb4381a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43d127640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43819f440 0x7fb4381a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43d127640 1 -- 172.21.15.19:0/3543833957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4381021f0 msgr2=0x7fb438199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43d127640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4381021f0 0x7fb438199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.754+0000 7fb43d127640 1 -- 172.21.15.19:0/3543833957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4381a1ed0 con 0x7fb438108270 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.755+0000 7fb43d127640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb438108270 0x7fb43819a4c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d950 tx=0x7fb42800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.755+0000 7fb42effd640 1 -- 172.21.15.19:0/3543833957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428017070 con 0x7fb438108270 2024-05-09T01:34:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.755+0000 7fb42effd640 1 -- 172.21.15.19:0/3543833957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb428009c30 con 0x7fb438108270 2024-05-09T01:34:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.755+0000 7fb43fb8c640 1 -- 172.21.15.19:0/3543833957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4381a21c0 con 0x7fb438108270 2024-05-09T01:34:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.755+0000 7fb42effd640 1 -- 172.21.15.19:0/3543833957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42800f620 con 0x7fb438108270 2024-05-09T01:34:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.755+0000 7fb43fb8c640 1 -- 172.21.15.19:0/3543833957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4381a2760 con 0x7fb438108270 2024-05-09T01:34:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb446a3d740 1 Processor -- start 2024-05-09T01:34:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb446a3d740 1 -- start start 2024-05-09T01:34:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb42effd640 1 -- 172.21.15.19:0/3543833957 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb428010460 con 0x7fb438108270 2024-05-09T01:34:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32fc1b0 0x55b5d36a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 0x55b5d36a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 0x55b5d36abd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb446a3d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5d36047c0 con 0x55b5d32fc1b0 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb446a3d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b5d35a00e0 con 0x55b5d32db4d0 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb446a3d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5d3600780 con 0x55b5d32d32f0 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.758+0000 7fb43e129640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32fc1b0 0x55b5d36a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43e129640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32fc1b0 0x55b5d36a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49716/0 (socket says 172.21.15.19:49716) 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43e129640 1 -- 172.21.15.19:0/1875830585 learned_addr learned my addr 172.21.15.19:0/1875830585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43d127640 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 0x55b5d36abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43d928640 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 0x55b5d36a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43e129640 1 -- 172.21.15.19:0/1875830585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 msgr2=0x55b5d36a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43e129640 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 0x55b5d36a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43e129640 1 -- 172.21.15.19:0/1875830585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 msgr2=0x55b5d36abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43e129640 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 0x55b5d36abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43e129640 1 -- 172.21.15.19:0/1875830585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d35c97c0 con 0x55b5d32fc1b0 2024-05-09T01:34:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43d127640 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 0x55b5d36abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43e129640 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32fc1b0 0x55b5d36a4910 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb43000af20 tx=0x7fb43000e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c85bf666fa12cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb42effd640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb4240b2860 0x7fb4240b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb423fff640 1 -- 172.21.15.19:0/1875830585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430004b20 con 0x55b5d32fc1b0 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb43d928640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb4240b2860 0x7fb4240b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb42effd640 1 -- 172.21.15.19:0/3543833957 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb4280e1d50 con 0x7fb438108270 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb423fff640 1 -- 172.21.15.19:0/1875830585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb430004cc0 con 0x55b5d32fc1b0 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.759+0000 7fb423fff640 1 -- 172.21.15.19:0/1875830585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4300054c0 con 0x55b5d32fc1b0 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb446a3d740 1 -- 172.21.15.19:0/1875830585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32fc1b0 msgr2=0x55b5d36a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32fc1b0 0x55b5d36a4910 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb43000af20 tx=0x7fb43000e8e0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb43d928640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb4240b2860 0x7fb4240b4d70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb43402a930 tx=0x7fb4340314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb446a3d740 1 -- 172.21.15.19:0/1875830585 shutdown_connections 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 0x55b5d36abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 0x55b5d36a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1875830585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32fc1b0 0x55b5d36a4910 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb446a3d740 1 -- 172.21.15.19:0/1875830585 >> 172.21.15.19:0/1875830585 conn(0x55b5d32f0300 msgr2=0x55b5d369be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb446a3d740 1 -- 172.21.15.19:0/1875830585 shutdown_connections 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.760+0000 7fb446a3d740 1 -- 172.21.15.19:0/1875830585 wait complete. 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.761+0000 7fb446a3d740 1 -- start start 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.761+0000 7fb446a3d740 1 Processor -- start 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.761+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 0x55b5d36ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.761+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32db4d0 0x55b5d36aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.761+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32fc1b0 0x55b5d36b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.761+0000 7fb446a3d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5d36ac7e0 con 0x55b5d32db4d0 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.761+0000 7fb446a3d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b5d358b9c0 con 0x55b5d32d32f0 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.761+0000 7fb446a3d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5d36ac960 con 0x55b5d32fc1b0 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb446a3d740 10 client.0 _ll_register_callbacks cb 0x55b5d35b8020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32db4d0 0x55b5d36aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32db4d0 0x55b5d36aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49726/0 (socket says 172.21.15.19:49726) 2024-05-09T01:34:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d127640 1 -- 172.21.15.19:0/1635898860 learned_addr learned my addr 172.21.15.19:0/1635898860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d928640 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 0x55b5d36ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43e129640 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32fc1b0 0x55b5d36b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d127640 1 -- 172.21.15.19:0/1635898860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32fc1b0 msgr2=0x55b5d36b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d127640 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32fc1b0 0x55b5d36b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d127640 1 -- 172.21.15.19:0/1635898860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 msgr2=0x55b5d36ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d127640 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 0x55b5d36ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d127640 1 -- 172.21.15.19:0/1635898860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d35b21f0 con 0x55b5d32db4d0 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43e129640 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32fc1b0 0x55b5d36b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d928640 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 0x55b5d36ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb43d127640 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32db4d0 0x55b5d36aec10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb4280a6670 tx=0x7fb4280a9290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b300d07128f089f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb4217fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb4217fa640 1 -- 172.21.15.19:0/1635898860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428020050 con 0x55b5d32db4d0 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb4217fa640 1 -- 172.21.15.19:0/1635898860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb4280a8660 con 0x55b5d32db4d0 2024-05-09T01:34:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.762+0000 7fb446a3d740 1 -- 172.21.15.19:0/1635898860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb430004cc0 con 0x55b5d32db4d0 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb4217fa640 1 -- 172.21.15.19:0/1635898860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb428009de0 con 0x55b5d32db4d0 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb446a3d740 10 client.17820 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb446a3d740 1 -- 172.21.15.19:0/1635898860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5d3367790 con 0x55b5d32db4d0 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb4217fa640 1 -- 172.21.15.19:0/1635898860 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb430004cc0 con 0x55b5d32db4d0 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb4217fa640 1 -- 172.21.15.19:0/1635898860 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb4280a4d70 con 0x55b5d32db4d0 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb446a3d740 10 client.17820 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb446a3d740 10 client.17820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb446a3d740 20 client.17820 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b5d32bad80 0x55b5d36bc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb446a3d740 4 client.17820 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb446a3d740 1 -- 172.21.15.19:0/1635898860 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5d36123d0 con 0x55b5d32bad80 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.763+0000 7fb43d928640 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b5d32bad80 0x55b5d36bc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.764+0000 7fb4217fa640 10 client.17820 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.764+0000 7fb43d928640 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b5d32bad80 0x55b5d36bc030 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612659c6b50832c6 server_cookie=25e38d52651879dd in_seq=0 out_seq=0 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb4217fa640 1 -- 172.21.15.19:0/1635898860 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5d36123d0 con 0x55b5d32bad80 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb4217fa640 10 client.17820 handle_command_reply: tid=0 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 client.17820 shutdown 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 -- 172.21.15.19:0/1635898860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32db4d0 msgr2=0x55b5d36aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32db4d0 0x55b5d36aec10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb4280a6670 tx=0x7fb4280a9290 comp rx=0 tx=0).stop 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 -- 172.21.15.19:0/1635898860 shutdown_connections 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b5d32bad80 0x55b5d36bc030 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32fc1b0 0x55b5d36b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32db4d0 0x55b5d36aec10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb4217fa640 0 client.17820 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1635898860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 0x55b5d36ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 -- 172.21.15.19:0/1635898860 >> 172.21.15.19:0/1635898860 conn(0x55b5d32f0300 msgr2=0x55b5d369a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 -- 172.21.15.19:0/1635898860 shutdown_connections 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 1 -- 172.21.15.19:0/1635898860 wait complete. 2024-05-09T01:34:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.765+0000 7fb446a3d740 20 client.17820 trim_cache size 0 max 16384 2024-05-09T01:34:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb446a3d740 1 Processor -- start 2024-05-09T01:34:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb446a3d740 1 -- start start 2024-05-09T01:34:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d369ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 0x55b5d36b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32db4d0 0x55b5d369b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb446a3d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381a92e0 con 0x55b5d32bad80 2024-05-09T01:34:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb446a3d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381aae20 con 0x55b5d32d32f0 2024-05-09T01:34:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb446a3d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381a99b0 con 0x55b5d32db4d0 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb43d928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d369ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb43d928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d369ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49734/0 (socket says 172.21.15.19:49734) 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.781+0000 7fb43d928640 1 -- 172.21.15.19:0/1866540517 learned_addr learned my addr 172.21.15.19:0/1866540517 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb43d127640 1 --2- 172.21.15.19:0/1866540517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 0x55b5d36b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb43d928640 1 -- 172.21.15.19:0/1866540517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32db4d0 msgr2=0x55b5d369b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb43d928640 1 --2- 172.21.15.19:0/1866540517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32db4d0 0x55b5d369b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb43d928640 1 -- 172.21.15.19:0/1866540517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 msgr2=0x55b5d36b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb43d928640 1 --2- 172.21.15.19:0/1866540517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 0x55b5d36b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb43d928640 1 -- 172.21.15.19:0/1866540517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d33de5f0 con 0x55b5d32bad80 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb43d928640 1 --2- 172.21.15.19:0/1866540517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d369ae10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb434005f70 tx=0x7fb43403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a66e68409b4ed5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb4237fe640 1 -- 172.21.15.19:0/1866540517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434035070 con 0x55b5d32bad80 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb4237fe640 1 -- 172.21.15.19:0/1866540517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb434003e90 con 0x55b5d32bad80 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.782+0000 7fb4237fe640 1 -- 172.21.15.19:0/1866540517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434031800 con 0x55b5d32bad80 2024-05-09T01:34:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 -- 172.21.15.19:0/1866540517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 msgr2=0x55b5d369ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1866540517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d369ae10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb434005f70 tx=0x7fb43403f000 comp rx=0 tx=0).stop 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 -- 172.21.15.19:0/1866540517 shutdown_connections 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1866540517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32db4d0 0x55b5d369b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1866540517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32d32f0 0x55b5d36b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1866540517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d369ae10 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 -- 172.21.15.19:0/1866540517 >> 172.21.15.19:0/1866540517 conn(0x55b5d32f0300 msgr2=0x55b5d369a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 -- 172.21.15.19:0/1866540517 shutdown_connections 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 -- 172.21.15.19:0/1866540517 wait complete. 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 -- start start 2024-05-09T01:34:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.783+0000 7fb446a3d740 1 Processor -- start 2024-05-09T01:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d36b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 0x55b5d36b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb446a3d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 0x55b5d36b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb446a3d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381aa750 con 0x55b5d32bad80 2024-05-09T01:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb446a3d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381aa080 con 0x55b5d32db4d0 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb446a3d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381a72c0 con 0x55b5d32d32f0 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb446a3d740 10 client.0 _ll_register_callbacks cb 0x7fb42800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d36b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d36b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49746/0 (socket says 172.21.15.19:49746) 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d928640 1 -- 172.21.15.19:0/1964946913 learned_addr learned my addr 172.21.15.19:0/1964946913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d127640 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 0x55b5d36b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d928640 1 -- 172.21.15.19:0/1964946913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 msgr2=0x55b5d36b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d928640 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 0x55b5d36b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d928640 1 -- 172.21.15.19:0/1964946913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 msgr2=0x55b5d36b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d928640 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 0x55b5d36b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d928640 1 -- 172.21.15.19:0/1964946913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d35c8ed0 con 0x55b5d32bad80 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb43d928640 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d36b33a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb4340096c0 tx=0x7fb43404fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fcfa75ad5087 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.784+0000 7fb4217fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb4217fa640 1 -- 172.21.15.19:0/1964946913 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43405c060 con 0x55b5d32bad80 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb4217fa640 1 -- 172.21.15.19:0/1964946913 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb434003aa0 con 0x55b5d32bad80 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb446a3d740 1 -- 172.21.15.19:0/1964946913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb434003e90 con 0x55b5d32bad80 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb4217fa640 1 -- 172.21.15.19:0/1964946913 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb434003ca0 con 0x55b5d32bad80 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb446a3d740 10 client.17826 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb446a3d740 1 -- 172.21.15.19:0/1964946913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5d3724430 con 0x55b5d32bad80 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb4217fa640 1 -- 172.21.15.19:0/1964946913 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb434003e90 con 0x55b5d32bad80 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb4217fa640 1 -- 172.21.15.19:0/1964946913 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb43404fce0 con 0x55b5d32bad80 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb446a3d740 10 client.17826 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb446a3d740 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb446a3d740 20 client.17826 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b5d32fc1b0 0x55b5d3726d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb446a3d740 4 client.17826 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.785+0000 7fb446a3d740 1 -- 172.21.15.19:0/1964946913 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5d360c240 con 0x55b5d32fc1b0 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.786+0000 7fb43d127640 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b5d32fc1b0 0x55b5d3726d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.786+0000 7fb4217fa640 10 client.17826 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.786+0000 7fb43d127640 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b5d32fc1b0 0x55b5d3726d80 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd41a3444b891537 server_cookie=43cc97d54ad72eb6 in_seq=0 out_seq=0 2024-05-09T01:34:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb4217fa640 1 -- 172.21.15.19:0/1964946913 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b5d360c240 con 0x55b5d32fc1b0 2024-05-09T01:34:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb4217fa640 10 client.17826 handle_command_reply: tid=0 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 client.17826 shutdown 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 -- 172.21.15.19:0/1964946913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 msgr2=0x55b5d36b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d36b33a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb4340096c0 tx=0x7fb43404fae0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 -- 172.21.15.19:0/1964946913 shutdown_connections 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b5d32fc1b0 0x55b5d3726d80 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5d32db4d0 0x55b5d36b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d32d32f0 0x55b5d36b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 --2- 172.21.15.19:0/1964946913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5d32bad80 0x55b5d36b33a0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb4217fa640 0 client.17826 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 -- 172.21.15.19:0/1964946913 >> 172.21.15.19:0/1964946913 conn(0x55b5d32f0300 msgr2=0x55b5d369b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 -- 172.21.15.19:0/1964946913 shutdown_connections 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.801+0000 7fb446a3d740 1 -- 172.21.15.19:0/1964946913 wait complete. 2024-05-09T01:34:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb446a3d740 20 client.17826 trim_cache size 0 max 16384 2024-05-09T01:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:34:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 -- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb4240b2860 msgr2=0x7fb4240b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb4240b2860 0x7fb4240b4d70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb43402a930 tx=0x7fb4340314a0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 -- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb438108270 msgr2=0x7fb43819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb438108270 0x7fb43819a4c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d950 tx=0x7fb42800de10 comp rx=0 tx=0).stop 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 -- 172.21.15.19:0/3543833957 shutdown_connections 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb4240b2860 0x7fb4240b4d70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43819f440 0x7fb4381a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb438108270 0x7fb43819a4c0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 --2- 172.21.15.19:0/3543833957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4381021f0 0x7fb438199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 -- 172.21.15.19:0/3543833957 >> 172.21.15.19:0/3543833957 conn(0x7fb4380fd4f0 msgr2=0x7fb4380ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 -- 172.21.15.19:0/3543833957 shutdown_connections 2024-05-09T01:34:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:28.802+0000 7fb43fb8c640 1 -- 172.21.15.19:0/3543833957 wait complete. 2024-05-09T01:34:28.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:28 smithi059 ceph-mon[34351]: pgmap v646: 97 pgs: 97 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:30 smithi019 ceph-mon[30158]: pgmap v647: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:34:30.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:30 smithi196 ceph-mon[35041]: pgmap v647: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:34:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:30 smithi059 ceph-mon[34351]: pgmap v647: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:34:32.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:32 smithi059 ceph-mon[34351]: pgmap v648: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:32.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:32 smithi196 ceph-mon[35041]: pgmap v648: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:32 smithi019 ceph-mon[30158]: pgmap v648: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:34.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:34 smithi196 ceph-mon[35041]: pgmap v649: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:34:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:34 smithi059 ceph-mon[34351]: pgmap v649: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:34:34.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:34 smithi019 ceph-mon[30158]: pgmap v649: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:34:36.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:36 smithi196 ceph-mon[35041]: pgmap v650: 97 pgs: 97 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:36 smithi059 ceph-mon[34351]: pgmap v650: 97 pgs: 97 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:36 smithi019 ceph-mon[30158]: pgmap v650: 97 pgs: 97 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:38.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:38 smithi196 ceph-mon[35041]: pgmap v651: 97 pgs: 97 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:34:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:38 smithi059 ceph-mon[34351]: pgmap v651: 97 pgs: 97 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:34:38.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:38 smithi019 ceph-mon[30158]: pgmap v651: 97 pgs: 97 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:34:40.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:40 smithi196 ceph-mon[35041]: pgmap v652: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:40 smithi059 ceph-mon[34351]: pgmap v652: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:40 smithi019 ceph-mon[30158]: pgmap v652: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:41.773 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:34:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 -- 172.21.15.19:0/303964559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 msgr2=0x7fb518103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 --2- 172.21.15.19:0/303964559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 0x7fb518103870 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb514009900 tx=0x7fb51402a410 comp rx=0 tx=0).stop 2024-05-09T01:34:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 -- 172.21.15.19:0/303964559 shutdown_connections 2024-05-09T01:34:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 --2- 172.21.15.19:0/303964559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb518108ab0 0x7fb51810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 --2- 172.21.15.19:0/303964559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb518103db0 0x7fb5181041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 --2- 172.21.15.19:0/303964559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 0x7fb518103870 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 -- 172.21.15.19:0/303964559 >> 172.21.15.19:0/303964559 conn(0x7fb5180fdd60 msgr2=0x7fb518100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 -- 172.21.15.19:0/303964559 shutdown_connections 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 -- 172.21.15.19:0/303964559 wait complete. 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.912+0000 7fb51fab8640 1 Processor -- start 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51fab8640 1 -- start start 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51fab8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 0x7fb51819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51fab8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb518103db0 0x7fb51819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51fab8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb518108ab0 0x7fb5181a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51fab8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810f7a0 con 0x7fb518102b70 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51fab8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810f620 con 0x7fb518103db0 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51fab8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810f920 con 0x7fb518108ab0 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d854640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 0x7fb51819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d854640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 0x7fb51819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49010/0 (socket says 172.21.15.19:49010) 2024-05-09T01:34:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d854640 1 -- 172.21.15.19:0/4103620054 learned_addr learned my addr 172.21.15.19:0/4103620054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d053640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb518103db0 0x7fb51819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51e055640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb518108ab0 0x7fb5181a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d854640 1 -- 172.21.15.19:0/4103620054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb518108ab0 msgr2=0x7fb5181a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d854640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb518108ab0 0x7fb5181a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d854640 1 -- 172.21.15.19:0/4103620054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb518103db0 msgr2=0x7fb51819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d854640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb518103db0 0x7fb51819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d854640 1 -- 172.21.15.19:0/4103620054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5181a1ef0 con 0x7fb518102b70 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51e055640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb518108ab0 0x7fb5181a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d053640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb518103db0 0x7fb51819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.913+0000 7fb51d854640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 0x7fb51819a3b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb514006010 tx=0x7fb514004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.914+0000 7fb50effd640 1 -- 172.21.15.19:0/4103620054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51403f030 con 0x7fb518102b70 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.914+0000 7fb50effd640 1 -- 172.21.15.19:0/4103620054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb514031910 con 0x7fb518102b70 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.914+0000 7fb51fab8640 1 -- 172.21.15.19:0/4103620054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5181a2180 con 0x7fb518102b70 2024-05-09T01:34:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.914+0000 7fb50effd640 1 -- 172.21.15.19:0/4103620054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514035070 con 0x7fb518102b70 2024-05-09T01:34:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.914+0000 7fb51fab8640 1 -- 172.21.15.19:0/4103620054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181a26c0 con 0x7fb518102b70 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb52696d740 1 Processor -- start 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb52696d740 1 -- start start 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb50effd640 1 -- 172.21.15.19:0/4103620054 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb514042030 con 0x7fb518102b70 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb52696d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab7c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb52696d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7ab2f0 0x55b38ab7ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb52696d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7b34d0 0x55b38ab83d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb52696d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b38aadc7c0 con 0x55b38a7d41b0 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb52696d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b38aa780e0 con 0x55b38a7b34d0 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb52696d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b38aad8780 con 0x55b38a7ab2f0 2024-05-09T01:34:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb51e055640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab7c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb51e055640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab7c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49022/0 (socket says 172.21.15.19:49022) 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.917+0000 7fb51e055640 1 -- 172.21.15.19:0/638670479 learned_addr learned my addr 172.21.15.19:0/638670479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51d854640 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7ab2f0 0x55b38ab7ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51d053640 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7b34d0 0x55b38ab83d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51e055640 1 -- 172.21.15.19:0/638670479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7ab2f0 msgr2=0x55b38ab7ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51e055640 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7ab2f0 0x55b38ab7ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51e055640 1 -- 172.21.15.19:0/638670479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7b34d0 msgr2=0x55b38ab83d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51e055640 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7b34d0 0x55b38ab83d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51e055640 1 -- 172.21.15.19:0/638670479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b38aaa17c0 con 0x55b38a7d41b0 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51d053640 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7b34d0 0x55b38ab83d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51d854640 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7ab2f0 0x55b38ab7ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51e055640 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab7c910 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb510009f90 tx=0x7fb51000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915686d64a051e21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb50effd640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5040b2860 0x7fb5040b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb503fff640 1 -- 172.21.15.19:0/638670479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510003b70 con 0x55b38a7d41b0 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb503fff640 1 -- 172.21.15.19:0/638670479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb510003d10 con 0x55b38a7d41b0 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb503fff640 1 -- 172.21.15.19:0/638670479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510011960 con 0x55b38a7d41b0 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.918+0000 7fb51d854640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5040b2860 0x7fb5040b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.919+0000 7fb50effd640 1 -- 172.21.15.19:0/4103620054 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb514102630 con 0x7fb518102b70 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.919+0000 7fb52696d740 1 -- 172.21.15.19:0/638670479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 msgr2=0x55b38ab7c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.919+0000 7fb51d854640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5040b2860 0x7fb5040b4d70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55b38ab7dcf0 tx=0x7fb51402a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.919+0000 7fb52696d740 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab7c910 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb510009f90 tx=0x7fb51000f7f0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.920+0000 7fb52696d740 1 -- 172.21.15.19:0/638670479 shutdown_connections 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.920+0000 7fb52696d740 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7b34d0 0x55b38ab83d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.920+0000 7fb52696d740 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7ab2f0 0x55b38ab7ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.920+0000 7fb52696d740 1 --2- 172.21.15.19:0/638670479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab7c910 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.920+0000 7fb52696d740 1 -- 172.21.15.19:0/638670479 >> 172.21.15.19:0/638670479 conn(0x55b38a7c8300 msgr2=0x55b38ab73e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.920+0000 7fb52696d740 1 -- 172.21.15.19:0/638670479 shutdown_connections 2024-05-09T01:34:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.920+0000 7fb52696d740 1 -- 172.21.15.19:0/638670479 wait complete. 2024-05-09T01:34:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.920+0000 7fb52696d740 1 -- start start 2024-05-09T01:34:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.920+0000 7fb52696d740 1 Processor -- start 2024-05-09T01:34:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb52696d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab86650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb52696d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7b34d0 0x55b38ab86b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb52696d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7d41b0 0x55b38ab8daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb52696d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b38ab847e0 con 0x55b38a7b34d0 2024-05-09T01:34:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb52696d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b38aa639c0 con 0x55b38a7ab2f0 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb52696d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b38ab84960 con 0x55b38a7d41b0 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb52696d740 10 client.0 _ll_register_callbacks cb 0x55b38aa90020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51d053640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7b34d0 0x55b38ab86b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51d053640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7b34d0 0x55b38ab86b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49030/0 (socket says 172.21.15.19:49030) 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51d053640 1 -- 172.21.15.19:0/2769844437 learned_addr learned my addr 172.21.15.19:0/2769844437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51e055640 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab86650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51e055640 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7d41b0 0x55b38ab8daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51d053640 1 -- 172.21.15.19:0/2769844437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7d41b0 msgr2=0x55b38ab8daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51d053640 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7d41b0 0x55b38ab8daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51d053640 1 -- 172.21.15.19:0/2769844437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 msgr2=0x55b38ab86650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51d053640 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab86650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51d053640 1 -- 172.21.15.19:0/2769844437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b38aa8a1f0 con 0x55b38a7b34d0 2024-05-09T01:34:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb51d053640 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7b34d0 0x55b38ab86b90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb5080126f0 tx=0x7fb508012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dda78766ccbd607 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.921+0000 7fb5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb5017fa640 1 -- 172.21.15.19:0/2769844437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50801f060 con 0x55b38a7b34d0 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb5017fa640 1 -- 172.21.15.19:0/2769844437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb508010d60 con 0x55b38a7b34d0 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb52696d740 1 -- 172.21.15.19:0/2769844437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510003d10 con 0x55b38a7b34d0 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb5017fa640 1 -- 172.21.15.19:0/2769844437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb508025400 con 0x55b38a7b34d0 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb52696d740 10 client.17838 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb52696d740 1 -- 172.21.15.19:0/2769844437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b38a83f790 con 0x55b38a7b34d0 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb5017fa640 1 -- 172.21.15.19:0/2769844437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb510003d10 con 0x55b38a7b34d0 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb5017fa640 1 -- 172.21.15.19:0/2769844437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb508019030 con 0x55b38a7b34d0 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb52696d740 10 client.17838 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb52696d740 10 client.17838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb52696d740 20 client.17838 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.922+0000 7fb52696d740 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b38a792d80 0x55b38ab94040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.923+0000 7fb52696d740 4 client.17838 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.923+0000 7fb52696d740 1 -- 172.21.15.19:0/2769844437 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b38aaea3d0 con 0x55b38a792d80 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.923+0000 7fb51e055640 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b38a792d80 0x55b38ab94040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.923+0000 7fb51e055640 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b38a792d80 0x55b38ab94040 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db5db441dc575295 server_cookie=4c5ce52d1c219aef in_seq=0 out_seq=0 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.923+0000 7fb5017fa640 10 client.17838 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb5017fa640 1 -- 172.21.15.19:0/2769844437 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b38aaea3d0 con 0x55b38a792d80 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb5017fa640 10 client.17838 handle_command_reply: tid=0 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 client.17838 shutdown 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 -- 172.21.15.19:0/2769844437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7b34d0 msgr2=0x55b38ab86b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7b34d0 0x55b38ab86b90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb5080126f0 tx=0x7fb508012bb0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 -- 172.21.15.19:0/2769844437 shutdown_connections 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b38a792d80 0x55b38ab94040 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7d41b0 0x55b38ab8daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7b34d0 0x55b38ab86b90 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb5017fa640 0 client.17838 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 --2- 172.21.15.19:0/2769844437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab86650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 -- 172.21.15.19:0/2769844437 >> 172.21.15.19:0/2769844437 conn(0x55b38a7c8300 msgr2=0x55b38ab76da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 -- 172.21.15.19:0/2769844437 shutdown_connections 2024-05-09T01:34:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 1 -- 172.21.15.19:0/2769844437 wait complete. 2024-05-09T01:34:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.924+0000 7fb52696d740 20 client.17838 trim_cache size 0 max 16384 2024-05-09T01:34:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb52696d740 1 Processor -- start 2024-05-09T01:34:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb52696d740 1 -- start start 2024-05-09T01:34:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb52696d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab80a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb52696d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 0x55b38ab80f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb52696d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab99840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb52696d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181a9230 con 0x55b38a7d41b0 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb52696d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181aad70 con 0x55b38a7ab2f0 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb52696d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181a9900 con 0x55b38a7b34d0 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb51e055640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab99840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb51e055640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab99840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49040/0 (socket says 172.21.15.19:49040) 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb51e055640 1 -- 172.21.15.19:0/4209643825 learned_addr learned my addr 172.21.15.19:0/4209643825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb51e055640 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab80a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb51d053640 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 0x55b38ab80f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb51e055640 1 -- 172.21.15.19:0/4209643825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 msgr2=0x55b38ab80f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb51e055640 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 0x55b38ab80f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb51e055640 1 -- 172.21.15.19:0/4209643825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 msgr2=0x55b38ab80a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.940+0000 7fb51e055640 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab80a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.941+0000 7fb51e055640 1 -- 172.21.15.19:0/4209643825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b38a8b65f0 con 0x55b38a7d41b0 2024-05-09T01:34:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.941+0000 7fb51d053640 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 0x55b38ab80f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.941+0000 7fb51e055640 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab99840 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x55b38ab7dae0 tx=0x7fb510009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bdd95645187dd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.941+0000 7fb5037fe640 1 -- 172.21.15.19:0/4209643825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510003c00 con 0x55b38a7d41b0 2024-05-09T01:34:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.941+0000 7fb5037fe640 1 -- 172.21.15.19:0/4209643825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb510003da0 con 0x55b38a7d41b0 2024-05-09T01:34:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.941+0000 7fb5037fe640 1 -- 172.21.15.19:0/4209643825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510011880 con 0x55b38a7d41b0 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.942+0000 7fb52696d740 1 -- 172.21.15.19:0/4209643825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 msgr2=0x55b38ab99840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.942+0000 7fb52696d740 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab99840 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x55b38ab7dae0 tx=0x7fb510009b20 comp rx=0 tx=0).stop 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.942+0000 7fb52696d740 1 -- 172.21.15.19:0/4209643825 shutdown_connections 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.942+0000 7fb52696d740 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab99840 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.942+0000 7fb52696d740 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 0x55b38ab80f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.942+0000 7fb52696d740 1 --2- 172.21.15.19:0/4209643825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab80a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.942+0000 7fb52696d740 1 -- 172.21.15.19:0/4209643825 >> 172.21.15.19:0/4209643825 conn(0x55b38a7c8300 msgr2=0x55b38ab7c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.942+0000 7fb52696d740 1 -- 172.21.15.19:0/4209643825 shutdown_connections 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.942+0000 7fb52696d740 1 -- 172.21.15.19:0/4209643825 wait complete. 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb52696d740 1 -- start start 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb52696d740 1 Processor -- start 2024-05-09T01:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb52696d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab8b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb52696d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 0x55b38ab8b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb52696d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab8c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb52696d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181aa6a0 con 0x55b38a7d41b0 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb52696d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181a9fd0 con 0x55b38a7ab2f0 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb52696d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181a7210 con 0x55b38a7b34d0 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb52696d740 10 client.0 _ll_register_callbacks cb 0x7fb5080095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb51e055640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab8c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb51e055640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab8c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49052/0 (socket says 172.21.15.19:49052) 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb51e055640 1 -- 172.21.15.19:0/1391984320 learned_addr learned my addr 172.21.15.19:0/1391984320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb51e055640 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab8b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.943+0000 7fb51d053640 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 0x55b38ab8b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb51e055640 1 -- 172.21.15.19:0/1391984320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 msgr2=0x55b38ab8b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb51e055640 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 0x55b38ab8b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb51e055640 1 -- 172.21.15.19:0/1391984320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 msgr2=0x55b38ab8b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb51e055640 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab8b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb51e055640 1 -- 172.21.15.19:0/1391984320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b38aaa0ed0 con 0x55b38a7d41b0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb51e055640 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab8c8a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fb51002afb0 tx=0x7fb510009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c55dd46c10ec13f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb5017fa640 1 -- 172.21.15.19:0/1391984320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510011a40 con 0x55b38a7d41b0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb5017fa640 1 -- 172.21.15.19:0/1391984320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb510011be0 con 0x55b38a7d41b0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb52696d740 1 -- 172.21.15.19:0/1391984320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510003da0 con 0x55b38a7d41b0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb5017fa640 1 -- 172.21.15.19:0/1391984320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb510040050 con 0x55b38a7d41b0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb52696d740 10 client.17844 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb52696d740 1 -- 172.21.15.19:0/1391984320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b38ab8df20 con 0x55b38a7d41b0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.944+0000 7fb5017fa640 1 -- 172.21.15.19:0/1391984320 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb510003da0 con 0x55b38a7d41b0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb5017fa640 1 -- 172.21.15.19:0/1391984320 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb510005200 con 0x55b38a7d41b0 2024-05-09T01:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb52696d740 10 client.17844 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb52696d740 10 client.17844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb52696d740 20 client.17844 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb52696d740 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b38a7681c0 0x55b38abfe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb52696d740 4 client.17844 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb52696d740 1 -- 172.21.15.19:0/1391984320 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b38aae4240 con 0x55b38a7681c0 2024-05-09T01:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb51d053640 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b38a7681c0 0x55b38abfe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb5017fa640 10 client.17844 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.945+0000 7fb51d053640 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b38a7681c0 0x55b38abfe400 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a96d6929f69057c server_cookie=4d041d3c5e27be96 in_seq=0 out_seq=0 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.961+0000 7fb5017fa640 1 -- 172.21.15.19:0/1391984320 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b38aae4240 con 0x55b38a7681c0 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.961+0000 7fb5017fa640 10 client.17844 handle_command_reply: tid=0 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.961+0000 7fb52696d740 1 client.17844 shutdown 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.961+0000 7fb52696d740 1 -- 172.21.15.19:0/1391984320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 msgr2=0x55b38ab8c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.961+0000 7fb52696d740 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab8c8a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fb51002afb0 tx=0x7fb510009d20 comp rx=0 tx=0).stop 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb52696d740 1 -- 172.21.15.19:0/1391984320 shutdown_connections 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb52696d740 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55b38a7681c0 0x55b38abfe400 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb52696d740 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b38a7d41b0 0x55b38ab8c8a0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb52696d740 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b38a7b34d0 0x55b38ab8b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb52696d740 1 --2- 172.21.15.19:0/1391984320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38a7ab2f0 0x55b38ab8b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb5017fa640 0 client.17844 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb52696d740 1 -- 172.21.15.19:0/1391984320 >> 172.21.15.19:0/1391984320 conn(0x55b38a7c8300 msgr2=0x55b38ab7c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb52696d740 1 -- 172.21.15.19:0/1391984320 shutdown_connections 2024-05-09T01:34:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb52696d740 1 -- 172.21.15.19:0/1391984320 wait complete. 2024-05-09T01:34:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb52696d740 20 client.17844 trim_cache size 0 max 16384 2024-05-09T01:34:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:34:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:34:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:34:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 -- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5040b2860 msgr2=0x7fb5040b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5040b2860 0x7fb5040b4d70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55b38ab7dcf0 tx=0x7fb51402a410 comp rx=0 tx=0).stop 2024-05-09T01:34:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 -- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 msgr2=0x7fb51819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 0x7fb51819a3b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb514006010 tx=0x7fb514004270 comp rx=0 tx=0).stop 2024-05-09T01:34:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 -- 172.21.15.19:0/4103620054 shutdown_connections 2024-05-09T01:34:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb5040b2860 0x7fb5040b4d70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb518108ab0 0x7fb5181a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb518103db0 0x7fb51819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 --2- 172.21.15.19:0/4103620054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518102b70 0x7fb51819a3b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.962+0000 7fb51fab8640 1 -- 172.21.15.19:0/4103620054 >> 172.21.15.19:0/4103620054 conn(0x7fb5180fdd60 msgr2=0x7fb518104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.963+0000 7fb51fab8640 1 -- 172.21.15.19:0/4103620054 shutdown_connections 2024-05-09T01:34:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:41.963+0000 7fb51fab8640 1 -- 172.21.15.19:0/4103620054 wait complete. 2024-05-09T01:34:42.049 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:42.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:42 smithi196 ceph-mon[35041]: pgmap v653: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:42 smithi059 ceph-mon[34351]: pgmap v653: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:42.848 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:42 smithi019 ceph-mon[30158]: pgmap v653: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.800+0000 7f56cfc69700 1 -- 172.21.15.19:0/4041686129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 msgr2=0x7f56c8111160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.800+0000 7f56cfc69700 1 --2- 172.21.15.19:0/4041686129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 0x7f56c8111160 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f56c000b0a0 tx=0x7f56c000b3b0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.800+0000 7f56cfc69700 1 -- 172.21.15.19:0/4041686129 shutdown_connections 2024-05-09T01:34:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.800+0000 7f56cfc69700 1 --2- 172.21.15.19:0/4041686129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 0x7f56c8111160 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.800+0000 7f56cfc69700 1 --2- 172.21.15.19:0/4041686129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56c81092d0 0x7f56c81096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.800+0000 7f56cfc69700 1 --2- 172.21.15.19:0/4041686129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56c8102870 0x7f56c8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.800+0000 7f56cfc69700 1 -- 172.21.15.19:0/4041686129 >> 172.21.15.19:0/4041686129 conn(0x7f56c80fe520 msgr2=0x7f56c8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.800+0000 7f56cfc69700 1 -- 172.21.15.19:0/4041686129 shutdown_connections 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.800+0000 7f56cfc69700 1 -- 172.21.15.19:0/4041686129 wait complete. 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cfc69700 1 Processor -- start 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cfc69700 1 -- start start 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cfc69700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56c8102870 0x7f56c8103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cfc69700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 0x7f56c8103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cfc69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56c81092d0 0x7f56c81044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cfc69700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8114590 con 0x7f56c81031c0 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cfc69700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8114410 con 0x7f56c81092d0 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cfc69700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8114710 con 0x7f56c8102870 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cd204700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 0x7f56c8103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cd204700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 0x7f56c8103f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49066/0 (socket says 172.21.15.19:49066) 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cda05700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56c8102870 0x7f56c8103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56cd204700 1 -- 172.21.15.19:0/3457531566 learned_addr learned my addr 172.21.15.19:0/3457531566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.801+0000 7f56ce206700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56c81092d0 0x7f56c81044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56cd204700 1 -- 172.21.15.19:0/3457531566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56c8102870 msgr2=0x7f56c8103a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56cd204700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56c8102870 0x7f56c8103a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56cd204700 1 -- 172.21.15.19:0/3457531566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56c81092d0 msgr2=0x7f56c81044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56cd204700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56c81092d0 0x7f56c81044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56cd204700 1 -- 172.21.15.19:0/3457531566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c81a9130 con 0x7f56c81031c0 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56cd204700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 0x7f56c8103f90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c9e0 tx=0x7f56bc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56baffd700 1 -- 172.21.15.19:0/3457531566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc007aa0 con 0x7f56c81031c0 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56baffd700 1 -- 172.21.15.19:0/3457531566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56bc007c40 con 0x7f56c81031c0 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56cfc69700 1 -- 172.21.15.19:0/3457531566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c81a9420 con 0x7f56c81031c0 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56baffd700 1 -- 172.21.15.19:0/3457531566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00fe60 con 0x7f56c81031c0 2024-05-09T01:34:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.802+0000 7f56cfc69700 1 -- 172.21.15.19:0/3457531566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c81a9960 con 0x7f56c81031c0 2024-05-09T01:34:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.806+0000 7f56baffd700 1 -- 172.21.15.19:0/3457531566 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f56bc004870 con 0x7f56c81031c0 2024-05-09T01:34:43.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.806+0000 7f56baffd700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56b40b2880 0x7f56b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.806+0000 7f56de67fb80 1 Processor -- start 2024-05-09T01:34:43.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.806+0000 7f56baffd700 1 -- 172.21.15.19:0/3457531566 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f56bc0e2170 con 0x7f56c81031c0 2024-05-09T01:34:43.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.806+0000 7f56cda05700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56b40b2880 0x7f56b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.806+0000 7f56de67fb80 1 -- start start 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x5632833894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x563283389a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x563283383ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56de67fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5632832a5ff0 con 0x5632833836a0 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56de67fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5632832447f0 con 0x5632832f0160 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56de67fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5632832a1fb0 con 0x5632832064a0 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56cda05700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56b40b2880 0x7f56b40b4d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005950 tx=0x7f56c400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56ce206700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x563283383ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56ce206700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x563283383ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49080/0 (socket says 172.21.15.19:49080) 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56ce206700 1 -- 172.21.15.19:0/3829780264 learned_addr learned my addr 172.21.15.19:0/3829780264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56cda05700 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x5632833894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56cd204700 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x563283389a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56ce206700 1 -- 172.21.15.19:0/3829780264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 msgr2=0x563283389a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56ce206700 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x563283389a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56ce206700 1 -- 172.21.15.19:0/3829780264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 msgr2=0x5632833894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56ce206700 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x5632833894f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56ce206700 1 -- 172.21.15.19:0/3829780264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328323c370 con 0x5632833836a0 2024-05-09T01:34:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56cda05700 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x5632833894f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.807+0000 7f56ce206700 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x563283383ad0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009200 tx=0x7f56c0009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb3d5e94673a953 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.808+0000 7f56b3fff700 1 -- 172.21.15.19:0/3829780264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0020070 con 0x5632833836a0 2024-05-09T01:34:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.808+0000 7f56b3fff700 1 -- 172.21.15.19:0/3829780264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56c0003c00 con 0x5632833836a0 2024-05-09T01:34:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.808+0000 7f56b3fff700 1 -- 172.21.15.19:0/3829780264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00052a0 con 0x5632833836a0 2024-05-09T01:34:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 -- 172.21.15.19:0/3829780264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 msgr2=0x563283383ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x563283383ad0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009200 tx=0x7f56c0009230 comp rx=0 tx=0).stop 2024-05-09T01:34:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 -- 172.21.15.19:0/3829780264 shutdown_connections 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x563283383ad0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x5632833894f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3829780264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x563283389a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 -- 172.21.15.19:0/3829780264 >> 172.21.15.19:0/3829780264 conn(0x5632832666a0 msgr2=0x56328337e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 -- 172.21.15.19:0/3829780264 shutdown_connections 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 -- 172.21.15.19:0/3829780264 wait complete. 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 -- start start 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.809+0000 7f56de67fb80 1 Processor -- start 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x56328338d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x56328338d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x56328338dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56de67fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563283121ba0 con 0x5632833836a0 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56de67fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563283260150 con 0x5632832f0160 2024-05-09T01:34:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56de67fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5632833901d0 con 0x5632832064a0 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56de67fb80 10 client.0 _ll_register_callbacks cb 0x5632831176c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56ce206700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x56328338dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56ce206700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x56328338dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49084/0 (socket says 172.21.15.19:49084) 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56ce206700 1 -- 172.21.15.19:0/3044181450 learned_addr learned my addr 172.21.15.19:0/3044181450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56cda05700 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x56328338d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56cd204700 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x56328338d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56ce206700 1 -- 172.21.15.19:0/3044181450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 msgr2=0x56328338d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56ce206700 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x56328338d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56ce206700 1 -- 172.21.15.19:0/3044181450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 msgr2=0x56328338d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56ce206700 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x56328338d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.810+0000 7f56ce206700 1 -- 172.21.15.19:0/3044181450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563283121490 con 0x5632833836a0 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56ce206700 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x56328338dc20 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f56c0007c30 tx=0x7f56c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76a38b35fbc0e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56b17fa700 1 -- 172.21.15.19:0/3044181450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0020040 con 0x5632833836a0 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56b17fa700 1 -- 172.21.15.19:0/3044181450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f56c00050c0 con 0x5632833836a0 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56de67fb80 1 -- 172.21.15.19:0/3044181450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c0003c00 con 0x5632833836a0 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56b17fa700 1 -- 172.21.15.19:0/3044181450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56c0034070 con 0x5632833836a0 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56de67fb80 10 client.17856 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56de67fb80 1 -- 172.21.15.19:0/3044181450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56328339ab30 con 0x5632833836a0 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56b17fa700 1 -- 172.21.15.19:0/3044181450 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56c0003c00 con 0x5632833836a0 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56b17fa700 1 -- 172.21.15.19:0/3044181450 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f56c0011070 con 0x5632833836a0 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56de67fb80 10 client.17856 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56de67fb80 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.811+0000 7f56de67fb80 20 client.17856 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.812+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56328339d000 0x56328339f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.812+0000 7f56de67fb80 4 client.17856 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.812+0000 7f56de67fb80 1 -- 172.21.15.19:0/3044181450 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632832b39d0 con 0x56328339d000 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.812+0000 7f56cda05700 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56328339d000 0x56328339f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.812+0000 7f56cda05700 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56328339d000 0x56328339f420 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0835e341f30ca9 server_cookie=5e01e1c8a3168621 in_seq=0 out_seq=0 2024-05-09T01:34:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.812+0000 7f56b17fa700 10 client.17856 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56b17fa700 1 -- 172.21.15.19:0/3044181450 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5632832b39d0 con 0x56328339d000 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56b17fa700 10 client.17856 handle_command_reply: tid=0 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56de67fb80 1 client.17856 shutdown 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56de67fb80 1 -- 172.21.15.19:0/3044181450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 msgr2=0x56328338dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x56328338dc20 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f56c0007c30 tx=0x7f56c0009c00 comp rx=0 tx=0).stop 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56de67fb80 1 -- 172.21.15.19:0/3044181450 shutdown_connections 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56328339d000 0x56328339f420 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632833836a0 0x56328338dc20 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x56328338d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56b17fa700 0 client.17856 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3044181450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x56328338d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.813+0000 7f56de67fb80 1 -- 172.21.15.19:0/3044181450 >> 172.21.15.19:0/3044181450 conn(0x5632832666a0 msgr2=0x56328337e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:43.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.814+0000 7f56de67fb80 1 -- 172.21.15.19:0/3044181450 shutdown_connections 2024-05-09T01:34:43.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.814+0000 7f56de67fb80 1 -- 172.21.15.19:0/3044181450 wait complete. 2024-05-09T01:34:43.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.814+0000 7f56de67fb80 20 client.17856 trim_cache size 0 max 16384 2024-05-09T01:34:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.834+0000 7f56de67fb80 1 Processor -- start 2024-05-09T01:34:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.834+0000 7f56de67fb80 1 -- start start 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x563283387830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832f0160 0x563283399280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632833836a0 0x56328338a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56de67fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81b0d40 con 0x5632832f0160 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56de67fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81b2880 con 0x5632833836a0 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56de67fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81b1410 con 0x5632832064a0 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cd204700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832f0160 0x563283399280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cd204700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832f0160 0x563283399280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49100/0 (socket says 172.21.15.19:49100) 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cd204700 1 -- 172.21.15.19:0/3943172521 learned_addr learned my addr 172.21.15.19:0/3943172521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cda05700 1 --2- 172.21.15.19:0/3943172521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x563283387830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56ce206700 1 --2- 172.21.15.19:0/3943172521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632833836a0 0x56328338a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cd204700 1 -- 172.21.15.19:0/3943172521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 msgr2=0x563283387830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cd204700 1 --2- 172.21.15.19:0/3943172521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x563283387830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cd204700 1 -- 172.21.15.19:0/3943172521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632833836a0 msgr2=0x56328338a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cd204700 1 --2- 172.21.15.19:0/3943172521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632833836a0 0x56328338a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cd204700 1 -- 172.21.15.19:0/3943172521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328338b710 con 0x5632832f0160 2024-05-09T01:34:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56cd204700 1 --2- 172.21.15.19:0/3943172521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832f0160 0x563283399280 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0074a0 tx=0x7f56bc0b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c839cb3273b2f1f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56b37fe700 1 -- 172.21.15.19:0/3943172521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0a9e70 con 0x5632832f0160 2024-05-09T01:34:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56b37fe700 1 -- 172.21.15.19:0/3943172521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56bc0a9770 con 0x5632832f0160 2024-05-09T01:34:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.835+0000 7f56b37fe700 1 -- 172.21.15.19:0/3943172521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0b23a0 con 0x5632832f0160 2024-05-09T01:34:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.836+0000 7f56de67fb80 1 -- 172.21.15.19:0/3943172521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832f0160 msgr2=0x563283399280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.836+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3943172521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832f0160 0x563283399280 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0074a0 tx=0x7f56bc0b2040 comp rx=0 tx=0).stop 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.836+0000 7f56de67fb80 1 -- 172.21.15.19:0/3943172521 shutdown_connections 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.836+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3943172521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632833836a0 0x56328338a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.836+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3943172521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632832064a0 0x563283387830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.836+0000 7f56de67fb80 1 --2- 172.21.15.19:0/3943172521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832f0160 0x563283399280 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.836+0000 7f56de67fb80 1 -- 172.21.15.19:0/3943172521 >> 172.21.15.19:0/3943172521 conn(0x5632832666a0 msgr2=0x5632833882a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.836+0000 7f56de67fb80 1 -- 172.21.15.19:0/3943172521 shutdown_connections 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.836+0000 7f56de67fb80 1 -- 172.21.15.19:0/3943172521 wait complete. 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56de67fb80 1 -- start start 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56de67fb80 1 Processor -- start 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832064a0 0x563283400090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x5632834005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56de67fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632833836a0 0x5632833a2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56de67fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81b21b0 con 0x5632832064a0 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56de67fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81b1ae0 con 0x5632832f0160 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56de67fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56c81aece0 con 0x5632833836a0 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56de67fb80 10 client.0 _ll_register_callbacks cb 0x563283388ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56cda05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832064a0 0x563283400090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56cda05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832064a0 0x563283400090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49116/0 (socket says 172.21.15.19:49116) 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56cda05700 1 -- 172.21.15.19:0/4056914187 learned_addr learned my addr 172.21.15.19:0/4056914187 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56cd204700 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x5632834005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.837+0000 7f56ce206700 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632833836a0 0x5632833a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56cda05700 1 -- 172.21.15.19:0/4056914187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632833836a0 msgr2=0x5632833a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56cda05700 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632833836a0 0x5632833a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56cda05700 1 -- 172.21.15.19:0/4056914187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 msgr2=0x5632834005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56cda05700 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x5632834005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56cda05700 1 -- 172.21.15.19:0/4056914187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632833a2740 con 0x5632832064a0 2024-05-09T01:34:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56cda05700 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832064a0 0x563283400090 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f56c400a950 tx=0x7f56c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb560e31d02cb5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56b17fa700 1 -- 172.21.15.19:0/4056914187 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4016070 con 0x5632832064a0 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56b17fa700 1 -- 172.21.15.19:0/4056914187 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f56c4004a30 con 0x5632832064a0 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56de67fb80 1 -- 172.21.15.19:0/4056914187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56bc0a9770 con 0x5632832064a0 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56b17fa700 1 -- 172.21.15.19:0/4056914187 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56c4004c30 con 0x5632832064a0 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56de67fb80 10 client.17862 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.838+0000 7f56de67fb80 1 -- 172.21.15.19:0/4056914187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563283407800 con 0x5632832064a0 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.839+0000 7f56b17fa700 1 -- 172.21.15.19:0/4056914187 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56bc0a9770 con 0x5632832064a0 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.839+0000 7f56b17fa700 1 -- 172.21.15.19:0/4056914187 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f56c402b310 con 0x5632832064a0 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.839+0000 7f56de67fb80 10 client.17862 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.839+0000 7f56de67fb80 10 client.17862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.839+0000 7f56de67fb80 20 client.17862 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.839+0000 7f56de67fb80 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563283407ef0 0x56328340a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.839+0000 7f56de67fb80 4 client.17862 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.839+0000 7f56de67fb80 1 -- 172.21.15.19:0/4056914187 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5632832ada70 con 0x563283407ef0 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.839+0000 7f56cd204700 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563283407ef0 0x56328340a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56b17fa700 10 client.17862 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56cd204700 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563283407ef0 0x56328340a310 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=255863c00698bf1b server_cookie=c2721808174a6241 in_seq=0 out_seq=0 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56b17fa700 1 -- 172.21.15.19:0/4056914187 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5632832ada70 con 0x563283407ef0 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56b17fa700 10 client.17862 handle_command_reply: tid=0 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 client.17862 shutdown 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 -- 172.21.15.19:0/4056914187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832064a0 msgr2=0x563283400090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832064a0 0x563283400090 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f56c400a950 tx=0x7f56c40047c0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 -- 172.21.15.19:0/4056914187 shutdown_connections 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x563283407ef0 0x56328340a310 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56b17fa700 0 client.17862 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5632833836a0 0x5632833a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5632832064a0 0x563283400090 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 --2- 172.21.15.19:0/4056914187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5632832f0160 0x5632834005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 -- 172.21.15.19:0/4056914187 >> 172.21.15.19:0/4056914187 conn(0x5632832666a0 msgr2=0x5632833882a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 -- 172.21.15.19:0/4056914187 shutdown_connections 2024-05-09T01:34:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 1 -- 172.21.15.19:0/4056914187 wait complete. 2024-05-09T01:34:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.840+0000 7f56de67fb80 20 client.17862 trim_cache size 0 max 16384 2024-05-09T01:34:43.842 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:34:43.842 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:34:43.842 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:34:43.842 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:34:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 -- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56b40b2880 msgr2=0x7f56b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56b40b2880 0x7f56b40b4d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005950 tx=0x7f56c400a300 comp rx=0 tx=0).stop 2024-05-09T01:34:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 -- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 msgr2=0x7f56c8103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 0x7f56c8103f90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c9e0 tx=0x7f56bc00ccf0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 -- 172.21.15.19:0/3457531566 shutdown_connections 2024-05-09T01:34:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56c81092d0 0x7f56c81044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56c8102870 0x7f56c8103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f56b40b2880 0x7f56b40b4d90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 --2- 172.21.15.19:0/3457531566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c81031c0 0x7f56c8103f90 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 -- 172.21.15.19:0/3457531566 >> 172.21.15.19:0/3457531566 conn(0x7f56c80fe520 msgr2=0x7f56c810a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 -- 172.21.15.19:0/3457531566 shutdown_connections 2024-05-09T01:34:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:43.841+0000 7f56cfc69700 1 -- 172.21.15.19:0/3457531566 wait complete. 2024-05-09T01:34:44.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:34:44.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:34:44.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:44 smithi196 ceph-mon[35041]: pgmap v654: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:34:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:44 smithi059 ceph-mon[34351]: pgmap v654: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:34:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:44.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:44 smithi019 ceph-mon[30158]: pgmap v654: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-05-09T01:34:46.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.529+0000 7fe50399e700 1 -- 172.21.15.19:0/914071943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 msgr2=0x7fe4fc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:46.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.529+0000 7fe50399e700 1 --2- 172.21.15.19:0/914071943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 0x7fe4fc1100a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400b0d0 tx=0x7fe4f400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.529+0000 7fe50399e700 1 -- 172.21.15.19:0/914071943 shutdown_connections 2024-05-09T01:34:46.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.529+0000 7fe50399e700 1 --2- 172.21.15.19:0/914071943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 0x7fe4fc1100a0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.529+0000 7fe50399e700 1 --2- 172.21.15.19:0/914071943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4fc1034c0 0x7fe4fc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.529+0000 7fe50399e700 1 --2- 172.21.15.19:0/914071943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc107df0 0x7fe4fc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.529+0000 7fe50399e700 1 -- 172.21.15.19:0/914071943 >> 172.21.15.19:0/914071943 conn(0x7fe4fc0fedf0 msgr2=0x7fe4fc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:46.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.529+0000 7fe50399e700 1 -- 172.21.15.19:0/914071943 shutdown_connections 2024-05-09T01:34:46.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.529+0000 7fe50399e700 1 -- 172.21.15.19:0/914071943 wait complete. 2024-05-09T01:34:46.526 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:46 smithi019 ceph-mon[30158]: pgmap v655: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:46.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe50399e700 1 Processor -- start 2024-05-09T01:34:46.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe50399e700 1 -- start start 2024-05-09T01:34:46.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe50399e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4fc1034c0 0x7fe4fc19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:46.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe50399e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc107df0 0x7fe4fc19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:46.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe50399e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 0x7fe4fc1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:46.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe50399e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc113370 con 0x7fe4fc108740 2024-05-09T01:34:46.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe50399e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1131f0 con 0x7fe4fc107df0 2024-05-09T01:34:46.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe50399e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1134f0 con 0x7fe4fc1034c0 2024-05-09T01:34:46.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe501f3b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 0x7fe4fc1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe501f3b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 0x7fe4fc1a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49130/0 (socket says 172.21.15.19:49130) 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe501f3b700 1 -- 172.21.15.19:0/3052606220 learned_addr learned my addr 172.21.15.19:0/3052606220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe500f39700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc107df0 0x7fe4fc19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.530+0000 7fe50173a700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4fc1034c0 0x7fe4fc19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe501f3b700 1 -- 172.21.15.19:0/3052606220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4fc1034c0 msgr2=0x7fe4fc19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe501f3b700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4fc1034c0 0x7fe4fc19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe501f3b700 1 -- 172.21.15.19:0/3052606220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc107df0 msgr2=0x7fe4fc19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe501f3b700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc107df0 0x7fe4fc19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe501f3b700 1 -- 172.21.15.19:0/3052606220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc1a1e70 con 0x7fe4fc108740 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe501f3b700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 0x7fe4fc1a17d0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009850 tx=0x7fe4f4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe4f27fc700 1 -- 172.21.15.19:0/3052606220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f401f070 con 0x7fe4fc108740 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe4f27fc700 1 -- 172.21.15.19:0/3052606220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f4004c50 con 0x7fe4fc108740 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc1a2100 con 0x7fe4fc108740 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe4f27fc700 1 -- 172.21.15.19:0/3052606220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f401b690 con 0x7fe4fc108740 2024-05-09T01:34:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.531+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc1a25e0 con 0x7fe4fc108740 2024-05-09T01:34:46.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.533+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4fc0512c0 con 0x7fe4fc108740 2024-05-09T01:34:46.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.534+0000 7fe4f27fc700 1 -- 172.21.15.19:0/3052606220 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe4f4022070 con 0x7fe4fc108740 2024-05-09T01:34:46.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.535+0000 7fe4f27fc700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe4e80b2880 0x7fe4e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:46.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.535+0000 7fe50173a700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe4e80b2880 0x7fe4e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:46.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.535+0000 7fe4f27fc700 1 -- 172.21.15.19:0/3052606220 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe4f400f020 con 0x7fe4fc108740 2024-05-09T01:34:46.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.535+0000 7fe50173a700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe4e80b2880 0x7fe4e80b4d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8005950 tx=0x7fe4f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:46.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.540+0000 7fe4f27fc700 1 -- 172.21.15.19:0/3052606220 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe4f4029020 con 0x7fe4fc108740 2024-05-09T01:34:46.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:46 smithi196 ceph-mon[35041]: pgmap v655: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.802+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4fc02e5f0 con 0x7fe4fc108740 2024-05-09T01:34:46.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.803+0000 7fe4f27fc700 1 -- 172.21.15.19:0/3052606220 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fe4f4009cb0 con 0x7fe4fc108740 2024-05-09T01:34:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:34:46.799 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.805+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe4e80b2880 msgr2=0x7fe4e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe4e80b2880 0x7fe4e80b4d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8005950 tx=0x7fe4f800b410 comp rx=0 tx=0).stop 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 msgr2=0x7fe4fc1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 0x7fe4fc1a17d0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009850 tx=0x7fe4f4007a20 comp rx=0 tx=0).stop 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 shutdown_connections 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe4e80b2880 0x7fe4e80b4d90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc108740 0x7fe4fc1a17d0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4fc1034c0 0x7fe4fc19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 --2- 172.21.15.19:0/3052606220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc107df0 0x7fe4fc19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 >> 172.21.15.19:0/3052606220 conn(0x7fe4fc0fedf0 msgr2=0x7fe4fc10ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 shutdown_connections 2024-05-09T01:34:46.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:46.806+0000 7fe50399e700 1 -- 172.21.15.19:0/3052606220 wait complete. 2024-05-09T01:34:46.803 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:34:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:46 smithi059 ceph-mon[34351]: pgmap v655: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:47.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:34:47.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:47 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3052606220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:34:47.666 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:47.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:47 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3052606220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:34:47.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:47 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3052606220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:34:48.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:48 smithi196 ceph-mon[35041]: pgmap v656: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:48.724 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:48 smithi019 ceph-mon[30158]: pgmap v656: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:48 smithi059 ceph-mon[34351]: pgmap v656: 97 pgs: 97 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:34:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.283+0000 7f0124104700 1 -- 172.21.15.19:0/1884696433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c109850 msgr2=0x7f011c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.283+0000 7f0124104700 1 --2- 172.21.15.19:0/1884696433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c109850 0x7f011c1111b0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f01140099c0 tx=0x7f0114009cd0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.284+0000 7f0124104700 1 -- 172.21.15.19:0/1884696433 shutdown_connections 2024-05-09T01:34:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.284+0000 7f0124104700 1 --2- 172.21.15.19:0/1884696433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c109850 0x7f011c1111b0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.284+0000 7f0124104700 1 --2- 172.21.15.19:0/1884696433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f011c1045d0 0x7f011c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.284+0000 7f0124104700 1 --2- 172.21.15.19:0/1884696433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f011c108f00 0x7f011c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.284+0000 7f0124104700 1 -- 172.21.15.19:0/1884696433 >> 172.21.15.19:0/1884696433 conn(0x7f011c0ffea0 msgr2=0x7f011c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.284+0000 7f0124104700 1 -- 172.21.15.19:0/1884696433 shutdown_connections 2024-05-09T01:34:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.284+0000 7f0124104700 1 -- 172.21.15.19:0/1884696433 wait complete. 2024-05-09T01:34:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.284+0000 7f0124104700 1 Processor -- start 2024-05-09T01:34:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f0124104700 1 -- start start 2024-05-09T01:34:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f0124104700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f011c1045d0 0x7f011c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f0124104700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c108f00 0x7f011c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f0124104700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f011c109850 0x7f011c077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f0124104700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f011c114480 con 0x7f011c108f00 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f0124104700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f011c114300 con 0x7f011c109850 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f0124104700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f011c114600 con 0x7f011c1045d0 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f012169f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c108f00 0x7f011c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f012169f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c108f00 0x7f011c077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49148/0 (socket says 172.21.15.19:49148) 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f012169f700 1 -- 172.21.15.19:0/2068376623 learned_addr learned my addr 172.21.15.19:0/2068376623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f0121ea0700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f011c1045d0 0x7f011c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f01226a1700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f011c109850 0x7f011c077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f012169f700 1 -- 172.21.15.19:0/2068376623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f011c1045d0 msgr2=0x7f011c076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f012169f700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f011c1045d0 0x7f011c076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f012169f700 1 -- 172.21.15.19:0/2068376623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f011c109850 msgr2=0x7f011c077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f012169f700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f011c109850 0x7f011c077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.285+0000 7f012169f700 1 -- 172.21.15.19:0/2068376623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c1a9120 con 0x7f011c108f00 2024-05-09T01:34:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.286+0000 7f012169f700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c108f00 0x7f011c077110 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f011000d980 tx=0x7f011000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.286+0000 7f011affd700 1 -- 172.21.15.19:0/2068376623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110017070 con 0x7f011c108f00 2024-05-09T01:34:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.286+0000 7f0124104700 1 -- 172.21.15.19:0/2068376623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c1a9410 con 0x7f011c108f00 2024-05-09T01:34:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.286+0000 7f011affd700 1 -- 172.21.15.19:0/2068376623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f011000fd40 con 0x7f011c108f00 2024-05-09T01:34:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.286+0000 7f011affd700 1 -- 172.21.15.19:0/2068376623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011000e3b0 con 0x7f011c108f00 2024-05-09T01:34:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.286+0000 7f0124104700 1 -- 172.21.15.19:0/2068376623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c1a99a0 con 0x7f011c108f00 2024-05-09T01:34:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.289+0000 7f011affd700 1 -- 172.21.15.19:0/2068376623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f011000e550 con 0x7f011c108f00 2024-05-09T01:34:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.290+0000 7f011affd700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01040b2880 0x7f01040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.290+0000 7f0121ea0700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01040b2880 0x7f01040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.290+0000 7f011affd700 1 -- 172.21.15.19:0/2068376623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f01100e1c80 con 0x7f011c108f00 2024-05-09T01:34:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.290+0000 7f0121ea0700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01040b2880 0x7f01040b4d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f010c0060b0 tx=0x7f010c00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.290+0000 7f0132b1ab80 1 Processor -- start 2024-05-09T01:34:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f0132b1ab80 1 -- start start 2024-05-09T01:34:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 0x562e1b2bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 0x562e1b2b5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e1b1d80a0 con 0x562e1b1384a0 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562e1b17d9a0 con 0x562e1b2b5400 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e1b1d4060 con 0x562e1b1b2d00 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f012169f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f012169f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2bb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49164/0 (socket says 172.21.15.19:49164) 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f012169f700 1 -- 172.21.15.19:0/1296874049 learned_addr learned my addr 172.21.15.19:0/1296874049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f01226a1700 1 --2- 172.21.15.19:0/1296874049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 0x562e1b2b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f0121ea0700 1 --2- 172.21.15.19:0/1296874049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 0x562e1b2bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f012169f700 1 -- 172.21.15.19:0/1296874049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 msgr2=0x562e1b2bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f012169f700 1 --2- 172.21.15.19:0/1296874049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 0x562e1b2bb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f012169f700 1 -- 172.21.15.19:0/1296874049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 msgr2=0x562e1b2b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f012169f700 1 --2- 172.21.15.19:0/1296874049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 0x562e1b2b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.291+0000 7f012169f700 1 -- 172.21.15.19:0/1296874049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1b16e500 con 0x562e1b1384a0 2024-05-09T01:34:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.292+0000 7f012169f700 1 --2- 172.21.15.19:0/1296874049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2bb780 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f01100a64f0 tx=0x7f01100a8ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b783193fa1d39448 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:49.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.292+0000 7f0103fff700 1 -- 172.21.15.19:0/1296874049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110020070 con 0x562e1b1384a0 2024-05-09T01:34:49.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.292+0000 7f0103fff700 1 -- 172.21.15.19:0/1296874049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01100a9280 con 0x562e1b1384a0 2024-05-09T01:34:49.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.292+0000 7f0103fff700 1 -- 172.21.15.19:0/1296874049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011000f890 con 0x562e1b1384a0 2024-05-09T01:34:49.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1296874049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 msgr2=0x562e1b2bb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:49.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1296874049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2bb780 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f01100a64f0 tx=0x7f01100a8ae0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1296874049 shutdown_connections 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1296874049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 0x562e1b2b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1296874049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 0x562e1b2bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1296874049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2bb780 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1296874049 >> 172.21.15.19:0/1296874049 conn(0x562e1b13bf70 msgr2=0x562e1b2b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1296874049 shutdown_connections 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1296874049 wait complete. 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 -- start start 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.293+0000 7f0132b1ab80 1 Processor -- start 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1384a0 0x562e1b2beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 0x562e1b2bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b2b5400 0x562e1b2bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562e1b2b60a0 con 0x562e1b2b5400 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562e1b172b20 con 0x562e1b1b2d00 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e1b2b6220 con 0x562e1b1384a0 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f0132b1ab80 10 client.0 _ll_register_callbacks cb 0x562e1b0496c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f01226a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b2b5400 0x562e1b2bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f01226a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b2b5400 0x562e1b2bfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49172/0 (socket says 172.21.15.19:49172) 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f01226a1700 1 -- 172.21.15.19:0/2281804031 learned_addr learned my addr 172.21.15.19:0/2281804031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f0121ea0700 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1384a0 0x562e1b2beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f012169f700 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 0x562e1b2bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f01226a1700 1 -- 172.21.15.19:0/2281804031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1384a0 msgr2=0x562e1b2beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f01226a1700 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1384a0 0x562e1b2beff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f01226a1700 1 -- 172.21.15.19:0/2281804031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 msgr2=0x562e1b2bf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f01226a1700 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 0x562e1b2bf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f01226a1700 1 -- 172.21.15.19:0/2281804031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1b053490 con 0x562e1b2b5400 2024-05-09T01:34:49.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.294+0000 7f012169f700 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 0x562e1b2bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.295+0000 7f01226a1700 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b2b5400 0x562e1b2bfa70 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0114015000 tx=0x7f0114003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3f0af716d8f36b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.295+0000 7f01017fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.295+0000 7f01017fa700 1 -- 172.21.15.19:0/2281804031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114005580 con 0x562e1b2b5400 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.295+0000 7f01017fa700 1 -- 172.21.15.19:0/2281804031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f011400b790 con 0x562e1b2b5400 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.295+0000 7f0132b1ab80 1 -- 172.21.15.19:0/2281804031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01100a9280 con 0x562e1b2b5400 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.295+0000 7f01017fa700 1 -- 172.21.15.19:0/2281804031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f011400b990 con 0x562e1b2b5400 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.295+0000 7f0132b1ab80 10 client.17880 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.295+0000 7f0132b1ab80 1 -- 172.21.15.19:0/2281804031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e1b2cc960 con 0x562e1b2b5400 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.295+0000 7f01017fa700 1 -- 172.21.15.19:0/2281804031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01100a9280 con 0x562e1b2b5400 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f01017fa700 1 -- 172.21.15.19:0/2281804031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0114023ab0 con 0x562e1b2b5400 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f0132b1ab80 10 client.17880 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f0132b1ab80 10 client.17880 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f0132b1ab80 20 client.17880 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:49.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x562e1b2cefa0 0x562e1b2d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f0132b1ab80 4 client.17880 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f0132b1ab80 1 -- 172.21.15.19:0/2281804031 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e1b1e5a80 con 0x562e1b2cefa0 2024-05-09T01:34:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f0121ea0700 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x562e1b2cefa0 0x562e1b2d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f01017fa700 10 client.17880 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:34:49.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.296+0000 7f0121ea0700 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x562e1b2cefa0 0x562e1b2d13f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a45224c788936de9 server_cookie=6e104bf227948318 in_seq=0 out_seq=0 2024-05-09T01:34:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f01017fa700 1 -- 172.21.15.19:0/2281804031 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e1b1e5a80 con 0x562e1b2cefa0 2024-05-09T01:34:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f01017fa700 10 client.17880 handle_command_reply: tid=0 2024-05-09T01:34:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f0132b1ab80 1 client.17880 shutdown 2024-05-09T01:34:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f0132b1ab80 1 -- 172.21.15.19:0/2281804031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b2b5400 msgr2=0x562e1b2bfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b2b5400 0x562e1b2bfa70 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0114015000 tx=0x7f0114003760 comp rx=0 tx=0).stop 2024-05-09T01:34:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f0132b1ab80 1 -- 172.21.15.19:0/2281804031 shutdown_connections 2024-05-09T01:34:49.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x562e1b2cefa0 0x562e1b2d13f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b2b5400 0x562e1b2bfa70 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1384a0 0x562e1b2beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f01017fa700 0 client.17880 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:34:49.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/2281804031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 0x562e1b2bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.297+0000 7f0132b1ab80 1 -- 172.21.15.19:0/2281804031 >> 172.21.15.19:0/2281804031 conn(0x562e1b13bf70 msgr2=0x562e1b2a9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:49.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.298+0000 7f0132b1ab80 1 -- 172.21.15.19:0/2281804031 shutdown_connections 2024-05-09T01:34:49.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.298+0000 7f0132b1ab80 1 -- 172.21.15.19:0/2281804031 wait complete. 2024-05-09T01:34:49.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.298+0000 7f0132b1ab80 20 client.17880 trim_cache size 0 max 16384 2024-05-09T01:34:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0132b1ab80 1 Processor -- start 2024-05-09T01:34:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0132b1ab80 1 -- start start 2024-05-09T01:34:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2aa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 0x562e1b2cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b2b5400 0x562e1b2d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1b0d90 con 0x562e1b1384a0 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1b28d0 con 0x562e1b1b2d00 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1b1460 con 0x562e1b2b5400 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0121ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0121ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2aa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49186/0 (socket says 172.21.15.19:49186) 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0121ea0700 1 -- 172.21.15.19:0/1254451216 learned_addr learned my addr 172.21.15.19:0/1254451216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f012169f700 1 --2- 172.21.15.19:0/1254451216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 0x562e1b2cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0121ea0700 1 -- 172.21.15.19:0/1254451216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b2b5400 msgr2=0x562e1b2d6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0121ea0700 1 --2- 172.21.15.19:0/1254451216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b2b5400 0x562e1b2d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0121ea0700 1 -- 172.21.15.19:0/1254451216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 msgr2=0x562e1b2cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0121ea0700 1 --2- 172.21.15.19:0/1254451216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 0x562e1b2cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0121ea0700 1 -- 172.21.15.19:0/1254451216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1b2bbaf0 con 0x562e1b1384a0 2024-05-09T01:34:49.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.313+0000 7f0121ea0700 1 --2- 172.21.15.19:0/1254451216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2aa700 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x562e1b2d1b50 tx=0x7f010c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9641ceba9bd19714 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:49.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f01037fe700 1 -- 172.21.15.19:0/1254451216 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c016da0 con 0x562e1b1384a0 2024-05-09T01:34:49.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f01037fe700 1 -- 172.21.15.19:0/1254451216 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f010c004bb0 con 0x562e1b1384a0 2024-05-09T01:34:49.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f01037fe700 1 -- 172.21.15.19:0/1254451216 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c005280 con 0x562e1b1384a0 2024-05-09T01:34:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1254451216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 msgr2=0x562e1b2aa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1254451216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2aa700 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x562e1b2d1b50 tx=0x7f010c012000 comp rx=0 tx=0).stop 2024-05-09T01:34:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1254451216 shutdown_connections 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1254451216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b2b5400 0x562e1b2d6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1254451216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2aa700 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1254451216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b1b2d00 0x562e1b2cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.314+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1254451216 >> 172.21.15.19:0/1254451216 conn(0x562e1b13bf70 msgr2=0x562e1b2a9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1254451216 shutdown_connections 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1254451216 wait complete. 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 -- start start 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 Processor -- start 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 0x562e1b2c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 0x562e1b337760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1b2200 con 0x562e1b1384a0 2024-05-09T01:34:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1b1b30 con 0x562e1b2b5400 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.315+0000 7f0132b1ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1aed30 con 0x562e1b1b2d00 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0132b1ab80 10 client.0 _ll_register_callbacks cb 0x562e1b2bad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0121ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0121ea0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49202/0 (socket says 172.21.15.19:49202) 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0121ea0700 1 -- 172.21.15.19:0/1428413163 learned_addr learned my addr 172.21.15.19:0/1428413163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f012169f700 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 0x562e1b2c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f01226a1700 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 0x562e1b337760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0121ea0700 1 -- 172.21.15.19:0/1428413163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 msgr2=0x562e1b2c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0121ea0700 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 0x562e1b2c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0121ea0700 1 -- 172.21.15.19:0/1428413163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 msgr2=0x562e1b337760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0121ea0700 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 0x562e1b337760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0121ea0700 1 -- 172.21.15.19:0/1428413163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1b2c4e20 con 0x562e1b1384a0 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0121ea0700 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2c40c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f010c0164e0 tx=0x7f010c02abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed1c18adeafda49 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f01017fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f01017fa700 1 -- 172.21.15.19:0/1428413163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c02a5e0 con 0x562e1b1384a0 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f01017fa700 1 -- 172.21.15.19:0/1428413163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f010c033440 con 0x562e1b1384a0 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.316+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1428413163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f010c004bb0 con 0x562e1b1384a0 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f01017fa700 1 -- 172.21.15.19:0/1428413163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f010c033640 con 0x562e1b1384a0 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f0132b1ab80 10 client.17886 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1428413163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e1b338bf0 con 0x562e1b1384a0 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f01017fa700 1 -- 172.21.15.19:0/1428413163 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f010c004bb0 con 0x562e1b1384a0 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f01017fa700 1 -- 172.21.15.19:0/1428413163 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f010c0249c0 con 0x562e1b1384a0 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f0132b1ab80 10 client.17886 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f0132b1ab80 10 client.17886 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f0132b1ab80 20 client.17886 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x562e1b339270 0x562e1b33b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f0132b1ab80 4 client.17886 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1428413163 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e1b1dfb20 con 0x562e1b339270 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.317+0000 7f012169f700 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x562e1b339270 0x562e1b33b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f01017fa700 10 client.17886 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f012169f700 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x562e1b339270 0x562e1b33b690 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=63b2b90f465dd9d8 server_cookie=8dca51210401b386 in_seq=0 out_seq=0 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f01017fa700 1 -- 172.21.15.19:0/1428413163 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e1b1dfb20 con 0x562e1b339270 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f01017fa700 10 client.17886 handle_command_reply: tid=0 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f0132b1ab80 1 client.17886 shutdown 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1428413163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 msgr2=0x562e1b2c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2c40c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f010c0164e0 tx=0x7f010c02abe0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1428413163 shutdown_connections 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x562e1b339270 0x562e1b33b690 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1b2b5400 0x562e1b337760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e1b1384a0 0x562e1b2c40c0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f01017fa700 0 client.17886 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:34:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f0132b1ab80 1 --2- 172.21.15.19:0/1428413163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e1b1b2d00 0x562e1b2c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.318+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1428413163 >> 172.21.15.19:0/1428413163 conn(0x562e1b13bf70 msgr2=0x562e1b2b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1428413163 shutdown_connections 2024-05-09T01:34:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0132b1ab80 1 -- 172.21.15.19:0/1428413163 wait complete. 2024-05-09T01:34:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0132b1ab80 20 client.17886 trim_cache size 0 max 16384 2024-05-09T01:34:49.320 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0124104700 1 -- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01040b2880 msgr2=0x7f01040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0124104700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01040b2880 0x7f01040b4d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f010c0060b0 tx=0x7f010c00c040 comp rx=0 tx=0).stop 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0124104700 1 -- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c108f00 msgr2=0x7f011c077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0124104700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c108f00 0x7f011c077110 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f011000d980 tx=0x7f011000dc90 comp rx=0 tx=0).stop 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0124104700 1 -- 172.21.15.19:0/2068376623 shutdown_connections 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0124104700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f011c109850 0x7f011c077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.319+0000 7f0124104700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f011c1045d0 0x7f011c076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.320+0000 7f0124104700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f01040b2880 0x7f01040b4d90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.320+0000 7f0124104700 1 --2- 172.21.15.19:0/2068376623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f011c108f00 0x7f011c077110 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.320+0000 7f0124104700 1 -- 172.21.15.19:0/2068376623 >> 172.21.15.19:0/2068376623 conn(0x7f011c0ffea0 msgr2=0x7f011c108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.320+0000 7f0124104700 1 -- 172.21.15.19:0/2068376623 shutdown_connections 2024-05-09T01:34:49.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:49.320+0000 7f0124104700 1 -- 172.21.15.19:0/2068376623 wait complete. 2024-05-09T01:34:50.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:34:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:50.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:50 smithi196 ceph-mon[35041]: pgmap v657: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:50.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:50 smithi019 ceph-mon[30158]: pgmap v657: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:50 smithi059 ceph-mon[34351]: pgmap v657: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:34:51.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.936+0000 7ff4b4292700 1 -- 172.21.15.19:0/1172340128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac1034c0 msgr2=0x7ff4ac1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:51.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.936+0000 7ff4b4292700 1 --2- 172.21.15.19:0/1172340128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac1034c0 0x7ff4ac1038d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80099c0 tx=0x7ff4a8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.936+0000 7ff4b4292700 1 -- 172.21.15.19:0/1172340128 shutdown_connections 2024-05-09T01:34:51.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.936+0000 7ff4b4292700 1 --2- 172.21.15.19:0/1172340128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac108740 0x7ff4ac1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.936+0000 7ff4b4292700 1 --2- 172.21.15.19:0/1172340128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac1034c0 0x7ff4ac1038d0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.936+0000 7ff4b4292700 1 --2- 172.21.15.19:0/1172340128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac107df0 0x7ff4ac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.936+0000 7ff4b4292700 1 -- 172.21.15.19:0/1172340128 >> 172.21.15.19:0/1172340128 conn(0x7ff4ac0fedf0 msgr2=0x7ff4ac101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.936+0000 7ff4b4292700 1 -- 172.21.15.19:0/1172340128 shutdown_connections 2024-05-09T01:34:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.936+0000 7ff4b4292700 1 -- 172.21.15.19:0/1172340128 wait complete. 2024-05-09T01:34:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.937+0000 7ff4b4292700 1 Processor -- start 2024-05-09T01:34:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.937+0000 7ff4b4292700 1 -- start start 2024-05-09T01:34:51.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.937+0000 7ff4b4292700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac1034c0 0x7ff4ac19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b4292700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac107df0 0x7ff4ac19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b4292700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac108740 0x7ff4ac19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b4292700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac113370 con 0x7ff4ac108740 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b4292700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1131f0 con 0x7ff4ac107df0 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b4292700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1134f0 con 0x7ff4ac1034c0 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b282f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac108740 0x7ff4ac19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b282f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac108740 0x7ff4ac19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49494/0 (socket says 172.21.15.19:49494) 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b282f700 1 -- 172.21.15.19:0/728337550 learned_addr learned my addr 172.21.15.19:0/728337550 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b202e700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac1034c0 0x7ff4ac19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b182d700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac107df0 0x7ff4ac19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b282f700 1 -- 172.21.15.19:0/728337550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac1034c0 msgr2=0x7ff4ac19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b282f700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac1034c0 0x7ff4ac19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b282f700 1 -- 172.21.15.19:0/728337550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac107df0 msgr2=0x7ff4ac19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b282f700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac107df0 0x7ff4ac19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b282f700 1 -- 172.21.15.19:0/728337550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac1a6f30 con 0x7ff4ac108740 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b202e700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac1034c0 0x7ff4ac19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b182d700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac107df0 0x7ff4ac19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.938+0000 7ff4b282f700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac108740 0x7ff4ac19c710 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400ef30 tx=0x7ff4a400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.939+0000 7ff4a37fe700 1 -- 172.21.15.19:0/728337550 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40092e0 con 0x7ff4ac108740 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.939+0000 7ff4b4292700 1 -- 172.21.15.19:0/728337550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac1a71c0 con 0x7ff4ac108740 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.939+0000 7ff4a37fe700 1 -- 172.21.15.19:0/728337550 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a4004bb0 con 0x7ff4ac108740 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.939+0000 7ff4a37fe700 1 -- 172.21.15.19:0/728337550 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4004280 con 0x7ff4ac108740 2024-05-09T01:34:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.939+0000 7ff4b4292700 1 -- 172.21.15.19:0/728337550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac1a76b0 con 0x7ff4ac108740 2024-05-09T01:34:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.941+0000 7ff4a37fe700 1 -- 172.21.15.19:0/728337550 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff4a401b030 con 0x7ff4ac108740 2024-05-09T01:34:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.942+0000 7ff4a37fe700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4980b2880 0x7ff4980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.942+0000 7ff4b202e700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4980b2880 0x7ff4980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.942+0000 7ff4a37fe700 1 -- 172.21.15.19:0/728337550 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff4a40e1ec0 con 0x7ff4ac108740 2024-05-09T01:34:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.942+0000 7ff4c2ca8b80 1 Processor -- start 2024-05-09T01:34:51.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4c2ca8b80 1 -- start start 2024-05-09T01:34:51.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b202e700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4980b2880 0x7ff4980b4d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8000c00 tx=0x7ff4a801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:51.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x560228179240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x560228179780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228173400 0x560228173810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5602280960a0 con 0x560228070d00 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56022803b9a0 con 0x560227ff64a0 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560228092060 con 0x560228173400 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b202e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x560228179240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b202e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x560228179240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49496/0 (socket says 172.21.15.19:49496) 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b202e700 1 -- 172.21.15.19:0/3762896410 learned_addr learned my addr 172.21.15.19:0/3762896410 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b182d700 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x560228179780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b282f700 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228173400 0x560228173810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b202e700 1 -- 172.21.15.19:0/3762896410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228173400 msgr2=0x560228173810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b202e700 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228173400 0x560228173810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b202e700 1 -- 172.21.15.19:0/3762896410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 msgr2=0x560228179780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b202e700 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x560228179780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b202e700 1 -- 172.21.15.19:0/3762896410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022802c500 con 0x560228070d00 2024-05-09T01:34:51.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.943+0000 7ff4b182d700 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x560228179780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:51.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.944+0000 7ff4b202e700 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x560228179240 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8015000 tx=0x7ff4a801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5157e520fda97893 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:51.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.944+0000 7ff4a0d47700 1 -- 172.21.15.19:0/3762896410 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800b790 con 0x560228070d00 2024-05-09T01:34:51.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.944+0000 7ff4a0d47700 1 -- 172.21.15.19:0/3762896410 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a800bde0 con 0x560228070d00 2024-05-09T01:34:51.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.944+0000 7ff4a0d47700 1 -- 172.21.15.19:0/3762896410 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a802fd70 con 0x560228070d00 2024-05-09T01:34:51.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3762896410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 msgr2=0x560228179240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:51.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x560228179240 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8015000 tx=0x7ff4a801e000 comp rx=0 tx=0).stop 2024-05-09T01:34:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3762896410 shutdown_connections 2024-05-09T01:34:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228173400 0x560228173810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x560228179240 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3762896410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x560228179780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3762896410 >> 172.21.15.19:0/3762896410 conn(0x560227ff9f70 msgr2=0x56022816e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3762896410 shutdown_connections 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3762896410 wait complete. 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.945+0000 7ff4c2ca8b80 1 -- start start 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4c2ca8b80 1 Processor -- start 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560227ff64a0 0x56022817d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228070d00 0x56022817d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560228173400 0x56022817daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560228174100 con 0x560227ff64a0 2024-05-09T01:34:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560228030b20 con 0x560228173400 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560228189200 con 0x560228070d00 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4c2ca8b80 10 client.0 _ll_register_callbacks cb 0x560227f076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4b202e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560227ff64a0 0x56022817d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4b202e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560227ff64a0 0x56022817d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49506/0 (socket says 172.21.15.19:49506) 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4b202e700 1 -- 172.21.15.19:0/4290173877 learned_addr learned my addr 172.21.15.19:0/4290173877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4b182d700 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228070d00 0x56022817d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.946+0000 7ff4b282f700 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560228173400 0x56022817daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4b202e700 1 -- 172.21.15.19:0/4290173877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228070d00 msgr2=0x56022817d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4b202e700 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228070d00 0x56022817d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4b202e700 1 -- 172.21.15.19:0/4290173877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560228173400 msgr2=0x56022817daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4b202e700 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560228173400 0x56022817daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4b202e700 1 -- 172.21.15.19:0/4290173877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560227f11490 con 0x560227ff64a0 2024-05-09T01:34:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4b182d700 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228070d00 0x56022817d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4b282f700 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560228173400 0x56022817daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4b202e700 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560227ff64a0 0x56022817d020 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80053d0 tx=0x7ff4a8005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcb2f3fd5ed3ef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff495ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff495ffb700 1 -- 172.21.15.19:0/4290173877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a802fc00 con 0x560227ff64a0 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff495ffb700 1 -- 172.21.15.19:0/4290173877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff4a802fda0 con 0x560227ff64a0 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/4290173877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a800bde0 con 0x560227ff64a0 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff495ffb700 1 -- 172.21.15.19:0/4290173877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4a800b790 con 0x560227ff64a0 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4c2ca8b80 10 client.17898 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.947+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/4290173877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56022818ab00 con 0x560227ff64a0 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.948+0000 7ff495ffb700 1 -- 172.21.15.19:0/4290173877 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a800bde0 con 0x560227ff64a0 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.948+0000 7ff495ffb700 1 -- 172.21.15.19:0/4290173877 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff4a8022070 con 0x560227ff64a0 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.948+0000 7ff4c2ca8b80 10 client.17898 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.948+0000 7ff4c2ca8b80 10 client.17898 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.948+0000 7ff4c2ca8b80 20 client.17898 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.948+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56022818cfd0 0x56022818f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.948+0000 7ff4c2ca8b80 4 client.17898 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.948+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/4290173877 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602280a3a80 con 0x56022818cfd0 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.948+0000 7ff4b182d700 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56022818cfd0 0x56022818f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.949+0000 7ff495ffb700 10 client.17898 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.949+0000 7ff4b182d700 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56022818cfd0 0x56022818f3f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c5b6816d8b029fa server_cookie=83dd5879e244126c in_seq=0 out_seq=0 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.949+0000 7ff495ffb700 1 -- 172.21.15.19:0/4290173877 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5602280a3a80 con 0x56022818cfd0 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.949+0000 7ff495ffb700 10 client.17898 handle_command_reply: tid=0 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.949+0000 7ff4c2ca8b80 1 client.17898 shutdown 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/4290173877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560227ff64a0 msgr2=0x56022817d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560227ff64a0 0x56022817d020 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80053d0 tx=0x7ff4a8005400 comp rx=0 tx=0).stop 2024-05-09T01:34:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/4290173877 shutdown_connections 2024-05-09T01:34:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x56022818cfd0 0x56022818f3f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560228173400 0x56022817daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560227ff64a0 0x56022817d020 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff495ffb700 0 client.17898 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:34:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/4290173877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560228070d00 0x56022817d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/4290173877 >> 172.21.15.19:0/4290173877 conn(0x560227ff9f70 msgr2=0x560228167ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/4290173877 shutdown_connections 2024-05-09T01:34:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/4290173877 wait complete. 2024-05-09T01:34:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.950+0000 7ff4c2ca8b80 20 client.17898 trim_cache size 0 max 16384 2024-05-09T01:34:51.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.965+0000 7ff4c2ca8b80 1 Processor -- start 2024-05-09T01:34:51.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.965+0000 7ff4c2ca8b80 1 -- start start 2024-05-09T01:34:51.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.965+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 0x560228168a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.965+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x56022817f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.965+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x56022817cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.965+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1aec70 con 0x560228070d00 2024-05-09T01:34:51.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.965+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1b07b0 con 0x560227ff64a0 2024-05-09T01:34:51.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.965+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1af340 con 0x560227ff9f70 2024-05-09T01:34:51.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.966+0000 7ff4b282f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x56022817cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.966+0000 7ff4b282f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x56022817cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49512/0 (socket says 172.21.15.19:49512) 2024-05-09T01:34:51.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.966+0000 7ff4b282f700 1 -- 172.21.15.19:0/3250462809 learned_addr learned my addr 172.21.15.19:0/3250462809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:51.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.966+0000 7ff4b202e700 1 --2- 172.21.15.19:0/3250462809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 0x560228168a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.967+0000 7ff4b182d700 1 --2- 172.21.15.19:0/3250462809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x56022817f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.967+0000 7ff4b282f700 1 -- 172.21.15.19:0/3250462809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 msgr2=0x560228168a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:51.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.967+0000 7ff4b282f700 1 --2- 172.21.15.19:0/3250462809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 0x560228168a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.967+0000 7ff4b282f700 1 -- 172.21.15.19:0/3250462809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 msgr2=0x56022817f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.967+0000 7ff4b282f700 1 --2- 172.21.15.19:0/3250462809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x56022817f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.967+0000 7ff4b282f700 1 -- 172.21.15.19:0/3250462809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022816a9d0 con 0x560228070d00 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.967+0000 7ff4b282f700 1 --2- 172.21.15.19:0/3250462809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x56022817cfe0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40df9f0 tx=0x7ff4a40a9a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac04b1632023ad41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.967+0000 7ff497fff700 1 -- 172.21.15.19:0/3250462809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40c3070 con 0x560228070d00 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.968+0000 7ff497fff700 1 -- 172.21.15.19:0/3250462809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a40a9c50 con 0x560228070d00 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.968+0000 7ff497fff700 1 -- 172.21.15.19:0/3250462809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40dfe60 con 0x560228070d00 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.968+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3250462809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 msgr2=0x56022817cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.968+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3250462809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x56022817cfe0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40df9f0 tx=0x7ff4a40a9a90 comp rx=0 tx=0).stop 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.969+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3250462809 shutdown_connections 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.969+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3250462809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x56022817cfe0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.969+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3250462809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 0x560228168a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.969+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3250462809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x56022817f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.969+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3250462809 >> 172.21.15.19:0/3250462809 conn(0x560228173ae0 msgr2=0x560228178170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.969+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3250462809 shutdown_connections 2024-05-09T01:34:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.969+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3250462809 wait complete. 2024-05-09T01:34:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.969+0000 7ff4c2ca8b80 1 -- start start 2024-05-09T01:34:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.969+0000 7ff4c2ca8b80 1 Processor -- start 2024-05-09T01:34:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x56022818fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 0x56022818ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4c2ca8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x5602281fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1b00e0 con 0x560228070d00 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1afa10 con 0x560227ff64a0 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4c2ca8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1acc10 con 0x560227ff9f70 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4c2ca8b80 10 client.0 _ll_register_callbacks cb 0x560227f076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4b202e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x56022818fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4b282f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x5602281fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4b282f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x5602281fe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49522/0 (socket says 172.21.15.19:49522) 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4b282f700 1 -- 172.21.15.19:0/3143116651 learned_addr learned my addr 172.21.15.19:0/3143116651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.970+0000 7ff4b182d700 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 0x56022818ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff4b182d700 1 -- 172.21.15.19:0/3143116651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 msgr2=0x56022818fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff4b182d700 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x56022818fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff4b182d700 1 -- 172.21.15.19:0/3143116651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 msgr2=0x5602281fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff4b182d700 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x5602281fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff4b182d700 1 -- 172.21.15.19:0/3143116651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560228179c70 con 0x560227ff9f70 2024-05-09T01:34:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff4b282f700 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x5602281fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff4b182d700 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 0x56022818ff50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x56022816fa00 tx=0x7ff49c011e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=434efd86e0926804 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff495ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff495ffb700 1 -- 172.21.15.19:0/3143116651 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c004520 con 0x560227ff9f70 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff495ffb700 1 -- 172.21.15.19:0/3143116651 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff49c010870 con 0x560227ff9f70 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3143116651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a40a9c50 con 0x560227ff9f70 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.971+0000 7ff495ffb700 1 -- 172.21.15.19:0/3143116651 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff49c02c070 con 0x560227ff9f70 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff4c2ca8b80 10 client.27145 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3143116651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602281ff390 con 0x560227ff9f70 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff495ffb700 1 -- 172.21.15.19:0/3143116651 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a40a9c50 con 0x560227ff9f70 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff495ffb700 1 -- 172.21.15.19:0/3143116651 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff49c003680 con 0x560227ff9f70 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff4c2ca8b80 10 client.27145 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff4c2ca8b80 10 client.27145 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff4c2ca8b80 20 client.27145 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x560228173400 0x560228201ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff4c2ca8b80 4 client.27145 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.972+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3143116651 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56022809db20 con 0x560228173400 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.973+0000 7ff4b202e700 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x560228173400 0x560228201ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.973+0000 7ff495ffb700 10 client.27145 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.973+0000 7ff4b202e700 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x560228173400 0x560228201ad0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fa3126ecf9245a47 server_cookie=831238d4b65b7cea in_seq=0 out_seq=0 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff495ffb700 1 -- 172.21.15.19:0/3143116651 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56022809db20 con 0x560228173400 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff495ffb700 10 client.27145 handle_command_reply: tid=0 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 client.27145 shutdown 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3143116651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 msgr2=0x56022818ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 0x56022818ff50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x56022816fa00 tx=0x7ff49c011e60 comp rx=0 tx=0).stop 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3143116651 shutdown_connections 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x560228173400 0x560228201ad0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560228070d00 0x5602281fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560227ff64a0 0x56022818fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 --2- 172.21.15.19:0/3143116651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560227ff9f70 0x56022818ff50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3143116651 >> 172.21.15.19:0/3143116651 conn(0x560228173ae0 msgr2=0x560228178170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:51.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff495ffb700 0 client.27145 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:34:51.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3143116651 shutdown_connections 2024-05-09T01:34:51.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 1 -- 172.21.15.19:0/3143116651 wait complete. 2024-05-09T01:34:51.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.974+0000 7ff4c2ca8b80 20 client.27145 trim_cache size 0 max 16384 2024-05-09T01:34:51.976 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:34:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 -- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4980b2880 msgr2=0x7ff4980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4980b2880 0x7ff4980b4d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8000c00 tx=0x7ff4a801a040 comp rx=0 tx=0).stop 2024-05-09T01:34:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 -- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac108740 msgr2=0x7ff4ac19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac108740 0x7ff4ac19c710 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400ef30 tx=0x7ff4a400c450 comp rx=0 tx=0).stop 2024-05-09T01:34:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 -- 172.21.15.19:0/728337550 shutdown_connections 2024-05-09T01:34:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff4980b2880 0x7ff4980b4d90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4ac108740 0x7ff4ac19c710 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac1034c0 0x7ff4ac19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 --2- 172.21.15.19:0/728337550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac107df0 0x7ff4ac19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 -- 172.21.15.19:0/728337550 >> 172.21.15.19:0/728337550 conn(0x7ff4ac0fedf0 msgr2=0x7ff4ac107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 -- 172.21.15.19:0/728337550 shutdown_connections 2024-05-09T01:34:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:51.975+0000 7ff4b4292700 1 -- 172.21.15.19:0/728337550 wait complete. 2024-05-09T01:34:52.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:52 smithi019 ceph-mon[30158]: pgmap v658: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:52.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:52 smithi196 ceph-mon[35041]: pgmap v658: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:52.743 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:34:52.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:52 smithi059 ceph-mon[34351]: pgmap v658: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:53.004 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 -- 172.21.15.19:0/1880736914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 msgr2=0x7f0ee4103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 --2- 172.21.15.19:0/1880736914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 0x7f0ee4103960 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00099c0 tx=0x7f0ee0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 -- 172.21.15.19:0/1880736914 shutdown_connections 2024-05-09T01:34:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 --2- 172.21.15.19:0/1880736914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ee4108780 0x7f0ee41100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 --2- 172.21.15.19:0/1880736914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 0x7f0ee4103960 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 --2- 172.21.15.19:0/1880736914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ee4107e30 0x7f0ee4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 -- 172.21.15.19:0/1880736914 >> 172.21.15.19:0/1880736914 conn(0x7f0ee4078e00 msgr2=0x7f0ee4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:54.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 -- 172.21.15.19:0/1880736914 shutdown_connections 2024-05-09T01:34:54.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 -- 172.21.15.19:0/1880736914 wait complete. 2024-05-09T01:34:54.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 Processor -- start 2024-05-09T01:34:54.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.651+0000 7f0eeb782700 1 -- start start 2024-05-09T01:34:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0eeb782700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 0x7f0ee419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0eeb782700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ee4107e30 0x7f0ee419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0eeb782700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ee4108780 0x7f0ee419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0eeb782700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee4114550 con 0x7f0ee4103550 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0eeb782700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee41143d0 con 0x7f0ee4108780 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0eeb782700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee41146d0 con 0x7f0ee4107e30 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee951e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 0x7f0ee419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee951e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 0x7f0ee419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49530/0 (socket says 172.21.15.19:49530) 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee951e700 1 -- 172.21.15.19:0/3340204561 learned_addr learned my addr 172.21.15.19:0/3340204561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ee4108780 0x7f0ee419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ee4107e30 0x7f0ee419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee951e700 1 -- 172.21.15.19:0/3340204561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ee4107e30 msgr2=0x7f0ee419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee951e700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ee4107e30 0x7f0ee419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee951e700 1 -- 172.21.15.19:0/3340204561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ee4108780 msgr2=0x7f0ee419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee951e700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ee4108780 0x7f0ee419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee951e700 1 -- 172.21.15.19:0/3340204561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee419fc80 con 0x7f0ee4103550 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ee4107e30 0x7f0ee419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.652+0000 7f0ee951e700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 0x7f0ee419ae60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009990 tx=0x7f0ee00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.653+0000 7f0eda7fc700 1 -- 172.21.15.19:0/3340204561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0027070 con 0x7f0ee4103550 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.653+0000 7f0eda7fc700 1 -- 172.21.15.19:0/3340204561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee0004190 con 0x7f0ee4103550 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.653+0000 7f0eeb782700 1 -- 172.21.15.19:0/3340204561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee419ff10 con 0x7f0ee4103550 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.653+0000 7f0eda7fc700 1 -- 172.21.15.19:0/3340204561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000f460 con 0x7f0ee4103550 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.653+0000 7f0eeb782700 1 -- 172.21.15.19:0/3340204561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee41a0440 con 0x7f0ee4103550 2024-05-09T01:34:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.656+0000 7f0eda7fc700 1 -- 172.21.15.19:0/3340204561 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0ee002a070 con 0x7f0ee4103550 2024-05-09T01:34:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.657+0000 7f0eda7fc700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ed00b2900 0x7f0ed00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.657+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ed00b2900 0x7f0ed00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.657+0000 7f0efa198b80 1 Processor -- start 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.657+0000 7f0eda7fc700 1 -- 172.21.15.19:0/3340204561 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0ee00e33f0 con 0x7f0ee4103550 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.657+0000 7f0efa198b80 1 -- start start 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 0x555f411f8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410754a0 0x555f411f8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 0x555f411f2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0efa198b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555f411150a0 con 0x555f410754a0 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0efa198b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f410ba9a0 con 0x555f411f2400 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0efa198b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f41111060 con 0x555f410efd00 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ed00b2900 0x7f0ed00b4e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4005fd0 tx=0x7f0ed4005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410754a0 0x555f411f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410754a0 0x555f411f8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49540/0 (socket says 172.21.15.19:49540) 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 -- 172.21.15.19:0/561527240 learned_addr learned my addr 172.21.15.19:0/561527240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee951e700 1 --2- 172.21.15.19:0/561527240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 0x555f411f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/561527240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 0x555f411f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 -- 172.21.15.19:0/561527240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 msgr2=0x555f411f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/561527240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 0x555f411f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 -- 172.21.15.19:0/561527240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 msgr2=0x555f411f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/561527240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 0x555f411f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 -- 172.21.15.19:0/561527240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f410ab500 con 0x555f410754a0 2024-05-09T01:34:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.658+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/561527240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410754a0 0x555f411f8780 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400dbd0 tx=0x7f0ed4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b300da1f2ddfaec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.659+0000 7f0ecf7fe700 1 -- 172.21.15.19:0/561527240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4020070 con 0x555f410754a0 2024-05-09T01:34:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.659+0000 7f0ecf7fe700 1 -- 172.21.15.19:0/561527240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed401c450 con 0x555f410754a0 2024-05-09T01:34:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.659+0000 7f0ecf7fe700 1 -- 172.21.15.19:0/561527240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400fe60 con 0x555f410754a0 2024-05-09T01:34:54.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 -- 172.21.15.19:0/561527240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410754a0 msgr2=0x555f411f8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:54.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 --2- 172.21.15.19:0/561527240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410754a0 0x555f411f8780 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400dbd0 tx=0x7f0ed4010000 comp rx=0 tx=0).stop 2024-05-09T01:34:54.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 -- 172.21.15.19:0/561527240 shutdown_connections 2024-05-09T01:34:54.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 --2- 172.21.15.19:0/561527240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 0x555f411f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 --2- 172.21.15.19:0/561527240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 0x555f411f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 --2- 172.21.15.19:0/561527240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410754a0 0x555f411f8780 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 -- 172.21.15.19:0/561527240 >> 172.21.15.19:0/561527240 conn(0x555f41078f70 msgr2=0x555f411ed220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 -- 172.21.15.19:0/561527240 shutdown_connections 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 -- 172.21.15.19:0/561527240 wait complete. 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 -- start start 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.660+0000 7f0efa198b80 1 Processor -- start 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410754a0 0x555f411fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 0x555f411fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f411fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0efa198b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555f411f3160 con 0x555f411f2400 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0efa198b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f410afb20 con 0x555f410754a0 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0efa198b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f411ff020 con 0x555f410efd00 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0efa198b80 10 client.0 _ll_register_callbacks cb 0x555f40f866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee9d1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f411fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee9d1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f411fc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49554/0 (socket says 172.21.15.19:49554) 2024-05-09T01:34:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee9d1f700 1 -- 172.21.15.19:0/1219461095 learned_addr learned my addr 172.21.15.19:0/1219461095 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee951e700 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410754a0 0x555f411fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 0x555f411fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee9d1f700 1 -- 172.21.15.19:0/1219461095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 msgr2=0x555f411fc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 0x555f411fc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee9d1f700 1 -- 172.21.15.19:0/1219461095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410754a0 msgr2=0x555f411fbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410754a0 0x555f411fbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee9d1f700 1 -- 172.21.15.19:0/1219461095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f40f90490 con 0x555f411f2400 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 0x555f411fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.661+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f411fc9e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0edc008f20 tx=0x7f0edc008f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adc1e2f8714be16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0eccff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0eccff9700 1 -- 172.21.15.19:0/1219461095 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc010c20 con 0x555f411f2400 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0eccff9700 1 -- 172.21.15.19:0/1219461095 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0edc010dc0 con 0x555f411f2400 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0efa198b80 1 -- 172.21.15.19:0/1219461095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed401c450 con 0x555f411f2400 2024-05-09T01:34:54.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0eccff9700 1 -- 172.21.15.19:0/1219461095 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0edc016470 con 0x555f411f2400 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0efa198b80 10 client.17916 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0efa198b80 1 -- 172.21.15.19:0/1219461095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f41209990 con 0x555f411f2400 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0eccff9700 1 -- 172.21.15.19:0/1219461095 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed401c450 con 0x555f411f2400 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0eccff9700 1 -- 172.21.15.19:0/1219461095 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0edc01f6d0 con 0x555f411f2400 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0efa198b80 10 client.17916 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0efa198b80 10 client.17916 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0efa198b80 20 client.17916 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f4120bfd0 0x555f4120e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0efa198b80 4 client.17916 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.662+0000 7f0efa198b80 1 -- 172.21.15.19:0/1219461095 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f41122a80 con 0x555f4120bfd0 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.663+0000 7f0ee951e700 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f4120bfd0 0x555f4120e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.663+0000 7f0ee951e700 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f4120bfd0 0x555f4120e440 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee2e00126359434c server_cookie=76044200cb035897 in_seq=0 out_seq=0 2024-05-09T01:34:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.663+0000 7f0eccff9700 10 client.17916 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:34:54.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.664+0000 7f0eccff9700 1 -- 172.21.15.19:0/1219461095 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f41122a80 con 0x555f4120bfd0 2024-05-09T01:34:54.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.664+0000 7f0eccff9700 10 client.17916 handle_command_reply: tid=0 2024-05-09T01:34:54.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.664+0000 7f0efa198b80 1 client.17916 shutdown 2024-05-09T01:34:54.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:54 smithi019 ceph-mon[30158]: pgmap v659: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:34:54.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.664+0000 7f0efa198b80 1 -- 172.21.15.19:0/1219461095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 msgr2=0x555f411fc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:54.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.664+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f411fc9e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0edc008f20 tx=0x7f0edc008f50 comp rx=0 tx=0).stop 2024-05-09T01:34:54.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.664+0000 7f0efa198b80 1 -- 172.21.15.19:0/1219461095 shutdown_connections 2024-05-09T01:34:54.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.664+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f411fc9e0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.664+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f4120bfd0 0x555f4120e440 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.664+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410754a0 0x555f411fbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.665+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1219461095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410efd00 0x555f411fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.665+0000 7f0efa198b80 1 -- 172.21.15.19:0/1219461095 >> 172.21.15.19:0/1219461095 conn(0x555f41078f70 msgr2=0x555f411e6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.665+0000 7f0eccff9700 0 client.17916 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:34:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.665+0000 7f0efa198b80 1 -- 172.21.15.19:0/1219461095 shutdown_connections 2024-05-09T01:34:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.665+0000 7f0efa198b80 1 -- 172.21.15.19:0/1219461095 wait complete. 2024-05-09T01:34:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.665+0000 7f0efa198b80 20 client.17916 trim_cache size 0 max 16384 2024-05-09T01:34:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.680+0000 7f0efa198b80 1 Processor -- start 2024-05-09T01:34:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.680+0000 7f0efa198b80 1 -- start start 2024-05-09T01:34:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.680+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 0x555f411fb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.680+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410efd00 0x555f411fc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.680+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f41207e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.680+0000 7f0efa198b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee41aeae0 con 0x555f411f2400 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.680+0000 7f0efa198b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee41b0620 con 0x555f410efd00 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.680+0000 7f0efa198b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee41af1b0 con 0x555f410754a0 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.680+0000 7f0ee9d1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f41207e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee9d1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f41207e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49568/0 (socket says 172.21.15.19:49568) 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee9d1f700 1 -- 172.21.15.19:0/2700897043 learned_addr learned my addr 172.21.15.19:0/2700897043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee951e700 1 --2- 172.21.15.19:0/2700897043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 0x555f411fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/2700897043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410efd00 0x555f411fc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee9d1f700 1 -- 172.21.15.19:0/2700897043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 msgr2=0x555f411fb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/2700897043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 0x555f411fb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee9d1f700 1 -- 172.21.15.19:0/2700897043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410efd00 msgr2=0x555f411fc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/2700897043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410efd00 0x555f411fc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee9d1f700 1 -- 172.21.15.19:0/2700897043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f411fe800 con 0x555f411f2400 2024-05-09T01:34:54.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/2700897043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f41207e60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0edc01fd30 tx=0x7f0edc007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2400a9ccbf42c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:54.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0eceffd700 1 -- 172.21.15.19:0/2700897043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc02f070 con 0x555f411f2400 2024-05-09T01:34:54.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0eceffd700 1 -- 172.21.15.19:0/2700897043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0edc01b070 con 0x555f411f2400 2024-05-09T01:34:54.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.681+0000 7f0eceffd700 1 -- 172.21.15.19:0/2700897043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc01f890 con 0x555f411f2400 2024-05-09T01:34:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.682+0000 7f0efa198b80 1 -- 172.21.15.19:0/2700897043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 msgr2=0x555f41207e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.682+0000 7f0efa198b80 1 --2- 172.21.15.19:0/2700897043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f41207e60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0edc01fd30 tx=0x7f0edc007b70 comp rx=0 tx=0).stop 2024-05-09T01:34:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.682+0000 7f0efa198b80 1 -- 172.21.15.19:0/2700897043 shutdown_connections 2024-05-09T01:34:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.682+0000 7f0efa198b80 1 --2- 172.21.15.19:0/2700897043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f411f2400 0x555f41207e60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.682+0000 7f0efa198b80 1 --2- 172.21.15.19:0/2700897043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 0x555f411fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.682+0000 7f0efa198b80 1 --2- 172.21.15.19:0/2700897043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f410efd00 0x555f411fc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.682+0000 7f0efa198b80 1 -- 172.21.15.19:0/2700897043 >> 172.21.15.19:0/2700897043 conn(0x555f41078f70 msgr2=0x555f411f7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.682+0000 7f0efa198b80 1 -- 172.21.15.19:0/2700897043 shutdown_connections 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.682+0000 7f0efa198b80 1 -- 172.21.15.19:0/2700897043 wait complete. 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0efa198b80 1 -- start start 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0efa198b80 1 Processor -- start 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 0x555f412041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410efd00 0x555f412046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0efa198b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 0x555f41212c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0efa198b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee41aff50 con 0x555f410efd00 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0efa198b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee41af880 con 0x555f411f2400 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0efa198b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee41aca80 con 0x555f410754a0 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0efa198b80 10 client.0 _ll_register_callbacks cb 0x555f40f866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0ee8d1d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410efd00 0x555f412046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0ee8d1d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410efd00 0x555f412046e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49584/0 (socket says 172.21.15.19:49584) 2024-05-09T01:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0ee8d1d700 1 -- 172.21.15.19:0/1794382139 learned_addr learned my addr 172.21.15.19:0/1794382139 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 0x555f41212c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.683+0000 7f0ee951e700 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 0x555f412041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0ee8d1d700 1 -- 172.21.15.19:0/1794382139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 msgr2=0x555f412041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 0x555f412041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0ee8d1d700 1 -- 172.21.15.19:0/1794382139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 msgr2=0x555f41212c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 0x555f41212c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0ee8d1d700 1 -- 172.21.15.19:0/1794382139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f41213180 con 0x555f410efd00 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0ee9d1f700 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 0x555f41212c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0ee8d1d700 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410efd00 0x555f412046e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400dfd0 tx=0x7f0ed400fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae39d4d9f4b746 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0eccff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0eccff9700 1 -- 172.21.15.19:0/1794382139 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4020040 con 0x555f410efd00 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0eccff9700 1 -- 172.21.15.19:0/1794382139 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0ed401c650 con 0x555f410efd00 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0efa198b80 1 -- 172.21.15.19:0/1794382139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0edc01b070 con 0x555f410efd00 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0eccff9700 1 -- 172.21.15.19:0/1794382139 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0ed400fe40 con 0x555f410efd00 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0efa198b80 10 client.17922 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0efa198b80 1 -- 172.21.15.19:0/1794382139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f41275ee0 con 0x555f410efd00 2024-05-09T01:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.684+0000 7f0eccff9700 1 -- 172.21.15.19:0/1794382139 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0edc01b070 con 0x555f410efd00 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0eccff9700 1 -- 172.21.15.19:0/1794382139 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0ed4031480 con 0x555f410efd00 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0efa198b80 10 client.17922 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0efa198b80 10 client.17922 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0efa198b80 20 client.17922 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f41276350 0x555f41278770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0efa198b80 4 client.17922 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0efa198b80 1 -- 172.21.15.19:0/1794382139 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f4111cb20 con 0x555f41276350 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0ee951e700 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f41276350 0x555f41278770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0eccff9700 10 client.17922 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.685+0000 7f0ee951e700 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f41276350 0x555f41278770 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76ce08c1c60b9030 server_cookie=3fa359ed7c3d4bba in_seq=0 out_seq=0 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0eccff9700 1 -- 172.21.15.19:0/1794382139 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f4111cb20 con 0x555f41276350 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0eccff9700 10 client.17922 handle_command_reply: tid=0 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 client.17922 shutdown 2024-05-09T01:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 -- 172.21.15.19:0/1794382139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410efd00 msgr2=0x555f412046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410efd00 0x555f412046e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400dfd0 tx=0x7f0ed400fa90 comp rx=0 tx=0).stop 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 -- 172.21.15.19:0/1794382139 shutdown_connections 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x555f41276350 0x555f41278770 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f411f2400 0x555f41212c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f410754a0 0x555f412041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 --2- 172.21.15.19:0/1794382139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f410efd00 0x555f412046e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 -- 172.21.15.19:0/1794382139 >> 172.21.15.19:0/1794382139 conn(0x555f41078f70 msgr2=0x555f411f5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0eccff9700 0 client.17922 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 -- 172.21.15.19:0/1794382139 shutdown_connections 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 1 -- 172.21.15.19:0/1794382139 wait complete. 2024-05-09T01:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.686+0000 7f0efa198b80 20 client.17922 trim_cache size 0 max 16384 2024-05-09T01:34:54.689 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 -- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ed00b2900 msgr2=0x7f0ed00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ed00b2900 0x7f0ed00b4e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4005fd0 tx=0x7f0ed4005e20 comp rx=0 tx=0).stop 2024-05-09T01:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 -- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 msgr2=0x7f0ee419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 0x7f0ee419ae60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009990 tx=0x7f0ee00047b0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 -- 172.21.15.19:0/3340204561 shutdown_connections 2024-05-09T01:34:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ee4108780 0x7f0ee419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0ed00b2900 0x7f0ed00b4e10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee4103550 0x7f0ee419ae60 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 --2- 172.21.15.19:0/3340204561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ee4107e30 0x7f0ee419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 -- 172.21.15.19:0/3340204561 >> 172.21.15.19:0/3340204561 conn(0x7f0ee4078e00 msgr2=0x7f0ee4100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 -- 172.21.15.19:0/3340204561 shutdown_connections 2024-05-09T01:34:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:54.687+0000 7f0eeb782700 1 -- 172.21.15.19:0/3340204561 wait complete. 2024-05-09T01:34:54.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:54 smithi196 ceph-mon[35041]: pgmap v659: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:34:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:54 smithi059 ceph-mon[34351]: pgmap v659: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:34:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.065+0000 7f201c03b640 1 -- 172.21.15.19:0/2664807772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014102b50 msgr2=0x7f2014103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.065+0000 7f201c03b640 1 --2- 172.21.15.19:0/2664807772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014102b50 0x7f2014103850 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f2010009900 tx=0x7f201002a410 comp rx=0 tx=0).stop 2024-05-09T01:34:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.065+0000 7f201c03b640 1 -- 172.21.15.19:0/2664807772 shutdown_connections 2024-05-09T01:34:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.065+0000 7f201c03b640 1 --2- 172.21.15.19:0/2664807772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2014108a90 0x7f201410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.065+0000 7f201c03b640 1 --2- 172.21.15.19:0/2664807772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2014103d90 0x7f20141041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.065+0000 7f201c03b640 1 --2- 172.21.15.19:0/2664807772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014102b50 0x7f2014103850 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.065+0000 7f201c03b640 1 -- 172.21.15.19:0/2664807772 >> 172.21.15.19:0/2664807772 conn(0x7f20140fdd40 msgr2=0x7f20141001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.066+0000 7f201c03b640 1 -- 172.21.15.19:0/2664807772 shutdown_connections 2024-05-09T01:34:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.066+0000 7f201c03b640 1 -- 172.21.15.19:0/2664807772 wait complete. 2024-05-09T01:34:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.066+0000 7f201c03b640 1 Processor -- start 2024-05-09T01:34:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.066+0000 7f201c03b640 1 -- start start 2024-05-09T01:34:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.066+0000 7f201c03b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2014102b50 0x7f201406af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f201c03b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014103d90 0x7f201419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f201c03b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2014108a90 0x7f20141a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f201c03b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f201410e630 con 0x7f2014103d90 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f201c03b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f201410e4b0 con 0x7f2014102b50 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f201c03b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f201410e7b0 con 0x7f2014108a90 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f20195d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014103d90 0x7f201419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f20195d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014103d90 0x7f201419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49598/0 (socket says 172.21.15.19:49598) 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f20195d6640 1 -- 172.21.15.19:0/3009975512 learned_addr learned my addr 172.21.15.19:0/3009975512 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f2019dd7640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2014102b50 0x7f201406af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f201a5d8640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2014108a90 0x7f20141a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f20195d6640 1 -- 172.21.15.19:0/3009975512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2014108a90 msgr2=0x7f20141a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f20195d6640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2014108a90 0x7f20141a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f20195d6640 1 -- 172.21.15.19:0/3009975512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2014102b50 msgr2=0x7f201406af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f20195d6640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2014102b50 0x7f201406af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f20195d6640 1 -- 172.21.15.19:0/3009975512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20141a1ea0 con 0x7f2014103d90 2024-05-09T01:34:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f20195d6640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014103d90 0x7f201419f130 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f200400d950 tx=0x7f200400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f200affd640 1 -- 172.21.15.19:0/3009975512 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004017070 con 0x7f2014103d90 2024-05-09T01:34:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f201c03b640 1 -- 172.21.15.19:0/3009975512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20141a2190 con 0x7f2014103d90 2024-05-09T01:34:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f200affd640 1 -- 172.21.15.19:0/3009975512 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2004009c30 con 0x7f2014103d90 2024-05-09T01:34:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f200affd640 1 -- 172.21.15.19:0/3009975512 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200400f620 con 0x7f2014103d90 2024-05-09T01:34:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.067+0000 7f201c03b640 1 -- 172.21.15.19:0/3009975512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20141a26e0 con 0x7f2014103d90 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.070+0000 7f2022fc9740 1 Processor -- start 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.070+0000 7f2022fc9740 1 -- start start 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.070+0000 7f200affd640 1 -- 172.21.15.19:0/3009975512 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2004010460 con 0x7f2014103d90 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.070+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 0x55ea96a8a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.070+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 0x55ea96a8aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.070+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966c54d0 0x55ea96a91980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.070+0000 7f2022fc9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ea969ee7c0 con 0x55ea966c54d0 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.070+0000 7f2022fc9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ea9698a0e0 con 0x55ea966bd2f0 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.070+0000 7f2022fc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea969ea780 con 0x55ea966e61b0 2024-05-09T01:34:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f20195d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966c54d0 0x55ea96a91980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f20195d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966c54d0 0x55ea96a91980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49612/0 (socket says 172.21.15.19:49612) 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f20195d6640 1 -- 172.21.15.19:0/3790061629 learned_addr learned my addr 172.21.15.19:0/3790061629 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f2019dd7640 1 --2- 172.21.15.19:0/3790061629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 0x55ea96a8aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f201a5d8640 1 --2- 172.21.15.19:0/3790061629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 0x55ea96a8a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f20195d6640 1 -- 172.21.15.19:0/3790061629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 msgr2=0x55ea96a8a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f20195d6640 1 --2- 172.21.15.19:0/3790061629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 0x55ea96a8a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f20195d6640 1 -- 172.21.15.19:0/3790061629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 msgr2=0x55ea96a8aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f20195d6640 1 --2- 172.21.15.19:0/3790061629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 0x55ea96a8aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f20195d6640 1 -- 172.21.15.19:0/3790061629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea969b37c0 con 0x55ea966c54d0 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f200affd640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f20000b27e0 0x7f20000b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f20195d6640 1 --2- 172.21.15.19:0/3790061629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966c54d0 0x55ea96a91980 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f20040e5f70 tx=0x7f20040ea570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c71c667b16eea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f201a5d8640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f20000b27e0 0x7f20000b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f200affd640 1 -- 172.21.15.19:0/3009975512 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f20040e1e60 con 0x7f2014103d90 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.071+0000 7f1ffffff640 1 -- 172.21.15.19:0/3790061629 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20040a5320 con 0x55ea966c54d0 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f1ffffff640 1 -- 172.21.15.19:0/3790061629 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20040a8f70 con 0x55ea966c54d0 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f1ffffff640 1 -- 172.21.15.19:0/3790061629 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20040a8bd0 con 0x55ea966c54d0 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f2022fc9740 1 -- 172.21.15.19:0/3790061629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966c54d0 msgr2=0x55ea96a91980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f201a5d8640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f20000b27e0 0x7f20000b4cf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55ea96a8b6d0 tx=0x7f200c009620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3790061629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966c54d0 0x55ea96a91980 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f20040e5f70 tx=0x7f20040ea570 comp rx=0 tx=0).stop 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f2022fc9740 1 -- 172.21.15.19:0/3790061629 shutdown_connections 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3790061629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966c54d0 0x55ea96a91980 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3790061629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 0x55ea96a8aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3790061629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 0x55ea96a8a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.072+0000 7f2022fc9740 1 -- 172.21.15.19:0/3790061629 >> 172.21.15.19:0/3790061629 conn(0x55ea966da300 msgr2=0x55ea96a85e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.073+0000 7f2022fc9740 1 -- 172.21.15.19:0/3790061629 shutdown_connections 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.073+0000 7f2022fc9740 1 -- 172.21.15.19:0/3790061629 wait complete. 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.073+0000 7f2022fc9740 1 -- start start 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.073+0000 7f2022fc9740 1 Processor -- start 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.073+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966bd2f0 0x55ea96a94260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.073+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 0x55ea96a947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.073+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.073+0000 7f2022fc9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ea96a923d0 con 0x55ea966e61b0 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f2022fc9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ea969759c0 con 0x55ea966c54d0 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f2022fc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea96a92550 con 0x55ea966bd2f0 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f2022fc9740 10 client.0 _ll_register_callbacks cb 0x55ea969a2020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f201a5d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f201a5d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49616/0 (socket says 172.21.15.19:49616) 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f201a5d8640 1 -- 172.21.15.19:0/2093421101 learned_addr learned my addr 172.21.15.19:0/2093421101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f20195d6640 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 0x55ea96a947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f2019dd7640 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966bd2f0 0x55ea96a94260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f201a5d8640 1 -- 172.21.15.19:0/2093421101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966bd2f0 msgr2=0x55ea96a94260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f201a5d8640 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966bd2f0 0x55ea96a94260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f201a5d8640 1 -- 172.21.15.19:0/2093421101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 msgr2=0x55ea96a947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f201a5d8640 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 0x55ea96a947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f201a5d8640 1 -- 172.21.15.19:0/2093421101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9699c1f0 con 0x55ea966e61b0 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f201a5d8640 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9b700 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f200c012870 tx=0x7f200c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8681491809c8be73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f1ffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/2093421101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c00fae0 con 0x55ea966e61b0 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/2093421101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f200c00fc80 con 0x55ea966e61b0 2024-05-09T01:34:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.074+0000 7f2022fc9740 1 -- 172.21.15.19:0/2093421101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20040a8f70 con 0x55ea966e61b0 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/2093421101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f200c01f420 con 0x55ea966e61b0 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f2022fc9740 10 client.17934 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f2022fc9740 1 -- 172.21.15.19:0/2093421101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea96751790 con 0x55ea966e61b0 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/2093421101 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20040a8f70 con 0x55ea966e61b0 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/2093421101 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f200c01d570 con 0x55ea966e61b0 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f2022fc9740 10 client.17934 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f2022fc9740 10 client.17934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f2022fc9740 20 client.17934 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f2022fc9740 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ea966a4d80 0x55ea96aa1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f2022fc9740 4 client.17934 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f2022fc9740 1 -- 172.21.15.19:0/2093421101 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea969fc3d0 con 0x55ea966a4d80 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.075+0000 7f2019dd7640 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ea966a4d80 0x55ea96aa1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.076+0000 7f1ffd7fa640 10 client.17934 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.076+0000 7f2019dd7640 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ea966a4d80 0x55ea96aa1c40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93d67b857d376847 server_cookie=df16b6ba1c15b6bc in_seq=0 out_seq=0 2024-05-09T01:34:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/2093421101 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea969fc3d0 con 0x55ea966a4d80 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f1ffd7fa640 10 client.17934 handle_command_reply: tid=0 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 client.17934 shutdown 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 -- 172.21.15.19:0/2093421101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 msgr2=0x55ea96a9b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9b700 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f200c012870 tx=0x7f200c004a40 comp rx=0 tx=0).stop 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 -- 172.21.15.19:0/2093421101 shutdown_connections 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ea966a4d80 0x55ea96aa1c40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9b700 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 0x55ea96a947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 --2- 172.21.15.19:0/2093421101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966bd2f0 0x55ea96a94260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f1ffd7fa640 0 client.17934 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 -- 172.21.15.19:0/2093421101 >> 172.21.15.19:0/2093421101 conn(0x55ea966da300 msgr2=0x55ea96a88da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 -- 172.21.15.19:0/2093421101 shutdown_connections 2024-05-09T01:34:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 1 -- 172.21.15.19:0/2093421101 wait complete. 2024-05-09T01:34:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.077+0000 7f2022fc9740 20 client.17934 trim_cache size 0 max 16384 2024-05-09T01:34:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2022fc9740 1 Processor -- start 2024-05-09T01:34:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2022fc9740 1 -- start start 2024-05-09T01:34:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966bd2f0 0x55ea96a928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 0x55ea96a92e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 0x55ea96a8e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2022fc9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20141ab520 con 0x55ea966bd2f0 2024-05-09T01:34:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2022fc9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20141ad060 con 0x55ea966c54d0 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2022fc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20141abbf0 con 0x55ea966e61b0 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2019dd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966bd2f0 0x55ea96a928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2019dd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966bd2f0 0x55ea96a928f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49624/0 (socket says 172.21.15.19:49624) 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f2019dd7640 1 -- 172.21.15.19:0/4150037251 learned_addr learned my addr 172.21.15.19:0/4150037251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.092+0000 7f20195d6640 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 0x55ea96a92e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f201a5d8640 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 0x55ea96a8e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f2019dd7640 1 -- 172.21.15.19:0/4150037251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 msgr2=0x55ea96a8e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f2019dd7640 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 0x55ea96a8e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f2019dd7640 1 -- 172.21.15.19:0/4150037251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 msgr2=0x55ea96a92e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f2019dd7640 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 0x55ea96a92e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f2019dd7640 1 -- 172.21.15.19:0/4150037251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea967c85f0 con 0x55ea966bd2f0 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f201a5d8640 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 0x55ea96a8e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f20195d6640 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 0x55ea96a92e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f2019dd7640 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966bd2f0 0x55ea96a928f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2010009900 tx=0x7f2010000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a510072d6a50d082 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f1fff7fe640 1 -- 172.21.15.19:0/4150037251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201004a020 con 0x55ea966bd2f0 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f1fff7fe640 1 -- 172.21.15.19:0/4150037251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20100463b0 con 0x55ea966bd2f0 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.093+0000 7f1fff7fe640 1 -- 172.21.15.19:0/4150037251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010035040 con 0x55ea966bd2f0 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 -- 172.21.15.19:0/4150037251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966bd2f0 msgr2=0x55ea96a928f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966bd2f0 0x55ea96a928f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2010009900 tx=0x7f2010000bc0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 -- 172.21.15.19:0/4150037251 shutdown_connections 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966e61b0 0x55ea96a8e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966c54d0 0x55ea96a92e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 --2- 172.21.15.19:0/4150037251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966bd2f0 0x55ea96a928f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 -- 172.21.15.19:0/4150037251 >> 172.21.15.19:0/4150037251 conn(0x55ea966da300 msgr2=0x55ea96a8d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 -- 172.21.15.19:0/4150037251 shutdown_connections 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 -- 172.21.15.19:0/4150037251 wait complete. 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 -- start start 2024-05-09T01:34:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.094+0000 7f2022fc9740 1 Processor -- start 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 0x55ea96a98ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966c54d0 0x55ea96a99530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f2022fc9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f2022fc9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20141ac990 con 0x55ea966e61b0 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f2022fc9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20141ac2c0 con 0x55ea966bd2f0 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f2022fc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20141a9500 con 0x55ea966c54d0 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f2022fc9740 10 client.0 _ll_register_callbacks cb 0x7f200c009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f201a5d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f201a5d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49634/0 (socket says 172.21.15.19:49634) 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f201a5d8640 1 -- 172.21.15.19:0/3976251595 learned_addr learned my addr 172.21.15.19:0/3976251595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f2019dd7640 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 0x55ea96a98ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f20195d6640 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966c54d0 0x55ea96a99530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f201a5d8640 1 -- 172.21.15.19:0/3976251595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966c54d0 msgr2=0x55ea96a99530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f201a5d8640 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966c54d0 0x55ea96a99530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f201a5d8640 1 -- 172.21.15.19:0/3976251595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 msgr2=0x55ea96a98ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f201a5d8640 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 0x55ea96a98ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f201a5d8640 1 -- 172.21.15.19:0/3976251595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea969b2ed0 con 0x55ea966e61b0 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f20195d6640 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966c54d0 0x55ea96a99530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f201a5d8640 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9a460 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f200c000f80 tx=0x7f200c004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435b046d8b949bd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.095+0000 7f1ffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/3976251595 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c00f730 con 0x55ea966e61b0 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/3976251595 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f200c00f8d0 con 0x55ea966e61b0 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f2022fc9740 1 -- 172.21.15.19:0/3976251595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20100463b0 con 0x55ea966e61b0 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/3976251595 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f200c00fad0 con 0x55ea966e61b0 2024-05-09T01:34:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f2022fc9740 10 client.17940 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f2022fc9740 1 -- 172.21.15.19:0/3976251595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea96a9ba50 con 0x55ea966e61b0 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/3976251595 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20100463b0 con 0x55ea966e61b0 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/3976251595 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f200c005040 con 0x55ea966e61b0 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f2022fc9740 10 client.17940 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f2022fc9740 10 client.17940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f2022fc9740 20 client.17940 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ea9667a1c0 0x55ea96b0c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f2022fc9740 4 client.17940 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.096+0000 7f2022fc9740 1 -- 172.21.15.19:0/3976251595 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea969f6240 con 0x55ea9667a1c0 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.097+0000 7f2019dd7640 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ea9667a1c0 0x55ea96b0c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.097+0000 7f2019dd7640 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ea9667a1c0 0x55ea96b0c0d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5efb0bc5eb99215 server_cookie=387f3b1cf007aaa2 in_seq=0 out_seq=0 2024-05-09T01:34:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.097+0000 7f1ffd7fa640 10 client.17940 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f1ffd7fa640 1 -- 172.21.15.19:0/3976251595 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea969f6240 con 0x55ea9667a1c0 2024-05-09T01:34:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f1ffd7fa640 10 client.17940 handle_command_reply: tid=0 2024-05-09T01:34:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 client.17940 shutdown 2024-05-09T01:34:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 -- 172.21.15.19:0/3976251595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 msgr2=0x55ea96a9a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9a460 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f200c000f80 tx=0x7f200c004ea0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 -- 172.21.15.19:0/3976251595 shutdown_connections 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55ea9667a1c0 0x55ea96b0c0d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea966e61b0 0x55ea96a9a460 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea966c54d0 0x55ea96a99530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f1ffd7fa640 0 client.17940 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 --2- 172.21.15.19:0/3976251595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea966bd2f0 0x55ea96a98ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 -- 172.21.15.19:0/3976251595 >> 172.21.15.19:0/3976251595 conn(0x55ea966da300 msgr2=0x55ea96a8d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 -- 172.21.15.19:0/3976251595 shutdown_connections 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 1 -- 172.21.15.19:0/3976251595 wait complete. 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.113+0000 7f2022fc9740 20 client.17940 trim_cache size 0 max 16384 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:34:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:34:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:34:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 -- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f20000b27e0 msgr2=0x7f20000b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f20000b27e0 0x7f20000b4cf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55ea96a8b6d0 tx=0x7f200c009620 comp rx=0 tx=0).stop 2024-05-09T01:34:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 -- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014103d90 msgr2=0x7f201419f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014103d90 0x7f201419f130 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f200400d950 tx=0x7f200400de10 comp rx=0 tx=0).stop 2024-05-09T01:34:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 -- 172.21.15.19:0/3009975512 shutdown_connections 2024-05-09T01:34:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f20000b27e0 0x7f20000b4cf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2014108a90 0x7f20141a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2014103d90 0x7f201419f130 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 --2- 172.21.15.19:0/3009975512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2014102b50 0x7f201406af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 -- 172.21.15.19:0/3009975512 >> 172.21.15.19:0/3009975512 conn(0x7f20140fdd40 msgr2=0x7f2014104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 -- 172.21.15.19:0/3009975512 shutdown_connections 2024-05-09T01:34:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:34:55.114+0000 7f201c03b640 1 -- 172.21.15.19:0/3009975512 wait complete. 2024-05-09T01:34:55.494 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:34:55.746 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:56.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:56 smithi196 ceph-mon[35041]: pgmap v660: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:56.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:56 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:34:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:56 smithi059 ceph-mon[34351]: pgmap v660: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:56 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:34:56.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:56 smithi019 ceph-mon[30158]: pgmap v660: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:34:56.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:56 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:34:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.376+0000 7ff79ffa1700 1 -- 172.21.15.19:0/3668991870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7981045e0 msgr2=0x7ff7981049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.376+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/3668991870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7981045e0 0x7ff7981049f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff7940099c0 tx=0x7ff794009cd0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.376+0000 7ff79ffa1700 1 -- 172.21.15.19:0/3668991870 shutdown_connections 2024-05-09T01:34:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.376+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/3668991870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff798109860 0x7ff7981111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.376+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/3668991870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7981045e0 0x7ff7981049f0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.376+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/3668991870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff798108f10 0x7ff798109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.376+0000 7ff79ffa1700 1 -- 172.21.15.19:0/3668991870 >> 172.21.15.19:0/3668991870 conn(0x7ff7980ffe70 msgr2=0x7ff7981022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.376+0000 7ff79ffa1700 1 -- 172.21.15.19:0/3668991870 shutdown_connections 2024-05-09T01:34:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.376+0000 7ff79ffa1700 1 -- 172.21.15.19:0/3668991870 wait complete. 2024-05-09T01:34:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.377+0000 7ff79ffa1700 1 Processor -- start 2024-05-09T01:34:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.377+0000 7ff79ffa1700 1 -- start start 2024-05-09T01:34:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79ffa1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7981045e0 0x7ff79819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79ffa1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff798108f10 0x7ff79819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79ffa1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff798109860 0x7ff79819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79ffa1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981144e0 con 0x7ff798108f10 2024-05-09T01:34:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79ffa1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff798114360 con 0x7ff7981045e0 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79ffa1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff798114660 con 0x7ff798109860 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79e53e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff798109860 0x7ff79819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79d53c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff798108f10 0x7ff79819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79e53e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff798109860 0x7ff79819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35122/0 (socket says 172.21.15.19:35122) 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79e53e700 1 -- 172.21.15.19:0/85654916 learned_addr learned my addr 172.21.15.19:0/85654916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.378+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7981045e0 0x7ff79819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff79e53e700 1 -- 172.21.15.19:0/85654916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7981045e0 msgr2=0x7ff79819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff79e53e700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7981045e0 0x7ff79819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff79e53e700 1 -- 172.21.15.19:0/85654916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff798108f10 msgr2=0x7ff79819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff79e53e700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff798108f10 0x7ff79819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff79e53e700 1 -- 172.21.15.19:0/85654916 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79819fd40 con 0x7ff798109860 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff79e53e700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff798109860 0x7ff79819f6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff79000ef30 tx=0x7ff79000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff78affd700 1 -- 172.21.15.19:0/85654916 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900092e0 con 0x7ff798109860 2024-05-09T01:34:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff78affd700 1 -- 172.21.15.19:0/85654916 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff790004bb0 con 0x7ff798109860 2024-05-09T01:34:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff79ffa1700 1 -- 172.21.15.19:0/85654916 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79819ffd0 con 0x7ff798109860 2024-05-09T01:34:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff78affd700 1 -- 172.21.15.19:0/85654916 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790004020 con 0x7ff798109860 2024-05-09T01:34:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.379+0000 7ff79ffa1700 1 -- 172.21.15.19:0/85654916 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7981a7950 con 0x7ff798109860 2024-05-09T01:34:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.382+0000 7ff78affd700 1 -- 172.21.15.19:0/85654916 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff79001b030 con 0x7ff798109860 2024-05-09T01:34:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.383+0000 7ff78affd700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7840b2900 0x7ff7840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.383+0000 7ff78affd700 1 -- 172.21.15.19:0/85654916 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff7900e1c40 con 0x7ff798109860 2024-05-09T01:34:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.383+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7840b2900 0x7ff7840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.383+0000 7ff7ae9b7b80 1 Processor -- start 2024-05-09T01:34:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff7ae9b7b80 1 -- start start 2024-05-09T01:34:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7840b2900 0x7ff7840b4e10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff794000c00 tx=0x7ff79401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 0x55873767f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587374fc4a0 0x55873767f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558737679400 0x558737679810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55873759c0a0 con 0x558737679400 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5587375419a0 con 0x5587374fc4a0 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558737598060 con 0x558737576d00 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff79dd3d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 0x55873767f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff79e53e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558737679400 0x558737679810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff79dd3d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 0x55873767f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35132/0 (socket says 172.21.15.19:35132) 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.384+0000 7ff79dd3d700 1 -- 172.21.15.19:0/839005763 learned_addr learned my addr 172.21.15.19:0/839005763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff79d53c700 1 --2- 172.21.15.19:0/839005763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587374fc4a0 0x55873767f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff79e53e700 1 -- 172.21.15.19:0/839005763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 msgr2=0x55873767f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff79e53e700 1 --2- 172.21.15.19:0/839005763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 0x55873767f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff79e53e700 1 -- 172.21.15.19:0/839005763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587374fc4a0 msgr2=0x55873767f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff79e53e700 1 --2- 172.21.15.19:0/839005763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587374fc4a0 0x55873767f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff79e53e700 1 -- 172.21.15.19:0/839005763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558737532500 con 0x558737679400 2024-05-09T01:34:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff79e53e700 1 --2- 172.21.15.19:0/839005763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558737679400 0x558737679810 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ff7900df6c0 tx=0x7ff79000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0df136d4772be4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff783fff700 1 -- 172.21.15.19:0/839005763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900a9270 con 0x558737679400 2024-05-09T01:34:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff783fff700 1 -- 172.21.15.19:0/839005763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7900ad620 con 0x558737679400 2024-05-09T01:34:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.385+0000 7ff783fff700 1 -- 172.21.15.19:0/839005763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900dfb80 con 0x558737679400 2024-05-09T01:34:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.386+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/839005763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558737679400 msgr2=0x558737679810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.386+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/839005763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558737679400 0x558737679810 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ff7900df6c0 tx=0x7ff79000ca60 comp rx=0 tx=0).stop 2024-05-09T01:34:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.386+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/839005763 shutdown_connections 2024-05-09T01:34:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.386+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/839005763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558737679400 0x558737679810 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.386+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/839005763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 0x55873767f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.386+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/839005763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587374fc4a0 0x55873767f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.386+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/839005763 >> 172.21.15.19:0/839005763 conn(0x5587374fff70 msgr2=0x558737674220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.386+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/839005763 shutdown_connections 2024-05-09T01:34:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.386+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/839005763 wait complete. 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.387+0000 7ff7ae9b7b80 1 -- start start 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.387+0000 7ff7ae9b7b80 1 Processor -- start 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.387+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x5587376830b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.387+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x5587376835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.387+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 0x558737683b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.387+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55873767a100 con 0x5587374fc4a0 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.387+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558737536b20 con 0x558737576d00 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.387+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55873768f200 con 0x558737679400 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.387+0000 7ff79dd3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x5587376830b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff7ae9b7b80 10 client.0 _ll_register_callbacks cb 0x55873740d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79dd3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x5587376830b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49664/0 (socket says 172.21.15.19:49664) 2024-05-09T01:34:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79dd3d700 1 -- 172.21.15.19:0/256622939 learned_addr learned my addr 172.21.15.19:0/256622939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79d53c700 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x5587376835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79e53e700 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 0x558737683b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79dd3d700 1 -- 172.21.15.19:0/256622939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 msgr2=0x558737683b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 0x558737683b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79dd3d700 1 -- 172.21.15.19:0/256622939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 msgr2=0x5587376835f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x5587376835f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79dd3d700 1 -- 172.21.15.19:0/256622939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558737417490 con 0x5587374fc4a0 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79d53c700 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x5587376835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x5587376830b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ff794003760 tx=0x7ff794005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b5a92cda17b8aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff7817fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:34:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff7817fa700 1 -- 172.21.15.19:0/256622939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79403e060 con 0x5587374fc4a0 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff7817fa700 1 -- 172.21.15.19:0/256622939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff794003a90 con 0x5587374fc4a0 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.388+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/256622939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7900ad620 con 0x5587374fc4a0 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7817fa700 1 -- 172.21.15.19:0/256622939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff794003c90 con 0x5587374fc4a0 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7ae9b7b80 10 client.17952 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/256622939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558737690220 con 0x5587374fc4a0 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7817fa700 1 -- 172.21.15.19:0/256622939 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7900ad620 con 0x5587374fc4a0 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7817fa700 1 -- 172.21.15.19:0/256622939 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff79400f5d0 con 0x5587374fc4a0 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7ae9b7b80 10 client.17952 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7ae9b7b80 10 client.17952 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7ae9b7b80 20 client.17952 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5587376928b0 0x558737692cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7ae9b7b80 4 client.17952 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:34:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/256622939 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587375a9a80 con 0x5587376928b0 2024-05-09T01:34:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.389+0000 7ff79d53c700 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5587376928b0 0x558737692cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.390+0000 7ff79d53c700 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5587376928b0 0x558737692cc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84cb6263f625f0aa server_cookie=d201c1941a9418af in_seq=0 out_seq=0 2024-05-09T01:34:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.390+0000 7ff7817fa700 10 client.17952 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:34:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7817fa700 1 -- 172.21.15.19:0/256622939 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587375a9a80 con 0x5587376928b0 2024-05-09T01:34:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7817fa700 10 client.17952 handle_command_reply: tid=0 2024-05-09T01:34:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 client.17952 shutdown 2024-05-09T01:34:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/256622939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 msgr2=0x5587376830b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x5587376830b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ff794003760 tx=0x7ff794005040 comp rx=0 tx=0).stop 2024-05-09T01:34:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/256622939 shutdown_connections 2024-05-09T01:34:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5587376928b0 0x558737692cc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 0x558737683b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x5587376830b0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7817fa700 0 client.17952 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:34:57.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/256622939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x5587376835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/256622939 >> 172.21.15.19:0/256622939 conn(0x5587374fff70 msgr2=0x55873766dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:57.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/256622939 shutdown_connections 2024-05-09T01:34:57.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.391+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/256622939 wait complete. 2024-05-09T01:34:57.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.392+0000 7ff7ae9b7b80 20 client.17952 trim_cache size 0 max 16384 2024-05-09T01:34:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.407+0000 7ff7ae9b7b80 1 Processor -- start 2024-05-09T01:34:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.407+0000 7ff7ae9b7b80 1 -- start start 2024-05-09T01:34:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.407+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873766dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.407+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 0x55873766e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.407+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737679400 0x558737683070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.407+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981aca70 con 0x5587374fc4a0 2024-05-09T01:34:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.407+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981ae5b0 con 0x558737679400 2024-05-09T01:34:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.407+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981ad140 con 0x558737576d00 2024-05-09T01:34:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79dd3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873766dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79dd3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873766dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49678/0 (socket says 172.21.15.19:49678) 2024-05-09T01:34:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79dd3d700 1 -- 172.21.15.19:0/4235051273 learned_addr learned my addr 172.21.15.19:0/4235051273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79e53e700 1 --2- 172.21.15.19:0/4235051273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737679400 0x558737683070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79d53c700 1 --2- 172.21.15.19:0/4235051273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 0x55873766e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79dd3d700 1 -- 172.21.15.19:0/4235051273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 msgr2=0x55873766e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/4235051273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 0x55873766e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79dd3d700 1 -- 172.21.15.19:0/4235051273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737679400 msgr2=0x558737683070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/4235051273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737679400 0x558737683070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79dd3d700 1 -- 172.21.15.19:0/4235051273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558737682e80 con 0x5587374fc4a0 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/4235051273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873766dff0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff7940098e0 tx=0x7ff794009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb97897d452ccafc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff7837fe700 1 -- 172.21.15.19:0/4235051273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79403e040 con 0x5587374fc4a0 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff7837fe700 1 -- 172.21.15.19:0/4235051273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79400b6b0 con 0x5587374fc4a0 2024-05-09T01:34:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.408+0000 7ff7837fe700 1 -- 172.21.15.19:0/4235051273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7940396c0 con 0x5587374fc4a0 2024-05-09T01:34:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.409+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/4235051273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 msgr2=0x55873766dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.409+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/4235051273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873766dff0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff7940098e0 tx=0x7ff794009f80 comp rx=0 tx=0).stop 2024-05-09T01:34:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.409+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/4235051273 shutdown_connections 2024-05-09T01:34:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.409+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/4235051273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737679400 0x558737683070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.409+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/4235051273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873766dff0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.409+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/4235051273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737576d00 0x55873766e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.409+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/4235051273 >> 172.21.15.19:0/4235051273 conn(0x5587374fff70 msgr2=0x55873767e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.409+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/4235051273 shutdown_connections 2024-05-09T01:34:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.409+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/4235051273 wait complete. 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.410+0000 7ff7ae9b7b80 1 -- start start 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.410+0000 7ff7ae9b7b80 1 Processor -- start 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873769c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x558737696f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff7ae9b7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 0x558737697440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981adee0 con 0x5587374fc4a0 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981ad810 con 0x558737576d00 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff7ae9b7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981aaa30 con 0x558737679400 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff7ae9b7b80 10 client.0 _ll_register_callbacks cb 0x55873740d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79d53c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x558737696f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79dd3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873769c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79d53c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x558737696f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44514/0 (socket says 172.21.15.19:44514) 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79dd3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873769c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49688/0 (socket says 172.21.15.19:49688) 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79d53c700 1 -- 172.21.15.19:0/3442004070 learned_addr learned my addr 172.21.15.19:0/3442004070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79e53e700 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 0x558737697440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79d53c700 1 -- 172.21.15.19:0/3442004070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 msgr2=0x558737697440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79d53c700 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 0x558737697440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79d53c700 1 -- 172.21.15.19:0/3442004070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 msgr2=0x55873769c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79d53c700 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873769c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79d53c700 1 -- 172.21.15.19:0/3442004070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873769c970 con 0x558737576d00 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79e53e700 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 0x558737697440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.411+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873769c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff79d53c700 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x558737696f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff798103410 tx=0x7ff78c010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fbf32d6844a1ff6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:34:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7817fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7817fa700 1 -- 172.21.15.19:0/3442004070 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c004220 con 0x558737576d00 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7817fa700 1 -- 172.21.15.19:0/3442004070 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff78c0043c0 con 0x558737576d00 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/3442004070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79400b6b0 con 0x558737576d00 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7817fa700 1 -- 172.21.15.19:0/3442004070 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff78c01e8f0 con 0x558737576d00 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7ae9b7b80 10 client.27311 fetch_fsmap learned FSMap version 28 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/3442004070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587376fd640 con 0x558737576d00 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7817fa700 1 -- 172.21.15.19:0/3442004070 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79400b6b0 con 0x558737576d00 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7817fa700 1 -- 172.21.15.19:0/3442004070 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff78c005020 con 0x558737576d00 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7ae9b7b80 10 client.27311 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.412+0000 7ff7ae9b7b80 10 client.27311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.413+0000 7ff7ae9b7b80 20 client.27311 populate_metadata read hostname 'smithi019' 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.413+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5587376fddd0 0x5587377001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.413+0000 7ff7ae9b7b80 4 client.27311 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:34:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.413+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/3442004070 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587375a3b20 con 0x5587376fddd0 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.413+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5587376fddd0 0x5587377001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.413+0000 7ff7817fa700 10 client.27311 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.413+0000 7ff79dd3d700 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5587376fddd0 0x5587377001f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2cfacd7757d0166 server_cookie=949d91f99741429b in_seq=0 out_seq=0 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7817fa700 1 -- 172.21.15.19:0/3442004070 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587375a3b20 con 0x5587376fddd0 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7817fa700 10 client.27311 handle_command_reply: tid=0 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 client.27311 shutdown 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/3442004070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 msgr2=0x558737696f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x558737696f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff798103410 tx=0x7ff78c010cc0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/3442004070 shutdown_connections 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5587376fddd0 0x5587377001f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558737679400 0x558737697440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7817fa700 0 client.27311 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587374fc4a0 0x55873769c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 --2- 172.21.15.19:0/3442004070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737576d00 0x558737696f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/3442004070 >> 172.21.15.19:0/3442004070 conn(0x5587374fff70 msgr2=0x55873767e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/3442004070 shutdown_connections 2024-05-09T01:34:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 1 -- 172.21.15.19:0/3442004070 wait complete. 2024-05-09T01:34:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.414+0000 7ff7ae9b7b80 20 client.27311 trim_cache size 0 max 16384 2024-05-09T01:34:57.417 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 -- 172.21.15.19:0/85654916 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7840b2900 msgr2=0x7ff7840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7840b2900 0x7ff7840b4e10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff794000c00 tx=0x7ff79401a040 comp rx=0 tx=0).stop 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 -- 172.21.15.19:0/85654916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff798109860 msgr2=0x7ff79819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff798109860 0x7ff79819f6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff79000ef30 tx=0x7ff79000c3a0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 -- 172.21.15.19:0/85654916 shutdown_connections 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff798109860 0x7ff79819f6a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7981045e0 0x7ff79819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff7840b2900 0x7ff7840b4e10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 --2- 172.21.15.19:0/85654916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff798108f10 0x7ff79819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.415+0000 7ff79ffa1700 1 -- 172.21.15.19:0/85654916 >> 172.21.15.19:0/85654916 conn(0x7ff7980ffe70 msgr2=0x7ff7981086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:34:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.416+0000 7ff79ffa1700 1 -- 172.21.15.19:0/85654916 shutdown_connections 2024-05-09T01:34:57.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:34:57.416+0000 7ff79ffa1700 1 -- 172.21.15.19:0/85654916 wait complete. 2024-05-09T01:34:58.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:58 smithi019 ceph-mon[30158]: pgmap v661: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:58.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:34:58 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:58 smithi196 ceph-mon[35041]: pgmap v661: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:34:58 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:34:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:58 smithi059 ceph-mon[34351]: pgmap v661: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:34:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:34:58 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:34:59.041 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:34:59.513 INFO:tasks.workunit.client.0.smithi019.stdout:375/600: open fd = 3 2024-05-09T01:34:59.513 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.513 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.513 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.513 INFO:tasks.workunit.client.0.smithi019.stdout:376/600: open fd = 3 2024-05-09T01:34:59.513 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.513 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.513 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:377/600: open fd = 3 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:378/600: open fd = 3 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:379/600: open fd = 3 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.514 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:380/600: open fd = 3 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:381/600: open fd = 3 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:382/600: open fd = 3 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.515 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.516 INFO:tasks.workunit.client.0.smithi019.stdout:383/600: open fd = 3 2024-05-09T01:34:59.516 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.516 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.516 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.516 INFO:tasks.workunit.client.0.smithi019.stdout:384/600: open fd = 3 2024-05-09T01:34:59.516 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.516 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.516 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.516 INFO:tasks.workunit.client.0.smithi019.stdout:385/600: open fd = 3 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:386/600: open fd = 3 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:387/600: open fd = 3 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:388/600: open fd = 3 2024-05-09T01:34:59.517 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:389/600: open fd = 3 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:390/600: open fd = 3 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:391/600: open fd = 3 2024-05-09T01:34:59.518 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:392/600: open fd = 3 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:393/600: open fd = 3 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:394/600: open fd = 3 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.519 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:395/600: open fd = 3 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:396/600: open fd = 3 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:397/600: open fd = 3 2024-05-09T01:34:59.520 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:398/600: open fd = 3 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:399/600: open fd = 3 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:400/600: open fd = 3 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.521 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:401/600: open fd = 3 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:402/600: open fd = 3 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:403/600: open fd = 3 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.522 INFO:tasks.workunit.client.0.smithi019.stdout:404/600: open fd = 3 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:405/600: open fd = 3 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:406/600: open fd = 3 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.523 INFO:tasks.workunit.client.0.smithi019.stdout:407/600: open fd = 3 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:408/600: open fd = 3 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:409/600: open fd = 3 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:410/600: open fd = 3 2024-05-09T01:34:59.524 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:411/600: open fd = 3 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:412/600: open fd = 3 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:413/600: open fd = 3 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.525 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:414/600: open fd = 3 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:415/600: open fd = 3 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:416/600: open fd = 3 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:417/600: open fd = 3 2024-05-09T01:34:59.526 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:418/600: open fd = 3 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:419/600: open fd = 3 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.527 INFO:tasks.workunit.client.0.smithi019.stdout:420/600: open fd = 3 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:421/600: open fd = 3 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:422/600: open fd = 3 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.528 INFO:tasks.workunit.client.0.smithi019.stdout:423/600: open fd = 3 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:424/600: open fd = 3 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:425/600: open fd = 3 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.529 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:426/600: open fd = 3 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:427/600: open fd = 3 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:428/600: open fd = 3 2024-05-09T01:34:59.530 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:429/600: open fd = 3 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:430/600: open fd = 3 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:431/600: open fd = 3 2024-05-09T01:34:59.531 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:432/600: open fd = 3 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:433/600: open fd = 3 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:434/600: open fd = 3 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.532 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.533 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.533 INFO:tasks.workunit.client.0.smithi019.stdout:435/600: open fd = 3 2024-05-09T01:34:59.533 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.533 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.533 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:34:59.533 INFO:tasks.workunit.client.0.smithi019.stdout:436/600: open fd = 3 2024-05-09T01:34:59.533 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.533 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:34:59.533 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:35:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:00 smithi019 ceph-mon[30158]: pgmap v662: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:35:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:00 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:35:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:00 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:35:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:00 smithi196 ceph-mon[35041]: pgmap v662: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:35:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:00 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:35:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:00 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:35:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:00 smithi059 ceph-mon[34351]: pgmap v662: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:35:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:00 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:35:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:00 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:35:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.851+0000 7f06112a0700 1 -- 172.21.15.19:0/2929697170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c107e10 msgr2=0x7f060c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.851+0000 7f06112a0700 1 --2- 172.21.15.19:0/2929697170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c107e10 0x7f060c108220 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f06000099c0 tx=0x7f0600009cd0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.851+0000 7f06112a0700 1 -- 172.21.15.19:0/2929697170 shutdown_connections 2024-05-09T01:35:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.851+0000 7f06112a0700 1 --2- 172.21.15.19:0/2929697170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f060c108760 0x7f060c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.851+0000 7f06112a0700 1 --2- 172.21.15.19:0/2929697170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060c1034e0 0x7f060c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.851+0000 7f06112a0700 1 --2- 172.21.15.19:0/2929697170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c107e10 0x7f060c108220 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.851+0000 7f06112a0700 1 -- 172.21.15.19:0/2929697170 >> 172.21.15.19:0/2929697170 conn(0x7f060c0fef30 msgr2=0x7f060c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.851+0000 7f06112a0700 1 -- 172.21.15.19:0/2929697170 shutdown_connections 2024-05-09T01:35:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.851+0000 7f06112a0700 1 -- 172.21.15.19:0/2929697170 wait complete. 2024-05-09T01:35:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f06112a0700 1 Processor -- start 2024-05-09T01:35:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f06112a0700 1 -- start start 2024-05-09T01:35:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f06112a0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060c1034e0 0x7f060c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f06112a0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f060c107e10 0x7f060c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f06112a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c108760 0x7f060c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f06112a0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f060c113370 con 0x7f060c108760 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f06112a0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1131f0 con 0x7f060c107e10 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f06112a0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1134f0 con 0x7f060c1034e0 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c108760 0x7f060c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c108760 0x7f060c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59554/0 (socket says 172.21.15.19:59554) 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f060b7fe700 1 -- 172.21.15.19:0/1107401083 learned_addr learned my addr 172.21.15.19:0/1107401083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.852+0000 7f060a7fc700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f060c107e10 0x7f060c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f060affd700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060c1034e0 0x7f060c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f060b7fe700 1 -- 172.21.15.19:0/1107401083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060c1034e0 msgr2=0x7f060c19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f060b7fe700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060c1034e0 0x7f060c19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f060b7fe700 1 -- 172.21.15.19:0/1107401083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f060c107e10 msgr2=0x7f060c19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f060b7fe700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f060c107e10 0x7f060c19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f060b7fe700 1 -- 172.21.15.19:0/1107401083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c19fda0 con 0x7f060c108760 2024-05-09T01:35:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f060b7fe700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c108760 0x7f060c19f700 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0604007ee0 tx=0x7f060400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f05f3fff700 1 -- 172.21.15.19:0/1107401083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060400dd20 con 0x7f060c108760 2024-05-09T01:35:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f06112a0700 1 -- 172.21.15.19:0/1107401083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c1a0090 con 0x7f060c108760 2024-05-09T01:35:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f05f3fff700 1 -- 172.21.15.19:0/1107401083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0604004020 con 0x7f060c108760 2024-05-09T01:35:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f05f3fff700 1 -- 172.21.15.19:0/1107401083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604015660 con 0x7f060c108760 2024-05-09T01:35:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.853+0000 7f06112a0700 1 -- 172.21.15.19:0/1107401083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c1a79b0 con 0x7f060c108760 2024-05-09T01:35:00.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.856+0000 7f05f3fff700 1 -- 172.21.15.19:0/1107401083 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f06040041d0 con 0x7f060c108760 2024-05-09T01:35:00.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.857+0000 7f05f3fff700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f05f40b2880 0x7f05f40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.857+0000 7f060affd700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f05f40b2880 0x7f05f40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.857+0000 7f05f3fff700 1 -- 172.21.15.19:0/1107401083 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f06040e1b80 con 0x7f060c108760 2024-05-09T01:35:00.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.857+0000 7f061fcb6b80 1 Processor -- start 2024-05-09T01:35:00.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.857+0000 7f060affd700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f05f40b2880 0x7f05f40b4d90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005950 tx=0x7f05fc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.857+0000 7f061fcb6b80 1 -- start start 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 0x56222de5ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de5f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56222dd7d0a0 con 0x56222dcdd4a0 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56222dd229a0 con 0x56222de631e0 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56222dd79060 con 0x56222dd57d00 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de5f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de5f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59562/0 (socket says 172.21.15.19:59562) 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060a7fc700 1 -- 172.21.15.19:0/1162959048 learned_addr learned my addr 172.21.15.19:0/1162959048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060b7fe700 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060affd700 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 0x56222de5ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060a7fc700 1 -- 172.21.15.19:0/1162959048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 msgr2=0x56222de5ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060a7fc700 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 0x56222de5ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060a7fc700 1 -- 172.21.15.19:0/1162959048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 msgr2=0x56222de635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060a7fc700 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de635f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060a7fc700 1 -- 172.21.15.19:0/1162959048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56222dd13500 con 0x56222dcdd4a0 2024-05-09T01:35:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060b7fe700 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f060a7fc700 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de5f030 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f06000099c0 tx=0x7f0600009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a4f9b63b7bb71d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f05f17fa700 1 -- 172.21.15.19:0/1162959048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600003ae0 con 0x56222dcdd4a0 2024-05-09T01:35:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.858+0000 7f05f17fa700 1 -- 172.21.15.19:0/1162959048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0600005580 con 0x56222dcdd4a0 2024-05-09T01:35:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.859+0000 7f05f17fa700 1 -- 172.21.15.19:0/1162959048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060000bc90 con 0x56222dcdd4a0 2024-05-09T01:35:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.859+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1162959048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 msgr2=0x56222de5f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.859+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de5f030 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f06000099c0 tx=0x7f0600009630 comp rx=0 tx=0).stop 2024-05-09T01:35:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.859+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1162959048 shutdown_connections 2024-05-09T01:35:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.859+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.859+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 0x56222de5ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.859+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/1162959048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de5f030 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.859+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1162959048 >> 172.21.15.19:0/1162959048 conn(0x56222dce0f70 msgr2=0x56222de55220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.859+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1162959048 shutdown_connections 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1162959048 wait complete. 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 1 -- start start 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 1 Processor -- start 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dd57d00 0x56222de68830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 0x56222de68d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56222de63e40 con 0x56222dcdd4a0 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56222dd17b20 con 0x56222dd57d00 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56222de63fc0 con 0x56222de631e0 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f061fcb6b80 10 client.0 _ll_register_callbacks cb 0x56222dbee6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.860+0000 7f060affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de682f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59564/0 (socket says 172.21.15.19:59564) 2024-05-09T01:35:00.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060affd700 1 -- 172.21.15.19:0/828160979 learned_addr learned my addr 172.21.15.19:0/828160979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060a7fc700 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dd57d00 0x56222de68830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060b7fe700 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 0x56222de68d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060affd700 1 -- 172.21.15.19:0/828160979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 msgr2=0x56222de68d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060affd700 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 0x56222de68d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060affd700 1 -- 172.21.15.19:0/828160979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dd57d00 msgr2=0x56222de68830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060affd700 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dd57d00 0x56222de68830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060affd700 1 -- 172.21.15.19:0/828160979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222dbf8490 con 0x56222dcdd4a0 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f060affd700 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de682f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0116f0 tx=0x7f05fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f038d840071b0b0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f05eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f05eeff5700 1 -- 172.21.15.19:0/828160979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc004a30 con 0x56222dcdd4a0 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f05eeff5700 1 -- 172.21.15.19:0/828160979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f05fc004bd0 con 0x56222dcdd4a0 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.861+0000 7f061fcb6b80 1 -- 172.21.15.19:0/828160979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0600005580 con 0x56222dcdd4a0 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f05eeff5700 1 -- 172.21.15.19:0/828160979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05fc004dd0 con 0x56222dcdd4a0 2024-05-09T01:35:00.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f061fcb6b80 10 client.17970 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f061fcb6b80 1 -- 172.21.15.19:0/828160979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56222de75d00 con 0x56222dcdd4a0 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f05eeff5700 1 -- 172.21.15.19:0/828160979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0600005580 con 0x56222dcdd4a0 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f05eeff5700 1 -- 172.21.15.19:0/828160979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f05fc01fa10 con 0x56222dcdd4a0 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f061fcb6b80 10 client.17970 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f061fcb6b80 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f061fcb6b80 20 client.17970 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56222de78390 0x56222de7a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f061fcb6b80 4 client.17970 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f061fcb6b80 1 -- 172.21.15.19:0/828160979 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56222dd8aa80 con 0x56222de78390 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.862+0000 7f060a7fc700 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56222de78390 0x56222de7a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.863+0000 7f05eeff5700 10 client.17970 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.863+0000 7f060a7fc700 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56222de78390 0x56222de7a810 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=399d9b124dea093f server_cookie=4cb3c2f3a8ba4a29 in_seq=0 out_seq=0 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f05eeff5700 1 -- 172.21.15.19:0/828160979 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56222dd8aa80 con 0x56222de78390 2024-05-09T01:35:00.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f05eeff5700 10 client.17970 handle_command_reply: tid=0 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 client.17970 shutdown 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 -- 172.21.15.19:0/828160979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 msgr2=0x56222de682f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de682f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0116f0 tx=0x7f05fc0047c0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 -- 172.21.15.19:0/828160979 shutdown_connections 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56222de78390 0x56222de7a810 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 0x56222de68d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de682f0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/828160979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dd57d00 0x56222de68830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 -- 172.21.15.19:0/828160979 >> 172.21.15.19:0/828160979 conn(0x56222dce0f70 msgr2=0x56222de4eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f05eeff5700 0 client.17970 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 -- 172.21.15.19:0/828160979 shutdown_connections 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 1 -- 172.21.15.19:0/828160979 wait complete. 2024-05-09T01:35:00.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.864+0000 7f061fcb6b80 20 client.17970 trim_cache size 0 max 16384 2024-05-09T01:35:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f061fcb6b80 1 Processor -- start 2024-05-09T01:35:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f061fcb6b80 1 -- start start 2024-05-09T01:35:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dcdd4a0 0x56222de55330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dd57d00 0x56222de74200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 0x56222de80010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1ac9f0 con 0x56222dd57d00 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1ae530 con 0x56222dcdd4a0 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1ad0c0 con 0x56222de631e0 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f060a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dd57d00 0x56222de74200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f060affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dcdd4a0 0x56222de55330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f060a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dd57d00 0x56222de74200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59576/0 (socket says 172.21.15.19:59576) 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.880+0000 7f060a7fc700 1 -- 172.21.15.19:0/1692592156 learned_addr learned my addr 172.21.15.19:0/1692592156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f060b7fe700 1 --2- 172.21.15.19:0/1692592156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 0x56222de80010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f060a7fc700 1 -- 172.21.15.19:0/1692592156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 msgr2=0x56222de80010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f060a7fc700 1 --2- 172.21.15.19:0/1692592156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 0x56222de80010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f060a7fc700 1 -- 172.21.15.19:0/1692592156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dcdd4a0 msgr2=0x56222de55330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f060a7fc700 1 --2- 172.21.15.19:0/1692592156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dcdd4a0 0x56222de55330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f060a7fc700 1 -- 172.21.15.19:0/1692592156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56222de55f40 con 0x56222dd57d00 2024-05-09T01:35:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f060affd700 1 --2- 172.21.15.19:0/1692592156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dcdd4a0 0x56222de55330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f060a7fc700 1 --2- 172.21.15.19:0/1692592156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dd57d00 0x56222de74200 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0600009990 tx=0x7f0600009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4251ff6724d250 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f05f0ff9700 1 -- 172.21.15.19:0/1692592156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600034070 con 0x56222dd57d00 2024-05-09T01:35:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f05f0ff9700 1 -- 172.21.15.19:0/1692592156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f060000bd60 con 0x56222dd57d00 2024-05-09T01:35:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.881+0000 7f05f0ff9700 1 -- 172.21.15.19:0/1692592156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600003d60 con 0x56222dd57d00 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1692592156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dd57d00 msgr2=0x56222de74200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/1692592156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dd57d00 0x56222de74200 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0600009990 tx=0x7f0600009cd0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1692592156 shutdown_connections 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/1692592156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222de631e0 0x56222de80010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/1692592156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222dcdd4a0 0x56222de55330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/1692592156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dd57d00 0x56222de74200 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1692592156 >> 172.21.15.19:0/1692592156 conn(0x56222dce0f70 msgr2=0x56222de54c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1692592156 shutdown_connections 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 -- 172.21.15.19:0/1692592156 wait complete. 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.882+0000 7f061fcb6b80 1 -- start start 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f061fcb6b80 1 Processor -- start 2024-05-09T01:35:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de70a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 0x56222de658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f061fcb6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de65e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1ade60 con 0x56222dcdd4a0 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1ad790 con 0x56222de631e0 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f061fcb6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1aa990 con 0x56222dd57d00 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f061fcb6b80 10 client.0 _ll_register_callbacks cb 0x56222de5f640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de70a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de70a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59578/0 (socket says 172.21.15.19:59578) 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060affd700 1 -- 172.21.15.19:0/2244305340 learned_addr learned my addr 172.21.15.19:0/2244305340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060a7fc700 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 0x56222de658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060b7fe700 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de65e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060affd700 1 -- 172.21.15.19:0/2244305340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 msgr2=0x56222de658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060affd700 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 0x56222de658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060affd700 1 -- 172.21.15.19:0/2244305340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 msgr2=0x56222de65e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060affd700 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de65e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060affd700 1 -- 172.21.15.19:0/2244305340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222de64700 con 0x56222dcdd4a0 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.883+0000 7f060b7fe700 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de65e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f060affd700 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de70a70 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005d90 tx=0x7f05fc004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c2d83076f5c0fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f05eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f05eeff5700 1 -- 172.21.15.19:0/2244305340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc037070 con 0x56222dcdd4a0 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f061fcb6b80 1 -- 172.21.15.19:0/2244305340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f060000bd60 con 0x56222dcdd4a0 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f05eeff5700 1 -- 172.21.15.19:0/2244305340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f05fc032ca0 con 0x56222dcdd4a0 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f05eeff5700 1 -- 172.21.15.19:0/2244305340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05fc011d90 con 0x56222dcdd4a0 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f061fcb6b80 10 client.17976 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f061fcb6b80 1 -- 172.21.15.19:0/2244305340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56222dee1d10 con 0x56222dcdd4a0 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f05eeff5700 1 -- 172.21.15.19:0/2244305340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f060000bd60 con 0x56222dcdd4a0 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f05eeff5700 1 -- 172.21.15.19:0/2244305340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f05fc0052f0 con 0x56222dcdd4a0 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f061fcb6b80 10 client.17976 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.884+0000 7f061fcb6b80 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f061fcb6b80 20 client.17976 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56222dee2360 0x56222dee4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f061fcb6b80 4 client.17976 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f061fcb6b80 1 -- 172.21.15.19:0/2244305340 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56222dd84b20 con 0x56222dee2360 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f060a7fc700 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56222dee2360 0x56222dee4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f05eeff5700 10 client.17976 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f060a7fc700 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56222dee2360 0x56222dee4780 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab5e054125e084c2 server_cookie=32d096e4ed83849e in_seq=0 out_seq=0 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f05eeff5700 1 -- 172.21.15.19:0/2244305340 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56222dd84b20 con 0x56222dee2360 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f05eeff5700 10 client.17976 handle_command_reply: tid=0 2024-05-09T01:35:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.885+0000 7f061fcb6b80 1 client.17976 shutdown 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 -- 172.21.15.19:0/2244305340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 msgr2=0x56222de70a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de70a70 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005d90 tx=0x7f05fc004b90 comp rx=0 tx=0).stop 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 -- 172.21.15.19:0/2244305340 shutdown_connections 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56222dee2360 0x56222dee4780 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56222de631e0 0x56222de65e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56222dcdd4a0 0x56222de70a70 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 --2- 172.21.15.19:0/2244305340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56222dd57d00 0x56222de658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 -- 172.21.15.19:0/2244305340 >> 172.21.15.19:0/2244305340 conn(0x56222dce0f70 msgr2=0x56222de4eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f05eeff5700 0 client.17976 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 -- 172.21.15.19:0/2244305340 shutdown_connections 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 1 -- 172.21.15.19:0/2244305340 wait complete. 2024-05-09T01:35:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f061fcb6b80 20 client.17976 trim_cache size 0 max 16384 2024-05-09T01:35:00.890 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:35:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f06112a0700 1 -- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f05f40b2880 msgr2=0x7f05f40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f06112a0700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f05f40b2880 0x7f05f40b4d90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005950 tx=0x7f05fc00b410 comp rx=0 tx=0).stop 2024-05-09T01:35:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f06112a0700 1 -- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c108760 msgr2=0x7f060c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.886+0000 7f06112a0700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c108760 0x7f060c19f700 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0604007ee0 tx=0x7f060400d3b0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.887+0000 7f06112a0700 1 -- 172.21.15.19:0/1107401083 shutdown_connections 2024-05-09T01:35:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.887+0000 7f06112a0700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f05f40b2880 0x7f05f40b4d90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.887+0000 7f06112a0700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f060c108760 0x7f060c19f700 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.887+0000 7f06112a0700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060c1034e0 0x7f060c19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.887+0000 7f06112a0700 1 --2- 172.21.15.19:0/1107401083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f060c107e10 0x7f060c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.887+0000 7f06112a0700 1 -- 172.21.15.19:0/1107401083 >> 172.21.15.19:0/1107401083 conn(0x7f060c0fef30 msgr2=0x7f060c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.887+0000 7f06112a0700 1 -- 172.21.15.19:0/1107401083 shutdown_connections 2024-05-09T01:35:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:00.887+0000 7f06112a0700 1 -- 172.21.15.19:0/1107401083 wait complete. 2024-05-09T01:35:01.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:35:01.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:35:01.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:35:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:35:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:35:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:35:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:35:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:35:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:35:02.656 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:35:02.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:35:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:02 smithi059 ceph-mon[34351]: pgmap v663: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:02 smithi019 ceph-mon[30158]: pgmap v663: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:02.921 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:02.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:02 smithi196 ceph-mon[35041]: pgmap v663: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 -- 172.21.15.19:0/948060064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c105e10 msgr2=0x7f275c10e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 --2- 172.21.15.19:0/948060064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c105e10 0x7f275c10e360 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2744009990 tx=0x7f2744009ca0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 -- 172.21.15.19:0/948060064 shutdown_connections 2024-05-09T01:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 --2- 172.21.15.19:0/948060064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f275c10e8f0 0x7f275c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 --2- 172.21.15.19:0/948060064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f275c1054c0 0x7f275c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 --2- 172.21.15.19:0/948060064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c105e10 0x7f275c10e360 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 -- 172.21.15.19:0/948060064 >> 172.21.15.19:0/948060064 conn(0x7f275c0fedc0 msgr2=0x7f275c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 -- 172.21.15.19:0/948060064 shutdown_connections 2024-05-09T01:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 -- 172.21.15.19:0/948060064 wait complete. 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 Processor -- start 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.613+0000 7f27613a2700 1 -- start start 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f27613a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c1054c0 0x7f275c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f27613a2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f275c105e10 0x7f275c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f27613a2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f275c10e8f0 0x7f275c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f27613a2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f275c114450 con 0x7f275c1054c0 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f27613a2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1142d0 con 0x7f275c105e10 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f27613a2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1145d0 con 0x7f275c10e8f0 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c1054c0 0x7f275c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c1054c0 0x7f275c19ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59598/0 (socket says 172.21.15.19:59598) 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275affd700 1 -- 172.21.15.19:0/3308175619 learned_addr learned my addr 172.21.15.19:0/3308175619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275b7fe700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f275c10e8f0 0x7f275c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f2753fff700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f275c105e10 0x7f275c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275affd700 1 -- 172.21.15.19:0/3308175619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f275c10e8f0 msgr2=0x7f275c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275affd700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f275c10e8f0 0x7f275c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275affd700 1 -- 172.21.15.19:0/3308175619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f275c105e10 msgr2=0x7f275c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275affd700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f275c105e10 0x7f275c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275affd700 1 -- 172.21.15.19:0/3308175619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c19fcb0 con 0x7f275c1054c0 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f2753fff700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f275c105e10 0x7f275c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.614+0000 7f275affd700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c1054c0 0x7f275c19ade0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f274c00d950 tx=0x7f274c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.615+0000 7f2758ff9700 1 -- 172.21.15.19:0/3308175619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c016070 con 0x7f275c1054c0 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.615+0000 7f2758ff9700 1 -- 172.21.15.19:0/3308175619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f274c009df0 con 0x7f275c1054c0 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.615+0000 7f27613a2700 1 -- 172.21.15.19:0/3308175619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275c19ffa0 con 0x7f275c1054c0 2024-05-09T01:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.615+0000 7f2758ff9700 1 -- 172.21.15.19:0/3308175619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c00f3c0 con 0x7f275c1054c0 2024-05-09T01:35:04.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.615+0000 7f27613a2700 1 -- 172.21.15.19:0/3308175619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275c1a78c0 con 0x7f275c1054c0 2024-05-09T01:35:04.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.617+0000 7f2758ff9700 1 -- 172.21.15.19:0/3308175619 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f274c019070 con 0x7f275c1054c0 2024-05-09T01:35:04.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.618+0000 7f2758ff9700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f27480b2900 0x7f27480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.618+0000 7f2753fff700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f27480b2900 0x7f27480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.618+0000 7f2758ff9700 1 -- 172.21.15.19:0/3308175619 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f274c0e1e30 con 0x7f275c1054c0 2024-05-09T01:35:04.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f276fdb8b80 1 Processor -- start 2024-05-09T01:35:04.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f276fdb8b80 1 -- start start 2024-05-09T01:35:04.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f2753fff700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f27480b2900 0x7f27480b4e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2744003800 tx=0x7f274401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131ecfd80 0x55e131fd6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 0x55e131fd70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fdb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e131ef5120 con 0x55e131fdb260 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e131e9aa20 con 0x55e131e55520 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e131ef10e0 con 0x55e131ecfd80 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f275b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fdb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f275b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fdb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59608/0 (socket says 172.21.15.19:59608) 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.619+0000 7f2753fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 0x55e131fd70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f275b7fe700 1 -- 172.21.15.19:0/3453726948 learned_addr learned my addr 172.21.15.19:0/3453726948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f275b7fe700 1 -- 172.21.15.19:0/3453726948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131ecfd80 msgr2=0x55e131fd6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f275b7fe700 1 --2- 172.21.15.19:0/3453726948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131ecfd80 0x55e131fd6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f275b7fe700 1 -- 172.21.15.19:0/3453726948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 msgr2=0x55e131fd70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f275b7fe700 1 --2- 172.21.15.19:0/3453726948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 0x55e131fd70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f275b7fe700 1 -- 172.21.15.19:0/3453726948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e131e8b580 con 0x55e131fdb260 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f275b7fe700 1 --2- 172.21.15.19:0/3453726948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fdb670 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f2754003fa0 tx=0x7f2754009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594a6ec2e6332f8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f27517fa700 1 -- 172.21.15.19:0/3453726948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754004bb0 con 0x55e131fdb260 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f27517fa700 1 -- 172.21.15.19:0/3453726948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2754004d50 con 0x55e131fdb260 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.620+0000 7f27517fa700 1 -- 172.21.15.19:0/3453726948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754007d20 con 0x55e131fdb260 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.621+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3453726948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 msgr2=0x55e131fdb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.621+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3453726948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fdb670 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f2754003fa0 tx=0x7f2754009ba0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.621+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3453726948 shutdown_connections 2024-05-09T01:35:04.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.621+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3453726948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fdb670 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.621+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3453726948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131ecfd80 0x55e131fd6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.621+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3453726948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 0x55e131fd70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.621+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3453726948 >> 172.21.15.19:0/3453726948 conn(0x55e131e58ff0 msgr2=0x55e131fcd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:04.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.621+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3453726948 shutdown_connections 2024-05-09T01:35:04.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.621+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3453726948 wait complete. 2024-05-09T01:35:04.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.622+0000 7f276fdb8b80 1 -- start start 2024-05-09T01:35:04.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.622+0000 7f276fdb8b80 1 Processor -- start 2024-05-09T01:35:04.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.622+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131e55520 0x55e131fe0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.622+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131ecfd80 0x55e131fe09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.622+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131fdb260 0x55e131fe0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.622+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e131fdbf80 con 0x55e131ecfd80 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.622+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e131e8fba0 con 0x55e131fdb260 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.622+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e131fe34f0 con 0x55e131e55520 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f276fdb8b80 10 client.0 _ll_register_callbacks cb 0x55e131d66740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f2753fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131ecfd80 0x55e131fe09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f2753fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131ecfd80 0x55e131fe09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59618/0 (socket says 172.21.15.19:59618) 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f2753fff700 1 -- 172.21.15.19:0/3830202091 learned_addr learned my addr 172.21.15.19:0/3830202091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f275b7fe700 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131fdb260 0x55e131fe0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f275affd700 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131e55520 0x55e131fe0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f2753fff700 1 -- 172.21.15.19:0/3830202091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131e55520 msgr2=0x55e131fe0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f2753fff700 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131e55520 0x55e131fe0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f2753fff700 1 -- 172.21.15.19:0/3830202091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131fdb260 msgr2=0x55e131fe0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f2753fff700 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131fdb260 0x55e131fe0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f2753fff700 1 -- 172.21.15.19:0/3830202091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e131d70510 con 0x55e131ecfd80 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f2753fff700 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131ecfd80 0x55e131fe09d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c00 tx=0x7f2744009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8978fe3ca74c2fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f27427fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f27427fc700 1 -- 172.21.15.19:0/3830202091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744034060 con 0x55e131ecfd80 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f27427fc700 1 -- 172.21.15.19:0/3830202091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2744003c70 con 0x55e131ecfd80 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.623+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3830202091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2754004d50 con 0x55e131ecfd80 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f27427fc700 1 -- 172.21.15.19:0/3830202091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f274400fe20 con 0x55e131ecfd80 2024-05-09T01:35:04.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f276fdb8b80 10 client.17988 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3830202091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e131fedea0 con 0x55e131ecfd80 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f27427fc700 1 -- 172.21.15.19:0/3830202091 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2754004d50 con 0x55e131ecfd80 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f27427fc700 1 -- 172.21.15.19:0/3830202091 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2744022070 con 0x55e131ecfd80 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f276fdb8b80 10 client.17988 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f276fdb8b80 10 client.17988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f276fdb8b80 20 client.17988 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e131ff08e0 0x55e131ff2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f276fdb8b80 4 client.17988 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3830202091 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e131f02b00 con 0x55e131ff08e0 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.624+0000 7f275affd700 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e131ff08e0 0x55e131ff2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.625+0000 7f27427fc700 10 client.17988 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.625+0000 7f275affd700 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e131ff08e0 0x55e131ff2d30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b157d2e634516b95 server_cookie=bd01ccf30e68a7f4 in_seq=0 out_seq=0 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f27427fc700 1 -- 172.21.15.19:0/3830202091 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e131f02b00 con 0x55e131ff08e0 2024-05-09T01:35:04.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f27427fc700 10 client.17988 handle_command_reply: tid=0 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 client.17988 shutdown 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3830202091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131ecfd80 msgr2=0x55e131fe09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131ecfd80 0x55e131fe09d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c00 tx=0x7f2744009ca0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3830202091 shutdown_connections 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e131ff08e0 0x55e131ff2d30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131fdb260 0x55e131fe0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131e55520 0x55e131fe0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3830202091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131ecfd80 0x55e131fe09d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3830202091 >> 172.21.15.19:0/3830202091 conn(0x55e131e58ff0 msgr2=0x55e131fd1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f27427fc700 0 client.17988 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3830202091 shutdown_connections 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3830202091 wait complete. 2024-05-09T01:35:04.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.626+0000 7f276fdb8b80 20 client.17988 trim_cache size 0 max 16384 2024-05-09T01:35:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f276fdb8b80 1 Processor -- start 2024-05-09T01:35:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f276fdb8b80 1 -- start start 2024-05-09T01:35:04.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:04 smithi019 ceph-mon[30158]: pgmap v664: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:35:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 0x55e131fc7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131ecfd80 0x55e131fe0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fec1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1acb30 con 0x55e131fdb260 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1ae670 con 0x55e131e55520 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1ad200 con 0x55e131ecfd80 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f275b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fec1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f275b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fec1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59622/0 (socket says 172.21.15.19:59622) 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.647+0000 7f275b7fe700 1 -- 172.21.15.19:0/2189839947 learned_addr learned my addr 172.21.15.19:0/2189839947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f275affd700 1 --2- 172.21.15.19:0/2189839947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 0x55e131fc7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f2753fff700 1 --2- 172.21.15.19:0/2189839947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131ecfd80 0x55e131fe0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f275b7fe700 1 -- 172.21.15.19:0/2189839947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131ecfd80 msgr2=0x55e131fe0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f275b7fe700 1 --2- 172.21.15.19:0/2189839947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131ecfd80 0x55e131fe0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f275b7fe700 1 -- 172.21.15.19:0/2189839947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 msgr2=0x55e131fc7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f275b7fe700 1 --2- 172.21.15.19:0/2189839947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 0x55e131fc7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f275b7fe700 1 -- 172.21.15.19:0/2189839947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e131fe2d30 con 0x55e131fdb260 2024-05-09T01:35:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f275b7fe700 1 --2- 172.21.15.19:0/2189839947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fec1b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2754000f80 tx=0x7f2754007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9222fe98a6680306 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f2750ff9700 1 -- 172.21.15.19:0/2189839947 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754007970 con 0x55e131fdb260 2024-05-09T01:35:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f2750ff9700 1 -- 172.21.15.19:0/2189839947 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2754023070 con 0x55e131fdb260 2024-05-09T01:35:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.648+0000 7f2750ff9700 1 -- 172.21.15.19:0/2189839947 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754015480 con 0x55e131fdb260 2024-05-09T01:35:04.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.649+0000 7f276fdb8b80 1 -- 172.21.15.19:0/2189839947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 msgr2=0x55e131fec1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:04.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.649+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/2189839947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fec1b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2754000f80 tx=0x7f2754007690 comp rx=0 tx=0).stop 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.649+0000 7f276fdb8b80 1 -- 172.21.15.19:0/2189839947 shutdown_connections 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.649+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/2189839947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131fdb260 0x55e131fec1b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.649+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/2189839947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131e55520 0x55e131fc7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.649+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/2189839947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131ecfd80 0x55e131fe0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.649+0000 7f276fdb8b80 1 -- 172.21.15.19:0/2189839947 >> 172.21.15.19:0/2189839947 conn(0x55e131e58ff0 msgr2=0x55e131fdab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.649+0000 7f276fdb8b80 1 -- 172.21.15.19:0/2189839947 shutdown_connections 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.649+0000 7f276fdb8b80 1 -- 172.21.15.19:0/2189839947 wait complete. 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.650+0000 7f276fdb8b80 1 -- start start 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.650+0000 7f276fdb8b80 1 Processor -- start 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.650+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131e55520 0x55e131fe84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.650+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131ecfd80 0x55e131fe8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.650+0000 7f276fdb8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131fdb260 0x55e131fe5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f276fdb8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1adfa0 con 0x55e131e55520 2024-05-09T01:35:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f275affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131e55520 0x55e131fe84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f275affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131e55520 0x55e131fe84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59628/0 (socket says 172.21.15.19:59628) 2024-05-09T01:35:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f275affd700 1 -- 172.21.15.19:0/3095970318 learned_addr learned my addr 172.21.15.19:0/3095970318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1ad8d0 con 0x55e131ecfd80 2024-05-09T01:35:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1aaad0 con 0x55e131fdb260 2024-05-09T01:35:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f2753fff700 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131ecfd80 0x55e131fe8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f276fdb8b80 10 client.0 _ll_register_callbacks cb 0x55e131d66740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f275affd700 1 -- 172.21.15.19:0/3095970318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131fdb260 msgr2=0x55e131fe5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f275affd700 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131fdb260 0x55e131fe5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f275affd700 1 -- 172.21.15.19:0/3095970318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131ecfd80 msgr2=0x55e131fe8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f275affd700 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131ecfd80 0x55e131fe8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f275affd700 1 -- 172.21.15.19:0/3095970318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e131fe59d0 con 0x55e131e55520 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f275affd700 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131e55520 0x55e131fe84c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f274c0a90d0 tx=0x7f274c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3b7a20b6b1fb4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f27427fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f27427fc700 1 -- 172.21.15.19:0/3095970318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c020040 con 0x55e131e55520 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f27427fc700 1 -- 172.21.15.19:0/3095970318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f274c010710 con 0x55e131e55520 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.651+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2754023070 con 0x55e131e55520 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f27427fc700 1 -- 172.21.15.19:0/3095970318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f274c0ad780 con 0x55e131e55520 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f276fdb8b80 10 client.17994 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e13205b7f0 con 0x55e131e55520 2024-05-09T01:35:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f27427fc700 1 -- 172.21.15.19:0/3095970318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2754023070 con 0x55e131e55520 2024-05-09T01:35:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f27427fc700 1 -- 172.21.15.19:0/3095970318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f274c0a8630 con 0x55e131e55520 2024-05-09T01:35:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f276fdb8b80 10 client.17994 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f276fdb8b80 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f276fdb8b80 20 client.17994 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e13205be80 0x55e13205e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f276fdb8b80 4 client.17994 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:35:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e131efcba0 con 0x55e13205be80 2024-05-09T01:35:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.652+0000 7f2753fff700 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e13205be80 0x55e13205e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.653+0000 7f27427fc700 10 client.17994 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.653+0000 7f2753fff700 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e13205be80 0x55e13205e2a0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95587426ef03ed02 server_cookie=e280e0b56c414dca in_seq=0 out_seq=0 2024-05-09T01:35:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f27427fc700 1 -- 172.21.15.19:0/3095970318 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e131efcba0 con 0x55e13205be80 2024-05-09T01:35:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f27427fc700 10 client.17994 handle_command_reply: tid=0 2024-05-09T01:35:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f276fdb8b80 1 client.17994 shutdown 2024-05-09T01:35:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131e55520 msgr2=0x55e131fe84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131e55520 0x55e131fe84c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f274c0a90d0 tx=0x7f274c005c80 comp rx=0 tx=0).stop 2024-05-09T01:35:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 shutdown_connections 2024-05-09T01:35:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e13205be80 0x55e13205e2a0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e131fdb260 0x55e131fe5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e131e55520 0x55e131fe84c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f276fdb8b80 1 --2- 172.21.15.19:0/3095970318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e131ecfd80 0x55e131fe8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f27427fc700 0 client.17994 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.654+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 >> 172.21.15.19:0/3095970318 conn(0x55e131e58ff0 msgr2=0x55e131fdab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 shutdown_connections 2024-05-09T01:35:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f276fdb8b80 1 -- 172.21.15.19:0/3095970318 wait complete. 2024-05-09T01:35:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f276fdb8b80 20 client.17994 trim_cache size 0 max 16384 2024-05-09T01:35:04.661 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:35:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:35:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "e7c3b7fa-f199-4749-bee4-61c521bedbfe", 2024-05-09T01:35:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:35:04.663 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:35:04.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 -- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f27480b2900 msgr2=0x7f27480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f27480b2900 0x7f27480b4e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2744003800 tx=0x7f274401a040 comp rx=0 tx=0).stop 2024-05-09T01:35:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 -- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c1054c0 msgr2=0x7f275c19ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c1054c0 0x7f275c19ade0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f274c00d950 tx=0x7f274c00dc60 comp rx=0 tx=0).stop 2024-05-09T01:35:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 -- 172.21.15.19:0/3308175619 shutdown_connections 2024-05-09T01:35:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f275c10e8f0 0x7f275c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f27480b2900 0x7f27480b4e10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c1054c0 0x7f275c19ade0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 --2- 172.21.15.19:0/3308175619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f275c105e10 0x7f275c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 -- 172.21.15.19:0/3308175619 >> 172.21.15.19:0/3308175619 conn(0x7f275c0fedc0 msgr2=0x7f275c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.655+0000 7f27613a2700 1 -- 172.21.15.19:0/3308175619 shutdown_connections 2024-05-09T01:35:04.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:04.656+0000 7f27613a2700 1 -- 172.21.15.19:0/3308175619 wait complete. 2024-05-09T01:35:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:04 smithi059 ceph-mon[34351]: pgmap v664: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:35:04.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:04 smithi196 ceph-mon[35041]: pgmap v664: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:35:05.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:35:05.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:05 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:35:05.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:05 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:05.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:05 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:35:05.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:05 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:05 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:35:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:05 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:05 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:35:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:05 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:05 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:35:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:05 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:05 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:35:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:05 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:06 smithi059 ceph-mon[34351]: pgmap v665: 97 pgs: 97 active+clean; 2.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:35:06.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:06 smithi019 ceph-mon[30158]: pgmap v665: 97 pgs: 97 active+clean; 2.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:35:06.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:06 smithi196 ceph-mon[35041]: pgmap v665: 97 pgs: 97 active+clean; 2.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:35:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.281+0000 7f5c513c9700 1 -- 172.21.15.19:0/2430454137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 msgr2=0x7f5c4c1102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.281+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2430454137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 0x7f5c4c1102d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400b0d0 tx=0x7f5c4400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.281+0000 7f5c513c9700 1 -- 172.21.15.19:0/2430454137 shutdown_connections 2024-05-09T01:35:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.281+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2430454137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 0x7f5c4c1102d0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.281+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2430454137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c103c10 0x7f5c4c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.281+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2430454137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108020 0x7f5c4c108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.281+0000 7f5c513c9700 1 -- 172.21.15.19:0/2430454137 >> 172.21.15.19:0/2430454137 conn(0x7f5c4c0fe540 msgr2=0x7f5c4c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.281+0000 7f5c513c9700 1 -- 172.21.15.19:0/2430454137 shutdown_connections 2024-05-09T01:35:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.281+0000 7f5c513c9700 1 -- 172.21.15.19:0/2430454137 wait complete. 2024-05-09T01:35:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c513c9700 1 Processor -- start 2024-05-09T01:35:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c513c9700 1 -- start start 2024-05-09T01:35:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c513c9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c103c10 0x7f5c4c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c513c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108020 0x7f5c4c19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c513c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 0x7f5c4c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c513c9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1135a0 con 0x7f5c4c108970 2024-05-09T01:35:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c513c9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c113420 con 0x7f5c4c108020 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c513c9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c113720 con 0x7f5c4c103c10 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c4b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 0x7f5c4c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c4b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 0x7f5c4c19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59656/0 (socket says 172.21.15.19:59656) 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c4b7fe700 1 -- 172.21.15.19:0/2102169980 learned_addr learned my addr 172.21.15.19:0/2102169980 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108020 0x7f5c4c19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.282+0000 7f5c4affd700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c103c10 0x7f5c4c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c4b7fe700 1 -- 172.21.15.19:0/2102169980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c103c10 msgr2=0x7f5c4c19afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c4b7fe700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c103c10 0x7f5c4c19afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c4b7fe700 1 -- 172.21.15.19:0/2102169980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108020 msgr2=0x7f5c4c19b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c4b7fe700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108020 0x7f5c4c19b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c4b7fe700 1 -- 172.21.15.19:0/2102169980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c19fe70 con 0x7f5c4c108970 2024-05-09T01:35:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108020 0x7f5c4c19b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c4b7fe700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 0x7f5c4c19f7d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5c440095c0 tx=0x7f5c44007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c33fff700 1 -- 172.21.15.19:0/2102169980 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4401f070 con 0x7f5c4c108970 2024-05-09T01:35:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c513c9700 1 -- 172.21.15.19:0/2102169980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4c1a0100 con 0x7f5c4c108970 2024-05-09T01:35:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c33fff700 1 -- 172.21.15.19:0/2102169980 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c44003c00 con 0x7f5c4c108970 2024-05-09T01:35:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c33fff700 1 -- 172.21.15.19:0/2102169980 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4401b6a0 con 0x7f5c4c108970 2024-05-09T01:35:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.283+0000 7f5c513c9700 1 -- 172.21.15.19:0/2102169980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4c1a05e0 con 0x7f5c4c108970 2024-05-09T01:35:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.286+0000 7f5c33fff700 1 -- 172.21.15.19:0/2102169980 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5c44022070 con 0x7f5c4c108970 2024-05-09T01:35:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c33fff700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2880 0x7f5c340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c33fff700 1 -- 172.21.15.19:0/2102169980 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5c4400f020 con 0x7f5c4c108970 2024-05-09T01:35:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c4affd700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2880 0x7f5c340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c5fddfb80 1 Processor -- start 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c5fddfb80 1 -- start start 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c4affd700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2880 0x7f5c340b4d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c005950 tx=0x7f5c3c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df31ad00 0x5611df421ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 0x5611df422030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 0x5611df4265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611df3400a0 con 0x5611df31ad00 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5611df2e59a0 con 0x5611df4261e0 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5611df33c060 con 0x5611df2a04a0 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.287+0000 7f5c4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df31ad00 0x5611df421ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df31ad00 0x5611df421ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59660/0 (socket says 172.21.15.19:59660) 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4affd700 1 -- 172.21.15.19:0/747939188 learned_addr learned my addr 172.21.15.19:0/747939188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4b7fe700 1 --2- 172.21.15.19:0/747939188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 0x5611df4265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/747939188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 0x5611df422030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4affd700 1 -- 172.21.15.19:0/747939188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 msgr2=0x5611df422030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4affd700 1 --2- 172.21.15.19:0/747939188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 0x5611df422030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4affd700 1 -- 172.21.15.19:0/747939188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 msgr2=0x5611df4265f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4affd700 1 --2- 172.21.15.19:0/747939188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 0x5611df4265f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4affd700 1 -- 172.21.15.19:0/747939188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611df2d6500 con 0x5611df31ad00 2024-05-09T01:35:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c4affd700 1 --2- 172.21.15.19:0/747939188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df31ad00 0x5611df421ad0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00fd70 tx=0x7f5c3c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63dc833ab50f119 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c317fa700 1 -- 172.21.15.19:0/747939188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c022070 con 0x5611df31ad00 2024-05-09T01:35:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c317fa700 1 -- 172.21.15.19:0/747939188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c011bb0 con 0x5611df31ad00 2024-05-09T01:35:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.288+0000 7f5c317fa700 1 -- 172.21.15.19:0/747939188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c01d610 con 0x5611df31ad00 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.289+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/747939188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df31ad00 msgr2=0x5611df421ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.289+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/747939188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df31ad00 0x5611df421ad0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00fd70 tx=0x7f5c3c012000 comp rx=0 tx=0).stop 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.289+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/747939188 shutdown_connections 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.289+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/747939188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 0x5611df4265f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.289+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/747939188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df31ad00 0x5611df421ad0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.289+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/747939188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 0x5611df422030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.289+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/747939188 >> 172.21.15.19:0/747939188 conn(0x5611df2a3f70 msgr2=0x5611df418220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.289+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/747939188 shutdown_connections 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.289+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/747939188 wait complete. 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c5fddfb80 1 -- start start 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c5fddfb80 1 Processor -- start 2024-05-09T01:35:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df42b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df31ad00 0x5611df42b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 0x5611df42be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611df426f00 con 0x5611df2a04a0 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5611df2dab20 con 0x5611df4261e0 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5611df42e470 con 0x5611df31ad00 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df42b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df42b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59672/0 (socket says 172.21.15.19:59672) 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c4affd700 1 -- 172.21.15.19:0/2953705291 learned_addr learned my addr 172.21.15.19:0/2953705291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c5fddfb80 10 client.0 _ll_register_callbacks cb 0x5611df1b16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.290+0000 7f5c4b7fe700 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 0x5611df42be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df31ad00 0x5611df42b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c4affd700 1 -- 172.21.15.19:0/2953705291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df31ad00 msgr2=0x5611df42b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:07.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c4affd700 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df31ad00 0x5611df42b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c4affd700 1 -- 172.21.15.19:0/2953705291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 msgr2=0x5611df42be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c4affd700 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 0x5611df42be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c4affd700 1 -- 172.21.15.19:0/2953705291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611df1bb490 con 0x5611df2a04a0 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df31ad00 0x5611df42b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c4b7fe700 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 0x5611df42be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c4affd700 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df42b390 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00b4d0 tx=0x7f5c3c011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7149945b68e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c2eff5700 1 -- 172.21.15.19:0/2953705291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c035060 con 0x5611df2a04a0 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c2eff5700 1 -- 172.21.15.19:0/2953705291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5c3c005260 con 0x5611df2a04a0 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/2953705291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c3c011bb0 con 0x5611df2a04a0 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.291+0000 7f5c2eff5700 1 -- 172.21.15.19:0/2953705291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c3c022070 con 0x5611df2a04a0 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c5fddfb80 10 client.18006 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/2953705291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611df438d20 con 0x5611df2a04a0 2024-05-09T01:35:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c2eff5700 1 -- 172.21.15.19:0/2953705291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c011bb0 con 0x5611df2a04a0 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c2eff5700 1 -- 172.21.15.19:0/2953705291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5c3c0049e0 con 0x5611df2a04a0 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c5fddfb80 10 client.18006 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c5fddfb80 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c5fddfb80 20 client.18006 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5611df43b2a0 0x5611df43d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c5fddfb80 4 client.18006 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/2953705291 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611df34da80 con 0x5611df43b2a0 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.292+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5611df43b2a0 0x5611df43d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.293+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5611df43b2a0 0x5611df43d6c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865c0aea70337ca6 server_cookie=56b2fabf260dd963 in_seq=0 out_seq=0 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.293+0000 7f5c2eff5700 10 client.18006 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.293+0000 7f5c2eff5700 1 -- 172.21.15.19:0/2953705291 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5611df34da80 con 0x5611df43b2a0 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.293+0000 7f5c2eff5700 10 client.18006 handle_command_reply: tid=0 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 client.18006 shutdown 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/2953705291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 msgr2=0x5611df42b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df42b390 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00b4d0 tx=0x7f5c3c011f90 comp rx=0 tx=0).stop 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/2953705291 shutdown_connections 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5611df43b2a0 0x5611df43d6c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df4261e0 0x5611df42be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df42b390 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/2953705291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df31ad00 0x5611df42b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c2eff5700 0 client.18006 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/2953705291 >> 172.21.15.19:0/2953705291 conn(0x5611df2a3f70 msgr2=0x5611df411ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/2953705291 shutdown_connections 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/2953705291 wait complete. 2024-05-09T01:35:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.294+0000 7f5c5fddfb80 20 client.18006 trim_cache size 0 max 16384 2024-05-09T01:35:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.314+0000 7f5c5fddfb80 1 Processor -- start 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.314+0000 7f5c5fddfb80 1 -- start start 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df4375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df41ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df4261e0 0x5611df42bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1aca20 con 0x5611df2a04a0 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1ae560 con 0x5611df31ad00 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1ad0f0 con 0x5611df4261e0 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df4375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df4375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59676/0 (socket says 172.21.15.19:59676) 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4affd700 1 -- 172.21.15.19:0/3883538230 learned_addr learned my addr 172.21.15.19:0/3883538230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4b7fe700 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df4261e0 0x5611df42bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df41ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4affd700 1 -- 172.21.15.19:0/3883538230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df4261e0 msgr2=0x5611df42bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4affd700 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df4261e0 0x5611df42bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4affd700 1 -- 172.21.15.19:0/3883538230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 msgr2=0x5611df41ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4affd700 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df41ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4affd700 1 -- 172.21.15.19:0/3883538230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611df4149d0 con 0x5611df2a04a0 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4b7fe700 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df4261e0 0x5611df42bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df41ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.315+0000 7f5c4affd700 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df4375b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00f6f0 tx=0x7f5c3c00fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fb8b736713269c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c30ff9700 1 -- 172.21.15.19:0/3883538230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c015070 con 0x5611df2a04a0 2024-05-09T01:35:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c30ff9700 1 -- 172.21.15.19:0/3883538230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c004710 con 0x5611df2a04a0 2024-05-09T01:35:07.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c30ff9700 1 -- 172.21.15.19:0/3883538230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c035070 con 0x5611df2a04a0 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/3883538230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 msgr2=0x5611df4375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df4375b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00f6f0 tx=0x7f5c3c00fcf0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/3883538230 shutdown_connections 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df4261e0 0x5611df42bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df2a04a0 0x5611df4375b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/3883538230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df41ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/3883538230 >> 172.21.15.19:0/3883538230 conn(0x5611df2a3f70 msgr2=0x5611df42a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/3883538230 shutdown_connections 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.316+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/3883538230 wait complete. 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c5fddfb80 1 -- start start 2024-05-09T01:35:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c5fddfb80 1 Processor -- start 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 0x5611df423a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df423f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c5fddfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df4261e0 0x5611df4244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1ade90 con 0x5611df4261e0 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1ad7c0 con 0x5611df31ad00 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c5fddfb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1aa9c0 con 0x5611df2a04a0 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c5fddfb80 10 client.0 _ll_register_callbacks cb 0x5611df1b16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c4b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df4261e0 0x5611df4244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c4a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df423f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c4b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df4261e0 0x5611df4244a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59678/0 (socket says 172.21.15.19:59678) 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.317+0000 7f5c4b7fe700 1 -- 172.21.15.19:0/1102642169 learned_addr learned my addr 172.21.15.19:0/1102642169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c4affd700 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 0x5611df423a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c4a7fc700 1 -- 172.21.15.19:0/1102642169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 msgr2=0x5611df423a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 0x5611df423a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c4a7fc700 1 -- 172.21.15.19:0/1102642169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df4261e0 msgr2=0x5611df4244a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df4261e0 0x5611df4244a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c4a7fc700 1 -- 172.21.15.19:0/1102642169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611df4a47e0 con 0x5611df31ad00 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c4b7fe700 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df4261e0 0x5611df4244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c4a7fc700 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df423f60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5611df426f90 tx=0x7f5c4000ddc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80368a4778889c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c2eff5700 1 -- 172.21.15.19:0/1102642169 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40021020 con 0x5611df31ad00 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c2eff5700 1 -- 172.21.15.19:0/1102642169 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5c40003d70 con 0x5611df31ad00 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/1102642169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c3c004710 con 0x5611df31ad00 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c2eff5700 1 -- 172.21.15.19:0/1102642169 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c4002c050 con 0x5611df31ad00 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c5fddfb80 10 client.27359 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.318+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/1102642169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611df4a5820 con 0x5611df31ad00 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.319+0000 7f5c2eff5700 1 -- 172.21.15.19:0/1102642169 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c004710 con 0x5611df31ad00 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.319+0000 7f5c2eff5700 1 -- 172.21.15.19:0/1102642169 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5c40011900 con 0x5611df31ad00 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.319+0000 7f5c5fddfb80 10 client.27359 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.319+0000 7f5c5fddfb80 10 client.27359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.319+0000 7f5c5fddfb80 20 client.27359 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.319+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5611df4a5f50 0x5611df4a8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.319+0000 7f5c5fddfb80 4 client.27359 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.319+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/1102642169 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5611df347b20 con 0x5611df4a5f50 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.319+0000 7f5c4affd700 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5611df4a5f50 0x5611df4a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c4affd700 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5611df4a5f50 0x5611df4a8370 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85733f190c9c5ca8 server_cookie=d005b5068d4c0d3f in_seq=0 out_seq=0 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c2eff5700 10 client.27359 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c2eff5700 1 -- 172.21.15.19:0/1102642169 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5611df347b20 con 0x5611df4a5f50 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c2eff5700 10 client.27359 handle_command_reply: tid=0 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 client.27359 shutdown 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/1102642169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 msgr2=0x5611df423f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df423f60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5611df426f90 tx=0x7f5c4000ddc0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/1102642169 shutdown_connections 2024-05-09T01:35:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5611df4a5f50 0x5611df4a8370 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611df4261e0 0x5611df4244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5611df2a04a0 0x5611df423a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c2eff5700 0 client.27359 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 --2- 172.21.15.19:0/1102642169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611df31ad00 0x5611df423f60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/1102642169 >> 172.21.15.19:0/1102642169 conn(0x5611df2a3f70 msgr2=0x5611df42c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/1102642169 shutdown_connections 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 1 -- 172.21.15.19:0/1102642169 wait complete. 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.320+0000 7f5c5fddfb80 20 client.27359 trim_cache size 0 max 16384 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:35:07.323 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:35:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 -- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2880 msgr2=0x7f5c340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2880 0x7f5c340b4d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c005950 tx=0x7f5c3c0058e0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 -- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 msgr2=0x7f5c4c19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 0x7f5c4c19f7d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5c440095c0 tx=0x7f5c44007ad0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 -- 172.21.15.19:0/2102169980 shutdown_connections 2024-05-09T01:35:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f5c340b2880 0x7f5c340b4d90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c4c108970 0x7f5c4c19f7d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c4c103c10 0x7f5c4c19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 --2- 172.21.15.19:0/2102169980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c4c108020 0x7f5c4c19b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 -- 172.21.15.19:0/2102169980 >> 172.21.15.19:0/2102169980 conn(0x7f5c4c0fe540 msgr2=0x7f5c4c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 -- 172.21.15.19:0/2102169980 shutdown_connections 2024-05-09T01:35:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:07.321+0000 7f5c513c9700 1 -- 172.21.15.19:0/2102169980 wait complete. 2024-05-09T01:35:07.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:07 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:35:07.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:07 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:35:07.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:07 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:35:08.028 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:35:08.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:35:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.216+0000 7f351b669640 1 -- 172.21.15.19:0/1027830177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514103dc0 msgr2=0x7f35141041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.216+0000 7f351b669640 1 --2- 172.21.15.19:0/1027830177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514103dc0 0x7f35141041d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3504009900 tx=0x7f350402a3f0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.217+0000 7f351b669640 1 -- 172.21.15.19:0/1027830177 shutdown_connections 2024-05-09T01:35:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.217+0000 7f351b669640 1 --2- 172.21.15.19:0/1027830177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514108ac0 0x7f351410aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.217+0000 7f351b669640 1 --2- 172.21.15.19:0/1027830177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514103dc0 0x7f35141041d0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.217+0000 7f351b669640 1 --2- 172.21.15.19:0/1027830177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3514102b80 0x7f3514103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.217+0000 7f351b669640 1 -- 172.21.15.19:0/1027830177 >> 172.21.15.19:0/1027830177 conn(0x7f35140fdd90 msgr2=0x7f3514100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.217+0000 7f351b669640 1 -- 172.21.15.19:0/1027830177 shutdown_connections 2024-05-09T01:35:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.217+0000 7f351b669640 1 -- 172.21.15.19:0/1027830177 wait complete. 2024-05-09T01:35:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.217+0000 7f351b669640 1 Processor -- start 2024-05-09T01:35:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.217+0000 7f351b669640 1 -- start start 2024-05-09T01:35:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f351b669640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3514102b80 0x7f351406bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f351b669640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514103dc0 0x7f351419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f351b669640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514108ac0 0x7f35141a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f351b669640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f351410e6d0 con 0x7f3514108ac0 2024-05-09T01:35:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f351b669640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f351410e550 con 0x7f3514102b80 2024-05-09T01:35:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f351b669640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f351410e850 con 0x7f3514103dc0 2024-05-09T01:35:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3519c06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514108ac0 0x7f35141a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3519c06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514108ac0 0x7f35141a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59694/0 (socket says 172.21.15.19:59694) 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3519c06640 1 -- 172.21.15.19:0/1069038433 learned_addr learned my addr 172.21.15.19:0/1069038433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3519405640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3514102b80 0x7f351406bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3518c04640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514103dc0 0x7f351419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3519c06640 1 -- 172.21.15.19:0/1069038433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514103dc0 msgr2=0x7f351419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3519c06640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514103dc0 0x7f351419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3519c06640 1 -- 172.21.15.19:0/1069038433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3514102b80 msgr2=0x7f351406bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3519c06640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3514102b80 0x7f351406bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.218+0000 7f3519c06640 1 -- 172.21.15.19:0/1069038433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35141a1e60 con 0x7f3514108ac0 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.219+0000 7f3519c06640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514108ac0 0x7f35141a1640 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f350c00eb10 tx=0x7f350c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.219+0000 7f350a7fc640 1 -- 172.21.15.19:0/1069038433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c004b20 con 0x7f3514108ac0 2024-05-09T01:35:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.219+0000 7f350a7fc640 1 -- 172.21.15.19:0/1069038433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c004cc0 con 0x7f3514108ac0 2024-05-09T01:35:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.219+0000 7f351b669640 1 -- 172.21.15.19:0/1069038433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35141a2150 con 0x7f3514108ac0 2024-05-09T01:35:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.219+0000 7f350a7fc640 1 -- 172.21.15.19:0/1069038433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c010670 con 0x7f3514108ac0 2024-05-09T01:35:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.219+0000 7f351b669640 1 -- 172.21.15.19:0/1069038433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35141a26a0 con 0x7f3514108ac0 2024-05-09T01:35:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.223+0000 7f35225f0740 1 Processor -- start 2024-05-09T01:35:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.223+0000 7f350a7fc640 1 -- 172.21.15.19:0/1069038433 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f350c010830 con 0x7f3514108ac0 2024-05-09T01:35:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.223+0000 7f35225f0740 1 -- start start 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.223+0000 7f35225f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689795d1b0 0x556897d05910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.223+0000 7f35225f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 0x556897d05e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.223+0000 7f35225f0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689793c4d0 0x556897d0cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.223+0000 7f35225f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556897c657c0 con 0x55689795d1b0 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.223+0000 7f35225f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556897c010e0 con 0x5568979342f0 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.223+0000 7f35225f0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556897c61780 con 0x55689793c4d0 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689795d1b0 0x556897d05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689795d1b0 0x556897d05910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59708/0 (socket says 172.21.15.19:59708) 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 -- 172.21.15.19:0/456742370 learned_addr learned my addr 172.21.15.19:0/456742370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3519405640 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 0x556897d05e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689793c4d0 0x556897d0cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 -- 172.21.15.19:0/456742370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689793c4d0 msgr2=0x556897d0cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689793c4d0 0x556897d0cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 -- 172.21.15.19:0/456742370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 msgr2=0x556897d05e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 0x556897d05e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 -- 172.21.15.19:0/456742370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556897c2a7c0 con 0x55689795d1b0 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3519405640 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 0x556897d05e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f350a7fc640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f35000b2860 0x7f35000b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3518c04640 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689795d1b0 0x556897d05910 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3504009230 tx=0x7f3504005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f1a383d68937ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f3519405640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f35000b2860 0x7f35000b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.224+0000 7f350a7fc640 1 -- 172.21.15.19:0/1069038433 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f350c0e28d0 con 0x7f3514108ac0 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.225+0000 7f34ff7fe640 1 -- 172.21.15.19:0/456742370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504043020 con 0x55689795d1b0 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.225+0000 7f34ff7fe640 1 -- 172.21.15.19:0/456742370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35040046c0 con 0x55689795d1b0 2024-05-09T01:35:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.225+0000 7f34ff7fe640 1 -- 172.21.15.19:0/456742370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504035070 con 0x55689795d1b0 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 -- 172.21.15.19:0/456742370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689795d1b0 msgr2=0x556897d05910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f3519405640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f35000b2860 0x7f35000b4d70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x556897d06cf0 tx=0x7f3510005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689795d1b0 0x556897d05910 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3504009230 tx=0x7f3504005dd0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 -- 172.21.15.19:0/456742370 shutdown_connections 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689793c4d0 0x556897d0cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 0x556897d05e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 --2- 172.21.15.19:0/456742370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689795d1b0 0x556897d05910 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 -- 172.21.15.19:0/456742370 >> 172.21.15.19:0/456742370 conn(0x556897951300 msgr2=0x556897cfce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 -- 172.21.15.19:0/456742370 shutdown_connections 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 -- 172.21.15.19:0/456742370 wait complete. 2024-05-09T01:35:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 -- start start 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.226+0000 7f35225f0740 1 Processor -- start 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f35225f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d0f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f35225f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55689793c4d0 0x556897d0fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f35225f0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689795d1b0 0x556897d16bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f35225f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556897d0d780 con 0x5568979342f0 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f35225f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556897bec9c0 con 0x55689793c4d0 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f35225f0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556897d0d900 con 0x55689795d1b0 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f35225f0740 10 client.0 _ll_register_callbacks cb 0x556897c19020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519405640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d0f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519405640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d0f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59710/0 (socket says 172.21.15.19:59710) 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519405640 1 -- 172.21.15.19:0/75373807 learned_addr learned my addr 172.21.15.19:0/75373807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3518c04640 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55689793c4d0 0x556897d0fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519c06640 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689795d1b0 0x556897d16bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519405640 1 -- 172.21.15.19:0/75373807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689795d1b0 msgr2=0x556897d16bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519405640 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689795d1b0 0x556897d16bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519405640 1 -- 172.21.15.19:0/75373807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55689793c4d0 msgr2=0x556897d0fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519405640 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55689793c4d0 0x556897d0fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519405640 1 -- 172.21.15.19:0/75373807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556897c131f0 con 0x5568979342f0 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3518c04640 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55689793c4d0 0x556897d0fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f3519405640 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d0f780 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f3510011c30 tx=0x7f35100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d889716eecdd830 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.227+0000 7f34fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f34fcff9640 1 -- 172.21.15.19:0/75373807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510010ce0 con 0x5568979342f0 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f34fcff9640 1 -- 172.21.15.19:0/75373807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3510005020 con 0x5568979342f0 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f35225f0740 1 -- 172.21.15.19:0/75373807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35040046c0 con 0x5568979342f0 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f34fcff9640 1 -- 172.21.15.19:0/75373807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f351002e070 con 0x5568979342f0 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f35225f0740 10 client.18024 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f35225f0740 1 -- 172.21.15.19:0/75373807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568979c8790 con 0x5568979342f0 2024-05-09T01:35:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f34fcff9640 1 -- 172.21.15.19:0/75373807 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35040046c0 con 0x5568979342f0 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f34fcff9640 1 -- 172.21.15.19:0/75373807 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3510028400 con 0x5568979342f0 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f35225f0740 10 client.18024 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f35225f0740 10 client.18024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f35225f0740 20 client.18024 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f35225f0740 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55689791bd80 0x556897d1d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f35225f0740 4 client.18024 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.228+0000 7f35225f0740 1 -- 172.21.15.19:0/75373807 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556897c733d0 con 0x55689791bd80 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.229+0000 7f3518c04640 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55689791bd80 0x556897d1d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.229+0000 7f34fcff9640 10 client.18024 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.229+0000 7f3518c04640 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55689791bd80 0x556897d1d170 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354528bfc9d8c171 server_cookie=2e1799a3667391f in_seq=0 out_seq=0 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f34fcff9640 1 -- 172.21.15.19:0/75373807 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556897c733d0 con 0x55689791bd80 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f34fcff9640 10 client.18024 handle_command_reply: tid=0 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 client.18024 shutdown 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 -- 172.21.15.19:0/75373807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 msgr2=0x556897d0f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d0f780 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f3510011c30 tx=0x7f35100046c0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 -- 172.21.15.19:0/75373807 shutdown_connections 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55689791bd80 0x556897d1d170 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689795d1b0 0x556897d16bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f34fcff9640 0 client.18024 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55689793c4d0 0x556897d0fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 --2- 172.21.15.19:0/75373807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d0f780 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 -- 172.21.15.19:0/75373807 >> 172.21.15.19:0/75373807 conn(0x556897951300 msgr2=0x556897cfb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 -- 172.21.15.19:0/75373807 shutdown_connections 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.230+0000 7f35225f0740 1 -- 172.21.15.19:0/75373807 wait complete. 2024-05-09T01:35:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.231+0000 7f35225f0740 20 client.18024 trim_cache size 0 max 16384 2024-05-09T01:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.247+0000 7f35225f0740 1 Processor -- start 2024-05-09T01:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.247+0000 7f35225f0740 1 -- start start 2024-05-09T01:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.247+0000 7f35225f0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556897951300 0x556897d0fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.247+0000 7f35225f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689791bd80 0x556897d16b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.247+0000 7f35225f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 0x556897d17080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.247+0000 7f35225f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35141a91f0 con 0x55689791bd80 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.247+0000 7f35225f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35141aad30 con 0x5568979342f0 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.247+0000 7f35225f0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35141a98c0 con 0x556897951300 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3518c04640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689791bd80 0x556897d16b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3518c04640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689791bd80 0x556897d16b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59726/0 (socket says 172.21.15.19:59726) 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3518c04640 1 -- 172.21.15.19:0/2506646060 learned_addr learned my addr 172.21.15.19:0/2506646060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3519c06640 1 --2- 172.21.15.19:0/2506646060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 0x556897d17080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3519405640 1 --2- 172.21.15.19:0/2506646060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556897951300 0x556897d0fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3518c04640 1 -- 172.21.15.19:0/2506646060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556897951300 msgr2=0x556897d0fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3518c04640 1 --2- 172.21.15.19:0/2506646060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556897951300 0x556897d0fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3518c04640 1 -- 172.21.15.19:0/2506646060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 msgr2=0x556897d17080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3518c04640 1 --2- 172.21.15.19:0/2506646060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 0x556897d17080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3518c04640 1 -- 172.21.15.19:0/2506646060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556897a3f5f0 con 0x55689791bd80 2024-05-09T01:35:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f3518c04640 1 --2- 172.21.15.19:0/2506646060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689791bd80 0x556897d16b40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f3504031440 tx=0x7f3504004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dc01bdba7f84b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f34feffd640 1 -- 172.21.15.19:0/2506646060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504043070 con 0x55689791bd80 2024-05-09T01:35:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f34feffd640 1 -- 172.21.15.19:0/2506646060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3504056070 con 0x55689791bd80 2024-05-09T01:35:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.248+0000 7f34feffd640 1 -- 172.21.15.19:0/2506646060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504035040 con 0x55689791bd80 2024-05-09T01:35:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.249+0000 7f35225f0740 1 -- 172.21.15.19:0/2506646060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689791bd80 msgr2=0x556897d16b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.249+0000 7f35225f0740 1 --2- 172.21.15.19:0/2506646060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689791bd80 0x556897d16b40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f3504031440 tx=0x7f3504004310 comp rx=0 tx=0).stop 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.249+0000 7f35225f0740 1 -- 172.21.15.19:0/2506646060 shutdown_connections 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.249+0000 7f35225f0740 1 --2- 172.21.15.19:0/2506646060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568979342f0 0x556897d17080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.249+0000 7f35225f0740 1 --2- 172.21.15.19:0/2506646060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689791bd80 0x556897d16b40 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.249+0000 7f35225f0740 1 --2- 172.21.15.19:0/2506646060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556897951300 0x556897d0fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.249+0000 7f35225f0740 1 -- 172.21.15.19:0/2506646060 >> 172.21.15.19:0/2506646060 conn(0x556897d0a450 msgr2=0x556897d0f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.249+0000 7f35225f0740 1 -- 172.21.15.19:0/2506646060 shutdown_connections 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.250+0000 7f35225f0740 1 -- 172.21.15.19:0/2506646060 wait complete. 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.250+0000 7f35225f0740 1 -- start start 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.250+0000 7f35225f0740 1 Processor -- start 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.250+0000 7f35225f0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689791bd80 0x556897d14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.250+0000 7f35225f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d14bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f35225f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556897951300 0x556897d15a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f35225f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35141aa660 con 0x5568979342f0 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f35225f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35141a9f90 con 0x556897951300 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f35225f0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35141a71d0 con 0x55689791bd80 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f35225f0740 10 client.0 _ll_register_callbacks cb 0x7f3510009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3518c04640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d14bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3518c04640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d14bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59742/0 (socket says 172.21.15.19:59742) 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3518c04640 1 -- 172.21.15.19:0/561735899 learned_addr learned my addr 172.21.15.19:0/561735899 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3519c06640 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556897951300 0x556897d15a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3519405640 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689791bd80 0x556897d14670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3518c04640 1 -- 172.21.15.19:0/561735899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689791bd80 msgr2=0x556897d14670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3518c04640 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689791bd80 0x556897d14670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3518c04640 1 -- 172.21.15.19:0/561735899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556897951300 msgr2=0x556897d15a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3518c04640 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556897951300 0x556897d15a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3518c04640 1 -- 172.21.15.19:0/561735899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556897c29ed0 con 0x5568979342f0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3519405640 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689791bd80 0x556897d14670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f3518c04640 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d14bb0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f3504009680 tx=0x7f350402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82e8c7f6f5c950f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f34fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f34fcff9640 1 -- 172.21.15.19:0/561735899 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504043070 con 0x5568979342f0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f34fcff9640 1 -- 172.21.15.19:0/561735899 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3504005200 con 0x5568979342f0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.251+0000 7f35225f0740 1 -- 172.21.15.19:0/561735899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3504056070 con 0x5568979342f0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f34fcff9640 1 -- 172.21.15.19:0/561735899 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3504035040 con 0x5568979342f0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f35225f0740 10 client.18030 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f35225f0740 1 -- 172.21.15.19:0/561735899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556897d170f0 con 0x5568979342f0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f34fcff9640 1 -- 172.21.15.19:0/561735899 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3504056070 con 0x5568979342f0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f34fcff9640 1 -- 172.21.15.19:0/561735899 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f35040035f0 con 0x5568979342f0 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f35225f0740 10 client.18030 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f35225f0740 10 client.18030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f35225f0740 20 client.18030 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f35225f0740 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55689793c4d0 0x556897d87640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f35225f0740 4 client.18030 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:35:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.252+0000 7f35225f0740 1 -- 172.21.15.19:0/561735899 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556897c6d240 con 0x55689793c4d0 2024-05-09T01:35:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.253+0000 7f3519405640 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55689793c4d0 0x556897d87640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.253+0000 7f3519405640 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55689793c4d0 0x556897d87640 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d88912f2c028db server_cookie=8e729f6554b56180 in_seq=0 out_seq=0 2024-05-09T01:35:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.253+0000 7f34fcff9640 10 client.18030 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f34fcff9640 1 -- 172.21.15.19:0/561735899 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556897c6d240 con 0x55689793c4d0 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f34fcff9640 10 client.18030 handle_command_reply: tid=0 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f35225f0740 1 client.18030 shutdown 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f35225f0740 1 -- 172.21.15.19:0/561735899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 msgr2=0x556897d14bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f35225f0740 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d14bb0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f3504009680 tx=0x7f350402a3f0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f35225f0740 1 -- 172.21.15.19:0/561735899 shutdown_connections 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f35225f0740 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55689793c4d0 0x556897d87640 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f35225f0740 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556897951300 0x556897d15a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f35225f0740 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568979342f0 0x556897d14bb0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f34fcff9640 0 client.18030 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f35225f0740 1 --2- 172.21.15.19:0/561735899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55689791bd80 0x556897d14670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.268+0000 7f35225f0740 1 -- 172.21.15.19:0/561735899 >> 172.21.15.19:0/561735899 conn(0x556897d0a450 msgr2=0x556897d0f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f35225f0740 1 -- 172.21.15.19:0/561735899 shutdown_connections 2024-05-09T01:35:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f35225f0740 1 -- 172.21.15.19:0/561735899 wait complete. 2024-05-09T01:35:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f35225f0740 20 client.18030 trim_cache size 0 max 16384 2024-05-09T01:35:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:35:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:35:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:35:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 -- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f35000b2860 msgr2=0x7f35000b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f35000b2860 0x7f35000b4d70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x556897d06cf0 tx=0x7f3510005f90 comp rx=0 tx=0).stop 2024-05-09T01:35:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 -- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514108ac0 msgr2=0x7f35141a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514108ac0 0x7f35141a1640 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f350c00eb10 tx=0x7f350c00c2d0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 -- 172.21.15.19:0/1069038433 shutdown_connections 2024-05-09T01:35:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f35000b2860 0x7f35000b4d70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3514108ac0 0x7f35141a1640 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514103dc0 0x7f351419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 --2- 172.21.15.19:0/1069038433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3514102b80 0x7f351406bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 -- 172.21.15.19:0/1069038433 >> 172.21.15.19:0/1069038433 conn(0x7f35140fdd90 msgr2=0x7f3514104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 -- 172.21.15.19:0/1069038433 shutdown_connections 2024-05-09T01:35:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:08.269+0000 7f351b669640 1 -- 172.21.15.19:0/1069038433 wait complete. 2024-05-09T01:35:08.287 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:08.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:08 smithi019 ceph-mon[30158]: pgmap v666: 97 pgs: 97 active+clean; 2.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:08 smithi059 ceph-mon[34351]: pgmap v666: 97 pgs: 97 active+clean; 2.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:08.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:08 smithi196 ceph-mon[35041]: pgmap v666: 97 pgs: 97 active+clean; 2.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.896+0000 7f301b7e0700 1 -- 172.21.15.19:0/1701714666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3014109880 msgr2=0x7f30141111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.896+0000 7f301b7e0700 1 --2- 172.21.15.19:0/1701714666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3014109880 0x7f30141111e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f300c00b0a0 tx=0x7f300c00b3b0 comp rx=0 tx=0).stop 2024-05-09T01:35:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.896+0000 7f301b7e0700 1 -- 172.21.15.19:0/1701714666 shutdown_connections 2024-05-09T01:35:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.896+0000 7f301b7e0700 1 --2- 172.21.15.19:0/1701714666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3014109880 0x7f30141111e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.896+0000 7f301b7e0700 1 --2- 172.21.15.19:0/1701714666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3014104650 0x7f3014104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.896+0000 7f301b7e0700 1 --2- 172.21.15.19:0/1701714666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3014108f30 0x7f3014109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.896+0000 7f301b7e0700 1 -- 172.21.15.19:0/1701714666 >> 172.21.15.19:0/1701714666 conn(0x7f3014079f00 msgr2=0x7f301407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.896+0000 7f301b7e0700 1 -- 172.21.15.19:0/1701714666 shutdown_connections 2024-05-09T01:35:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.896+0000 7f301b7e0700 1 -- 172.21.15.19:0/1701714666 wait complete. 2024-05-09T01:35:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f301b7e0700 1 Processor -- start 2024-05-09T01:35:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f301b7e0700 1 -- start start 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f301b7e0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3014104650 0x7f3014075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f301b7e0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3014108f30 0x7f3014076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f301b7e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3014109880 0x7f3014076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f301b7e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30141144b0 con 0x7f3014109880 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f301b7e0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3014114330 con 0x7f3014108f30 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f301b7e0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3014114630 con 0x7f3014104650 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f3019d7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3014109880 0x7f3014076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f3019d7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3014109880 0x7f3014076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42468/0 (socket says 172.21.15.19:42468) 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f3019d7d700 1 -- 172.21.15.19:0/2450106781 learned_addr learned my addr 172.21.15.19:0/2450106781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f3018d7b700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3014108f30 0x7f3014076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.897+0000 7f301957c700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3014104650 0x7f3014075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f3019d7d700 1 -- 172.21.15.19:0/2450106781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3014104650 msgr2=0x7f3014075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f3019d7d700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3014104650 0x7f3014075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f3019d7d700 1 -- 172.21.15.19:0/2450106781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3014108f30 msgr2=0x7f3014076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f3019d7d700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3014108f30 0x7f3014076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f3019d7d700 1 -- 172.21.15.19:0/2450106781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30141a9160 con 0x7f3014109880 2024-05-09T01:35:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f3019d7d700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3014109880 0x7f3014076540 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f300c00b780 tx=0x7f300c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f300a7fc700 1 -- 172.21.15.19:0/2450106781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c021070 con 0x7f3014109880 2024-05-09T01:35:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f300a7fc700 1 -- 172.21.15.19:0/2450106781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300c007b80 con 0x7f3014109880 2024-05-09T01:35:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30141a93f0 con 0x7f3014109880 2024-05-09T01:35:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f300a7fc700 1 -- 172.21.15.19:0/2450106781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c005040 con 0x7f3014109880 2024-05-09T01:35:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.898+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30141a98a0 con 0x7f3014109880 2024-05-09T01:35:09.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.899+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30140512c0 con 0x7f3014109880 2024-05-09T01:35:09.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.901+0000 7f300a7fc700 1 -- 172.21.15.19:0/2450106781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f300c024070 con 0x7f3014109880 2024-05-09T01:35:09.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.901+0000 7f300a7fc700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30000b2880 0x7f30000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:09.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.902+0000 7f301957c700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30000b2880 0x7f30000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:09.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.902+0000 7f300a7fc700 1 -- 172.21.15.19:0/2450106781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f300c00f020 con 0x7f3014109880 2024-05-09T01:35:09.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.902+0000 7f301957c700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30000b2880 0x7f30000b4d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3010009c60 tx=0x7f3010009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:09.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:09.906+0000 7f300a7fc700 1 -- 172.21.15.19:0/2450106781 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f300c02b020 con 0x7f3014109880 2024-05-09T01:35:10.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.168+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30141a9a70 con 0x7f3014109880 2024-05-09T01:35:10.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.169+0000 7f300a7fc700 1 -- 172.21.15.19:0/2450106781 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f300c0ab3d0 con 0x7f3014109880 2024-05-09T01:35:10.166 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:35:10.166 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:35:10.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30000b2880 msgr2=0x7f30000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:10.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30000b2880 0x7f30000b4d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3010009c60 tx=0x7f3010009260 comp rx=0 tx=0).stop 2024-05-09T01:35:10.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3014109880 msgr2=0x7f3014076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:10.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3014109880 0x7f3014076540 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f300c00b780 tx=0x7f300c004d00 comp rx=0 tx=0).stop 2024-05-09T01:35:10.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 shutdown_connections 2024-05-09T01:35:10.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30000b2880 0x7f30000b4d90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3014109880 0x7f3014076540 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3014104650 0x7f3014075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 --2- 172.21.15.19:0/2450106781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3014108f30 0x7f3014076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 >> 172.21.15.19:0/2450106781 conn(0x7f3014079f00 msgr2=0x7f301410b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 shutdown_connections 2024-05-09T01:35:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:10.171+0000 7f301b7e0700 1 -- 172.21.15.19:0/2450106781 wait complete. 2024-05-09T01:35:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:35:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:10 smithi019 ceph-mon[30158]: pgmap v667: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-05-09T01:35:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:10 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2450106781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:35:10.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:10 smithi196 ceph-mon[35041]: pgmap v667: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-05-09T01:35:10.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:10 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2450106781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:35:10.981 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:35:11.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:10 smithi059 ceph-mon[34351]: pgmap v667: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-05-09T01:35:11.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:10 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2450106781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:35:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:12.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:12 smithi019 ceph-mon[30158]: pgmap v668: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:35:12.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:12 smithi196 ceph-mon[35041]: pgmap v668: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:35:13.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:12 smithi059 ceph-mon[34351]: pgmap v668: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.029+0000 7f4674b9e700 1 -- 172.21.15.19:0/1872125738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 msgr2=0x7f4670108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.029+0000 7f4674b9e700 1 --2- 172.21.15.19:0/1872125738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 0x7f4670108170 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f46640099c0 tx=0x7f4664009cd0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.029+0000 7f4674b9e700 1 -- 172.21.15.19:0/1872125738 shutdown_connections 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.029+0000 7f4674b9e700 1 --2- 172.21.15.19:0/1872125738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46701086b0 0x7f4670110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.029+0000 7f4674b9e700 1 --2- 172.21.15.19:0/1872125738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4670103a40 0x7f4670103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.029+0000 7f4674b9e700 1 --2- 172.21.15.19:0/1872125738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 0x7f4670108170 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.029+0000 7f4674b9e700 1 -- 172.21.15.19:0/1872125738 >> 172.21.15.19:0/1872125738 conn(0x7f4670078370 msgr2=0x7f46700787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.029+0000 7f4674b9e700 1 -- 172.21.15.19:0/1872125738 shutdown_connections 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.029+0000 7f4674b9e700 1 -- 172.21.15.19:0/1872125738 wait complete. 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.030+0000 7f4674b9e700 1 Processor -- start 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.030+0000 7f4674b9e700 1 -- start start 2024-05-09T01:35:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.030+0000 7f4674b9e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4670103a40 0x7f467019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.030+0000 7f4674b9e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 0x7f467019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.030+0000 7f4674b9e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46701086b0 0x7f467019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.030+0000 7f4674b9e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4670114430 con 0x7f4670107d60 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.030+0000 7f4674b9e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46701142b0 con 0x7f4670103a40 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.030+0000 7f4674b9e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46701145b0 con 0x7f46701086b0 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 0x7f467019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 0x7f467019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42496/0 (socket says 172.21.15.19:42496) 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466effd700 1 -- 172.21.15.19:0/4021441986 learned_addr learned my addr 172.21.15.19:0/4021441986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4670103a40 0x7f467019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466ffff700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46701086b0 0x7f467019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466effd700 1 -- 172.21.15.19:0/4021441986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46701086b0 msgr2=0x7f467019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466effd700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46701086b0 0x7f467019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466effd700 1 -- 172.21.15.19:0/4021441986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4670103a40 msgr2=0x7f467019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466effd700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4670103a40 0x7f467019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466effd700 1 -- 172.21.15.19:0/4021441986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467019fc30 con 0x7f4670107d60 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466effd700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 0x7f467019b300 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f46640098e0 tx=0x7f46640057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466cff9700 1 -- 172.21.15.19:0/4021441986 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664027070 con 0x7f4670107d60 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466cff9700 1 -- 172.21.15.19:0/4021441986 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4664003f20 con 0x7f4670107d60 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f4674b9e700 1 -- 172.21.15.19:0/4021441986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467019fec0 con 0x7f4670107d60 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f466cff9700 1 -- 172.21.15.19:0/4021441986 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664022d70 con 0x7f4670107d60 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.031+0000 7f4674b9e700 1 -- 172.21.15.19:0/4021441986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46701a03a0 con 0x7f4670107d60 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.034+0000 7f466cff9700 1 -- 172.21.15.19:0/4021441986 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f466402a070 con 0x7f4670107d60 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.035+0000 7f466cff9700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f46580b2880 0x7f46580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.035+0000 7f466cff9700 1 -- 172.21.15.19:0/4021441986 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f46640e3010 con 0x7f4670107d60 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.035+0000 7f466f7fe700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f46580b2880 0x7f46580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.035+0000 7f4684816b80 1 Processor -- start 2024-05-09T01:35:13.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.035+0000 7f4684816b80 1 -- start start 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f4684816b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fc87160 0x56152fd20640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f46580b2880 0x7f46580b4d90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4660007be0 tx=0x7f46600073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f4684816b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 0x56152fd20b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f4684816b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fd1a800 0x56152fd1ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f4684816b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56152fc3cff0 con 0x56152fc87160 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f4684816b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56152fbdb7f0 con 0x56152fb9d4a0 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f4684816b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56152fc38fb0 con 0x56152fd1a800 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fc87160 0x56152fd20640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fc87160 0x56152fd20640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42506/0 (socket says 172.21.15.19:42506) 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 -- 172.21.15.19:0/1938255959 learned_addr learned my addr 172.21.15.19:0/1938255959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466effd700 1 --2- 172.21.15.19:0/1938255959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 0x56152fd20b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 -- 172.21.15.19:0/1938255959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fd1a800 msgr2=0x56152fd1ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 --2- 172.21.15.19:0/1938255959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fd1a800 0x56152fd1ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 -- 172.21.15.19:0/1938255959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 msgr2=0x56152fd20b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 --2- 172.21.15.19:0/1938255959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 0x56152fd20b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 -- 172.21.15.19:0/1938255959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56152fbd3370 con 0x56152fc87160 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f466f7fe700 1 --2- 172.21.15.19:0/1938255959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fc87160 0x56152fd20640 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4660010e80 tx=0x7f4660013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a8fe9a5d241709 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f4655ffb700 1 -- 172.21.15.19:0/1938255959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660024070 con 0x56152fc87160 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f4655ffb700 1 -- 172.21.15.19:0/1938255959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4660015070 con 0x56152fc87160 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.036+0000 7f4655ffb700 1 -- 172.21.15.19:0/1938255959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660011920 con 0x56152fc87160 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.037+0000 7f4684816b80 1 -- 172.21.15.19:0/1938255959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fc87160 msgr2=0x56152fd20640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.037+0000 7f4684816b80 1 --2- 172.21.15.19:0/1938255959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fc87160 0x56152fd20640 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4660010e80 tx=0x7f4660013000 comp rx=0 tx=0).stop 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.037+0000 7f4684816b80 1 -- 172.21.15.19:0/1938255959 shutdown_connections 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.037+0000 7f4684816b80 1 --2- 172.21.15.19:0/1938255959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fd1a800 0x56152fd1ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.037+0000 7f4684816b80 1 --2- 172.21.15.19:0/1938255959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fc87160 0x56152fd20640 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.037+0000 7f4684816b80 1 --2- 172.21.15.19:0/1938255959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 0x56152fd20b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.037+0000 7f4684816b80 1 -- 172.21.15.19:0/1938255959 >> 172.21.15.19:0/1938255959 conn(0x56152fbfd6a0 msgr2=0x56152fd15620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.037+0000 7f4684816b80 1 -- 172.21.15.19:0/1938255959 shutdown_connections 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.037+0000 7f4684816b80 1 -- 172.21.15.19:0/1938255959 wait complete. 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.038+0000 7f4684816b80 1 -- start start 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.038+0000 7f4684816b80 1 Processor -- start 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.038+0000 7f4684816b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd24490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.038+0000 7f4684816b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 0x56152fd249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.038+0000 7f4684816b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fd1a800 0x56152fd24f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.038+0000 7f4684816b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56152fab8ba0 con 0x56152fb9d4a0 2024-05-09T01:35:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.038+0000 7f4684816b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56152fbf7150 con 0x56152fd1a800 2024-05-09T01:35:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.038+0000 7f4684816b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56152fd1b5f0 con 0x56152fc87160 2024-05-09T01:35:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.038+0000 7f4684816b80 10 client.0 _ll_register_callbacks cb 0x56152faae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd24490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd24490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42518/0 (socket says 172.21.15.19:42518) 2024-05-09T01:35:13.114 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466f7fe700 1 -- 172.21.15.19:0/1901441024 learned_addr learned my addr 172.21.15.19:0/1901441024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466ffff700 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fd1a800 0x56152fd24f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466effd700 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 0x56152fd249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466f7fe700 1 -- 172.21.15.19:0/1901441024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 msgr2=0x56152fd249d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466f7fe700 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 0x56152fd249d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466f7fe700 1 -- 172.21.15.19:0/1901441024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fd1a800 msgr2=0x56152fd24f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466f7fe700 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fd1a800 0x56152fd24f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466f7fe700 1 -- 172.21.15.19:0/1901441024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56152fab8490 con 0x56152fb9d4a0 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f466f7fe700 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd24490 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4660011480 tx=0x7f4660011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c5989ed63a8f8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f464effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:13.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f464effd700 1 -- 172.21.15.19:0/1901441024 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466002e020 con 0x56152fb9d4a0 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f464effd700 1 -- 172.21.15.19:0/1901441024 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4660003880 con 0x56152fb9d4a0 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f4684816b80 1 -- 172.21.15.19:0/1901441024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4660015070 con 0x56152fb9d4a0 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.039+0000 7f464effd700 1 -- 172.21.15.19:0/1901441024 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4660039050 con 0x56152fb9d4a0 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f4684816b80 10 client.18048 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f4684816b80 1 -- 172.21.15.19:0/1901441024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56152fd36180 con 0x56152fb9d4a0 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f464effd700 1 -- 172.21.15.19:0/1901441024 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4660015070 con 0x56152fb9d4a0 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f464effd700 1 -- 172.21.15.19:0/1901441024 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f466001fc50 con 0x56152fb9d4a0 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f4684816b80 10 client.18048 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f4684816b80 10 client.18048 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f4684816b80 20 client.18048 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f4684816b80 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56152fd38760 0x56152fd3ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f4684816b80 4 client.18048 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f4684816b80 1 -- 172.21.15.19:0/1901441024 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56152fc4a9d0 con 0x56152fd38760 2024-05-09T01:35:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.040+0000 7f466effd700 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56152fd38760 0x56152fd3ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.041+0000 7f466effd700 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56152fd38760 0x56152fd3ab80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bebf2ff5f8d18ff5 server_cookie=3b3e01835ab37a94 in_seq=0 out_seq=0 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.041+0000 7f464effd700 10 client.18048 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f464effd700 1 -- 172.21.15.19:0/1901441024 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56152fc4a9d0 con 0x56152fd38760 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f464effd700 10 client.18048 handle_command_reply: tid=0 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 client.18048 shutdown 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 -- 172.21.15.19:0/1901441024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 msgr2=0x56152fd24490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd24490 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4660011480 tx=0x7f4660011620 comp rx=0 tx=0).stop 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 -- 172.21.15.19:0/1901441024 shutdown_connections 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56152fd38760 0x56152fd3ab80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fd1a800 0x56152fd24f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd24490 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f464effd700 0 client.18048 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:35:13.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 --2- 172.21.15.19:0/1901441024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 0x56152fd249d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 -- 172.21.15.19:0/1901441024 >> 172.21.15.19:0/1901441024 conn(0x56152fbfd6a0 msgr2=0x56152fd15380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 -- 172.21.15.19:0/1901441024 shutdown_connections 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 1 -- 172.21.15.19:0/1901441024 wait complete. 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.042+0000 7f4684816b80 20 client.18048 trim_cache size 0 max 16384 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.057+0000 7f4684816b80 1 Processor -- start 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.057+0000 7f4684816b80 1 -- start start 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f4684816b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 0x56152fd22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f4684816b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 0x56152fd26930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f4684816b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fd1a800 0x56152fd24d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f4684816b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ac7b0 con 0x56152fd1a800 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f4684816b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ae2f0 con 0x56152fb9d4a0 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f4684816b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ace80 con 0x56152fc87160 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fd1a800 0x56152fd24d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fd1a800 0x56152fd24d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42532/0 (socket says 172.21.15.19:42532) 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466ffff700 1 -- 172.21.15.19:0/516027608 learned_addr learned my addr 172.21.15.19:0/516027608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466f7fe700 1 --2- 172.21.15.19:0/516027608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 0x56152fd22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466effd700 1 --2- 172.21.15.19:0/516027608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 0x56152fd26930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466ffff700 1 -- 172.21.15.19:0/516027608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 msgr2=0x56152fd26930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466ffff700 1 --2- 172.21.15.19:0/516027608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 0x56152fd26930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466ffff700 1 -- 172.21.15.19:0/516027608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 msgr2=0x56152fd22110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466ffff700 1 --2- 172.21.15.19:0/516027608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 0x56152fd22110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466ffff700 1 -- 172.21.15.19:0/516027608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56152fd25290 con 0x56152fd1a800 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f466ffff700 1 --2- 172.21.15.19:0/516027608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fd1a800 0x56152fd24d50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4668009d70 tx=0x7f46680077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca44a0f99676de9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f46557fa700 1 -- 172.21.15.19:0/516027608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466801a020 con 0x56152fd1a800 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f46557fa700 1 -- 172.21.15.19:0/516027608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4668016b10 con 0x56152fd1a800 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.058+0000 7f46557fa700 1 -- 172.21.15.19:0/516027608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668015b70 con 0x56152fd1a800 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.059+0000 7f4684816b80 1 -- 172.21.15.19:0/516027608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fd1a800 msgr2=0x56152fd24d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.059+0000 7f4684816b80 1 --2- 172.21.15.19:0/516027608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fd1a800 0x56152fd24d50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4668009d70 tx=0x7f46680077e0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.059+0000 7f4684816b80 1 -- 172.21.15.19:0/516027608 shutdown_connections 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.059+0000 7f4684816b80 1 --2- 172.21.15.19:0/516027608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fd1a800 0x56152fd24d50 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.059+0000 7f4684816b80 1 --2- 172.21.15.19:0/516027608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fb9d4a0 0x56152fd22110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.059+0000 7f4684816b80 1 --2- 172.21.15.19:0/516027608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fc87160 0x56152fd26930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.059+0000 7f4684816b80 1 -- 172.21.15.19:0/516027608 >> 172.21.15.19:0/516027608 conn(0x56152fbfd6a0 msgr2=0x56152fd22da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.059+0000 7f4684816b80 1 -- 172.21.15.19:0/516027608 shutdown_connections 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.059+0000 7f4684816b80 1 -- 172.21.15.19:0/516027608 wait complete. 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f4684816b80 1 -- start start 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f4684816b80 1 Processor -- start 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f4684816b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd3ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f4684816b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fc87160 0x56152fd3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f4684816b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fd1a800 0x56152fda0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f4684816b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46701adc20 con 0x56152fb9d4a0 2024-05-09T01:35:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f4684816b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ad550 con 0x56152fc87160 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f4684816b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46701aa750 con 0x56152fd1a800 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f4684816b80 10 client.0 _ll_register_callbacks cb 0x56152fd1b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f466f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd3ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f466f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd3ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42534/0 (socket says 172.21.15.19:42534) 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f466f7fe700 1 -- 172.21.15.19:0/2612619396 learned_addr learned my addr 172.21.15.19:0/2612619396 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.060+0000 7f466effd700 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fc87160 0x56152fd3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f466ffff700 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fd1a800 0x56152fda0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f466f7fe700 1 -- 172.21.15.19:0/2612619396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fd1a800 msgr2=0x56152fda0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f466f7fe700 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fd1a800 0x56152fda0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f466f7fe700 1 -- 172.21.15.19:0/2612619396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fc87160 msgr2=0x56152fd3b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f466f7fe700 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fc87160 0x56152fd3b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f466f7fe700 1 -- 172.21.15.19:0/2612619396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56152fda11e0 con 0x56152fb9d4a0 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f466effd700 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fc87160 0x56152fd3b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f466f7fe700 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd3ade0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f46600078d0 tx=0x7f466002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6506d93bb91e74 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f464effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f464effd700 1 -- 172.21.15.19:0/2612619396 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660042060 con 0x56152fb9d4a0 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f4684816b80 1 -- 172.21.15.19:0/2612619396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4668016b10 con 0x56152fb9d4a0 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f464effd700 1 -- 172.21.15.19:0/2612619396 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f466002ac80 con 0x56152fb9d4a0 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f464effd700 1 -- 172.21.15.19:0/2612619396 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f466002e040 con 0x56152fb9d4a0 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f4684816b80 10 client.18054 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.061+0000 7f4684816b80 1 -- 172.21.15.19:0/2612619396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56152fda2300 con 0x56152fb9d4a0 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.062+0000 7f464effd700 1 -- 172.21.15.19:0/2612619396 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4668016b10 con 0x56152fb9d4a0 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.062+0000 7f464effd700 1 -- 172.21.15.19:0/2612619396 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f466001f430 con 0x56152fb9d4a0 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.062+0000 7f4684816b80 10 client.18054 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.062+0000 7f4684816b80 10 client.18054 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.062+0000 7f4684816b80 20 client.18054 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.062+0000 7f4684816b80 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56152fda28f0 0x56152fda4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.062+0000 7f4684816b80 4 client.18054 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:35:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.062+0000 7f4684816b80 1 -- 172.21.15.19:0/2612619396 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56152fc44a70 con 0x56152fda28f0 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.062+0000 7f466effd700 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56152fda28f0 0x56152fda4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f464effd700 10 client.18054 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f466effd700 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56152fda28f0 0x56152fda4d10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=62812935916df740 server_cookie=930f531559f4671b in_seq=0 out_seq=0 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f464effd700 1 -- 172.21.15.19:0/2612619396 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56152fc44a70 con 0x56152fda28f0 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f464effd700 10 client.18054 handle_command_reply: tid=0 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 client.18054 shutdown 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 -- 172.21.15.19:0/2612619396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 msgr2=0x56152fd3ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd3ade0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f46600078d0 tx=0x7f466002a660 comp rx=0 tx=0).stop 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 -- 172.21.15.19:0/2612619396 shutdown_connections 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x56152fda28f0 0x56152fda4d10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56152fd1a800 0x56152fda0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56152fb9d4a0 0x56152fd3ade0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 --2- 172.21.15.19:0/2612619396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56152fc87160 0x56152fd3b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 -- 172.21.15.19:0/2612619396 >> 172.21.15.19:0/2612619396 conn(0x56152fbfd6a0 msgr2=0x56152fd1f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f464effd700 0 client.18054 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 -- 172.21.15.19:0/2612619396 shutdown_connections 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 1 -- 172.21.15.19:0/2612619396 wait complete. 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.063+0000 7f4684816b80 20 client.18054 trim_cache size 0 max 16384 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 -- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f46580b2880 msgr2=0x7f46580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f46580b2880 0x7f46580b4d90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4660007be0 tx=0x7f46600073d0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 -- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 msgr2=0x7f467019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 0x7f467019b300 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f46640098e0 tx=0x7f46640057c0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 -- 172.21.15.19:0/4021441986 shutdown_connections 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46701086b0 0x7f467019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4670103a40 0x7f467019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f46580b2880 0x7f46580b4d90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 --2- 172.21.15.19:0/4021441986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4670107d60 0x7f467019b300 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 -- 172.21.15.19:0/4021441986 >> 172.21.15.19:0/4021441986 conn(0x7f4670078370 msgr2=0x7f4670100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:13.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 -- 172.21.15.19:0/4021441986 shutdown_connections 2024-05-09T01:35:13.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:13.064+0000 7f4674b9e700 1 -- 172.21.15.19:0/4021441986 wait complete. 2024-05-09T01:35:13.928 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:35:14.181 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:14 smithi019 ceph-mon[30158]: pgmap v669: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:35:14.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:14 smithi196 ceph-mon[35041]: pgmap v669: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:35:15.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:14 smithi059 ceph-mon[34351]: pgmap v669: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:35:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.962+0000 7fa9bc04c700 1 -- 172.21.15.19:0/3970370681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b4103bc0 msgr2=0x7fa9b41080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.962+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/3970370681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b4103bc0 0x7fa9b41080f0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009990 tx=0x7fa9a4009ca0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.962+0000 7fa9bc04c700 1 -- 172.21.15.19:0/3970370681 shutdown_connections 2024-05-09T01:35:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.962+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/3970370681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9b41086c0 0x7fa9b4108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.962+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/3970370681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4103270 0x7fa9b4103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.962+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/3970370681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b4103bc0 0x7fa9b41080f0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.962+0000 7fa9bc04c700 1 -- 172.21.15.19:0/3970370681 >> 172.21.15.19:0/3970370681 conn(0x7fa9b40fef30 msgr2=0x7fa9b4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.962+0000 7fa9bc04c700 1 -- 172.21.15.19:0/3970370681 shutdown_connections 2024-05-09T01:35:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.962+0000 7fa9bc04c700 1 -- 172.21.15.19:0/3970370681 wait complete. 2024-05-09T01:35:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9bc04c700 1 Processor -- start 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9bc04c700 1 -- start start 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9bc04c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4103270 0x7fa9b41a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9bc04c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9b4103bc0 0x7fa9b41a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9bc04c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b41086c0 0x7fa9b41a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9bc04c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b411aa40 con 0x7fa9b41086c0 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9bc04c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b411a8c0 con 0x7fa9b4103bc0 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9bc04c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b411abc0 con 0x7fa9b4103270 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9ba5e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b41086c0 0x7fa9b41a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9ba5e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b41086c0 0x7fa9b41a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42558/0 (socket says 172.21.15.19:42558) 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9b9de8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4103270 0x7fa9b41a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9ba5e9700 1 -- 172.21.15.19:0/679976346 learned_addr learned my addr 172.21.15.19:0/679976346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.963+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9b4103bc0 0x7fa9b41a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9ba5e9700 1 -- 172.21.15.19:0/679976346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4103270 msgr2=0x7fa9b41a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4103270 0x7fa9b41a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9ba5e9700 1 -- 172.21.15.19:0/679976346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9b4103bc0 msgr2=0x7fa9b41a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9b4103bc0 0x7fa9b41a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9ba5e9700 1 -- 172.21.15.19:0/679976346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b41a62e0 con 0x7fa9b41086c0 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b41086c0 0x7fa9b41a5c40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac007ee0 tx=0x7fa9ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9aaffd700 1 -- 172.21.15.19:0/679976346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac004bb0 con 0x7fa9b41086c0 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9aaffd700 1 -- 172.21.15.19:0/679976346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ac004d50 con 0x7fa9b41086c0 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9bc04c700 1 -- 172.21.15.19:0/679976346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b41a65d0 con 0x7fa9b41086c0 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9aaffd700 1 -- 172.21.15.19:0/679976346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac005920 con 0x7fa9b41086c0 2024-05-09T01:35:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.964+0000 7fa9bc04c700 1 -- 172.21.15.19:0/679976346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b41adef0 con 0x7fa9b41086c0 2024-05-09T01:35:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.967+0000 7fa9aaffd700 1 -- 172.21.15.19:0/679976346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa9ac01a020 con 0x7fa9b41086c0 2024-05-09T01:35:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.968+0000 7fa9aaffd700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9a00b2880 0x7fa9a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.968+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9a00b2880 0x7fa9a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.968+0000 7fa9aaffd700 1 -- 172.21.15.19:0/679976346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa9ac0e18d0 con 0x7fa9b41086c0 2024-05-09T01:35:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.968+0000 7fa9caa62b80 1 Processor -- start 2024-05-09T01:35:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.968+0000 7fa9caa62b80 1 -- start start 2024-05-09T01:35:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.968+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9a00b2880 0x7fa9a00b4d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005950 tx=0x7fa9b000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6e40160 0x55afd6ed94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6ed9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6ed36a0 0x55afd6ed3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55afd6df5ff0 con 0x55afd6ed36a0 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55afd6d947f0 con 0x55afd6e40160 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55afd6df1fb0 con 0x55afd6d564a0 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9ba5e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6ed36a0 0x55afd6ed3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9ba5e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6ed36a0 0x55afd6ed3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42572/0 (socket says 172.21.15.19:42572) 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9ba5e9700 1 -- 172.21.15.19:0/3669968843 learned_addr learned my addr 172.21.15.19:0/3669968843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6e40160 0x55afd6ed94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6ed9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9ba5e9700 1 -- 172.21.15.19:0/3669968843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 msgr2=0x55afd6ed9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6ed9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9ba5e9700 1 -- 172.21.15.19:0/3669968843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6e40160 msgr2=0x55afd6ed94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:15.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6e40160 0x55afd6ed94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9ba5e9700 1 -- 172.21.15.19:0/3669968843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd6d8c370 con 0x55afd6ed36a0 2024-05-09T01:35:15.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6ed9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:35:15.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.969+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6ed36a0 0x55afd6ed3ad0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0a60e0 tx=0x7fa9ac0a8d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd4df728de2ddaf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:15.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.970+0000 7fa99ffff700 1 -- 172.21.15.19:0/3669968843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0a9570 con 0x55afd6ed36a0 2024-05-09T01:35:15.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.970+0000 7fa99ffff700 1 -- 172.21.15.19:0/3669968843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ac0a8e70 con 0x55afd6ed36a0 2024-05-09T01:35:15.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.970+0000 7fa99ffff700 1 -- 172.21.15.19:0/3669968843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0a9330 con 0x55afd6ed36a0 2024-05-09T01:35:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.970+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3669968843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6ed36a0 msgr2=0x55afd6ed3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.970+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6ed36a0 0x55afd6ed3ad0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0a60e0 tx=0x7fa9ac0a8d70 comp rx=0 tx=0).stop 2024-05-09T01:35:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.971+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3669968843 shutdown_connections 2024-05-09T01:35:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.971+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6ed36a0 0x55afd6ed3ad0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.971+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6e40160 0x55afd6ed94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.971+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/3669968843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6ed9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.971+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3669968843 >> 172.21.15.19:0/3669968843 conn(0x55afd6db66a0 msgr2=0x55afd6ece400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.971+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3669968843 shutdown_connections 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.971+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3669968843 wait complete. 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.971+0000 7fa9caa62b80 1 -- start start 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.971+0000 7fa9caa62b80 1 Processor -- start 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6edd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 0x55afd6eddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55afd6c71ba0 con 0x55afd6e40160 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55afd6db0150 con 0x55afd6ed36a0 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55afd6ee01d0 con 0x55afd6d564a0 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9caa62b80 10 client.0 _ll_register_callbacks cb 0x55afd6c676c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9b95e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9b95e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42576/0 (socket says 172.21.15.19:42576) 2024-05-09T01:35:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9b95e7700 1 -- 172.21.15.19:0/1083824461 learned_addr learned my addr 172.21.15.19:0/1083824461 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 0x55afd6eddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6edd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9b95e7700 1 -- 172.21.15.19:0/1083824461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 msgr2=0x55afd6edd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6edd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9b95e7700 1 -- 172.21.15.19:0/1083824461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 msgr2=0x55afd6eddc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 0x55afd6eddc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.972+0000 7fa9b95e7700 1 -- 172.21.15.19:0/1083824461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd6c71490 con 0x55afd6e40160 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.973+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd6f0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4015000 tx=0x7fa9a400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7b39b18027f36e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.973+0000 7fa99d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.973+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1083824461 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4027060 con 0x55afd6e40160 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.973+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1083824461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ac0a8e70 con 0x55afd6e40160 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.973+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1083824461 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa9a4003c70 con 0x55afd6e40160 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.973+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1083824461 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9a4003680 con 0x55afd6e40160 2024-05-09T01:35:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.973+0000 7fa9caa62b80 10 client.18066 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.973+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1083824461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afd6eeab00 con 0x55afd6e40160 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.973+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1083824461 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ac0a8e70 con 0x55afd6e40160 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.974+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1083824461 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa9a400f640 con 0x55afd6e40160 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.974+0000 7fa9caa62b80 10 client.18066 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.974+0000 7fa9caa62b80 10 client.18066 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.974+0000 7fa9caa62b80 20 client.18066 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.974+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55afd6eed1e0 0x55afd6eef630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.974+0000 7fa9caa62b80 4 client.18066 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.974+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1083824461 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afd6e039d0 con 0x55afd6eed1e0 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.974+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55afd6eed1e0 0x55afd6eef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.975+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55afd6eed1e0 0x55afd6eef630 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27d8ee0d068622ee server_cookie=9e150ec1f8a5f7c6 in_seq=0 out_seq=0 2024-05-09T01:35:15.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.975+0000 7fa99d7fa700 10 client.18066 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:35:15.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.975+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1083824461 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55afd6e039d0 con 0x55afd6eed1e0 2024-05-09T01:35:15.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.975+0000 7fa99d7fa700 10 client.18066 handle_command_reply: tid=0 2024-05-09T01:35:15.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.975+0000 7fa9caa62b80 1 client.18066 shutdown 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1083824461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 msgr2=0x55afd6edd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd6f0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4015000 tx=0x7fa9a400bf90 comp rx=0 tx=0).stop 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1083824461 shutdown_connections 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55afd6eed1e0 0x55afd6eef630 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 0x55afd6eddc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa99d7fa700 0 client.18066 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6edd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1083824461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd6f0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1083824461 >> 172.21.15.19:0/1083824461 conn(0x55afd6db66a0 msgr2=0x55afd6ece1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1083824461 shutdown_connections 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1083824461 wait complete. 2024-05-09T01:35:15.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.976+0000 7fa9caa62b80 20 client.18066 trim_cache size 0 max 16384 2024-05-09T01:35:15.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.991+0000 7fa9caa62b80 1 Processor -- start 2024-05-09T01:35:15.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.991+0000 7fa9caa62b80 1 -- start start 2024-05-09T01:35:15.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6ed88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 0x55afd6ee8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41b5290 con 0x55afd6e40160 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41b6dd0 con 0x55afd6ed36a0 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41b5960 con 0x55afd6d564a0 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b95e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b95e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42588/0 (socket says 172.21.15.19:42588) 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b95e7700 1 -- 172.21.15.19:0/3270899551 learned_addr learned my addr 172.21.15.19:0/3270899551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/3270899551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 0x55afd6ee8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/3270899551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6ed88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b95e7700 1 -- 172.21.15.19:0/3270899551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 msgr2=0x55afd6ed88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/3270899551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6ed88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b95e7700 1 -- 172.21.15.19:0/3270899551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 msgr2=0x55afd6ee8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/3270899551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 0x55afd6ee8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b95e7700 1 -- 172.21.15.19:0/3270899551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd6ed8e00 con 0x55afd6e40160 2024-05-09T01:35:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.992+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/3270899551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd740 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4006010 tx=0x7fa9a4005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c0351ed8bea263 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa99f7fe700 1 -- 172.21.15.19:0/3270899551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4003c70 con 0x55afd6e40160 2024-05-09T01:35:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa99f7fe700 1 -- 172.21.15.19:0/3270899551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a4009590 con 0x55afd6e40160 2024-05-09T01:35:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa99f7fe700 1 -- 172.21.15.19:0/3270899551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a402d570 con 0x55afd6e40160 2024-05-09T01:35:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3270899551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 msgr2=0x55afd6edd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/3270899551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd740 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4006010 tx=0x7fa9a4005ab0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3270899551 shutdown_connections 2024-05-09T01:35:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/3270899551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6ed36a0 0x55afd6ee8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/3270899551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6d564a0 0x55afd6ed88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/3270899551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6edd740 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.993+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3270899551 >> 172.21.15.19:0/3270899551 conn(0x55afd6db66a0 msgr2=0x55afd6ed81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3270899551 shutdown_connections 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 -- 172.21.15.19:0/3270899551 wait complete. 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 -- start start 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 Processor -- start 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6d564a0 0x55afd6ef45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6ef4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6ed36a0 0x55afd6ef5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41b6700 con 0x55afd6e40160 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41b6030 con 0x55afd6d564a0 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.994+0000 7fa9caa62b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41b3230 con 0x55afd6ed36a0 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9caa62b80 10 client.0 _ll_register_callbacks cb 0x55afd6c676c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b95e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6ef4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b95e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6ef4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42590/0 (socket says 172.21.15.19:42590) 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b95e7700 1 -- 172.21.15.19:0/1671891386 learned_addr learned my addr 172.21.15.19:0/1671891386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6d564a0 0x55afd6ef45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6ed36a0 0x55afd6ef5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b95e7700 1 -- 172.21.15.19:0/1671891386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6ed36a0 msgr2=0x55afd6ef5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6ed36a0 0x55afd6ef5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b95e7700 1 -- 172.21.15.19:0/1671891386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6d564a0 msgr2=0x55afd6ef45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6d564a0 0x55afd6ef45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b95e7700 1 -- 172.21.15.19:0/1671891386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd6f56940 con 0x55afd6e40160 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9ba5e9700 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6ed36a0 0x55afd6ef5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9b95e7700 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6ef4af0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40098e0 tx=0x7fa9a40057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bc95647cf95c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa99d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1671891386 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400f460 con 0x55afd6e40160 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1671891386 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa9a400f600 con 0x55afd6e40160 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.995+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1671891386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9a4009590 con 0x55afd6e40160 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1671891386 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9a4044050 con 0x55afd6e40160 2024-05-09T01:35:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa9caa62b80 10 client.18072 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1671891386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afd6f57a80 con 0x55afd6e40160 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1671891386 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a4009590 con 0x55afd6e40160 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1671891386 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa9a4022450 con 0x55afd6e40160 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa9caa62b80 10 client.18072 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa9caa62b80 10 client.18072 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa9caa62b80 20 client.18072 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55afd6f58050 0x55afd6f5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa9caa62b80 4 client.18072 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1671891386 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afd6dfda70 con 0x55afd6f58050 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.996+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55afd6f58050 0x55afd6f5a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9b9de8700 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55afd6f58050 0x55afd6f5a470 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=50f11259421473ff server_cookie=bc675a0c58f73b8f in_seq=0 out_seq=0 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa99d7fa700 10 client.18072 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:35:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa99d7fa700 1 -- 172.21.15.19:0/1671891386 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afd6dfda70 con 0x55afd6f58050 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa99d7fa700 10 client.18072 handle_command_reply: tid=0 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 client.18072 shutdown 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1671891386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 msgr2=0x55afd6ef4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6ef4af0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40098e0 tx=0x7fa9a40057e0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1671891386 shutdown_connections 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55afd6f58050 0x55afd6f5a470 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afd6ed36a0 0x55afd6ef5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afd6d564a0 0x55afd6ef45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 --2- 172.21.15.19:0/1671891386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55afd6e40160 0x55afd6ef4af0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa99d7fa700 0 client.18072 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1671891386 >> 172.21.15.19:0/1671891386 conn(0x55afd6db66a0 msgr2=0x55afd6edb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1671891386 shutdown_connections 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 1 -- 172.21.15.19:0/1671891386 wait complete. 2024-05-09T01:35:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.997+0000 7fa9caa62b80 20 client.18072 trim_cache size 0 max 16384 2024-05-09T01:35:16.000 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:35:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 -- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9a00b2880 msgr2=0x7fa9a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9a00b2880 0x7fa9a00b4d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005950 tx=0x7fa9b000b410 comp rx=0 tx=0).stop 2024-05-09T01:35:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 -- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b41086c0 msgr2=0x7fa9b41a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b41086c0 0x7fa9b41a5c40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac007ee0 tx=0x7fa9ac00d3b0 comp rx=0 tx=0).stop 2024-05-09T01:35:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 -- 172.21.15.19:0/679976346 shutdown_connections 2024-05-09T01:35:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa9a00b2880 0x7fa9a00b4d90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9b41086c0 0x7fa9b41a5c40 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4103270 0x7fa9b41a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 --2- 172.21.15.19:0/679976346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9b4103bc0 0x7fa9b41a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 -- 172.21.15.19:0/679976346 >> 172.21.15.19:0/679976346 conn(0x7fa9b40fef30 msgr2=0x7fa9b41060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 -- 172.21.15.19:0/679976346 shutdown_connections 2024-05-09T01:35:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:15.998+0000 7fa9bc04c700 1 -- 172.21.15.19:0/679976346 wait complete. 2024-05-09T01:35:16.717 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:35:16.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:16 smithi019 ceph-mon[30158]: pgmap v670: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-05-09T01:35:16.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:16 smithi196 ceph-mon[35041]: pgmap v670: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-05-09T01:35:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:17.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:16 smithi059 ceph-mon[34351]: pgmap v670: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-05-09T01:35:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.637+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/1461680653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109300 msgr2=0x7f0f48109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.637+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/1461680653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109300 0x7f0f48109710 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400b0a0 tx=0x7f0f4400b3b0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.637+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/1461680653 shutdown_connections 2024-05-09T01:35:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.637+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/1461680653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f481031d0 0x7f0f48111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.637+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/1461680653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48109300 0x7f0f48109710 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.637+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/1461680653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f48102880 0x7f0f48102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.637+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/1461680653 >> 172.21.15.19:0/1461680653 conn(0x7f0f480fe6f0 msgr2=0x7f0f48100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.637+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/1461680653 shutdown_connections 2024-05-09T01:35:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.637+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/1461680653 wait complete. 2024-05-09T01:35:18.636 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:18 smithi019 ceph-mon[30158]: pgmap v671: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:35:18.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4fd2b700 1 Processor -- start 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4fd2b700 1 -- start start 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4fd2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48102880 0x7f0f4819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4fd2b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f481031d0 0x7f0f4819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4fd2b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f48109300 0x7f0f4819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4fd2b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48114590 con 0x7f0f48102880 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4fd2b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48114410 con 0x7f0f481031d0 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4fd2b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48114710 con 0x7f0f48109300 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4dac7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48102880 0x7f0f4819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4dac7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48102880 0x7f0f4819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42612/0 (socket says 172.21.15.19:42612) 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4dac7700 1 -- 172.21.15.19:0/40553548 learned_addr learned my addr 172.21.15.19:0/40553548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4e2c8700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f48109300 0x7f0f4819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.638+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f481031d0 0x7f0f4819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f4dac7700 1 -- 172.21.15.19:0/40553548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f48109300 msgr2=0x7f0f4819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f48109300 0x7f0f4819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f4dac7700 1 -- 172.21.15.19:0/40553548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f481031d0 msgr2=0x7f0f4819b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f481031d0 0x7f0f4819b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f4dac7700 1 -- 172.21.15.19:0/40553548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4819fdc0 con 0x7f0f48102880 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48102880 0x7f0f4819afa0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f0f44000f80 tx=0x7f0f44004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f3affd700 1 -- 172.21.15.19:0/40553548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44021070 con 0x7f0f48102880 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/40553548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f481a0050 con 0x7f0f48102880 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f3affd700 1 -- 172.21.15.19:0/40553548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f44007b80 con 0x7f0f48102880 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f3affd700 1 -- 172.21.15.19:0/40553548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440052a0 con 0x7f0f48102880 2024-05-09T01:35:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.639+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/40553548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f481a0590 con 0x7f0f48102880 2024-05-09T01:35:18.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.643+0000 7f0f3affd700 1 -- 172.21.15.19:0/40553548 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0f44024070 con 0x7f0f48102880 2024-05-09T01:35:18.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f3affd700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f340b2880 0x7f0f340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f340b2880 0x7f0f340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f5e741b80 1 Processor -- start 2024-05-09T01:35:18.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f3affd700 1 -- 172.21.15.19:0/40553548 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0f4400f020 con 0x7f0f48102880 2024-05-09T01:35:18.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f5e741b80 1 -- start start 2024-05-09T01:35:18.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f340b2880 0x7f0f340b4d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0f481078c0 tx=0x7f0f3c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:18.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 0x55b077e384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e38a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e32ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b077d54ff0 con 0x55b077cb54a0 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b077cf37f0 con 0x55b077e326a0 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.644+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b077d50fb0 con 0x55b077d9f160 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4d2c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e38a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4d2c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e38a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42626/0 (socket says 172.21.15.19:42626) 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/1015328044 learned_addr learned my addr 172.21.15.19:0/1015328044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/1015328044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 0x55b077e384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4e2c8700 1 --2- 172.21.15.19:0/1015328044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e32ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/1015328044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 msgr2=0x55b077e384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/1015328044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 0x55b077e384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/1015328044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 msgr2=0x55b077e32ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:18.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/1015328044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e32ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/1015328044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b077ceb370 con 0x55b077cb54a0 2024-05-09T01:35:18.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/1015328044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e38a30 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00fe70 tx=0x7f0f3c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61df219d78ec60c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:18.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f33fff700 1 -- 172.21.15.19:0/1015328044 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c016070 con 0x55b077cb54a0 2024-05-09T01:35:18.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f33fff700 1 -- 172.21.15.19:0/1015328044 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3c014070 con 0x55b077cb54a0 2024-05-09T01:35:18.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.645+0000 7f0f33fff700 1 -- 172.21.15.19:0/1015328044 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c01d400 con 0x55b077cb54a0 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.646+0000 7f0f5e741b80 1 -- 172.21.15.19:0/1015328044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 msgr2=0x55b077e38a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.646+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/1015328044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e38a30 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00fe70 tx=0x7f0f3c012000 comp rx=0 tx=0).stop 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.646+0000 7f0f5e741b80 1 -- 172.21.15.19:0/1015328044 shutdown_connections 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.646+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/1015328044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e32ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.646+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/1015328044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 0x55b077e384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.646+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/1015328044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e38a30 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.646+0000 7f0f5e741b80 1 -- 172.21.15.19:0/1015328044 >> 172.21.15.19:0/1015328044 conn(0x55b077d156a0 msgr2=0x55b077e2d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.646+0000 7f0f5e741b80 1 -- 172.21.15.19:0/1015328044 shutdown_connections 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.646+0000 7f0f5e741b80 1 -- 172.21.15.19:0/1015328044 wait complete. 2024-05-09T01:35:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.647+0000 7f0f5e741b80 1 -- start start 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.647+0000 7f0f5e741b80 1 Processor -- start 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.647+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e3c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.647+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 0x55b077e3c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.647+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e3cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b077bd0ba0 con 0x55b077cb54a0 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b077d0f150 con 0x55b077e326a0 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b077e3f1d0 con 0x55b077d9f160 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f5e741b80 10 client.0 _ll_register_callbacks cb 0x55b077bc66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4dac7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e3c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4dac7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e3c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42634/0 (socket says 172.21.15.19:42634) 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4dac7700 1 -- 172.21.15.19:0/3336023309 learned_addr learned my addr 172.21.15.19:0/3336023309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4e2c8700 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e3cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 0x55b077e3c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4dac7700 1 -- 172.21.15.19:0/3336023309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 msgr2=0x55b077e3c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 0x55b077e3c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4dac7700 1 -- 172.21.15.19:0/3336023309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 msgr2=0x55b077e3cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e3cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4dac7700 1 -- 172.21.15.19:0/3336023309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b077bd0490 con 0x55b077cb54a0 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 0x55b077e3c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e3c110 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f0f44004fd0 tx=0x7f0f440aa600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d069d7c56805d5dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f317fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f317fa700 1 -- 172.21.15.19:0/3336023309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440aadc0 con 0x55b077cb54a0 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f317fa700 1 -- 172.21.15.19:0/3336023309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0f440b3910 con 0x55b077cb54a0 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.648+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3336023309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3c014070 con 0x55b077cb54a0 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f317fa700 1 -- 172.21.15.19:0/3336023309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0f4406f050 con 0x55b077cb54a0 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f5e741b80 10 client.18084 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3336023309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b077e49bb0 con 0x55b077cb54a0 2024-05-09T01:35:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f317fa700 1 -- 172.21.15.19:0/3336023309 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3c014070 con 0x55b077cb54a0 2024-05-09T01:35:18.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f317fa700 1 -- 172.21.15.19:0/3336023309 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0f44011070 con 0x55b077cb54a0 2024-05-09T01:35:18.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f5e741b80 10 client.18084 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:18.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f5e741b80 10 client.18084 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:35:18.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f5e741b80 20 client.18084 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:18.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b077e4c170 0x55b077e4e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f5e741b80 4 client.18084 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:18.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.649+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3336023309 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b077d629d0 con 0x55b077e4c170 2024-05-09T01:35:18.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.650+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b077e4c170 0x55b077e4e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.651+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b077e4c170 0x55b077e4e5f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9018bb694f0f8859 server_cookie=1b76ce16bbcdb579 in_seq=0 out_seq=0 2024-05-09T01:35:18.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.651+0000 7f0f317fa700 10 client.18084 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.652+0000 7f0f317fa700 1 -- 172.21.15.19:0/3336023309 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b077d629d0 con 0x55b077e4c170 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.652+0000 7f0f317fa700 10 client.18084 handle_command_reply: tid=0 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.652+0000 7f0f5e741b80 1 client.18084 shutdown 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.652+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3336023309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 msgr2=0x55b077e3c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.652+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e3c110 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f0f44004fd0 tx=0x7f0f440aa600 comp rx=0 tx=0).stop 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3336023309 shutdown_connections 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b077e4c170 0x55b077e4e5f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e3cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f317fa700 0 client.18084 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077cb54a0 0x55b077e3c110 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3336023309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077d9f160 0x55b077e3c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3336023309 >> 172.21.15.19:0/3336023309 conn(0x55b077d156a0 msgr2=0x55b077e2d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3336023309 shutdown_connections 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3336023309 wait complete. 2024-05-09T01:35:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.653+0000 7f0f5e741b80 20 client.18084 trim_cache size 0 max 16384 2024-05-09T01:35:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f5e741b80 1 Processor -- start 2024-05-09T01:35:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f5e741b80 1 -- start start 2024-05-09T01:35:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077cb54a0 0x55b077e36830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e37860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e3b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481b0ec0 con 0x55b077d9f160 2024-05-09T01:35:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481b2a00 con 0x55b077e326a0 2024-05-09T01:35:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481b1590 con 0x55b077cb54a0 2024-05-09T01:35:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f4d2c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e37860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f4d2c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e37860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42646/0 (socket says 172.21.15.19:42646) 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/3679614368 learned_addr learned my addr 172.21.15.19:0/3679614368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077cb54a0 0x55b077e36830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.668+0000 7f0f4e2c8700 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e3b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/3679614368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077cb54a0 msgr2=0x55b077e36830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077cb54a0 0x55b077e36830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/3679614368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 msgr2=0x55b077e3b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e3b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/3679614368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b077e2e2e0 con 0x55b077d9f160 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077cb54a0 0x55b077e36830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e37860 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00a4a0 tx=0x7f0f3c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f74409154a5e2f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f337fe700 1 -- 172.21.15.19:0/3679614368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c010c40 con 0x55b077d9f160 2024-05-09T01:35:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f337fe700 1 -- 172.21.15.19:0/3679614368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3c032e50 con 0x55b077d9f160 2024-05-09T01:35:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.669+0000 7f0f337fe700 1 -- 172.21.15.19:0/3679614368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c003cb0 con 0x55b077d9f160 2024-05-09T01:35:18.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3679614368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 msgr2=0x55b077e37860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:18.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e37860 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00a4a0 tx=0x7f0f3c016000 comp rx=0 tx=0).stop 2024-05-09T01:35:18.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3679614368 shutdown_connections 2024-05-09T01:35:18.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077e326a0 0x55b077e3b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077cb54a0 0x55b077e36830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/3679614368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e37860 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3679614368 >> 172.21.15.19:0/3679614368 conn(0x55b077d156a0 msgr2=0x55b077e3aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:18.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3679614368 shutdown_connections 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 -- 172.21.15.19:0/3679614368 wait complete. 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.670+0000 7f0f5e741b80 1 -- start start 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f5e741b80 1 Processor -- start 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077cb54a0 0x55b077e446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f5e741b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077e326a0 0x55b077e536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481b2330 con 0x55b077d9f160 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481b1c60 con 0x55b077cb54a0 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f5e741b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481aee60 con 0x55b077e326a0 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f5e741b80 10 client.0 _ll_register_callbacks cb 0x55b077bc66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f4d2c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f4dac7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077cb54a0 0x55b077e446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f4d2c6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e53170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42652/0 (socket says 172.21.15.19:42652) 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/2139813714 learned_addr learned my addr 172.21.15.19:0/2139813714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f4e2c8700 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077e326a0 0x55b077e536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/2139813714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077e326a0 msgr2=0x55b077e536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.671+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077e326a0 0x55b077e536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/2139813714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077cb54a0 msgr2=0x55b077e446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077cb54a0 0x55b077e446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f4d2c6700 1 -- 172.21.15.19:0/2139813714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b077e53bf0 con 0x55b077d9f160 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f4d2c6700 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e53170 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c005f00 tx=0x7f0f3c003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a31928df07a9fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f317fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f317fa700 1 -- 172.21.15.19:0/2139813714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c041060 con 0x55b077d9f160 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f317fa700 1 -- 172.21.15.19:0/2139813714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f0f3c010860 con 0x55b077d9f160 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f5e741b80 1 -- 172.21.15.19:0/2139813714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3c032e50 con 0x55b077d9f160 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f317fa700 1 -- 172.21.15.19:0/2139813714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c010a60 con 0x55b077d9f160 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f5e741b80 10 client.18090 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f5e741b80 1 -- 172.21.15.19:0/2139813714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b077eb6530 con 0x55b077d9f160 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.672+0000 7f0f317fa700 1 -- 172.21.15.19:0/2139813714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3c032e50 con 0x55b077d9f160 2024-05-09T01:35:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f317fa700 1 -- 172.21.15.19:0/2139813714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0f3c03c410 con 0x55b077d9f160 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f5e741b80 10 client.18090 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f5e741b80 10 client.18090 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f5e741b80 20 client.18090 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b077eb6b00 0x55b077eb8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f5e741b80 4 client.18090 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f5e741b80 1 -- 172.21.15.19:0/2139813714 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b077d5ca70 con 0x55b077eb6b00 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b077eb6b00 0x55b077eb8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f4dac7700 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b077eb6b00 0x55b077eb8f20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a41c3a5b620824d4 server_cookie=11b97f6d8bd431fc in_seq=0 out_seq=0 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.673+0000 7f0f317fa700 10 client.18090 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f317fa700 1 -- 172.21.15.19:0/2139813714 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b077d5ca70 con 0x55b077eb6b00 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f317fa700 10 client.18090 handle_command_reply: tid=0 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 client.18090 shutdown 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 -- 172.21.15.19:0/2139813714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 msgr2=0x55b077e53170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e53170 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c005f00 tx=0x7f0f3c003930 comp rx=0 tx=0).stop 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 -- 172.21.15.19:0/2139813714 shutdown_connections 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b077e326a0 0x55b077e536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x55b077eb6b00 0x55b077eb8f20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b077cb54a0 0x55b077e446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 --2- 172.21.15.19:0/2139813714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b077d9f160 0x55b077e53170 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 -- 172.21.15.19:0/2139813714 >> 172.21.15.19:0/2139813714 conn(0x55b077d156a0 msgr2=0x55b077e36590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f317fa700 0 client.18090 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 -- 172.21.15.19:0/2139813714 shutdown_connections 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 1 -- 172.21.15.19:0/2139813714 wait complete. 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.674+0000 7f0f5e741b80 20 client.18090 trim_cache size 0 max 16384 2024-05-09T01:35:18.675 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:35:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f340b2880 msgr2=0x7f0f340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f340b2880 0x7f0f340b4d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0f481078c0 tx=0x7f0f3c00a380 comp rx=0 tx=0).stop 2024-05-09T01:35:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48102880 msgr2=0x7f0f4819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48102880 0x7f0f4819afa0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f0f44000f80 tx=0x7f0f44004ce0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/40553548 shutdown_connections 2024-05-09T01:35:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f48109300 0x7f0f4819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f0f340b2880 0x7f0f340b4d90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f48102880 0x7f0f4819afa0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 --2- 172.21.15.19:0/40553548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f481031d0 0x7f0f4819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/40553548 >> 172.21.15.19:0/40553548 conn(0x7f0f480fe6f0 msgr2=0x7f0f4810a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/40553548 shutdown_connections 2024-05-09T01:35:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:18.675+0000 7f0f4fd2b700 1 -- 172.21.15.19:0/40553548 wait complete. 2024-05-09T01:35:18.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:18 smithi196 ceph-mon[35041]: pgmap v671: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:35:19.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:18 smithi059 ceph-mon[34351]: pgmap v671: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-09T01:35:19.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:35:19.703 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:20.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:20 smithi019 ceph-mon[30158]: pgmap v672: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:35:20.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:20 smithi196 ceph-mon[35041]: pgmap v672: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:35:21.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:20 smithi059 ceph-mon[34351]: pgmap v672: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:35:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.284+0000 7fced10dd700 1 -- 172.21.15.19:0/52324363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 msgr2=0x7fcecc1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.284+0000 7fced10dd700 1 --2- 172.21.15.19:0/52324363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 0x7fcecc1049e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0099c0 tx=0x7fcebc009cd0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.284+0000 7fced10dd700 1 -- 172.21.15.19:0/52324363 shutdown_connections 2024-05-09T01:35:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.284+0000 7fced10dd700 1 --2- 172.21.15.19:0/52324363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcecc109850 0x7fcecc1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.284+0000 7fced10dd700 1 --2- 172.21.15.19:0/52324363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 0x7fcecc1049e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.284+0000 7fced10dd700 1 --2- 172.21.15.19:0/52324363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcecc108f00 0x7fcecc109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.284+0000 7fced10dd700 1 -- 172.21.15.19:0/52324363 >> 172.21.15.19:0/52324363 conn(0x7fcecc0ffea0 msgr2=0x7fcecc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.284+0000 7fced10dd700 1 -- 172.21.15.19:0/52324363 shutdown_connections 2024-05-09T01:35:21.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.284+0000 7fced10dd700 1 -- 172.21.15.19:0/52324363 wait complete. 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.285+0000 7fced10dd700 1 Processor -- start 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.285+0000 7fced10dd700 1 -- start start 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fced10dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 0x7fcecc19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fced10dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcecc108f00 0x7fcecc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fced10dd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcecc109850 0x7fcecc19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fced10dd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc114480 con 0x7fcecc1045d0 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fced10dd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc114300 con 0x7fcecc108f00 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fced10dd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc114600 con 0x7fcecc109850 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 0x7fcecc19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 0x7fcecc19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56740/0 (socket says 172.21.15.19:56740) 2024-05-09T01:35:21.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecad9d700 1 -- 172.21.15.19:0/1352126461 learned_addr learned my addr 172.21.15.19:0/1352126461 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fceca59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcecc108f00 0x7fcecc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecb59e700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcecc109850 0x7fcecc19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecad9d700 1 -- 172.21.15.19:0/1352126461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcecc109850 msgr2=0x7fcecc19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecad9d700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcecc109850 0x7fcecc19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecad9d700 1 -- 172.21.15.19:0/1352126461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcecc108f00 msgr2=0x7fcecc19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecad9d700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcecc108f00 0x7fcecc19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecad9d700 1 -- 172.21.15.19:0/1352126461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc19fc60 con 0x7fcecc1045d0 2024-05-09T01:35:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.286+0000 7fcecad9d700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 0x7fcecc19ad90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0098e0 tx=0x7fcebc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.287+0000 7fceb3fff700 1 -- 172.21.15.19:0/1352126461 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc027070 con 0x7fcecc1045d0 2024-05-09T01:35:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.287+0000 7fced10dd700 1 -- 172.21.15.19:0/1352126461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc19fef0 con 0x7fcecc1045d0 2024-05-09T01:35:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.287+0000 7fceb3fff700 1 -- 172.21.15.19:0/1352126461 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcebc003f20 con 0x7fcecc1045d0 2024-05-09T01:35:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.287+0000 7fceb3fff700 1 -- 172.21.15.19:0/1352126461 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc022ca0 con 0x7fcecc1045d0 2024-05-09T01:35:21.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.287+0000 7fced10dd700 1 -- 172.21.15.19:0/1352126461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc1a7870 con 0x7fcecc1045d0 2024-05-09T01:35:21.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.289+0000 7fceb3fff700 1 -- 172.21.15.19:0/1352126461 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcebc02a070 con 0x7fcecc1045d0 2024-05-09T01:35:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.290+0000 7fceb3fff700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fceb40b2880 0x7fceb40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.290+0000 7fceb3fff700 1 -- 172.21.15.19:0/1352126461 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcebc0e30b0 con 0x7fcecc1045d0 2024-05-09T01:35:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.290+0000 7fceca59c700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fceb40b2880 0x7fceb40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.291+0000 7fceca59c700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fceb40b2880 0x7fceb40b4d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcecc107780 tx=0x7fcec000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.292+0000 7fcedfaf3b80 1 Processor -- start 2024-05-09T01:35:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.292+0000 7fcedfaf3b80 1 -- start start 2024-05-09T01:35:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 0x5636225fe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x5636225fea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56362251aff0 con 0x5636225f86a0 2024-05-09T01:35:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5636224b97f0 con 0x563622565160 2024-05-09T01:35:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563622516fb0 con 0x56362247b4a0 2024-05-09T01:35:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcecb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcecb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56756/0 (socket says 172.21.15.19:56756) 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcecb59e700 1 -- 172.21.15.19:0/4231187122 learned_addr learned my addr 172.21.15.19:0/4231187122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fceca59c700 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x5636225fea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcecad9d700 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 0x5636225fe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcecb59e700 1 -- 172.21.15.19:0/4231187122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 msgr2=0x5636225fea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcecb59e700 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x5636225fea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcecb59e700 1 -- 172.21.15.19:0/4231187122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 msgr2=0x5636225fe4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcecb59e700 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 0x5636225fe4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fcecb59e700 1 -- 172.21.15.19:0/4231187122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636224b1370 con 0x5636225f86a0 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.293+0000 7fceca59c700 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x5636225fea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.294+0000 7fcecb59e700 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f8ad0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcec4003fa0 tx=0x7fcec4009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68fede56c5b03e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.294+0000 7fceb17fa700 1 -- 172.21.15.19:0/4231187122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4004bb0 con 0x5636225f86a0 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.294+0000 7fceb17fa700 1 -- 172.21.15.19:0/4231187122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcec4004d50 con 0x5636225f86a0 2024-05-09T01:35:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.294+0000 7fceb17fa700 1 -- 172.21.15.19:0/4231187122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4007da0 con 0x5636225f86a0 2024-05-09T01:35:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/4231187122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 msgr2=0x5636225f8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f8ad0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcec4003fa0 tx=0x7fcec4009ba0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/4231187122 shutdown_connections 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f8ad0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 0x5636225fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/4231187122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x5636225fea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/4231187122 >> 172.21.15.19:0/4231187122 conn(0x5636224db6a0 msgr2=0x5636225f3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/4231187122 shutdown_connections 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/4231187122 wait complete. 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 -- start start 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.295+0000 7fcedfaf3b80 1 Processor -- start 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56362247b4a0 0x563622602200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563622565160 0x563622602740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x563622602c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563622396ba0 con 0x5636225f86a0 2024-05-09T01:35:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5636224d5150 con 0x56362247b4a0 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636226051d0 con 0x563622565160 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcedfaf3b80 10 client.0 _ll_register_callbacks cb 0x56362238c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x563622602c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x563622602c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56758/0 (socket says 172.21.15.19:56758) 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecb59e700 1 -- 172.21.15.19:0/2689090914 learned_addr learned my addr 172.21.15.19:0/2689090914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56362247b4a0 0x563622602200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fceca59c700 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563622565160 0x563622602740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecb59e700 1 -- 172.21.15.19:0/2689090914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563622565160 msgr2=0x563622602740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecb59e700 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563622565160 0x563622602740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecb59e700 1 -- 172.21.15.19:0/2689090914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56362247b4a0 msgr2=0x563622602200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecb59e700 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56362247b4a0 0x563622602200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecb59e700 1 -- 172.21.15.19:0/2689090914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563622396490 con 0x5636225f86a0 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.296+0000 7fcecad9d700 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56362247b4a0 0x563622602200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fcecb59e700 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x563622602c80 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fcec4003b40 tx=0x7fcec4007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8aadcdeb08612e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fceaeff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fceaeff5700 1 -- 172.21.15.19:0/2689090914 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4022070 con 0x5636225f86a0 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fceaeff5700 1 -- 172.21.15.19:0/2689090914 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcec4004450 con 0x5636225f86a0 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2689090914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec4004d50 con 0x5636225f86a0 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fceaeff5700 1 -- 172.21.15.19:0/2689090914 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcec402e050 con 0x5636225f86a0 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fcedfaf3b80 10 client.18102 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2689090914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56362260fbe0 con 0x5636225f86a0 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fceaeff5700 1 -- 172.21.15.19:0/2689090914 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcec4004d50 con 0x5636225f86a0 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.297+0000 7fceaeff5700 1 -- 172.21.15.19:0/2689090914 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcec401e410 con 0x5636225f86a0 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.298+0000 7fcedfaf3b80 10 client.18102 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.298+0000 7fcedfaf3b80 10 client.18102 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.298+0000 7fcedfaf3b80 20 client.18102 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.298+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x563622612110 0x563622614530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.298+0000 7fcedfaf3b80 4 client.18102 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.298+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2689090914 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636225289d0 con 0x563622612110 2024-05-09T01:35:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.298+0000 7fcecad9d700 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x563622612110 0x563622614530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.299+0000 7fceaeff5700 10 client.18102 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:35:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.299+0000 7fcecad9d700 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x563622612110 0x563622614530 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1596ccba685d85c server_cookie=3cd2ae99644298c in_seq=0 out_seq=0 2024-05-09T01:35:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.299+0000 7fceaeff5700 1 -- 172.21.15.19:0/2689090914 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636225289d0 con 0x563622612110 2024-05-09T01:35:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.299+0000 7fceaeff5700 10 client.18102 handle_command_reply: tid=0 2024-05-09T01:35:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.299+0000 7fcedfaf3b80 1 client.18102 shutdown 2024-05-09T01:35:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2689090914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 msgr2=0x563622602c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x563622602c80 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fcec4003b40 tx=0x7fcec4007ec0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2689090914 shutdown_connections 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x563622612110 0x563622614530 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x563622602c80 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56362247b4a0 0x563622602200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2689090914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563622565160 0x563622602740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2689090914 >> 172.21.15.19:0/2689090914 conn(0x5636224db6a0 msgr2=0x5636225f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fceaeff5700 0 client.18102 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2689090914 shutdown_connections 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2689090914 wait complete. 2024-05-09T01:35:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.300+0000 7fcedfaf3b80 20 client.18102 trim_cache size 0 max 16384 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcedfaf3b80 1 Processor -- start 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcedfaf3b80 1 -- start start 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x5636225fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 0x56362260e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1aeb80 con 0x5636225f86a0 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1b06c0 con 0x563622565160 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1af250 con 0x56362247b4a0 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcecb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcecb59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56774/0 (socket says 172.21.15.19:56774) 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcecb59e700 1 -- 172.21.15.19:0/2470951724 learned_addr learned my addr 172.21.15.19:0/2470951724 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fcecad9d700 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x5636225fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.316+0000 7fceca59c700 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 0x56362260e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fcecb59e700 1 -- 172.21.15.19:0/2470951724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 msgr2=0x5636225fc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fcecb59e700 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x5636225fc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fcecb59e700 1 -- 172.21.15.19:0/2470951724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 msgr2=0x56362260e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fcecb59e700 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 0x56362260e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fcecb59e700 1 -- 172.21.15.19:0/2470951724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636225f4d00 con 0x5636225f86a0 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fceca59c700 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 0x56362260e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fcecb59e700 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f7da0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcec4000f80 tx=0x7fcec4007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed729953c799d255 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fceb0ff9700 1 -- 172.21.15.19:0/2470951724 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec402b070 con 0x5636225f86a0 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fceb0ff9700 1 -- 172.21.15.19:0/2470951724 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcec4004900 con 0x5636225f86a0 2024-05-09T01:35:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.317+0000 7fceb0ff9700 1 -- 172.21.15.19:0/2470951724 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4022050 con 0x5636225f86a0 2024-05-09T01:35:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.318+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2470951724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 msgr2=0x5636225f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.318+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f7da0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcec4000f80 tx=0x7fcec4007da0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.318+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2470951724 shutdown_connections 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.318+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636225f86a0 0x5636225f7da0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.318+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x5636225fc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.318+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2470951724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563622565160 0x56362260e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.318+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2470951724 >> 172.21.15.19:0/2470951724 conn(0x5636224db6a0 msgr2=0x5636225f7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.318+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2470951724 shutdown_connections 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.318+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2470951724 wait complete. 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fcedfaf3b80 1 -- start start 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fcedfaf3b80 1 Processor -- start 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x563622674aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563622565160 0x563622674fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fcedfaf3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636225f86a0 0x5636226126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1afff0 con 0x563622565160 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1af920 con 0x5636225f86a0 2024-05-09T01:35:21.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fcedfaf3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1acb20 con 0x56362247b4a0 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fcedfaf3b80 10 client.0 _ll_register_callbacks cb 0x56362238c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.319+0000 7fceca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563622565160 0x563622674fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563622565160 0x563622674fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56784/0 (socket says 172.21.15.19:56784) 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceca59c700 1 -- 172.21.15.19:0/2592684893 learned_addr learned my addr 172.21.15.19:0/2592684893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fcecad9d700 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x563622674aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fcecb59e700 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636225f86a0 0x5636226126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceca59c700 1 -- 172.21.15.19:0/2592684893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 msgr2=0x563622674aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceca59c700 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x563622674aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceca59c700 1 -- 172.21.15.19:0/2592684893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636225f86a0 msgr2=0x5636226126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceca59c700 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636225f86a0 0x5636226126f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceca59c700 1 -- 172.21.15.19:0/2592684893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563622612d30 con 0x563622565160 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceca59c700 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563622565160 0x563622674fe0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fcec0005d90 tx=0x7fcec0013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd9c61a1d565593 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceaeff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceaeff5700 1 -- 172.21.15.19:0/2592684893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec002e060 con 0x563622565160 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2592684893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec4004900 con 0x563622565160 2024-05-09T01:35:21.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.320+0000 7fceaeff5700 1 -- 172.21.15.19:0/2592684893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcec001f840 con 0x563622565160 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fceaeff5700 1 -- 172.21.15.19:0/2592684893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcec0011e30 con 0x563622565160 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fcedfaf3b80 10 client.18108 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2592684893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56362267c0a0 con 0x563622565160 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fceaeff5700 1 -- 172.21.15.19:0/2592684893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcec4004900 con 0x563622565160 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fceaeff5700 1 -- 172.21.15.19:0/2592684893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcec0029bf0 con 0x563622565160 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fcedfaf3b80 10 client.18108 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fcedfaf3b80 10 client.18108 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fcedfaf3b80 20 client.18108 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x56362267c7e0 0x56362267ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fcedfaf3b80 4 client.18108 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2592684893 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563622522a70 con 0x56362267c7e0 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.321+0000 7fcecad9d700 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x56362267c7e0 0x56362267ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcecad9d700 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x56362267c7e0 0x56362267ec00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bbe31be386fef36 server_cookie=41e3ecfcae2900ac in_seq=0 out_seq=0 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fceaeff5700 10 client.18108 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fceaeff5700 1 -- 172.21.15.19:0/2592684893 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563622522a70 con 0x56362267c7e0 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fceaeff5700 10 client.18108 handle_command_reply: tid=0 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 client.18108 shutdown 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2592684893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563622565160 msgr2=0x563622674fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563622565160 0x563622674fe0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fcec0005d90 tx=0x7fcec0013070 comp rx=0 tx=0).stop 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2592684893 shutdown_connections 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x56362267c7e0 0x56362267ec00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636225f86a0 0x5636226126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fceaeff5700 0 client.18108 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362247b4a0 0x563622674aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 --2- 172.21.15.19:0/2592684893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563622565160 0x563622674fe0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2592684893 >> 172.21.15.19:0/2592684893 conn(0x5636224db6a0 msgr2=0x5636225fbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2592684893 shutdown_connections 2024-05-09T01:35:21.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 1 -- 172.21.15.19:0/2592684893 wait complete. 2024-05-09T01:35:21.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.322+0000 7fcedfaf3b80 20 client.18108 trim_cache size 0 max 16384 2024-05-09T01:35:21.324 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 -- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fceb40b2880 msgr2=0x7fceb40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fceb40b2880 0x7fceb40b4d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcecc107780 tx=0x7fcec000b410 comp rx=0 tx=0).stop 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 -- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 msgr2=0x7fcecc19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 0x7fcecc19ad90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0098e0 tx=0x7fcebc0057c0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 -- 172.21.15.19:0/1352126461 shutdown_connections 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcecc109850 0x7fcecc19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fceb40b2880 0x7fceb40b4d90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcecc1045d0 0x7fcecc19ad90 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 --2- 172.21.15.19:0/1352126461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcecc108f00 0x7fcecc19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 -- 172.21.15.19:0/1352126461 >> 172.21.15.19:0/1352126461 conn(0x7fcecc0ffea0 msgr2=0x7fcecc108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 -- 172.21.15.19:0/1352126461 shutdown_connections 2024-05-09T01:35:21.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:21.323+0000 7fced10dd700 1 -- 172.21.15.19:0/1352126461 wait complete. 2024-05-09T01:35:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.380+0000 7f02788a8640 1 -- 172.21.15.19:0/265372938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274103c70 msgr2=0x7f0274104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.380+0000 7f02788a8640 1 --2- 172.21.15.19:0/265372938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274103c70 0x7f0274104970 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0264009ca0 tx=0x7f026402a8c0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.381+0000 7f02788a8640 1 -- 172.21.15.19:0/265372938 shutdown_connections 2024-05-09T01:35:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.381+0000 7f02788a8640 1 --2- 172.21.15.19:0/265372938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0274109bb0 0x7f027410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.381+0000 7f02788a8640 1 --2- 172.21.15.19:0/265372938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0274104eb0 0x7f02741052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.381+0000 7f02788a8640 1 --2- 172.21.15.19:0/265372938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274103c70 0x7f0274104970 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.381+0000 7f02788a8640 1 -- 172.21.15.19:0/265372938 >> 172.21.15.19:0/265372938 conn(0x7f02740fee60 msgr2=0x7f0274101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.381+0000 7f02788a8640 1 -- 172.21.15.19:0/265372938 shutdown_connections 2024-05-09T01:35:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.381+0000 7f02788a8640 1 -- 172.21.15.19:0/265372938 wait complete. 2024-05-09T01:35:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f02788a8640 1 Processor -- start 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f02788a8640 1 -- start start 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f02788a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0274103c70 0x7f027419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f02788a8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0274104eb0 0x7f027419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f02788a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274109bb0 0x7f02741a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f02788a8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f027410f750 con 0x7f0274109bb0 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f02788a8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f027410f5d0 con 0x7f0274104eb0 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f02788a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f027410f8d0 con 0x7f0274103c70 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0272d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274109bb0 0x7f02741a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0272d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274109bb0 0x7f02741a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56808/0 (socket says 172.21.15.19:56808) 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0272d9d640 1 -- 172.21.15.19:0/2440049904 learned_addr learned my addr 172.21.15.19:0/2440049904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f027259c640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0274103c70 0x7f027419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0271d9b640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0274104eb0 0x7f027419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0272d9d640 1 -- 172.21.15.19:0/2440049904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0274103c70 msgr2=0x7f027419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0272d9d640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0274103c70 0x7f027419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0272d9d640 1 -- 172.21.15.19:0/2440049904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0274104eb0 msgr2=0x7f027419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0272d9d640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0274104eb0 0x7f027419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0272d9d640 1 -- 172.21.15.19:0/2440049904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02741a1ef0 con 0x7f0274109bb0 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.382+0000 7f0271d9b640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0274104eb0 0x7f027419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.383+0000 7f0272d9d640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274109bb0 0x7f02741a1880 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f026c00d950 tx=0x7f026c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.383+0000 7f02637fe640 1 -- 172.21.15.19:0/2440049904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c017030 con 0x7f0274109bb0 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.383+0000 7f02637fe640 1 -- 172.21.15.19:0/2440049904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026c004020 con 0x7f0274109bb0 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.383+0000 7f02788a8640 1 -- 172.21.15.19:0/2440049904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02741a21e0 con 0x7f0274109bb0 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.383+0000 7f02637fe640 1 -- 172.21.15.19:0/2440049904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c003b50 con 0x7f0274109bb0 2024-05-09T01:35:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.383+0000 7f02788a8640 1 -- 172.21.15.19:0/2440049904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02741a26e0 con 0x7f0274109bb0 2024-05-09T01:35:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.386+0000 7f027f839740 1 Processor -- start 2024-05-09T01:35:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.386+0000 7f02637fe640 1 -- 172.21.15.19:0/2440049904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f026c01a070 con 0x7f0274109bb0 2024-05-09T01:35:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.386+0000 7f027f839740 1 -- start start 2024-05-09T01:35:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f027f839740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 0x55bf4965e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f027f839740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 0x55bf4965ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f027f839740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf49665980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f027f839740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bf495c27c0 con 0x55bf492994d0 2024-05-09T01:35:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f027f839740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bf4955e0e0 con 0x55bf492ba1b0 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f027f839740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bf495be780 con 0x55bf492912f0 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf49665980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf49665980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56810/0 (socket says 172.21.15.19:56810) 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 -- 172.21.15.19:0/2790055149 learned_addr learned my addr 172.21.15.19:0/2790055149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f027259c640 1 --2- 172.21.15.19:0/2790055149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 0x55bf4965ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 --2- 172.21.15.19:0/2790055149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 0x55bf4965e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 -- 172.21.15.19:0/2790055149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 msgr2=0x55bf4965ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 --2- 172.21.15.19:0/2790055149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 0x55bf4965ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 -- 172.21.15.19:0/2790055149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 msgr2=0x55bf4965e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 --2- 172.21.15.19:0/2790055149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 0x55bf4965e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f02637fe640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f025c0b27e0 0x7f025c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 -- 172.21.15.19:0/2790055149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf495877c0 con 0x55bf492994d0 2024-05-09T01:35:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f027259c640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f025c0b27e0 0x7f025c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f0271d9b640 1 --2- 172.21.15.19:0/2790055149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf49665980 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f02680124c0 tx=0x7f026800b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb779e56e69d17a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.387+0000 7f02637fe640 1 -- 172.21.15.19:0/2440049904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f026c0e1a80 con 0x7f0274109bb0 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.388+0000 7f0260ff9640 1 -- 172.21.15.19:0/2790055149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268018030 con 0x55bf492994d0 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.388+0000 7f0260ff9640 1 -- 172.21.15.19:0/2790055149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02680054d0 con 0x55bf492994d0 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.388+0000 7f0260ff9640 1 -- 172.21.15.19:0/2790055149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026800e630 con 0x55bf492994d0 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 -- 172.21.15.19:0/2790055149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 msgr2=0x55bf49665980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027259c640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f025c0b27e0 0x7f025c0b4cf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0264000f80 tx=0x7f0264004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 --2- 172.21.15.19:0/2790055149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf49665980 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f02680124c0 tx=0x7f026800b620 comp rx=0 tx=0).stop 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 -- 172.21.15.19:0/2790055149 shutdown_connections 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 --2- 172.21.15.19:0/2790055149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf49665980 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 --2- 172.21.15.19:0/2790055149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 0x55bf4965ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 --2- 172.21.15.19:0/2790055149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 0x55bf4965e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 -- 172.21.15.19:0/2790055149 >> 172.21.15.19:0/2790055149 conn(0x55bf492ae300 msgr2=0x55bf49659e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 -- 172.21.15.19:0/2790055149 shutdown_connections 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 -- 172.21.15.19:0/2790055149 wait complete. 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 -- start start 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 Processor -- start 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492912f0 0x55bf49668320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.389+0000 7f027f839740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492994d0 0x55bf49668860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027f839740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 0x55bf4966f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027f839740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bf496663d0 con 0x55bf492912f0 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027f839740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bf495499c0 con 0x55bf492ba1b0 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027f839740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bf49666550 con 0x55bf492994d0 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027f839740 10 client.0 _ll_register_callbacks cb 0x55bf49576020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492912f0 0x55bf49668320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492912f0 0x55bf49668320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56812/0 (socket says 172.21.15.19:56812) 2024-05-09T01:35:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027259c640 1 -- 172.21.15.19:0/1149420161 learned_addr learned my addr 172.21.15.19:0/1149420161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f0271d9b640 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492994d0 0x55bf49668860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f0272d9d640 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 0x55bf4966f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027259c640 1 -- 172.21.15.19:0/1149420161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492994d0 msgr2=0x55bf49668860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027259c640 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492994d0 0x55bf49668860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027259c640 1 -- 172.21.15.19:0/1149420161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 msgr2=0x55bf4966f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027259c640 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 0x55bf4966f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027259c640 1 -- 172.21.15.19:0/1149420161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf495701f0 con 0x55bf492912f0 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f0272d9d640 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 0x55bf4966f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f027259c640 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492912f0 0x55bf49668320 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0264009ca0 tx=0x7f026402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b29e59ceac3609 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f0259ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.390+0000 7f0259ffb640 1 -- 172.21.15.19:0/1149420161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264004250 con 0x55bf492912f0 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.391+0000 7f0259ffb640 1 -- 172.21.15.19:0/1149420161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f02640043f0 con 0x55bf492912f0 2024-05-09T01:35:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.391+0000 7f027f839740 1 -- 172.21.15.19:0/1149420161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02680054d0 con 0x55bf492912f0 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.391+0000 7f0259ffb640 1 -- 172.21.15.19:0/1149420161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f02640045f0 con 0x55bf492912f0 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.391+0000 7f027f839740 10 client.18120 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.391+0000 7f027f839740 1 -- 172.21.15.19:0/1149420161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf49325790 con 0x55bf492912f0 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.391+0000 7f0259ffb640 1 -- 172.21.15.19:0/1149420161 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02680054d0 con 0x55bf492912f0 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f0259ffb640 1 -- 172.21.15.19:0/1149420161 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0264051070 con 0x55bf492912f0 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f027f839740 10 client.18120 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f027f839740 10 client.18120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f027f839740 20 client.18120 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f027f839740 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bf49278d80 0x55bf49675c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f027f839740 4 client.18120 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f027f839740 1 -- 172.21.15.19:0/1149420161 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf495d03d0 con 0x55bf49278d80 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f0271d9b640 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bf49278d80 0x55bf49675c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f0271d9b640 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bf49278d80 0x55bf49675c60 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2deb22529888d5bf server_cookie=8ab2a4c501f6c9c2 in_seq=0 out_seq=0 2024-05-09T01:35:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.392+0000 7f0259ffb640 10 client.18120 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f0259ffb640 1 -- 172.21.15.19:0/1149420161 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf495d03d0 con 0x55bf49278d80 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f0259ffb640 10 client.18120 handle_command_reply: tid=0 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 client.18120 shutdown 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 -- 172.21.15.19:0/1149420161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492912f0 msgr2=0x55bf49668320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492912f0 0x55bf49668320 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0264009ca0 tx=0x7f026402a8c0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 -- 172.21.15.19:0/1149420161 shutdown_connections 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bf49278d80 0x55bf49675c60 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492ba1b0 0x55bf4966f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492994d0 0x55bf49668860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 --2- 172.21.15.19:0/1149420161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492912f0 0x55bf49668320 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 -- 172.21.15.19:0/1149420161 >> 172.21.15.19:0/1149420161 conn(0x55bf492ae300 msgr2=0x55bf49658700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f0259ffb640 0 client.18120 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.393+0000 7f027f839740 1 -- 172.21.15.19:0/1149420161 shutdown_connections 2024-05-09T01:35:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.394+0000 7f027f839740 1 -- 172.21.15.19:0/1149420161 wait complete. 2024-05-09T01:35:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.394+0000 7f027f839740 20 client.18120 trim_cache size 0 max 16384 2024-05-09T01:35:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f027f839740 1 Processor -- start 2024-05-09T01:35:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f027f839740 1 -- start start 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f027f839740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492912f0 0x55bf49668600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f027f839740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf4966f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f027f839740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492ba1b0 0x55bf4966fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f027f839740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02741ab4b0 con 0x55bf492994d0 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f027f839740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02741acff0 con 0x55bf492912f0 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f027f839740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02741abb80 con 0x55bf492ba1b0 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f0271d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf4966f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f0271d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf4966f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56818/0 (socket says 172.21.15.19:56818) 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f0271d9b640 1 -- 172.21.15.19:0/1242665739 learned_addr learned my addr 172.21.15.19:0/1242665739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f0271d9b640 1 -- 172.21.15.19:0/1242665739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492ba1b0 msgr2=0x55bf4966fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f0271d9b640 1 --2- 172.21.15.19:0/1242665739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492ba1b0 0x55bf4966fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f0271d9b640 1 -- 172.21.15.19:0/1242665739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492912f0 msgr2=0x55bf49668600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f0271d9b640 1 --2- 172.21.15.19:0/1242665739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492912f0 0x55bf49668600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f0271d9b640 1 -- 172.21.15.19:0/1242665739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf4965eb60 con 0x55bf492994d0 2024-05-09T01:35:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.409+0000 7f0271d9b640 1 --2- 172.21.15.19:0/1242665739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf4966f750 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0268005d00 tx=0x7f026800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745ea3a7111b82e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f025bfff640 1 -- 172.21.15.19:0/1242665739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268018040 con 0x55bf492994d0 2024-05-09T01:35:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f025bfff640 1 -- 172.21.15.19:0/1242665739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026802ae50 con 0x55bf492994d0 2024-05-09T01:35:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f025bfff640 1 -- 172.21.15.19:0/1242665739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02680049e0 con 0x55bf492994d0 2024-05-09T01:35:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f027f839740 1 -- 172.21.15.19:0/1242665739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 msgr2=0x55bf4966f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f027f839740 1 --2- 172.21.15.19:0/1242665739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf4966f750 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0268005d00 tx=0x7f026800b4a0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f027f839740 1 -- 172.21.15.19:0/1242665739 shutdown_connections 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f027f839740 1 --2- 172.21.15.19:0/1242665739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492ba1b0 0x55bf4966fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f027f839740 1 --2- 172.21.15.19:0/1242665739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492994d0 0x55bf4966f750 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f027f839740 1 --2- 172.21.15.19:0/1242665739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492912f0 0x55bf49668600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f027f839740 1 -- 172.21.15.19:0/1242665739 >> 172.21.15.19:0/1242665739 conn(0x55bf492ae300 msgr2=0x55bf49667ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f027f839740 1 -- 172.21.15.19:0/1242665739 shutdown_connections 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.410+0000 7f027f839740 1 -- 172.21.15.19:0/1242665739 wait complete. 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027f839740 1 -- start start 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027f839740 1 Processor -- start 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027f839740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 0x55bf4966ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027f839740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492994d0 0x55bf4966d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027f839740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492ba1b0 0x55bf4966e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027f839740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02741ac920 con 0x55bf492ba1b0 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027f839740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02741ac250 con 0x55bf492994d0 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027f839740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02741a9490 con 0x55bf492912f0 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027f839740 10 client.0 _ll_register_callbacks cb 0x7f0264009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f0272d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492ba1b0 0x55bf4966e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f027259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 0x55bf4966ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f0272d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492ba1b0 0x55bf4966e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56830/0 (socket says 172.21.15.19:56830) 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f0272d9d640 1 -- 172.21.15.19:0/1640128265 learned_addr learned my addr 172.21.15.19:0/1640128265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.411+0000 7f0271d9b640 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492994d0 0x55bf4966d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0272d9d640 1 -- 172.21.15.19:0/1640128265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 msgr2=0x55bf4966ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0272d9d640 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 0x55bf4966ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0272d9d640 1 -- 172.21.15.19:0/1640128265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492994d0 msgr2=0x55bf4966d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0272d9d640 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492994d0 0x55bf4966d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0272d9d640 1 -- 172.21.15.19:0/1640128265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf4939c5f0 con 0x55bf492ba1b0 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0271d9b640 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492994d0 0x55bf4966d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0272d9d640 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492ba1b0 0x55bf4966e2f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f026c005fd0 tx=0x7f026c0a8cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a2a7cb88e24b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0259ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0259ffb640 1 -- 172.21.15.19:0/1640128265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c0c2020 con 0x55bf492ba1b0 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0259ffb640 1 -- 172.21.15.19:0/1640128265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f026c0ad0d0 con 0x55bf492ba1b0 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f027f839740 1 -- 172.21.15.19:0/1640128265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026802ae50 con 0x55bf492ba1b0 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0259ffb640 1 -- 172.21.15.19:0/1640128265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f026c0b1ee0 con 0x55bf492ba1b0 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f027f839740 10 client.18126 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f027f839740 1 -- 172.21.15.19:0/1640128265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf49586ed0 con 0x55bf492ba1b0 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0259ffb640 1 -- 172.21.15.19:0/1640128265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026802ae50 con 0x55bf492ba1b0 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f0259ffb640 1 -- 172.21.15.19:0/1640128265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f026c068af0 con 0x55bf492ba1b0 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f027f839740 10 client.18126 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f027f839740 10 client.18126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:21.414 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f027f839740 20 client.18126 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f027f839740 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bf4924e1c0 0x55bf496e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f027f839740 4 client.18126 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.412+0000 7f027f839740 1 -- 172.21.15.19:0/1640128265 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf495ca240 con 0x55bf4924e1c0 2024-05-09T01:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.413+0000 7f027259c640 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bf4924e1c0 0x55bf496e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.413+0000 7f027259c640 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bf4924e1c0 0x55bf496e0020 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b745c939b25cd4 server_cookie=bc44e638adbc85c9 in_seq=0 out_seq=0 2024-05-09T01:35:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.413+0000 7f0259ffb640 10 client.18126 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f0259ffb640 1 -- 172.21.15.19:0/1640128265 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bf495ca240 con 0x55bf4924e1c0 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f0259ffb640 10 client.18126 handle_command_reply: tid=0 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f027f839740 1 client.18126 shutdown 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f027f839740 1 -- 172.21.15.19:0/1640128265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492ba1b0 msgr2=0x55bf4966e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f027f839740 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492ba1b0 0x55bf4966e2f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f026c005fd0 tx=0x7f026c0a8cd0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f027f839740 1 -- 172.21.15.19:0/1640128265 shutdown_connections 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f027f839740 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bf4924e1c0 0x55bf496e0020 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f027f839740 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bf492ba1b0 0x55bf4966e2f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f027f839740 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bf492994d0 0x55bf4966d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f0259ffb640 0 client.18126 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f027f839740 1 --2- 172.21.15.19:0/1640128265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf492912f0 0x55bf4966ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.429+0000 7f027f839740 1 -- 172.21.15.19:0/1640128265 >> 172.21.15.19:0/1640128265 conn(0x55bf492ae300 msgr2=0x55bf49659680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f027f839740 1 -- 172.21.15.19:0/1640128265 shutdown_connections 2024-05-09T01:35:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f027f839740 1 -- 172.21.15.19:0/1640128265 wait complete. 2024-05-09T01:35:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f027f839740 20 client.18126 trim_cache size 0 max 16384 2024-05-09T01:35:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:35:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:35:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:35:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 -- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f025c0b27e0 msgr2=0x7f025c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f025c0b27e0 0x7f025c0b4cf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0264000f80 tx=0x7f0264004a60 comp rx=0 tx=0).stop 2024-05-09T01:35:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 -- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274109bb0 msgr2=0x7f02741a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274109bb0 0x7f02741a1880 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f026c00d950 tx=0x7f026c00de10 comp rx=0 tx=0).stop 2024-05-09T01:35:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 -- 172.21.15.19:0/2440049904 shutdown_connections 2024-05-09T01:35:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f025c0b27e0 0x7f025c0b4cf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0274109bb0 0x7f02741a1880 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0274104eb0 0x7f027419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 --2- 172.21.15.19:0/2440049904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0274103c70 0x7f027419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 -- 172.21.15.19:0/2440049904 >> 172.21.15.19:0/2440049904 conn(0x7f02740fee60 msgr2=0x7f0274105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 -- 172.21.15.19:0/2440049904 shutdown_connections 2024-05-09T01:35:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:21.430+0000 7f02788a8640 1 -- 172.21.15.19:0/2440049904 wait complete. 2024-05-09T01:35:22.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:35:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:22.771 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:22 smithi019 ceph-mon[30158]: pgmap v673: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-05-09T01:35:22.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:22 smithi196 ceph-mon[35041]: pgmap v673: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-05-09T01:35:23.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:22 smithi059 ceph-mon[34351]: pgmap v673: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-05-09T01:35:24.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.196+0000 7f3085c6a700 1 -- 172.21.15.19:0/3971553312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3080108f20 msgr2=0x7f3080109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.196+0000 7f3085c6a700 1 --2- 172.21.15.19:0/3971553312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3080108f20 0x7f3080109330 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f30740099c0 tx=0x7f3074009cd0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.196+0000 7f3085c6a700 1 -- 172.21.15.19:0/3971553312 shutdown_connections 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.196+0000 7f3085c6a700 1 --2- 172.21.15.19:0/3971553312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3080109870 0x7f30801111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.196+0000 7f3085c6a700 1 --2- 172.21.15.19:0/3971553312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30801045f0 0x7f3080104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.196+0000 7f3085c6a700 1 --2- 172.21.15.19:0/3971553312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3080108f20 0x7f3080109330 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.196+0000 7f3085c6a700 1 -- 172.21.15.19:0/3971553312 >> 172.21.15.19:0/3971553312 conn(0x7f3080100040 msgr2=0x7f3080102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.196+0000 7f3085c6a700 1 -- 172.21.15.19:0/3971553312 shutdown_connections 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.196+0000 7f3085c6a700 1 -- 172.21.15.19:0/3971553312 wait complete. 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f3085c6a700 1 Processor -- start 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f3085c6a700 1 -- start start 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f3085c6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30801045f0 0x7f308019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f3085c6a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3080108f20 0x7f308019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f3085c6a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080109870 0x7f308019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f3085c6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30801144a0 con 0x7f30801045f0 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f3085c6a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3080114320 con 0x7f3080108f20 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f3085c6a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3080114620 con 0x7f3080109870 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f307f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30801045f0 0x7f308019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f307f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30801045f0 0x7f308019aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56842/0 (socket says 172.21.15.19:56842) 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f307f7fe700 1 -- 172.21.15.19:0/1324681266 learned_addr learned my addr 172.21.15.19:0/1324681266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f307ffff700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080109870 0x7f308019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.197+0000 7f307effd700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3080108f20 0x7f308019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f307f7fe700 1 -- 172.21.15.19:0/1324681266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080109870 msgr2=0x7f308019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080109870 0x7f308019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f307f7fe700 1 -- 172.21.15.19:0/1324681266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3080108f20 msgr2=0x7f308019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3080108f20 0x7f308019b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f307f7fe700 1 -- 172.21.15.19:0/1324681266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308019fda0 con 0x7f30801045f0 2024-05-09T01:35:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30801045f0 0x7f308019aed0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f307000d980 tx=0x7f307000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f307cff9700 1 -- 172.21.15.19:0/1324681266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070017070 con 0x7f30801045f0 2024-05-09T01:35:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f307cff9700 1 -- 172.21.15.19:0/1324681266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3070005020 con 0x7f30801045f0 2024-05-09T01:35:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f3085c6a700 1 -- 172.21.15.19:0/1324681266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30801a0090 con 0x7f30801045f0 2024-05-09T01:35:24.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f307cff9700 1 -- 172.21.15.19:0/1324681266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307000fa10 con 0x7f30801045f0 2024-05-09T01:35:24.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.198+0000 7f3085c6a700 1 -- 172.21.15.19:0/1324681266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30801a79b0 con 0x7f30801045f0 2024-05-09T01:35:24.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.201+0000 7f307cff9700 1 -- 172.21.15.19:0/1324681266 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f307000fbb0 con 0x7f30801045f0 2024-05-09T01:35:24.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.202+0000 7f3094680b80 1 Processor -- start 2024-05-09T01:35:24.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.202+0000 7f3094680b80 1 -- start start 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.202+0000 7f307cff9700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30680b2900 0x7f30680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.202+0000 7f307cff9700 1 -- 172.21.15.19:0/1324681266 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f30700e1fd0 con 0x7f30801045f0 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.202+0000 7f307ffff700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30680b2900 0x7f30680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.202+0000 7f3094680b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9afe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.202+0000 7f3094680b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9afedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f3094680b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9af8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f3094680b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5588c9a16ff0 con 0x5588c9af8ab0 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f3094680b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5588c99b57f0 con 0x5588c9a61160 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f3094680b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588c9a12fb0 con 0x5588c99774a0 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9af8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9af8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56844/0 (socket says 172.21.15.19:56844) 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 -- 172.21.15.19:0/1913367842 learned_addr learned my addr 172.21.15.19:0/1913367842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1913367842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9afe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307effd700 1 --2- 172.21.15.19:0/1913367842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9afedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 -- 172.21.15.19:0/1913367842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 msgr2=0x5588c9afedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 --2- 172.21.15.19:0/1913367842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9afedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 -- 172.21.15.19:0/1913367842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 msgr2=0x5588c9afe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 --2- 172.21.15.19:0/1913367842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9afe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 -- 172.21.15.19:0/1913367842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588c99ad370 con 0x5588c9af8ab0 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30680b2900 0x7f30680b4e10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f307800f930 tx=0x7f307800fcf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f307ffff700 1 --2- 172.21.15.19:0/1913367842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9af8ec0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f307800f400 tx=0x7f307801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0d1d03e5a3cfa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.203+0000 7f3065ffb700 1 -- 172.21.15.19:0/1913367842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078004db0 con 0x5588c9af8ab0 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.204+0000 7f3065ffb700 1 -- 172.21.15.19:0/1913367842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3078024690 con 0x5588c9af8ab0 2024-05-09T01:35:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.204+0000 7f3065ffb700 1 -- 172.21.15.19:0/1913367842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078008770 con 0x5588c9af8ab0 2024-05-09T01:35:24.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.204+0000 7f3094680b80 1 -- 172.21.15.19:0/1913367842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 msgr2=0x5588c9af8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.204+0000 7f3094680b80 1 --2- 172.21.15.19:0/1913367842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9af8ec0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f307800f400 tx=0x7f307801b000 comp rx=0 tx=0).stop 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.204+0000 7f3094680b80 1 -- 172.21.15.19:0/1913367842 shutdown_connections 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.204+0000 7f3094680b80 1 --2- 172.21.15.19:0/1913367842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9af8ec0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.204+0000 7f3094680b80 1 --2- 172.21.15.19:0/1913367842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9afe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.204+0000 7f3094680b80 1 --2- 172.21.15.19:0/1913367842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9afedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.204+0000 7f3094680b80 1 -- 172.21.15.19:0/1913367842 >> 172.21.15.19:0/1913367842 conn(0x5588c99d76a0 msgr2=0x5588c9aef400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.205+0000 7f3094680b80 1 -- 172.21.15.19:0/1913367842 shutdown_connections 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.205+0000 7f3094680b80 1 -- 172.21.15.19:0/1913367842 wait complete. 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.205+0000 7f3094680b80 1 -- start start 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.205+0000 7f3094680b80 1 Processor -- start 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f3094680b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9b024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f3094680b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9b02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f3094680b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9b02f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f3094680b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5588c9892ba0 con 0x5588c9af8ab0 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f3094680b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5588c99d1150 con 0x5588c9a61160 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f3094680b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588c9af9830 con 0x5588c99774a0 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f3094680b80 10 client.0 _ll_register_callbacks cb 0x5588c98886c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9b02f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9b02f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56848/0 (socket says 172.21.15.19:56848) 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307ffff700 1 -- 172.21.15.19:0/4196180523 learned_addr learned my addr 172.21.15.19:0/4196180523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307f7fe700 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9b024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307effd700 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9b02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307ffff700 1 -- 172.21.15.19:0/4196180523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 msgr2=0x5588c9b024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307ffff700 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9b024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307ffff700 1 -- 172.21.15.19:0/4196180523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 msgr2=0x5588c9b02a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:24.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307ffff700 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9b02a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307ffff700 1 -- 172.21.15.19:0/4196180523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588c9892490 con 0x5588c9af8ab0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307f7fe700 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9b024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f307ffff700 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9b02f50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3078024c80 tx=0x7f3078024d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a3fad0bec2e119 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f305effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.206+0000 7f305effd700 1 -- 172.21.15.19:0/4196180523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078008710 con 0x5588c9af8ab0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f305effd700 1 -- 172.21.15.19:0/4196180523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3078004a90 con 0x5588c9af8ab0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f3094680b80 1 -- 172.21.15.19:0/4196180523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3078024690 con 0x5588c9af8ab0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f305effd700 1 -- 172.21.15.19:0/4196180523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3078004cb0 con 0x5588c9af8ab0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f3094680b80 10 client.18138 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f3094680b80 1 -- 172.21.15.19:0/4196180523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588c9b0fdf0 con 0x5588c9af8ab0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f305effd700 1 -- 172.21.15.19:0/4196180523 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3078024690 con 0x5588c9af8ab0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f305effd700 1 -- 172.21.15.19:0/4196180523 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3078008a20 con 0x5588c9af8ab0 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f3094680b80 10 client.18138 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f3094680b80 10 client.18138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f3094680b80 20 client.18138 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f3094680b80 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5588c9b12380 0x5588c9b147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f3094680b80 4 client.18138 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f3094680b80 1 -- 172.21.15.19:0/4196180523 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588c9a249d0 con 0x5588c9b12380 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.207+0000 7f307f7fe700 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5588c9b12380 0x5588c9b147a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.208+0000 7f305effd700 10 client.18138 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.208+0000 7f307f7fe700 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5588c9b12380 0x5588c9b147a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9b1ddc4520f3d5 server_cookie=458d4092e2e0efc9 in_seq=0 out_seq=0 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f305effd700 1 -- 172.21.15.19:0/4196180523 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5588c9a249d0 con 0x5588c9b12380 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f305effd700 10 client.18138 handle_command_reply: tid=0 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 client.18138 shutdown 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 -- 172.21.15.19:0/4196180523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 msgr2=0x5588c9b02f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9b02f50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3078024c80 tx=0x7f3078024d60 comp rx=0 tx=0).stop 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 -- 172.21.15.19:0/4196180523 shutdown_connections 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5588c9b12380 0x5588c9b147a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9af8ab0 0x5588c9b02f50 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9b024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 --2- 172.21.15.19:0/4196180523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9b02a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 -- 172.21.15.19:0/4196180523 >> 172.21.15.19:0/4196180523 conn(0x5588c99d76a0 msgr2=0x5588c9aef1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f305effd700 0 client.18138 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 -- 172.21.15.19:0/4196180523 shutdown_connections 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.209+0000 7f3094680b80 1 -- 172.21.15.19:0/4196180523 wait complete. 2024-05-09T01:35:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.210+0000 7f3094680b80 20 client.18138 trim_cache size 0 max 16384 2024-05-09T01:35:24.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.224+0000 7f3094680b80 1 Processor -- start 2024-05-09T01:35:24.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.224+0000 7f3094680b80 1 -- start start 2024-05-09T01:35:24.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f3094680b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c99774a0 0x5588c9b02d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f3094680b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9b014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f3094680b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c9af8ab0 0x5588c9aff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f3094680b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30801acac0 con 0x5588c99774a0 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f3094680b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30801ae600 con 0x5588c9a61160 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f3094680b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30801ad190 con 0x5588c9af8ab0 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c99774a0 0x5588c9b02d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c99774a0 0x5588c9b02d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56852/0 (socket says 172.21.15.19:56852) 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307f7fe700 1 -- 172.21.15.19:0/1580243470 learned_addr learned my addr 172.21.15.19:0/1580243470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307ffff700 1 --2- 172.21.15.19:0/1580243470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c9af8ab0 0x5588c9aff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307effd700 1 --2- 172.21.15.19:0/1580243470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9b014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307f7fe700 1 -- 172.21.15.19:0/1580243470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c9af8ab0 msgr2=0x5588c9aff4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1580243470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c9af8ab0 0x5588c9aff4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307f7fe700 1 -- 172.21.15.19:0/1580243470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 msgr2=0x5588c9b014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:24.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1580243470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9b014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307f7fe700 1 -- 172.21.15.19:0/1580243470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588c9ae8f60 con 0x5588c99774a0 2024-05-09T01:35:24.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.225+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1580243470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c99774a0 0x5588c9b02d90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f30700dfb70 tx=0x7f30700dfa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ab43cfef3762e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:24.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f30657fa700 1 -- 172.21.15.19:0/1580243470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30700b24d0 con 0x5588c99774a0 2024-05-09T01:35:24.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f30657fa700 1 -- 172.21.15.19:0/1580243470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30700dfc60 con 0x5588c99774a0 2024-05-09T01:35:24.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f30657fa700 1 -- 172.21.15.19:0/1580243470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30700a98d0 con 0x5588c99774a0 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f3094680b80 1 -- 172.21.15.19:0/1580243470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c99774a0 msgr2=0x5588c9b02d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f3094680b80 1 --2- 172.21.15.19:0/1580243470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c99774a0 0x5588c9b02d90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f30700dfb70 tx=0x7f30700dfa20 comp rx=0 tx=0).stop 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f3094680b80 1 -- 172.21.15.19:0/1580243470 shutdown_connections 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f3094680b80 1 --2- 172.21.15.19:0/1580243470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c9af8ab0 0x5588c9aff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f3094680b80 1 --2- 172.21.15.19:0/1580243470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c99774a0 0x5588c9b02d90 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f3094680b80 1 --2- 172.21.15.19:0/1580243470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9a61160 0x5588c9b014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f3094680b80 1 -- 172.21.15.19:0/1580243470 >> 172.21.15.19:0/1580243470 conn(0x5588c99d76a0 msgr2=0x5588c9b00dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f3094680b80 1 -- 172.21.15.19:0/1580243470 shutdown_connections 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.226+0000 7f3094680b80 1 -- 172.21.15.19:0/1580243470 wait complete. 2024-05-09T01:35:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.227+0000 7f3094680b80 1 -- start start 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.227+0000 7f3094680b80 1 Processor -- start 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.227+0000 7f3094680b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9b14330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.227+0000 7f3094680b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9a61160 0x5588c9b14870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.227+0000 7f3094680b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9af8ab0 0x5588c9b7a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.227+0000 7f3094680b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30801adf30 con 0x5588c9a61160 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.227+0000 7f3094680b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30801ad860 con 0x5588c9af8ab0 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.227+0000 7f3094680b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30801aaa60 con 0x5588c99774a0 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.227+0000 7f3094680b80 10 client.0 _ll_register_callbacks cb 0x5588c98886c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9a61160 0x5588c9b14870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9a61160 0x5588c9b14870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56858/0 (socket says 172.21.15.19:56858) 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307effd700 1 -- 172.21.15.19:0/1540761147 learned_addr learned my addr 172.21.15.19:0/1540761147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9b14330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307ffff700 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9af8ab0 0x5588c9b7a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307effd700 1 -- 172.21.15.19:0/1540761147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 msgr2=0x5588c9b14330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307effd700 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9b14330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307effd700 1 -- 172.21.15.19:0/1540761147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9af8ab0 msgr2=0x5588c9b7a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307effd700 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9af8ab0 0x5588c9b7a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307effd700 1 -- 172.21.15.19:0/1540761147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588c9b15120 con 0x5588c9a61160 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f307effd700 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9a61160 0x5588c9b14870 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f308006c680 tx=0x7f3074005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d46e733beced0b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f305effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f305effd700 1 -- 172.21.15.19:0/1540761147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30740095d0 con 0x5588c9a61160 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f3094680b80 1 -- 172.21.15.19:0/1540761147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30700dfc60 con 0x5588c9a61160 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.228+0000 7f305effd700 1 -- 172.21.15.19:0/1540761147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f30740039d0 con 0x5588c9a61160 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f305effd700 1 -- 172.21.15.19:0/1540761147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3074003bd0 con 0x5588c9a61160 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f3094680b80 10 client.18144 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f3094680b80 1 -- 172.21.15.19:0/1540761147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588c9b7be90 con 0x5588c9a61160 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f305effd700 1 -- 172.21.15.19:0/1540761147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30700dfc60 con 0x5588c9a61160 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f305effd700 1 -- 172.21.15.19:0/1540761147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3074026610 con 0x5588c9a61160 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f3094680b80 10 client.18144 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f3094680b80 10 client.18144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f3094680b80 20 client.18144 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f3094680b80 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5588c9b7c5d0 0x5588c9b7e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f3094680b80 4 client.18144 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f3094680b80 1 -- 172.21.15.19:0/1540761147 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588c9a1ea70 con 0x5588c9b7c5d0 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.229+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5588c9b7c5d0 0x5588c9b7e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f305effd700 10 client.18144 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f307f7fe700 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5588c9b7c5d0 0x5588c9b7e9f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca8ee66fad4604b server_cookie=2ad4c22d40ddd73d in_seq=0 out_seq=0 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f305effd700 1 -- 172.21.15.19:0/1540761147 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588c9a1ea70 con 0x5588c9b7c5d0 2024-05-09T01:35:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f305effd700 10 client.18144 handle_command_reply: tid=0 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 client.18144 shutdown 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 -- 172.21.15.19:0/1540761147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9a61160 msgr2=0x5588c9b14870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9a61160 0x5588c9b14870 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f308006c680 tx=0x7f3074005790 comp rx=0 tx=0).stop 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 -- 172.21.15.19:0/1540761147 shutdown_connections 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5588c9b7c5d0 0x5588c9b7e9f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588c9af8ab0 0x5588c9b7a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588c99774a0 0x5588c9b14330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 --2- 172.21.15.19:0/1540761147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588c9a61160 0x5588c9b14870 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 -- 172.21.15.19:0/1540761147 >> 172.21.15.19:0/1540761147 conn(0x5588c99d76a0 msgr2=0x5588c9afca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f305effd700 0 client.18144 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 -- 172.21.15.19:0/1540761147 shutdown_connections 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.230+0000 7f3094680b80 1 -- 172.21.15.19:0/1540761147 wait complete. 2024-05-09T01:35:24.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3094680b80 20 client.18144 trim_cache size 0 max 16384 2024-05-09T01:35:24.233 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:35:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 -- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30680b2900 msgr2=0x7f30680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30680b2900 0x7f30680b4e10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f307800f930 tx=0x7f307800fcf0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 -- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30801045f0 msgr2=0x7f308019aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30801045f0 0x7f308019aed0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f307000d980 tx=0x7f307000dc90 comp rx=0 tx=0).stop 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 -- 172.21.15.19:0/1324681266 shutdown_connections 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3080109870 0x7f308019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f30680b2900 0x7f30680b4e10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30801045f0 0x7f308019aed0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 --2- 172.21.15.19:0/1324681266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3080108f20 0x7f308019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 -- 172.21.15.19:0/1324681266 >> 172.21.15.19:0/1324681266 conn(0x7f3080100040 msgr2=0x7f308010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 -- 172.21.15.19:0/1324681266 shutdown_connections 2024-05-09T01:35:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:24.231+0000 7f3085c6a700 1 -- 172.21.15.19:0/1324681266 wait complete. 2024-05-09T01:35:24.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:24 smithi019 ceph-mon[30158]: pgmap v674: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-09T01:35:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:24 smithi196 ceph-mon[35041]: pgmap v674: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-09T01:35:25.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:24 smithi059 ceph-mon[34351]: pgmap v674: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-09T01:35:26.109 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:35:26.109 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:35:26.376 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:26.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:26 smithi019 ceph-mon[30158]: pgmap v675: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:35:26.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:26 smithi196 ceph-mon[35041]: pgmap v675: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:35:27.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:26 smithi059 ceph-mon[34351]: pgmap v675: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:35:28.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.120+0000 7fe64f915700 1 -- 172.21.15.19:0/1777958126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe648103310 msgr2=0x7fe64810f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:28.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.120+0000 7fe64f915700 1 --2- 172.21.15.19:0/1777958126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe648103310 0x7fe64810f890 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe64000b0a0 tx=0x7fe64000b3b0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.120+0000 7fe64f915700 1 -- 172.21.15.19:0/1777958126 shutdown_connections 2024-05-09T01:35:28.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.120+0000 7fe64f915700 1 --2- 172.21.15.19:0/1777958126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe648103310 0x7fe64810f890 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.120+0000 7fe64f915700 1 --2- 172.21.15.19:0/1777958126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6481089f0 0x7fe648108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.120+0000 7fe64f915700 1 --2- 172.21.15.19:0/1777958126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6481029c0 0x7fe648102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.120+0000 7fe64f915700 1 -- 172.21.15.19:0/1777958126 >> 172.21.15.19:0/1777958126 conn(0x7fe6480fe520 msgr2=0x7fe648100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:28.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.120+0000 7fe64f915700 1 -- 172.21.15.19:0/1777958126 shutdown_connections 2024-05-09T01:35:28.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.120+0000 7fe64f915700 1 -- 172.21.15.19:0/1777958126 wait complete. 2024-05-09T01:35:28.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64f915700 1 Processor -- start 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64f915700 1 -- start start 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64f915700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6481029c0 0x7fe64819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64f915700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe648103310 0x7fe64819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64f915700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6481089f0 0x7fe64819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64f915700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe648112300 con 0x7fe6481029c0 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64f915700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe648112180 con 0x7fe648103310 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64f915700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe648112480 con 0x7fe6481089f0 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64d6b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6481029c0 0x7fe64819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64d6b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6481029c0 0x7fe64819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56874/0 (socket says 172.21.15.19:56874) 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64d6b1700 1 -- 172.21.15.19:0/2816269441 learned_addr learned my addr 172.21.15.19:0/2816269441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.121+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe648103310 0x7fe64819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64deb2700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6481089f0 0x7fe64819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64d6b1700 1 -- 172.21.15.19:0/2816269441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6481089f0 msgr2=0x7fe64819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6481089f0 0x7fe64819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64d6b1700 1 -- 172.21.15.19:0/2816269441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe648103310 msgr2=0x7fe64819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe648103310 0x7fe64819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64d6b1700 1 -- 172.21.15.19:0/2816269441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64819fcf0 con 0x7fe6481029c0 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64deb2700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6481089f0 0x7fe64819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6481029c0 0x7fe64819ae20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe64400ddf0 tx=0x7fe64400a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe63a7fc700 1 -- 172.21.15.19:0/2816269441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64401b070 con 0x7fe6481029c0 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64f915700 1 -- 172.21.15.19:0/2816269441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64819ffe0 con 0x7fe6481029c0 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe63a7fc700 1 -- 172.21.15.19:0/2816269441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe64400de20 con 0x7fe6481029c0 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe63a7fc700 1 -- 172.21.15.19:0/2816269441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440167c0 con 0x7fe6481029c0 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.122+0000 7fe64f915700 1 -- 172.21.15.19:0/2816269441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe648198950 con 0x7fe6481029c0 2024-05-09T01:35:28.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.125+0000 7fe63a7fc700 1 -- 172.21.15.19:0/2816269441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe64401e070 con 0x7fe6481029c0 2024-05-09T01:35:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.125+0000 7fe63a7fc700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6340b2880 0x7fe6340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6340b2880 0x7fe6340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe63a7fc700 1 -- 172.21.15.19:0/2816269441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe6440e1e00 con 0x7fe6481029c0 2024-05-09T01:35:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe65e32bb80 1 Processor -- start 2024-05-09T01:35:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe65e32bb80 1 -- start start 2024-05-09T01:35:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6340b2880 0x7fe6340b4d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe648106340 tx=0x7fe63c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924b10d80 0x55e924c192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c19800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924c13480 0x55e924c13890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e924b36120 con 0x55e924b10d80 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e924adba20 con 0x55e924a96520 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.126+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e924b320e0 con 0x55e924c13480 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64d6b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924b10d80 0x55e924c192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64d6b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924b10d80 0x55e924c192c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56878/0 (socket says 172.21.15.19:56878) 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64d6b1700 1 -- 172.21.15.19:0/3910538923 learned_addr learned my addr 172.21.15.19:0/3910538923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/3910538923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c19800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64deb2700 1 --2- 172.21.15.19:0/3910538923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924c13480 0x55e924c13890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64d6b1700 1 -- 172.21.15.19:0/3910538923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924c13480 msgr2=0x55e924c13890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/3910538923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924c13480 0x55e924c13890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64d6b1700 1 -- 172.21.15.19:0/3910538923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 msgr2=0x55e924c19800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/3910538923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c19800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64d6b1700 1 -- 172.21.15.19:0/3910538923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e924acc580 con 0x55e924b10d80 2024-05-09T01:35:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/3910538923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924b10d80 0x55e924c192c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fe6440a6670 tx=0x7fe6440a8c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00f99d6e03f4587 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe6337fe700 1 -- 172.21.15.19:0/3910538923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440a9400 con 0x55e924b10d80 2024-05-09T01:35:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe6337fe700 1 -- 172.21.15.19:0/3910538923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6440ad7b0 con 0x55e924b10d80 2024-05-09T01:35:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.127+0000 7fe6337fe700 1 -- 172.21.15.19:0/3910538923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440a8660 con 0x55e924b10d80 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.128+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3910538923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924b10d80 msgr2=0x55e924c192c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.128+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3910538923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924b10d80 0x55e924c192c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fe6440a6670 tx=0x7fe6440a8c40 comp rx=0 tx=0).stop 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.128+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3910538923 shutdown_connections 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.128+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3910538923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924c13480 0x55e924c13890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.128+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3910538923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924b10d80 0x55e924c192c0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.128+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3910538923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c19800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.128+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3910538923 >> 172.21.15.19:0/3910538923 conn(0x55e924a99ff0 msgr2=0x55e924c0e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.128+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3910538923 shutdown_connections 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.128+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3910538923 wait complete. 2024-05-09T01:35:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe65e32bb80 1 -- start start 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe65e32bb80 1 Processor -- start 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924a96520 0x55e924c1d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924c13480 0x55e924c1db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e924c14180 con 0x55e924a96520 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e924ad0ba0 con 0x55e924c13480 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e924c29280 con 0x55e924b10d80 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe65e32bb80 10 client.0 _ll_register_callbacks cb 0x55e9249a7740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe64d6b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924a96520 0x55e924c1d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe64d6b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924a96520 0x55e924c1d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56884/0 (socket says 172.21.15.19:56884) 2024-05-09T01:35:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.129+0000 7fe64d6b1700 1 -- 172.21.15.19:0/3822409391 learned_addr learned my addr 172.21.15.19:0/3822409391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe64deb2700 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924c13480 0x55e924c1db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe64d6b1700 1 -- 172.21.15.19:0/3822409391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 msgr2=0x55e924c1d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe64d6b1700 1 -- 172.21.15.19:0/3822409391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924c13480 msgr2=0x55e924c1db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924c13480 0x55e924c1db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe64d6b1700 1 -- 172.21.15.19:0/3822409391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9249b1510 con 0x55e924a96520 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924a96520 0x55e924c1d100 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe6440a5480 tx=0x7fe6440a9890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f13df74c7299a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe630ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe630ff9700 1 -- 172.21.15.19:0/3822409391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440a9400 con 0x55e924a96520 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3822409391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6440ad7b0 con 0x55e924a96520 2024-05-09T01:35:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe630ff9700 1 -- 172.21.15.19:0/3822409391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe6440b0d40 con 0x55e924a96520 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe630ff9700 1 -- 172.21.15.19:0/3822409391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6440a8660 con 0x55e924a96520 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe65e32bb80 10 client.18156 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3822409391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e924c2abf0 con 0x55e924a96520 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.130+0000 7fe630ff9700 1 -- 172.21.15.19:0/3822409391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6440ad7b0 con 0x55e924a96520 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.131+0000 7fe630ff9700 1 -- 172.21.15.19:0/3822409391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe64400b040 con 0x55e924a96520 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.131+0000 7fe65e32bb80 10 client.18156 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.131+0000 7fe65e32bb80 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.131+0000 7fe65e32bb80 20 client.18156 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.131+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e924c2d0c0 0x55e924c2f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.131+0000 7fe65e32bb80 4 client.18156 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.131+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3822409391 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e924b43b00 con 0x55e924c2d0c0 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.131+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e924c2d0c0 0x55e924c2f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e924c2d0c0 0x55e924c2f4e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328261dad1c4116a server_cookie=1079275dc22acdb9 in_seq=0 out_seq=0 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe630ff9700 10 client.18156 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe630ff9700 1 -- 172.21.15.19:0/3822409391 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e924b43b00 con 0x55e924c2d0c0 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe630ff9700 10 client.18156 handle_command_reply: tid=0 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe65e32bb80 1 client.18156 shutdown 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3822409391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924a96520 msgr2=0x55e924c1d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924a96520 0x55e924c1d100 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe6440a5480 tx=0x7fe6440a9890 comp rx=0 tx=0).stop 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3822409391 shutdown_connections 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e924c2d0c0 0x55e924c2f4e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924c13480 0x55e924c1db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924a96520 0x55e924c1d100 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe630ff9700 0 client.18156 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/3822409391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.132+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3822409391 >> 172.21.15.19:0/3822409391 conn(0x55e924a99ff0 msgr2=0x55e924c08070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.133+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3822409391 shutdown_connections 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.133+0000 7fe65e32bb80 1 -- 172.21.15.19:0/3822409391 wait complete. 2024-05-09T01:35:28.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.133+0000 7fe65e32bb80 20 client.18156 trim_cache size 0 max 16384 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.153+0000 7fe65e32bb80 1 Processor -- start 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.153+0000 7fe65e32bb80 1 -- start start 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c1f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c291f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6481b0fd0 con 0x55e924c13480 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6481b2b10 con 0x55e924a96520 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6481b16a0 con 0x55e924b10d80 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64d6b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c1f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64deb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c291f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64d6b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c1f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60976/0 (socket says 172.21.15.19:60976) 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64deb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c291f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56900/0 (socket says 172.21.15.19:56900) 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64d6b1700 1 -- 172.21.15.19:0/2536306567 learned_addr learned my addr 172.21.15.19:0/2536306567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64d6b1700 1 -- 172.21.15.19:0/2536306567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 msgr2=0x55e924c1daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64d6b1700 1 -- 172.21.15.19:0/2536306567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 msgr2=0x55e924c291f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c291f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64d6b1700 1 -- 172.21.15.19:0/2536306567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e924c1ced0 con 0x55e924a96520 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64deb2700 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c291f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.154+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c1f5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe64400ddc0 tx=0x7fe64400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79bccfcd4eebff05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe632ffd700 1 -- 172.21.15.19:0/2536306567 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440a9400 con 0x55e924a96520 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe632ffd700 1 -- 172.21.15.19:0/2536306567 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6440ad7b0 con 0x55e924a96520 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe632ffd700 1 -- 172.21.15.19:0/2536306567 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440a8800 con 0x55e924a96520 2024-05-09T01:35:28.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2536306567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 msgr2=0x55e924c1f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c1f5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe64400ddc0 tx=0x7fe64400edc0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2536306567 shutdown_connections 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c291f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c1f5a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2536306567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c1daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.155+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2536306567 >> 172.21.15.19:0/2536306567 conn(0x55e924a99ff0 msgr2=0x55e924c12ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2536306567 shutdown_connections 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2536306567 wait complete. 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 -- start start 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 Processor -- start 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c30020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c30560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c30aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6481b2440 con 0x55e924c13480 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6481b1d70 con 0x55e924a96520 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.156+0000 7fe65e32bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6481aef70 con 0x55e924b10d80 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe65e32bb80 10 client.0 _ll_register_callbacks cb 0x55e9249a7740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64deb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c30aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64deb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c30aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56906/0 (socket says 172.21.15.19:56906) 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64deb2700 1 -- 172.21.15.19:0/2880047453 learned_addr learned my addr 172.21.15.19:0/2880047453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c30020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c30560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64deb2700 1 -- 172.21.15.19:0/2880047453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 msgr2=0x55e924c30560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64deb2700 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c30560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64deb2700 1 -- 172.21.15.19:0/2880047453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 msgr2=0x55e924c30020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64deb2700 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c30020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64deb2700 1 -- 172.21.15.19:0/2880047453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e924c96610 con 0x55e924c13480 2024-05-09T01:35:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64ceb0700 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c30560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c30020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe64deb2700 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c30aa0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe640015000 tx=0x7fe640004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff8bc0fa6240614 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe630ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe630ff9700 1 -- 172.21.15.19:0/2880047453 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640029020 con 0x55e924c13480 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe630ff9700 1 -- 172.21.15.19:0/2880047453 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe6400077d0 con 0x55e924c13480 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.157+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2880047453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6440ad7b0 con 0x55e924c13480 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe630ff9700 1 -- 172.21.15.19:0/2880047453 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6400079d0 con 0x55e924c13480 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe65e32bb80 10 client.18162 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2880047453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e924c97770 con 0x55e924c13480 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe630ff9700 1 -- 172.21.15.19:0/2880047453 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6440ad7b0 con 0x55e924c13480 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe630ff9700 1 -- 172.21.15.19:0/2880047453 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe640024070 con 0x55e924c13480 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe65e32bb80 10 client.18162 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe65e32bb80 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe65e32bb80 20 client.18162 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e924c97ed0 0x55e924c9a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe65e32bb80 4 client.18162 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2880047453 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e924c07dc0 con 0x55e924c97ed0 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.158+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e924c97ed0 0x55e924c9a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.159+0000 7fe630ff9700 10 client.18162 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.159+0000 7fe64d6b1700 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e924c97ed0 0x55e924c9a2f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f658337a57da5fbc server_cookie=3e25de14d797b41e in_seq=0 out_seq=0 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe630ff9700 1 -- 172.21.15.19:0/2880047453 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e924c07dc0 con 0x55e924c97ed0 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe630ff9700 10 client.18162 handle_command_reply: tid=0 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 client.18162 shutdown 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2880047453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 msgr2=0x55e924c30aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c30aa0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe640015000 tx=0x7fe640004d30 comp rx=0 tx=0).stop 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2880047453 shutdown_connections 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e924c97ed0 0x55e924c9a2f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e924c13480 0x55e924c30aa0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e924a96520 0x55e924c30020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe630ff9700 0 client.18162 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:28.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 --2- 172.21.15.19:0/2880047453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e924b10d80 0x55e924c30560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2880047453 >> 172.21.15.19:0/2880047453 conn(0x55e924a99ff0 msgr2=0x55e924c1b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:28.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2880047453 shutdown_connections 2024-05-09T01:35:28.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.160+0000 7fe65e32bb80 1 -- 172.21.15.19:0/2880047453 wait complete. 2024-05-09T01:35:28.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe65e32bb80 20 client.18162 trim_cache size 0 max 16384 2024-05-09T01:35:28.164 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:35:28.164 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:35:28.164 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "6824b8e2-ae9a-4590-8674-da1c4e5e8614", 2024-05-09T01:35:28.165 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:35:28.165 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:35:28.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 -- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6340b2880 msgr2=0x7fe6340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:28.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6340b2880 0x7fe6340b4d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe648106340 tx=0x7fe63c009190 comp rx=0 tx=0).stop 2024-05-09T01:35:28.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 -- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6481029c0 msgr2=0x7fe64819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:28.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6481029c0 0x7fe64819ae20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe64400ddf0 tx=0x7fe64400a530 comp rx=0 tx=0).stop 2024-05-09T01:35:28.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 -- 172.21.15.19:0/2816269441 shutdown_connections 2024-05-09T01:35:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6481089f0 0x7fe64819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6340b2880 0x7fe6340b4d90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6481029c0 0x7fe64819ae20 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 --2- 172.21.15.19:0/2816269441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe648103310 0x7fe64819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 -- 172.21.15.19:0/2816269441 >> 172.21.15.19:0/2816269441 conn(0x7fe6480fe520 msgr2=0x7fe648106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 -- 172.21.15.19:0/2816269441 shutdown_connections 2024-05-09T01:35:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:28.161+0000 7fe64f915700 1 -- 172.21.15.19:0/2816269441 wait complete. 2024-05-09T01:35:28.891 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:28 smithi019 ceph-mon[30158]: pgmap v676: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:28.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:35:28.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:28 smithi196 ceph-mon[35041]: pgmap v676: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:29.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:28 smithi059 ceph-mon[34351]: pgmap v676: 97 pgs: 97 active+clean; 2.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:29.188 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:35:29.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:29 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:35:29.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:29 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:29.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:29 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:35:29.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:29 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:29.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:29 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:35:29.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:29 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:29.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:29 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:35:29.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:29 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:30.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:29 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:35:30.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:29 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:30.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:29 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:35:30.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:29 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:35:30.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:30 smithi019 ceph-mon[30158]: pgmap v677: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:35:30.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:30 smithi196 ceph-mon[35041]: pgmap v677: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:35:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.978+0000 7ff023897700 1 -- 172.21.15.19:0/1001625853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c109870 msgr2=0x7ff01c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.978+0000 7ff023897700 1 --2- 172.21.15.19:0/1001625853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c109870 0x7ff01c1111d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ff01400b0d0 tx=0x7ff01400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.979+0000 7ff023897700 1 -- 172.21.15.19:0/1001625853 shutdown_connections 2024-05-09T01:35:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.979+0000 7ff023897700 1 --2- 172.21.15.19:0/1001625853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c109870 0x7ff01c1111d0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.979+0000 7ff023897700 1 --2- 172.21.15.19:0/1001625853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff01c1045f0 0x7ff01c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.979+0000 7ff023897700 1 --2- 172.21.15.19:0/1001625853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff01c108f20 0x7ff01c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.979+0000 7ff023897700 1 -- 172.21.15.19:0/1001625853 >> 172.21.15.19:0/1001625853 conn(0x7ff01c0fffe0 msgr2=0x7ff01c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.979+0000 7ff023897700 1 -- 172.21.15.19:0/1001625853 shutdown_connections 2024-05-09T01:35:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.979+0000 7ff023897700 1 -- 172.21.15.19:0/1001625853 wait complete. 2024-05-09T01:35:30.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.979+0000 7ff023897700 1 Processor -- start 2024-05-09T01:35:30.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.979+0000 7ff023897700 1 -- start start 2024-05-09T01:35:30.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff023897700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c1045f0 0x7ff01c0775c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff023897700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff01c108f20 0x7ff01c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff023897700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff01c109870 0x7ff01c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff023897700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1144f0 con 0x7ff01c1045f0 2024-05-09T01:35:30.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff023897700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c114370 con 0x7ff01c108f20 2024-05-09T01:35:30.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff023897700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c114670 con 0x7ff01c109870 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021633700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c1045f0 0x7ff01c0775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021633700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c1045f0 0x7ff01c0775c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49570/0 (socket says 172.21.15.19:49570) 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021633700 1 -- 172.21.15.19:0/184045227 learned_addr learned my addr 172.21.15.19:0/184045227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff020e32700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff01c108f20 0x7ff01c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021e34700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff01c109870 0x7ff01c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021633700 1 -- 172.21.15.19:0/184045227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff01c109870 msgr2=0x7ff01c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021633700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff01c109870 0x7ff01c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021633700 1 -- 172.21.15.19:0/184045227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff01c108f20 msgr2=0x7ff01c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021633700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff01c108f20 0x7ff01c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021633700 1 -- 172.21.15.19:0/184045227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c076830 con 0x7ff01c1045f0 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021e34700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff01c109870 0x7ff01c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.980+0000 7ff021633700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c1045f0 0x7ff01c0775c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff01800d950 tx=0x7ff01800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.981+0000 7ff0127fc700 1 -- 172.21.15.19:0/184045227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018016070 con 0x7ff01c1045f0 2024-05-09T01:35:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.981+0000 7ff0127fc700 1 -- 172.21.15.19:0/184045227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff018009df0 con 0x7ff01c1045f0 2024-05-09T01:35:30.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.981+0000 7ff0127fc700 1 -- 172.21.15.19:0/184045227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01800f3c0 con 0x7ff01c1045f0 2024-05-09T01:35:30.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.981+0000 7ff023897700 1 -- 172.21.15.19:0/184045227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c1ad890 con 0x7ff01c1045f0 2024-05-09T01:35:30.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.981+0000 7ff023897700 1 -- 172.21.15.19:0/184045227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c1ade20 con 0x7ff01c1045f0 2024-05-09T01:35:30.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.984+0000 7ff0127fc700 1 -- 172.21.15.19:0/184045227 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff018019070 con 0x7ff01c1045f0 2024-05-09T01:35:30.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0127fc700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0080b2900 0x7ff0080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0322adb80 1 Processor -- start 2024-05-09T01:35:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff020e32700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0080b2900 0x7ff0080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0127fc700 1 -- 172.21.15.19:0/184045227 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff0180e1dd0 con 0x7ff01c1045f0 2024-05-09T01:35:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0322adb80 1 -- start start 2024-05-09T01:35:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 0x55709c0f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c0ef810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff020e32700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0080b2900 0x7ff0080b4e10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff01c108960 tx=0x7ff00c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0322adb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55709c0120a0 con 0x55709c0ef400 2024-05-09T01:35:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0322adb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55709bfb79a0 con 0x55709bf724a0 2024-05-09T01:35:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.985+0000 7ff0322adb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55709c00e060 con 0x55709bfecd00 2024-05-09T01:35:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff021633700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff020e32700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 0x55709c0f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff021633700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:57028/0 (socket says 172.21.15.19:57028) 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff021633700 1 -- 172.21.15.19:0/1330343783 learned_addr learned my addr 172.21.15.19:0/1330343783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff021e34700 1 --2- 172.21.15.19:0/1330343783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c0ef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff020e32700 1 -- 172.21.15.19:0/1330343783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 msgr2=0x55709c0f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff020e32700 1 --2- 172.21.15.19:0/1330343783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff020e32700 1 -- 172.21.15.19:0/1330343783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 msgr2=0x55709c0ef810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff020e32700 1 --2- 172.21.15.19:0/1330343783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c0ef810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff020e32700 1 -- 172.21.15.19:0/1330343783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55709bfa8500 con 0x55709bf724a0 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff020e32700 1 --2- 172.21.15.19:0/1330343783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 0x55709c0f5780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff01c06b540 tx=0x7ff00c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d20ff68ee674f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff0077fe700 1 -- 172.21.15.19:0/1330343783 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c011b90 con 0x55709bf724a0 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff0077fe700 1 -- 172.21.15.19:0/1330343783 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff00c011d30 con 0x55709bf724a0 2024-05-09T01:35:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.986+0000 7ff0077fe700 1 -- 172.21.15.19:0/1330343783 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c01c5f0 con 0x55709bf724a0 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.987+0000 7ff0322adb80 1 -- 172.21.15.19:0/1330343783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 msgr2=0x55709c0f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.987+0000 7ff0322adb80 1 --2- 172.21.15.19:0/1330343783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 0x55709c0f5780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff01c06b540 tx=0x7ff00c013000 comp rx=0 tx=0).stop 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.987+0000 7ff0322adb80 1 -- 172.21.15.19:0/1330343783 shutdown_connections 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.987+0000 7ff0322adb80 1 --2- 172.21.15.19:0/1330343783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c0ef810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.987+0000 7ff0322adb80 1 --2- 172.21.15.19:0/1330343783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.987+0000 7ff0322adb80 1 --2- 172.21.15.19:0/1330343783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 0x55709c0f5780 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.987+0000 7ff0322adb80 1 -- 172.21.15.19:0/1330343783 >> 172.21.15.19:0/1330343783 conn(0x55709bf75f70 msgr2=0x55709c0ea220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.987+0000 7ff0322adb80 1 -- 172.21.15.19:0/1330343783 shutdown_connections 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.988+0000 7ff0322adb80 1 -- 172.21.15.19:0/1330343783 wait complete. 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.988+0000 7ff0322adb80 1 -- start start 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.988+0000 7ff0322adb80 1 Processor -- start 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 0x55709c0f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bfecd00 0x55709c0f9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709c0ef400 0x55709c0f9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff0322adb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55709c0f0160 con 0x55709bfecd00 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff0322adb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55709bfacb20 con 0x55709bf724a0 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff0322adb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55709c0fc020 con 0x55709c0ef400 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff0322adb80 10 client.0 _ll_register_callbacks cb 0x55709be836c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff020e32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bfecd00 0x55709c0f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff020e32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bfecd00 0x55709c0f9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49578/0 (socket says 172.21.15.19:49578) 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff020e32700 1 -- 172.21.15.19:0/3475966442 learned_addr learned my addr 172.21.15.19:0/3475966442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff021633700 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 0x55709c0f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff021e34700 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709c0ef400 0x55709c0f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff020e32700 1 -- 172.21.15.19:0/3475966442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709c0ef400 msgr2=0x55709c0f9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff020e32700 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709c0ef400 0x55709c0f9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff020e32700 1 -- 172.21.15.19:0/3475966442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 msgr2=0x55709c0f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff020e32700 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 0x55709c0f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff020e32700 1 -- 172.21.15.19:0/3475966442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55709be8d490 con 0x55709bfecd00 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff020e32700 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bfecd00 0x55709c0f9440 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0059d0 tx=0x7ff00c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f98a44a1e33ba3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.989+0000 7ff004ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff004ff9700 1 -- 172.21.15.19:0/3475966442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c036060 con 0x55709bfecd00 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff0322adb80 1 -- 172.21.15.19:0/3475966442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff00c011d30 con 0x55709bfecd00 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff004ff9700 1 -- 172.21.15.19:0/3475966442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff00c005390 con 0x55709bfecd00 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff004ff9700 1 -- 172.21.15.19:0/3475966442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff00c005590 con 0x55709bfecd00 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff0322adb80 10 client.18174 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff0322adb80 1 -- 172.21.15.19:0/3475966442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55709c106910 con 0x55709bfecd00 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff004ff9700 1 -- 172.21.15.19:0/3475966442 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff00c011d30 con 0x55709bfecd00 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff004ff9700 1 -- 172.21.15.19:0/3475966442 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff00c01e750 con 0x55709bfecd00 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff0322adb80 10 client.18174 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff0322adb80 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff0322adb80 20 client.18174 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:30.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff0322adb80 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55709c108e40 0x55709c10b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:30.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff0322adb80 4 client.18174 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:30.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.990+0000 7ff0322adb80 1 -- 172.21.15.19:0/3475966442 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55709c01fa80 con 0x55709c108e40 2024-05-09T01:35:30.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.991+0000 7ff021633700 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55709c108e40 0x55709c10b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:30.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.991+0000 7ff021633700 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55709c108e40 0x55709c10b260 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552147a9234b8fc5 server_cookie=8fff9d41ed5e9a54 in_seq=0 out_seq=0 2024-05-09T01:35:30.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.991+0000 7ff004ff9700 10 client.18174 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff004ff9700 1 -- 172.21.15.19:0/3475966442 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55709c01fa80 con 0x55709c108e40 2024-05-09T01:35:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff004ff9700 10 client.18174 handle_command_reply: tid=0 2024-05-09T01:35:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 client.18174 shutdown 2024-05-09T01:35:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 -- 172.21.15.19:0/3475966442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bfecd00 msgr2=0x55709c0f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bfecd00 0x55709c0f9440 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0059d0 tx=0x7ff00c011c10 comp rx=0 tx=0).stop 2024-05-09T01:35:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 -- 172.21.15.19:0/3475966442 shutdown_connections 2024-05-09T01:35:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55709c108e40 0x55709c10b260 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709c0ef400 0x55709c0f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bf724a0 0x55709c0f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 --2- 172.21.15.19:0/3475966442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bfecd00 0x55709c0f9440 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff004ff9700 0 client.18174 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 -- 172.21.15.19:0/3475966442 >> 172.21.15.19:0/3475966442 conn(0x55709bf75f70 msgr2=0x55709c0e3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 -- 172.21.15.19:0/3475966442 shutdown_connections 2024-05-09T01:35:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 1 -- 172.21.15.19:0/3475966442 wait complete. 2024-05-09T01:35:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:30.992+0000 7ff0322adb80 20 client.18174 trim_cache size 0 max 16384 2024-05-09T01:35:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.013+0000 7ff0322adb80 1 Processor -- start 2024-05-09T01:35:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.013+0000 7ff0322adb80 1 -- start start 2024-05-09T01:35:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.013+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bf724a0 0x55709c1051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.013+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.013+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709c0ef400 0x55709c0f3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.013+0000 7ff0322adb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b53e0 con 0x55709bf724a0 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.013+0000 7ff0322adb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b6f20 con 0x55709c0ef400 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.013+0000 7ff0322adb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b5ab0 con 0x55709bfecd00 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021633700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bf724a0 0x55709c1051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021633700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bf724a0 0x55709c1051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49592/0 (socket says 172.21.15.19:49592) 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021633700 1 -- 172.21.15.19:0/93298341 learned_addr learned my addr 172.21.15.19:0/93298341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021e34700 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709c0ef400 0x55709c0f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff020e32700 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021633700 1 -- 172.21.15.19:0/93298341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 msgr2=0x55709c0f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021633700 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021633700 1 -- 172.21.15.19:0/93298341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709c0ef400 msgr2=0x55709c0f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021633700 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709c0ef400 0x55709c0f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021633700 1 -- 172.21.15.19:0/93298341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55709c0e69d0 con 0x55709bf724a0 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff020e32700 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021e34700 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709c0ef400 0x55709c0f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff021633700 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bf724a0 0x55709c1051c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff0180df900 tx=0x7ff018091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dad725435a55e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff006ffd700 1 -- 172.21.15.19:0/93298341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0180c3070 con 0x55709bf724a0 2024-05-09T01:35:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff006ffd700 1 -- 172.21.15.19:0/93298341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0180df9c0 con 0x55709bf724a0 2024-05-09T01:35:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.014+0000 7ff006ffd700 1 -- 172.21.15.19:0/93298341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018020040 con 0x55709bf724a0 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.015+0000 7ff0322adb80 1 -- 172.21.15.19:0/93298341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bf724a0 msgr2=0x55709c1051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.015+0000 7ff0322adb80 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bf724a0 0x55709c1051c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff0180df900 tx=0x7ff018091040 comp rx=0 tx=0).stop 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.015+0000 7ff0322adb80 1 -- 172.21.15.19:0/93298341 shutdown_connections 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.015+0000 7ff0322adb80 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709c0ef400 0x55709c0f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.015+0000 7ff0322adb80 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709bf724a0 0x55709c1051c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.015+0000 7ff0322adb80 1 --2- 172.21.15.19:0/93298341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bfecd00 0x55709c0f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.015+0000 7ff0322adb80 1 -- 172.21.15.19:0/93298341 >> 172.21.15.19:0/93298341 conn(0x55709bf75f70 msgr2=0x55709c0f2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.015+0000 7ff0322adb80 1 -- 172.21.15.19:0/93298341 shutdown_connections 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.015+0000 7ff0322adb80 1 -- 172.21.15.19:0/93298341 wait complete. 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff0322adb80 1 -- start start 2024-05-09T01:35:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff0322adb80 1 Processor -- start 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bf724a0 0x55709c10c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bfecd00 0x55709c10c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff0322adb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c10cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff0322adb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b6850 con 0x55709c0ef400 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff0322adb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b6180 con 0x55709bfecd00 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff0322adb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1b3380 con 0x55709bf724a0 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff0322adb80 10 client.0 _ll_register_callbacks cb 0x55709be836c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff021e34700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c10cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff021e34700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c10cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49600/0 (socket says 172.21.15.19:49600) 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff021e34700 1 -- 172.21.15.19:0/978208572 learned_addr learned my addr 172.21.15.19:0/978208572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.016+0000 7ff021633700 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bf724a0 0x55709c10c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff020e32700 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bfecd00 0x55709c10c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff021e34700 1 -- 172.21.15.19:0/978208572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bf724a0 msgr2=0x55709c10c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff021e34700 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bf724a0 0x55709c10c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff021e34700 1 -- 172.21.15.19:0/978208572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bfecd00 msgr2=0x55709c10c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff021e34700 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bfecd00 0x55709c10c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff021e34700 1 -- 172.21.15.19:0/978208572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55709c172550 con 0x55709c0ef400 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff021e34700 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c10cb10 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff01400b0d0 tx=0x7ff014004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd394735a23d95c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff004ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff004ff9700 1 -- 172.21.15.19:0/978208572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01402a060 con 0x55709c0ef400 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff004ff9700 1 -- 172.21.15.19:0/978208572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff0140077d0 con 0x55709c0ef400 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff0322adb80 1 -- 172.21.15.19:0/978208572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0180df9c0 con 0x55709c0ef400 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff004ff9700 1 -- 172.21.15.19:0/978208572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0140079d0 con 0x55709c0ef400 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff0322adb80 10 client.18180 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff0322adb80 1 -- 172.21.15.19:0/978208572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55709c1737c0 con 0x55709c0ef400 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.017+0000 7ff004ff9700 1 -- 172.21.15.19:0/978208572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0180df9c0 con 0x55709c0ef400 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.018+0000 7ff004ff9700 1 -- 172.21.15.19:0/978208572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff014024070 con 0x55709c0ef400 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.018+0000 7ff0322adb80 10 client.18180 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.018+0000 7ff0322adb80 10 client.18180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.018+0000 7ff0322adb80 20 client.18180 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.018+0000 7ff0322adb80 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55709c173ed0 0x55709c1762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.018+0000 7ff0322adb80 4 client.18180 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.018+0000 7ff0322adb80 1 -- 172.21.15.19:0/978208572 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55709c019b20 con 0x55709c173ed0 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.018+0000 7ff021633700 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55709c173ed0 0x55709c1762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff021633700 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55709c173ed0 0x55709c1762f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e4a9165ea0b666 server_cookie=b6d3d9bf41acb895 in_seq=0 out_seq=0 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff004ff9700 10 client.18180 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff004ff9700 1 -- 172.21.15.19:0/978208572 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55709c019b20 con 0x55709c173ed0 2024-05-09T01:35:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff004ff9700 10 client.18180 handle_command_reply: tid=0 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 client.18180 shutdown 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 -- 172.21.15.19:0/978208572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 msgr2=0x55709c10cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c10cb10 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff01400b0d0 tx=0x7ff014004cf0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 -- 172.21.15.19:0/978208572 shutdown_connections 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55709c173ed0 0x55709c1762f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55709c0ef400 0x55709c10cb10 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709bf724a0 0x55709c10c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 --2- 172.21.15.19:0/978208572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55709bfecd00 0x55709c10c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 -- 172.21.15.19:0/978208572 >> 172.21.15.19:0/978208572 conn(0x55709bf75f70 msgr2=0x55709c0fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff004ff9700 0 client.18180 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 -- 172.21.15.19:0/978208572 shutdown_connections 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 1 -- 172.21.15.19:0/978208572 wait complete. 2024-05-09T01:35:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.019+0000 7ff0322adb80 20 client.18180 trim_cache size 0 max 16384 2024-05-09T01:35:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:35:31.022 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T01:35:31.022 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T01:35:31.022 INFO:teuthology.orchestra.run.smithi019.stdout: "6824b8e2-ae9a-4590-8674-da1c4e5e8614": { 2024-05-09T01:35:31.022 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T01:35:31.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "6824b8e2-ae9a-4590-8674-da1c4e5e8614", 2024-05-09T01:35:31.022 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T01:35:31.022 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:35:31.023 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:35:31.023 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:35:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 -- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0080b2900 msgr2=0x7ff0080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0080b2900 0x7ff0080b4e10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff01c108960 tx=0x7ff00c00b410 comp rx=0 tx=0).stop 2024-05-09T01:35:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 -- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c1045f0 msgr2=0x7ff01c0775c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c1045f0 0x7ff01c0775c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff01800d950 tx=0x7ff01800dc60 comp rx=0 tx=0).stop 2024-05-09T01:35:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 -- 172.21.15.19:0/184045227 shutdown_connections 2024-05-09T01:35:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff01c109870 0x7ff01c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff0080b2900 0x7ff0080b4e10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff01c1045f0 0x7ff01c0775c0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 --2- 172.21.15.19:0/184045227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff01c108f20 0x7ff01c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 -- 172.21.15.19:0/184045227 >> 172.21.15.19:0/184045227 conn(0x7ff01c0fffe0 msgr2=0x7ff01c106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 -- 172.21.15.19:0/184045227 shutdown_connections 2024-05-09T01:35:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:35:31.020+0000 7ff023897700 1 -- 172.21.15.19:0/184045227 wait complete. 2024-05-09T01:35:31.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:30 smithi059 ceph-mon[34351]: pgmap v677: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:35:31.850 INFO:tasks.cephfs.filesystem:scrub status for tag:6824b8e2-ae9a-4590-8674-da1c4e5e8614 - {'path': '/', 'tag': '6824b8e2-ae9a-4590-8674-da1c4e5e8614', 'options': 'recursive,force'} 2024-05-09T01:35:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:32 smithi019 ceph-mon[30158]: pgmap v678: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:35:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:32 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:35:32.898 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:32 smithi059 ceph-mon[34351]: pgmap v678: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:35:32.898 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:32 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:35:32.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:32 smithi196 ceph-mon[35041]: pgmap v678: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:35:32.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:32 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:35:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.544+0000 7f4a56ffd640 1 -- 172.21.15.19:0/2561074495 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480318a0 con 0x7f4a58103c60 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.545+0000 7f4a5e70d640 1 -- 172.21.15.19:0/2561074495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58103c60 msgr2=0x7f4a58104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.545+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/2561074495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58103c60 0x7f4a58104960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009900 tx=0x7f4a4802a410 comp rx=0 tx=0).stop 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.545+0000 7f4a5e70d640 1 -- 172.21.15.19:0/2561074495 shutdown_connections 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.545+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/2561074495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a58109ba0 0x7f4a5810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.545+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/2561074495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58104ea0 0x7f4a581052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.545+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/2561074495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58103c60 0x7f4a58104960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.545+0000 7f4a5e70d640 1 -- 172.21.15.19:0/2561074495 >> 172.21.15.19:0/2561074495 conn(0x7f4a580fee70 msgr2=0x7f4a58101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.545+0000 7f4a5e70d640 1 -- 172.21.15.19:0/2561074495 shutdown_connections 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.545+0000 7f4a5e70d640 1 -- 172.21.15.19:0/2561074495 wait complete. 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.546+0000 7f4a5e70d640 1 Processor -- start 2024-05-09T01:35:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.546+0000 7f4a5e70d640 1 -- start start 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.546+0000 7f4a5e70d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58103c60 0x7f4a5819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.546+0000 7f4a5e70d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a58104ea0 0x7f4a5819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.546+0000 7f4a5e70d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109ba0 0x7f4a581a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.546+0000 7f4a5e70d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5810f760 con 0x7f4a58103c60 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.546+0000 7f4a5e70d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5810f5e0 con 0x7f4a58109ba0 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.546+0000 7f4a5e70d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5810f8e0 con 0x7f4a58104ea0 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a57fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58103c60 0x7f4a5819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a57fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58103c60 0x7f4a5819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49610/0 (socket says 172.21.15.19:49610) 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a57fff640 1 -- 172.21.15.19:0/3491544669 learned_addr learned my addr 172.21.15.19:0/3491544669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109ba0 0x7f4a581a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a577fe640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a58104ea0 0x7f4a5819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a57fff640 1 -- 172.21.15.19:0/3491544669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a58104ea0 msgr2=0x7f4a5819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a57fff640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a58104ea0 0x7f4a5819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a57fff640 1 -- 172.21.15.19:0/3491544669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109ba0 msgr2=0x7f4a581a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a57fff640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109ba0 0x7f4a581a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a57fff640 1 -- 172.21.15.19:0/3491544669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a581a1ea0 con 0x7f4a58103c60 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109ba0 0x7f4a581a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a57fff640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58103c60 0x7f4a5819a360 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006010 tx=0x7f4a48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.547+0000 7f4a557fa640 1 -- 172.21.15.19:0/3491544669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4803f030 con 0x7f4a58103c60 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.548+0000 7f4a5e70d640 1 -- 172.21.15.19:0/3491544669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a581a2130 con 0x7f4a58103c60 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.548+0000 7f4a557fa640 1 -- 172.21.15.19:0/3491544669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a48031910 con 0x7f4a58103c60 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.548+0000 7f4a5e70d640 1 -- 172.21.15.19:0/3491544669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a2670 con 0x7f4a58103c60 2024-05-09T01:35:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.548+0000 7f4a557fa640 1 -- 172.21.15.19:0/3491544669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48035070 con 0x7f4a58103c60 2024-05-09T01:35:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.550+0000 7f4a655c1740 1 Processor -- start 2024-05-09T01:35:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a655c1740 1 -- start start 2024-05-09T01:35:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a557fa640 1 -- 172.21.15.19:0/3491544669 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4a48042030 con 0x7f4a58103c60 2024-05-09T01:35:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8d01b0 0x55cccfc78910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8a72f0 0x55cccfc78e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8af4d0 0x55cccfc7fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a655c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cccfbd87c0 con 0x55cccf8af4d0 2024-05-09T01:35:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a655c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cccfb740e0 con 0x55cccf8d01b0 2024-05-09T01:35:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a655c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cccfbd4780 con 0x55cccf8a72f0 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8af4d0 0x55cccfc7fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8af4d0 0x55cccfc7fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49626/0 (socket says 172.21.15.19:49626) 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a577fe640 1 -- 172.21.15.19:0/1473916136 learned_addr learned my addr 172.21.15.19:0/1473916136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a57fff640 1 --2- 172.21.15.19:0/1473916136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8a72f0 0x55cccfc78e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/1473916136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8d01b0 0x55cccfc78910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a577fe640 1 -- 172.21.15.19:0/1473916136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8a72f0 msgr2=0x55cccfc78e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a577fe640 1 --2- 172.21.15.19:0/1473916136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8a72f0 0x55cccfc78e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a577fe640 1 -- 172.21.15.19:0/1473916136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8d01b0 msgr2=0x55cccfc78910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a577fe640 1 --2- 172.21.15.19:0/1473916136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8d01b0 0x55cccfc78910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.551+0000 7f4a577fe640 1 -- 172.21.15.19:0/1473916136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cccfb9d7c0 con 0x55cccf8af4d0 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.552+0000 7f4a577fe640 1 --2- 172.21.15.19:0/1473916136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8af4d0 0x55cccfc7fd90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009c90 tx=0x7f4a4c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bbbcf759fab1c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.552+0000 7f4a557fa640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a400b27e0 0x7f4a400b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.552+0000 7f4a3e7fc640 1 -- 172.21.15.19:0/1473916136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c005220 con 0x55cccf8af4d0 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.552+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a400b27e0 0x7f4a400b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.552+0000 7f4a3e7fc640 1 -- 172.21.15.19:0/1473916136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4c004c50 con 0x55cccf8af4d0 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.552+0000 7f4a3e7fc640 1 -- 172.21.15.19:0/1473916136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c019690 con 0x55cccf8af4d0 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.552+0000 7f4a557fa640 1 -- 172.21.15.19:0/3491544669 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4a48102570 con 0x7f4a58103c60 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 -- 172.21.15.19:0/1473916136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8af4d0 msgr2=0x55cccfc7fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a400b27e0 0x7f4a400b4cf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55cccfc79ae0 tx=0x7f4a50009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 --2- 172.21.15.19:0/1473916136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8af4d0 0x55cccfc7fd90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009c90 tx=0x7f4a4c011810 comp rx=0 tx=0).stop 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 -- 172.21.15.19:0/1473916136 shutdown_connections 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 --2- 172.21.15.19:0/1473916136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8af4d0 0x55cccfc7fd90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 --2- 172.21.15.19:0/1473916136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8a72f0 0x55cccfc78e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 --2- 172.21.15.19:0/1473916136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8d01b0 0x55cccfc78910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 -- 172.21.15.19:0/1473916136 >> 172.21.15.19:0/1473916136 conn(0x55cccf8c4300 msgr2=0x55cccfc6fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 -- 172.21.15.19:0/1473916136 shutdown_connections 2024-05-09T01:35:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 -- 172.21.15.19:0/1473916136 wait complete. 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.553+0000 7f4a655c1740 1 -- start start 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a655c1740 1 Processor -- start 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8a72f0 0x55cccfc82610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc82b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8d01b0 0x55cccfc89ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a655c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cccfc807e0 con 0x55cccf8d01b0 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a655c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cccfb5f9c0 con 0x55cccf8a72f0 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a655c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cccfc80960 con 0x55cccf8af4d0 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a655c1740 10 client.0 _ll_register_callbacks cb 0x55cccfb8c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a5ccaa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8d01b0 0x55cccfc89ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a5ccaa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8d01b0 0x55cccfc89ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49636/0 (socket says 172.21.15.19:49636) 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a5ccaa640 1 -- 172.21.15.19:0/3358497885 learned_addr learned my addr 172.21.15.19:0/3358497885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a57fff640 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8a72f0 0x55cccfc82610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a577fe640 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc82b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a5ccaa640 1 -- 172.21.15.19:0/3358497885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 msgr2=0x55cccfc82b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc82b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a5ccaa640 1 -- 172.21.15.19:0/3358497885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8a72f0 msgr2=0x55cccfc82610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8a72f0 0x55cccfc82610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a5ccaa640 1 -- 172.21.15.19:0/3358497885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cccfb861f0 con 0x55cccf8d01b0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8d01b0 0x55cccfc89ab0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f4a50003f50 tx=0x7f4a50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9451a81ca8e32b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.554+0000 7f4a377fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a377fe640 1 -- 172.21.15.19:0/3358497885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000fe60 con 0x55cccf8d01b0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a377fe640 1 -- 172.21.15.19:0/3358497885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4a50004750 con 0x55cccf8d01b0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a655c1740 1 -- 172.21.15.19:0/3358497885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4c004c50 con 0x55cccf8d01b0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a377fe640 1 -- 172.21.15.19:0/3358497885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a50029440 con 0x55cccf8d01b0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a655c1740 10 client.18192 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a655c1740 1 -- 172.21.15.19:0/3358497885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cccf93b790 con 0x55cccf8d01b0 2024-05-09T01:35:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a377fe640 1 -- 172.21.15.19:0/3358497885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4c004c50 con 0x55cccf8d01b0 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a377fe640 1 -- 172.21.15.19:0/3358497885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4a50027070 con 0x55cccf8d01b0 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a655c1740 10 client.18192 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a655c1740 10 client.18192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a655c1740 20 client.18192 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a655c1740 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55cccf88ed80 0x55cccfc900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a655c1740 4 client.18192 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.555+0000 7f4a655c1740 1 -- 172.21.15.19:0/3358497885 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cccfbe63d0 con 0x55cccf88ed80 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.556+0000 7f4a57fff640 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55cccf88ed80 0x55cccfc900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.556+0000 7f4a57fff640 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55cccf88ed80 0x55cccfc900b0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a2e5789483106c1 server_cookie=61264ad2ad5e8e3 in_seq=0 out_seq=0 2024-05-09T01:35:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.556+0000 7f4a377fe640 10 client.18192 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a377fe640 1 -- 172.21.15.19:0/3358497885 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cccfbe63d0 con 0x55cccf88ed80 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a377fe640 10 client.18192 handle_command_reply: tid=0 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 client.18192 shutdown 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 -- 172.21.15.19:0/3358497885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8d01b0 msgr2=0x55cccfc89ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8d01b0 0x55cccfc89ab0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f4a50003f50 tx=0x7f4a50004ab0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 -- 172.21.15.19:0/3358497885 shutdown_connections 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55cccf88ed80 0x55cccfc900b0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8d01b0 0x55cccfc89ab0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc82b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a377fe640 0 client.18192 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 --2- 172.21.15.19:0/3358497885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf8a72f0 0x55cccfc82610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 -- 172.21.15.19:0/3358497885 >> 172.21.15.19:0/3358497885 conn(0x55cccf8c4300 msgr2=0x55cccfc6e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 -- 172.21.15.19:0/3358497885 shutdown_connections 2024-05-09T01:35:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.557+0000 7f4a655c1740 1 -- 172.21.15.19:0/3358497885 wait complete. 2024-05-09T01:35:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.558+0000 7f4a655c1740 20 client.18192 trim_cache size 0 max 16384 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a655c1740 1 Processor -- start 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a655c1740 1 -- start start 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 0x55cccfc80d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc7ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a655c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581a92f0 con 0x55cccf8a72f0 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a655c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581aae30 con 0x55cccf88ed80 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a655c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581a99c0 con 0x55cccf8af4d0 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc81270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc81270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49646/0 (socket says 172.21.15.19:49646) 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a577fe640 1 -- 172.21.15.19:0/424182566 learned_addr learned my addr 172.21.15.19:0/424182566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a57fff640 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 0x55cccfc80d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc7ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a577fe640 1 -- 172.21.15.19:0/424182566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 msgr2=0x55cccfc7ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a577fe640 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc7ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a577fe640 1 -- 172.21.15.19:0/424182566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 msgr2=0x55cccfc80d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a577fe640 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 0x55cccfc80d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.573+0000 7f4a577fe640 1 -- 172.21.15.19:0/424182566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cccf9b25f0 con 0x55cccf8a72f0 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.574+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc7ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.574+0000 7f4a577fe640 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc81270 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009be0 tx=0x7f4a4c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4522db36d277aaa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.574+0000 7f4a3dffb640 1 -- 172.21.15.19:0/424182566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c005340 con 0x55cccf8a72f0 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.574+0000 7f4a3dffb640 1 -- 172.21.15.19:0/424182566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4c0054e0 con 0x55cccf8a72f0 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.574+0000 7f4a3dffb640 1 -- 172.21.15.19:0/424182566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c0197f0 con 0x55cccf8a72f0 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 -- 172.21.15.19:0/424182566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 msgr2=0x55cccfc81270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc81270 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009be0 tx=0x7f4a4c0099e0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 -- 172.21.15.19:0/424182566 shutdown_connections 2024-05-09T01:35:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc7ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc81270 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 --2- 172.21.15.19:0/424182566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 0x55cccfc80d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 -- 172.21.15.19:0/424182566 >> 172.21.15.19:0/424182566 conn(0x55cccf8c4300 msgr2=0x55cccfc82610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 -- 172.21.15.19:0/424182566 shutdown_connections 2024-05-09T01:35:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 -- 172.21.15.19:0/424182566 wait complete. 2024-05-09T01:35:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 -- start start 2024-05-09T01:35:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.575+0000 7f4a655c1740 1 Processor -- start 2024-05-09T01:35:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 0x55cccfc86210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc86750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a655c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc87680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a655c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581aa760 con 0x55cccf8a72f0 2024-05-09T01:35:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a655c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581aa090 con 0x55cccf88ed80 2024-05-09T01:35:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a655c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581a72d0 con 0x55cccf8af4d0 2024-05-09T01:35:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a655c1740 10 client.0 _ll_register_callbacks cb 0x7f4a50012ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc86750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc86750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49662/0 (socket says 172.21.15.19:49662) 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a577fe640 1 -- 172.21.15.19:0/110208877 learned_addr learned my addr 172.21.15.19:0/110208877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a57fff640 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 0x55cccfc86210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc87680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a577fe640 1 -- 172.21.15.19:0/110208877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 msgr2=0x55cccfc87680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a577fe640 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc87680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a577fe640 1 -- 172.21.15.19:0/110208877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 msgr2=0x55cccfc86210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a577fe640 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 0x55cccfc86210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a577fe640 1 -- 172.21.15.19:0/110208877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cccfb9ced0 con 0x55cccf8a72f0 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a5ccaa640 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc87680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a577fe640 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc86750 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c01b040 tx=0x7f4a4c004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f32de650c84257 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a377fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.576+0000 7f4a377fe640 1 -- 172.21.15.19:0/110208877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c005c50 con 0x55cccf8a72f0 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a377fe640 1 -- 172.21.15.19:0/110208877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4a4c019530 con 0x55cccf8a72f0 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a655c1740 1 -- 172.21.15.19:0/110208877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4c0054e0 con 0x55cccf8a72f0 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a377fe640 1 -- 172.21.15.19:0/110208877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a4c019730 con 0x55cccf8a72f0 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a655c1740 10 client.18198 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a655c1740 1 -- 172.21.15.19:0/110208877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cccfc88d00 con 0x55cccf8a72f0 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a377fe640 1 -- 172.21.15.19:0/110208877 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4c0054e0 con 0x55cccf8a72f0 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a377fe640 1 -- 172.21.15.19:0/110208877 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4a4c042870 con 0x55cccf8a72f0 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a655c1740 10 client.18198 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a655c1740 10 client.18198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a655c1740 20 client.18198 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a655c1740 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55cccf8d01b0 0x55cccfcfa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a655c1740 4 client.18198 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a655c1740 1 -- 172.21.15.19:0/110208877 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cccfbe0240 con 0x55cccf8d01b0 2024-05-09T01:35:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.577+0000 7f4a57fff640 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55cccf8d01b0 0x55cccfcfa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.578+0000 7f4a57fff640 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55cccf8d01b0 0x55cccfcfa470 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a62fd6a484c2399 server_cookie=8dd8eb0e00fe7917 in_seq=0 out_seq=0 2024-05-09T01:35:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.578+0000 7f4a377fe640 10 client.18198 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a377fe640 1 -- 172.21.15.19:0/110208877 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cccfbe0240 con 0x55cccf8d01b0 2024-05-09T01:35:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a377fe640 10 client.18198 handle_command_reply: tid=0 2024-05-09T01:35:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 client.18198 shutdown 2024-05-09T01:35:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 -- 172.21.15.19:0/110208877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 msgr2=0x55cccfc86750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc86750 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c01b040 tx=0x7f4a4c004870 comp rx=0 tx=0).stop 2024-05-09T01:35:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 -- 172.21.15.19:0/110208877 shutdown_connections 2024-05-09T01:35:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55cccf8d01b0 0x55cccfcfa470 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cccf8af4d0 0x55cccfc87680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cccf8a72f0 0x55cccfc86750 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a377fe640 0 client.18198 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 --2- 172.21.15.19:0/110208877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cccf88ed80 0x55cccfc86210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 -- 172.21.15.19:0/110208877 >> 172.21.15.19:0/110208877 conn(0x55cccf8c4300 msgr2=0x55cccfc78890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 -- 172.21.15.19:0/110208877 shutdown_connections 2024-05-09T01:35:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.593+0000 7f4a655c1740 1 -- 172.21.15.19:0/110208877 wait complete. 2024-05-09T01:35:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a655c1740 20 client.18198 trim_cache size 0 max 16384 2024-05-09T01:35:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:35:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:35:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:35:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 -- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a400b27e0 msgr2=0x7f4a400b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a400b27e0 0x7f4a400b4cf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55cccfc79ae0 tx=0x7f4a50009500 comp rx=0 tx=0).stop 2024-05-09T01:35:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 -- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58103c60 msgr2=0x7f4a5819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58103c60 0x7f4a5819a360 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006010 tx=0x7f4a48004270 comp rx=0 tx=0).stop 2024-05-09T01:35:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 -- 172.21.15.19:0/3491544669 shutdown_connections 2024-05-09T01:35:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f4a400b27e0 0x7f4a400b4cf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109ba0 0x7f4a581a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a58104ea0 0x7f4a5819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 --2- 172.21.15.19:0/3491544669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58103c60 0x7f4a5819a360 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 -- 172.21.15.19:0/3491544669 >> 172.21.15.19:0/3491544669 conn(0x7f4a580fee70 msgr2=0x7f4a58105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 -- 172.21.15.19:0/3491544669 shutdown_connections 2024-05-09T01:35:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:34.594+0000 7f4a5e70d640 1 -- 172.21.15.19:0/3491544669 wait complete. 2024-05-09T01:35:34.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:34 smithi019 ceph-mon[30158]: pgmap v679: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:34.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:34 smithi196 ceph-mon[35041]: pgmap v679: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:35.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:34 smithi059 ceph-mon[34351]: pgmap v679: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:36 smithi019 ceph-mon[30158]: pgmap v680: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:35:36.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:36 smithi196 ceph-mon[35041]: pgmap v680: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:35:37.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:36 smithi059 ceph-mon[34351]: pgmap v680: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:35:38.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:38 smithi196 ceph-mon[35041]: pgmap v681: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:35:39.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:38 smithi059 ceph-mon[34351]: pgmap v681: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:35:39.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:38 smithi019 ceph-mon[30158]: pgmap v681: 97 pgs: 97 active+clean; 2.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:35:40.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:40 smithi196 ceph-mon[35041]: pgmap v682: 97 pgs: 97 active+clean; 2.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:35:41.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:40 smithi059 ceph-mon[34351]: pgmap v682: 97 pgs: 97 active+clean; 2.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:35:41.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:40 smithi019 ceph-mon[30158]: pgmap v682: 97 pgs: 97 active+clean; 2.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:35:42.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:42 smithi196 ceph-mon[35041]: pgmap v683: 97 pgs: 97 active+clean; 2.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:35:43.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:42 smithi059 ceph-mon[34351]: pgmap v683: 97 pgs: 97 active+clean; 2.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:35:43.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:42 smithi019 ceph-mon[30158]: pgmap v683: 97 pgs: 97 active+clean; 2.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:35:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:44 smithi196 ceph-mon[35041]: pgmap v684: 97 pgs: 97 active+clean; 2.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:35:45.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:44 smithi059 ceph-mon[34351]: pgmap v684: 97 pgs: 97 active+clean; 2.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:35:45.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:44 smithi019 ceph-mon[30158]: pgmap v684: 97 pgs: 97 active+clean; 2.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:35:46.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:46 smithi196 ceph-mon[35041]: pgmap v685: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:35:47.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:46 smithi059 ceph-mon[34351]: pgmap v685: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:35:47.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:46 smithi019 ceph-mon[30158]: pgmap v685: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:35:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.709+0000 7fdab4861640 1 -- 172.21.15.19:0/2551512687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 msgr2=0x7fdab0106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.709+0000 7fdab4861640 1 --2- 172.21.15.19:0/2551512687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 0x7fdab0106b50 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009900 tx=0x7fdaa002a410 comp rx=0 tx=0).stop 2024-05-09T01:35:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.709+0000 7fdab4861640 1 -- 172.21.15.19:0/2551512687 shutdown_connections 2024-05-09T01:35:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.709+0000 7fdab4861640 1 --2- 172.21.15.19:0/2551512687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab01099f0 0x7fdab010be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.709+0000 7fdab4861640 1 --2- 172.21.15.19:0/2551512687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdab0107090 0x7fdab01094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.709+0000 7fdab4861640 1 --2- 172.21.15.19:0/2551512687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 0x7fdab0106b50 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.709+0000 7fdab4861640 1 -- 172.21.15.19:0/2551512687 >> 172.21.15.19:0/2551512687 conn(0x7fdab00fdd30 msgr2=0x7fdab01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.709+0000 7fdab4861640 1 -- 172.21.15.19:0/2551512687 shutdown_connections 2024-05-09T01:35:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.709+0000 7fdab4861640 1 -- 172.21.15.19:0/2551512687 wait complete. 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.710+0000 7fdab4861640 1 Processor -- start 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.710+0000 7fdab4861640 1 -- start start 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.710+0000 7fdab4861640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 0x7fdab019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.710+0000 7fdab4861640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0107090 0x7fdab019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.710+0000 7fdab4861640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdab01099f0 0x7fdab01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.710+0000 7fdab4861640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010f720 con 0x7fdab0104730 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.710+0000 7fdab4861640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010f5a0 con 0x7fdab01099f0 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.710+0000 7fdab4861640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010f8a0 con 0x7fdab0107090 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaae59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 0x7fdab019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaae59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 0x7fdab019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40124/0 (socket says 172.21.15.19:40124) 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaae59c640 1 -- 172.21.15.19:0/3985009847 learned_addr learned my addr 172.21.15.19:0/3985009847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdab01099f0 0x7fdab01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaadd9b640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0107090 0x7fdab019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaae59c640 1 -- 172.21.15.19:0/3985009847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0107090 msgr2=0x7fdab019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaae59c640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0107090 0x7fdab019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaae59c640 1 -- 172.21.15.19:0/3985009847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdab01099f0 msgr2=0x7fdab01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaae59c640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdab01099f0 0x7fdab01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaae59c640 1 -- 172.21.15.19:0/3985009847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab01a1e40 con 0x7fdab0104730 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdab01099f0 0x7fdab01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdaae59c640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 0x7fdab019a3b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0006010 tx=0x7fdaa0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fda9f7fe640 1 -- 172.21.15.19:0/3985009847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa003f030 con 0x7fdab0104730 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fda9f7fe640 1 -- 172.21.15.19:0/3985009847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa0003aa0 con 0x7fdab0104730 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fdab4861640 1 -- 172.21.15.19:0/3985009847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab01a20d0 con 0x7fdab0104730 2024-05-09T01:35:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.711+0000 7fda9f7fe640 1 -- 172.21.15.19:0/3985009847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0035070 con 0x7fdab0104730 2024-05-09T01:35:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.712+0000 7fdab4861640 1 -- 172.21.15.19:0/3985009847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01a25c0 con 0x7fdab0104730 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.714+0000 7fdabb7f7740 1 Processor -- start 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdabb7f7740 1 -- start start 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 0x55bc482f7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 0x55bc482f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fda9f7fe640 1 -- 172.21.15.19:0/3985009847 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdaa0042030 con 0x7fdab0104730 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc482fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc4825b7c0 con 0x55bc47f324d0 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bc481f70e0 con 0x55bc47f2a2f0 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc48257780 con 0x55bc47f531b0 2024-05-09T01:35:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdaadd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc482fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdaadd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc482fe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40136/0 (socket says 172.21.15.19:40136) 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdaadd9b640 1 -- 172.21.15.19:0/3171370993 learned_addr learned my addr 172.21.15.19:0/3171370993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 0x55bc482f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.715+0000 7fdaae59c640 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 0x55bc482f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fdaadd9b640 1 -- 172.21.15.19:0/3171370993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 msgr2=0x55bc482f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fdaadd9b640 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 0x55bc482f7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fdaadd9b640 1 -- 172.21.15.19:0/3171370993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 msgr2=0x55bc482f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fdaadd9b640 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 0x55bc482f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fdaadd9b640 1 -- 172.21.15.19:0/3171370993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc482207c0 con 0x55bc47f324d0 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fdaae59c640 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 0x55bc482f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fdaadd9b640 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc482fe980 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009c20 tx=0x7fdaa40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5f4f21450c156c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fda9cff9640 1 -- 172.21.15.19:0/3171370993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4012450 con 0x55bc47f324d0 2024-05-09T01:35:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fda9cff9640 1 -- 172.21.15.19:0/3171370993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa4012aa0 con 0x55bc47f324d0 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fda9f7fe640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda980b27e0 0x7fda980b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fda9cff9640 1 -- 172.21.15.19:0/3171370993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa400fa80 con 0x55bc47f324d0 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda980b27e0 0x7fda980b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.716+0000 7fda9f7fe640 1 -- 172.21.15.19:0/3985009847 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdaa01029e0 con 0x7fdab0104730 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3171370993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 msgr2=0x55bc482fe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda980b27e0 0x7fda980b4cf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55bc482f86d0 tx=0x7fdaa8007740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc482fe980 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009c20 tx=0x7fdaa40117b0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3171370993 shutdown_connections 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc482fe980 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 0x55bc482f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/3171370993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 0x55bc482f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3171370993 >> 172.21.15.19:0/3171370993 conn(0x55bc47f47300 msgr2=0x55bc482f2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3171370993 shutdown_connections 2024-05-09T01:35:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.717+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3171370993 wait complete. 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdabb7f7740 1 -- start start 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdabb7f7740 1 Processor -- start 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 0x55bc483011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc48301710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 0x55bc48308670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc482ff3d0 con 0x55bc47f324d0 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bc481e29c0 con 0x55bc47f2a2f0 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc482ff550 con 0x55bc47f531b0 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdabb7f7740 10 client.0 _ll_register_callbacks cb 0x55bc4820f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdaadd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc48301710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdaadd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc48301710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40144/0 (socket says 172.21.15.19:40144) 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdaadd9b640 1 -- 172.21.15.19:0/4028813136 learned_addr learned my addr 172.21.15.19:0/4028813136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 0x55bc48308670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.718+0000 7fdaae59c640 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 0x55bc483011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fdaadd9b640 1 -- 172.21.15.19:0/4028813136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 msgr2=0x55bc48308670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fdaadd9b640 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 0x55bc48308670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fdaadd9b640 1 -- 172.21.15.19:0/4028813136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 msgr2=0x55bc483011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fdaadd9b640 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 0x55bc483011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fdaadd9b640 1 -- 172.21.15.19:0/4028813136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc482091f0 con 0x55bc47f324d0 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fdaadd9b640 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc48301710 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4000c00 tx=0x7fdaa40098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d741b654d9bd63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fda95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fda95ffb640 1 -- 172.21.15.19:0/4028813136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4028020 con 0x55bc47f324d0 2024-05-09T01:35:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fda95ffb640 1 -- 172.21.15.19:0/4028813136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdaa4012450 con 0x55bc47f324d0 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fdabb7f7740 1 -- 172.21.15.19:0/4028813136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa4012aa0 con 0x55bc47f324d0 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fda95ffb640 1 -- 172.21.15.19:0/4028813136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdaa4033050 con 0x55bc47f324d0 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fdabb7f7740 10 client.18210 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fdabb7f7740 1 -- 172.21.15.19:0/4028813136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc47fbe790 con 0x55bc47f324d0 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.719+0000 7fda95ffb640 1 -- 172.21.15.19:0/4028813136 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa4012aa0 con 0x55bc47f324d0 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fda95ffb640 1 -- 172.21.15.19:0/4028813136 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdaa400f950 con 0x55bc47f324d0 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fdabb7f7740 10 client.18210 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fdabb7f7740 10 client.18210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fdabb7f7740 20 client.18210 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bc47f11d80 0x55bc4830e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fdabb7f7740 4 client.18210 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fdabb7f7740 1 -- 172.21.15.19:0/4028813136 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc482693d0 con 0x55bc47f11d80 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fdaae59c640 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bc47f11d80 0x55bc4830e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fda95ffb640 10 client.18210 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.720+0000 7fdaae59c640 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bc47f11d80 0x55bc4830e850 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43186e7a6599a8af server_cookie=bf7fbfc9ba14a003 in_seq=0 out_seq=0 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fda95ffb640 1 -- 172.21.15.19:0/4028813136 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc482693d0 con 0x55bc47f11d80 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fda95ffb640 10 client.18210 handle_command_reply: tid=0 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 client.18210 shutdown 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 -- 172.21.15.19:0/4028813136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 msgr2=0x55bc48301710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc48301710 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4000c00 tx=0x7fdaa40098b0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 -- 172.21.15.19:0/4028813136 shutdown_connections 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bc47f11d80 0x55bc4830e850 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f531b0 0x55bc48308670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fda95ffb640 0 client.18210 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f324d0 0x55bc48301710 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/4028813136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f2a2f0 0x55bc483011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 -- 172.21.15.19:0/4028813136 >> 172.21.15.19:0/4028813136 conn(0x55bc47f47300 msgr2=0x55bc482f5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.721+0000 7fdabb7f7740 1 -- 172.21.15.19:0/4028813136 shutdown_connections 2024-05-09T01:35:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.722+0000 7fdabb7f7740 1 -- 172.21.15.19:0/4028813136 wait complete. 2024-05-09T01:35:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.722+0000 7fdabb7f7740 20 client.18210 trim_cache size 0 max 16384 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdabb7f7740 1 Processor -- start 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdabb7f7740 1 -- start start 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 0x55bc48312fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 0x55bc48308590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48308ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01ab460 con 0x55bc47f531b0 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01acfa0 con 0x55bc47f324d0 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01abb30 con 0x55bc47f2a2f0 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdaaed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48308ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdaaed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48308ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40160/0 (socket says 172.21.15.19:40160) 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.737+0000 7fdaaed9d640 1 -- 172.21.15.19:0/3582949630 learned_addr learned my addr 172.21.15.19:0/3582949630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fdaae59c640 1 --2- 172.21.15.19:0/3582949630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 0x55bc48312fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fdaadd9b640 1 --2- 172.21.15.19:0/3582949630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 0x55bc48308590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fdaaed9d640 1 -- 172.21.15.19:0/3582949630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 msgr2=0x55bc48312fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/3582949630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 0x55bc48312fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fdaaed9d640 1 -- 172.21.15.19:0/3582949630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 msgr2=0x55bc48308590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/3582949630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 0x55bc48308590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fdaaed9d640 1 -- 172.21.15.19:0/3582949630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc480355f0 con 0x55bc47f531b0 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/3582949630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48308ad0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800e6d0 tx=0x7fdaa8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361eac2fd6fb10b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fda97fff640 1 -- 172.21.15.19:0/3582949630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8003970 con 0x55bc47f531b0 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fda97fff640 1 -- 172.21.15.19:0/3582949630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa8004020 con 0x55bc47f531b0 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.738+0000 7fda97fff640 1 -- 172.21.15.19:0/3582949630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8026780 con 0x55bc47f531b0 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3582949630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 msgr2=0x55bc48308ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/3582949630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48308ad0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800e6d0 tx=0x7fdaa8014000 comp rx=0 tx=0).stop 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3582949630 shutdown_connections 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/3582949630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48308ad0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/3582949630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 0x55bc48308590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/3582949630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 0x55bc48312fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3582949630 >> 172.21.15.19:0/3582949630 conn(0x55bc47f47300 msgr2=0x55bc482fb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3582949630 shutdown_connections 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 -- 172.21.15.19:0/3582949630 wait complete. 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 -- start start 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.739+0000 7fdabb7f7740 1 Processor -- start 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 0x55bc48305ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 0x55bc483061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdabb7f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48374b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01ac8d0 con 0x55bc47f531b0 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01ac200 con 0x55bc47f324d0 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdabb7f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01a9440 con 0x55bc47f2a2f0 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdabb7f7740 10 client.0 _ll_register_callbacks cb 0x7fdaa80075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaaed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48374b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaaed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48374b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40164/0 (socket says 172.21.15.19:40164) 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaaed9d640 1 -- 172.21.15.19:0/538256591 learned_addr learned my addr 172.21.15.19:0/538256591 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaae59c640 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 0x55bc48305ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaadd9b640 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 0x55bc483061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaaed9d640 1 -- 172.21.15.19:0/538256591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 msgr2=0x55bc48305ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 0x55bc48305ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaaed9d640 1 -- 172.21.15.19:0/538256591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 msgr2=0x55bc483061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 0x55bc483061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaaed9d640 1 -- 172.21.15.19:0/538256591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc4821fed0 con 0x55bc47f531b0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fdaaed9d640 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48374b60 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8007530 tx=0x7fdaa80045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90f04618d98e9e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.740+0000 7fda95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fda95ffb640 1 -- 172.21.15.19:0/538256591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8003c80 con 0x55bc47f531b0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fda95ffb640 1 -- 172.21.15.19:0/538256591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdaa8005040 con 0x55bc47f531b0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fdabb7f7740 1 -- 172.21.15.19:0/538256591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa8004020 con 0x55bc47f531b0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fda95ffb640 1 -- 172.21.15.19:0/538256591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdaa8003e20 con 0x55bc47f531b0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fdabb7f7740 10 client.18216 fetch_fsmap learned FSMap version 28 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fdabb7f7740 1 -- 172.21.15.19:0/538256591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc48375fa0 con 0x55bc47f531b0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fda95ffb640 1 -- 172.21.15.19:0/538256591 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa8004020 con 0x55bc47f531b0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fda95ffb640 1 -- 172.21.15.19:0/538256591 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdaa802f890 con 0x55bc47f531b0 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fdabb7f7740 10 client.18216 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fdabb7f7740 10 client.18216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fdabb7f7740 20 client.18216 populate_metadata read hostname 'smithi019' 2024-05-09T01:35:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bc47ee71c0 0x55bc483789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:35:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fdabb7f7740 4 client.18216 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:35:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.741+0000 7fdabb7f7740 1 -- 172.21.15.19:0/538256591 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc48263240 con 0x55bc47ee71c0 2024-05-09T01:35:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.742+0000 7fdaae59c640 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bc47ee71c0 0x55bc483789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:35:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.742+0000 7fdaae59c640 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bc47ee71c0 0x55bc483789e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f591d5709d6fcc server_cookie=151c98b53c695c46 in_seq=0 out_seq=0 2024-05-09T01:35:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.742+0000 7fda95ffb640 10 client.18216 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.758+0000 7fda95ffb640 1 -- 172.21.15.19:0/538256591 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bc48263240 con 0x55bc47ee71c0 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.758+0000 7fda95ffb640 10 client.18216 handle_command_reply: tid=0 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 client.18216 shutdown 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 -- 172.21.15.19:0/538256591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 msgr2=0x55bc48374b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48374b60 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8007530 tx=0x7fdaa80045c0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 -- 172.21.15.19:0/538256591 shutdown_connections 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55bc47ee71c0 0x55bc483789e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc47f531b0 0x55bc48374b60 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bc47f324d0 0x55bc483061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fda95ffb640 0 client.18216 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:35:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 --2- 172.21.15.19:0/538256591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc47f2a2f0 0x55bc48305ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 -- 172.21.15.19:0/538256591 >> 172.21.15.19:0/538256591 conn(0x55bc47f47300 msgr2=0x55bc482f5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 -- 172.21.15.19:0/538256591 shutdown_connections 2024-05-09T01:35:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 1 -- 172.21.15.19:0/538256591 wait complete. 2024-05-09T01:35:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.759+0000 7fdabb7f7740 20 client.18216 trim_cache size 0 max 16384 2024-05-09T01:35:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:35:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:35:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:35:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 -- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda980b27e0 msgr2=0x7fda980b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda980b27e0 0x7fda980b4cf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55bc482f86d0 tx=0x7fdaa8007740 comp rx=0 tx=0).stop 2024-05-09T01:35:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 -- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 msgr2=0x7fdab019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:35:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 0x7fdab019a3b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0006010 tx=0x7fdaa0004270 comp rx=0 tx=0).stop 2024-05-09T01:35:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 -- 172.21.15.19:0/3985009847 shutdown_connections 2024-05-09T01:35:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda980b27e0 0x7fda980b4cf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdab01099f0 0x7fdab01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0107090 0x7fdab019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 --2- 172.21.15.19:0/3985009847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab0104730 0x7fdab019a3b0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:35:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 -- 172.21.15.19:0/3985009847 >> 172.21.15.19:0/3985009847 conn(0x7fdab00fdd30 msgr2=0x7fdab0106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:35:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 -- 172.21.15.19:0/3985009847 shutdown_connections 2024-05-09T01:35:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:35:47.760+0000 7fdab4861640 1 -- 172.21.15.19:0/3985009847 wait complete. 2024-05-09T01:35:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:48 smithi196 ceph-mon[35041]: pgmap v686: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:49.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:48 smithi059 ceph-mon[34351]: pgmap v686: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:49.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:48 smithi019 ceph-mon[30158]: pgmap v686: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:50.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:50 smithi196 ceph-mon[35041]: pgmap v687: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:35:51.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:50 smithi059 ceph-mon[34351]: pgmap v687: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:35:51.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:50 smithi019 ceph-mon[30158]: pgmap v687: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:35:52.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:52 smithi196 ceph-mon[35041]: pgmap v688: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:53.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:52 smithi059 ceph-mon[34351]: pgmap v688: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:53.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:52 smithi019 ceph-mon[30158]: pgmap v688: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:35:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:54 smithi196 ceph-mon[35041]: pgmap v689: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:35:55.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:54 smithi059 ceph-mon[34351]: pgmap v689: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:35:55.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:54 smithi019 ceph-mon[30158]: pgmap v689: 97 pgs: 97 active+clean; 2.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:35:56.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:55 smithi059 ceph-mon[34351]: pgmap v690: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:35:56.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:55 smithi019 ceph-mon[30158]: pgmap v690: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:35:56.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:55 smithi196 ceph-mon[35041]: pgmap v690: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:35:58.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:35:58 smithi196 ceph-mon[35041]: pgmap v691: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:35:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:35:58 smithi059 ceph-mon[34351]: pgmap v691: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:35:58.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:35:58 smithi019 ceph-mon[30158]: pgmap v691: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:00.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:00 smithi196 ceph-mon[35041]: pgmap v692: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:36:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:00 smithi059 ceph-mon[34351]: pgmap v692: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:36:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.872+0000 7f2bb3861640 1 -- 172.21.15.19:0/1658286001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac104ec0 msgr2=0x7f2bac1052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.872+0000 7f2bb3861640 1 --2- 172.21.15.19:0/1658286001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac104ec0 0x7f2bac1052d0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009900 tx=0x7f2ba002a410 comp rx=0 tx=0).stop 2024-05-09T01:36:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.872+0000 7f2bb3861640 1 -- 172.21.15.19:0/1658286001 shutdown_connections 2024-05-09T01:36:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.872+0000 7f2bb3861640 1 --2- 172.21.15.19:0/1658286001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bac109bc0 0x7f2bac10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.872+0000 7f2bb3861640 1 --2- 172.21.15.19:0/1658286001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac104ec0 0x7f2bac1052d0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.872+0000 7f2bb3861640 1 --2- 172.21.15.19:0/1658286001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac103c80 0x7f2bac104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.872+0000 7f2bb3861640 1 -- 172.21.15.19:0/1658286001 >> 172.21.15.19:0/1658286001 conn(0x7f2bac0fee50 msgr2=0x7f2bac101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.872+0000 7f2bb3861640 1 -- 172.21.15.19:0/1658286001 shutdown_connections 2024-05-09T01:36:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.872+0000 7f2bb3861640 1 -- 172.21.15.19:0/1658286001 wait complete. 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb3861640 1 Processor -- start 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb3861640 1 -- start start 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb3861640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac103c80 0x7f2bac19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb3861640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bac104ec0 0x7f2bac19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb3861640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac109bc0 0x7f2bac1a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb3861640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10f7b0 con 0x7f2bac109bc0 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb3861640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10f630 con 0x7f2bac103c80 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb3861640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10f930 con 0x7f2bac104ec0 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb15fd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac103c80 0x7f2bac19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb1dfe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac109bc0 0x7f2bac1a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb0dfc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bac104ec0 0x7f2bac19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb15fd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac103c80 0x7f2bac19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38218/0 (socket says 172.21.15.19:38218) 2024-05-09T01:36:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb1dfe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac109bc0 0x7f2bac1a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43046/0 (socket says 172.21.15.19:43046) 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.873+0000 7f2bb15fd640 1 -- 172.21.15.19:0/3743146384 learned_addr learned my addr 172.21.15.19:0/3743146384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2bb15fd640 1 -- 172.21.15.19:0/3743146384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bac104ec0 msgr2=0x7f2bac19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bac104ec0 0x7f2bac19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2bb15fd640 1 -- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac109bc0 msgr2=0x7f2bac1a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac109bc0 0x7f2bac1a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2bb15fd640 1 -- 172.21.15.19:0/3743146384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac1a4070 con 0x7f2bac103c80 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac109bc0 0x7f2bac1a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac103c80 0x7f2bac19c530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00eb90 tx=0x7f2b9c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2baa7fc640 1 -- 172.21.15.19:0/3743146384 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c003b70 con 0x7f2bac103c80 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2bb3861640 1 -- 172.21.15.19:0/3743146384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bac1a4360 con 0x7f2bac103c80 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2baa7fc640 1 -- 172.21.15.19:0/3743146384 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c00ee90 con 0x7f2bac103c80 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2baa7fc640 1 -- 172.21.15.19:0/3743146384 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0197b0 con 0x7f2bac103c80 2024-05-09T01:36:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.874+0000 7f2bb3861640 1 -- 172.21.15.19:0/3743146384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bac1a4900 con 0x7f2bac103c80 2024-05-09T01:36:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.877+0000 7f2bba7fb740 1 Processor -- start 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.877+0000 7f2bba7fb740 1 -- start start 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.877+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dc4d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.877+0000 7f2baa7fc640 1 -- 172.21.15.19:0/3743146384 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2b9c004030 con 0x7f2bac103c80 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.877+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc4da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.877+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.877+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56137dbb17c0 con 0x56137d8802f0 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.877+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56137db4d0e0 con 0x56137d8884d0 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.877+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56137dbad780 con 0x56137d8a91b0 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb15fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc4da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb1dfe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dc4d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb0dfc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc54980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb15fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc4da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43058/0 (socket says 172.21.15.19:43058) 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb1dfe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dc4d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50696/0 (socket says 172.21.15.19:50696) 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb15fd640 1 -- 172.21.15.19:0/2152086178 learned_addr learned my addr 172.21.15.19:0/2152086178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb0dfc640 1 -- 172.21.15.19:0/2152086178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 msgr2=0x56137dc4d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/2152086178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dc4d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb0dfc640 1 -- 172.21.15.19:0/2152086178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 msgr2=0x56137dc4da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/2152086178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc4da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb0dfc640 1 -- 172.21.15.19:0/2152086178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137db767c0 con 0x56137d8884d0 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/2152086178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dc4d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/2152086178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc54980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005ab0 tx=0x7f2ba0009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f64edb4e8d085efa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2baa7fc640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b940b27e0 0x7f2b940b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2baa7fc640 1 -- 172.21.15.19:0/3743146384 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2b9c0e2a80 con 0x7f2bac103c80 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2b937fe640 1 -- 172.21.15.19:0/2152086178 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0035030 con 0x56137d8884d0 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.878+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b940b27e0 0x7f2b940b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.879+0000 7f2b937fe640 1 -- 172.21.15.19:0/2152086178 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ba0040070 con 0x56137d8884d0 2024-05-09T01:36:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.879+0000 7f2b937fe640 1 -- 172.21.15.19:0/2152086178 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0003850 con 0x56137d8884d0 2024-05-09T01:36:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 -- 172.21.15.19:0/2152086178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 msgr2=0x56137dc54980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b940b27e0 0x7f2b940b4cf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009fd0 tx=0x7f2ba4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/2152086178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc54980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005ab0 tx=0x7f2ba0009600 comp rx=0 tx=0).stop 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 -- 172.21.15.19:0/2152086178 shutdown_connections 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/2152086178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc54980 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/2152086178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc4da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/2152086178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dc4d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 -- 172.21.15.19:0/2152086178 >> 172.21.15.19:0/2152086178 conn(0x56137d89d300 msgr2=0x56137dc48e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 -- 172.21.15.19:0/2152086178 shutdown_connections 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 -- 172.21.15.19:0/2152086178 wait complete. 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 -- start start 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.880+0000 7f2bba7fb740 1 Processor -- start 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 0x56137dc571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8884d0 0x56137dc57710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8a91b0 0x56137dc5e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56137dc55370 con 0x56137d8884d0 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56137db389c0 con 0x56137d8a91b0 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56137dc554f0 con 0x56137d8802f0 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bba7fb740 10 client.0 _ll_register_callbacks cb 0x56137db65020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb0dfc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8884d0 0x56137dc57710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb0dfc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8884d0 0x56137dc57710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43062/0 (socket says 172.21.15.19:43062) 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb0dfc640 1 -- 172.21.15.19:0/1256466474 learned_addr learned my addr 172.21.15.19:0/1256466474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8a91b0 0x56137dc5e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 0x56137dc571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb0dfc640 1 -- 172.21.15.19:0/1256466474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 msgr2=0x56137dc571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 0x56137dc571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb0dfc640 1 -- 172.21.15.19:0/1256466474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8a91b0 msgr2=0x56137dc5e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8a91b0 0x56137dc5e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb0dfc640 1 -- 172.21.15.19:0/1256466474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137db5f1f0 con 0x56137d8884d0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.881+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8a91b0 0x56137dc5e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8884d0 0x56137dc57710 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0031f00 tx=0x7f2ba0031f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94d48fe9cb461fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2b90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2b90ff9640 1 -- 172.21.15.19:0/1256466474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0035040 con 0x56137d8884d0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2b90ff9640 1 -- 172.21.15.19:0/1256466474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2ba00315a0 con 0x56137d8884d0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2bba7fb740 1 -- 172.21.15.19:0/1256466474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ba0040070 con 0x56137d8884d0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2b90ff9640 1 -- 172.21.15.19:0/1256466474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ba00317a0 con 0x56137d8884d0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2bba7fb740 10 client.18228 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2bba7fb740 1 -- 172.21.15.19:0/1256466474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56137d914790 con 0x56137d8884d0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2b90ff9640 1 -- 172.21.15.19:0/1256466474 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ba0040070 con 0x56137d8884d0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2b90ff9640 1 -- 172.21.15.19:0/1256466474 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2ba0045500 con 0x56137d8884d0 2024-05-09T01:36:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2bba7fb740 10 client.18228 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.882+0000 7f2bba7fb740 10 client.18228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.883+0000 7f2bba7fb740 20 client.18228 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.883+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56137d867d80 0x56137dc64730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.883+0000 7f2bba7fb740 4 client.18228 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.883+0000 7f2bba7fb740 1 -- 172.21.15.19:0/1256466474 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56137dbbf3d0 con 0x56137d867d80 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.883+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56137d867d80 0x56137dc64730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.883+0000 7f2b90ff9640 10 client.18228 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.883+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56137d867d80 0x56137dc64730 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6342ab5fcf2d668 server_cookie=95e5782e40d6ee3e in_seq=0 out_seq=0 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2b90ff9640 1 -- 172.21.15.19:0/1256466474 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56137dbbf3d0 con 0x56137d867d80 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2b90ff9640 10 client.18228 handle_command_reply: tid=0 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 client.18228 shutdown 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 -- 172.21.15.19:0/1256466474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8884d0 msgr2=0x56137dc57710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8884d0 0x56137dc57710 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0031f00 tx=0x7f2ba0031f30 comp rx=0 tx=0).stop 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 -- 172.21.15.19:0/1256466474 shutdown_connections 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56137d867d80 0x56137dc64730 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8a91b0 0x56137dc5e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8884d0 0x56137dc57710 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2b90ff9640 0 client.18228 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/1256466474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 0x56137dc571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 -- 172.21.15.19:0/1256466474 >> 172.21.15.19:0/1256466474 conn(0x56137d89d300 msgr2=0x56137dc47700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 -- 172.21.15.19:0/1256466474 shutdown_connections 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 1 -- 172.21.15.19:0/1256466474 wait complete. 2024-05-09T01:36:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.885+0000 7f2bba7fb740 20 client.18228 trim_cache size 0 max 16384 2024-05-09T01:36:00.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:00 smithi019 ceph-mon[30158]: pgmap v692: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bba7fb740 1 Processor -- start 2024-05-09T01:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bba7fb740 1 -- start start 2024-05-09T01:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 0x56137dc68eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc5e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8a91b0 0x56137dc5ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac1ad5c0 con 0x56137d8a91b0 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac1af100 con 0x56137d8884d0 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac1adc90 con 0x56137d8802f0 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bb1dfe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8a91b0 0x56137dc5ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bb1dfe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8a91b0 0x56137dc5ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43072/0 (socket says 172.21.15.19:43072) 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bb1dfe640 1 -- 172.21.15.19:0/3696295916 learned_addr learned my addr 172.21.15.19:0/3696295916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc5e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.901+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 0x56137dc68eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2bb1dfe640 1 -- 172.21.15.19:0/3696295916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 msgr2=0x56137dc68eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 0x56137dc68eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2bb1dfe640 1 -- 172.21.15.19:0/3696295916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 msgr2=0x56137dc5e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc5e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2bb1dfe640 1 -- 172.21.15.19:0/3696295916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137d98b5f0 con 0x56137d8a91b0 2024-05-09T01:36:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc5e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8a91b0 0x56137dc5ead0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4012520 tx=0x7f2ba401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281734c376b5f76f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2b92ffd640 1 -- 172.21.15.19:0/3696295916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4004050 con 0x56137d8a91b0 2024-05-09T01:36:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2b92ffd640 1 -- 172.21.15.19:0/3696295916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ba4011df0 con 0x56137d8a91b0 2024-05-09T01:36:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.902+0000 7f2b92ffd640 1 -- 172.21.15.19:0/3696295916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4003b20 con 0x56137d8a91b0 2024-05-09T01:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.903+0000 7f2bba7fb740 1 -- 172.21.15.19:0/3696295916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8a91b0 msgr2=0x56137dc5ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.903+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8a91b0 0x56137dc5ead0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4012520 tx=0x7f2ba401a000 comp rx=0 tx=0).stop 2024-05-09T01:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.903+0000 7f2bba7fb740 1 -- 172.21.15.19:0/3696295916 shutdown_connections 2024-05-09T01:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.903+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8a91b0 0x56137dc5ead0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.903+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc5e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.903+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/3696295916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8802f0 0x56137dc68eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.904+0000 7f2bba7fb740 1 -- 172.21.15.19:0/3696295916 >> 172.21.15.19:0/3696295916 conn(0x56137d89d300 msgr2=0x56137dc51660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.904+0000 7f2bba7fb740 1 -- 172.21.15.19:0/3696295916 shutdown_connections 2024-05-09T01:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.904+0000 7f2bba7fb740 1 -- 172.21.15.19:0/3696295916 wait complete. 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.904+0000 7f2bba7fb740 1 -- start start 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.904+0000 7f2bba7fb740 1 Processor -- start 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc5bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc5c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bba7fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dccaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac1aea30 con 0x56137d8802f0 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac1ae360 con 0x56137d8884d0 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bba7fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac1ab5a0 con 0x56137d8a91b0 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bba7fb740 10 client.0 _ll_register_callbacks cb 0x7f2b9c0e6430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb15fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc5bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb15fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc5bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43080/0 (socket says 172.21.15.19:43080) 2024-05-09T01:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb15fd640 1 -- 172.21.15.19:0/961386173 learned_addr learned my addr 172.21.15.19:0/961386173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc5c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb1dfe640 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dccaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb15fd640 1 -- 172.21.15.19:0/961386173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 msgr2=0x56137dccaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dccaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb15fd640 1 -- 172.21.15.19:0/961386173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 msgr2=0x56137dc5c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc5c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb15fd640 1 -- 172.21.15.19:0/961386173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137db75ed0 con 0x56137d8802f0 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2bb15fd640 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc5bd20 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0a6100 tx=0x7f2b9c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682ad64408086ec0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.905+0000 7f2b90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2b90ff9640 1 -- 172.21.15.19:0/961386173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0eb040 con 0x56137d8802f0 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2b90ff9640 1 -- 172.21.15.19:0/961386173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2b9c0ae0c0 con 0x56137d8802f0 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bba7fb740 1 -- 172.21.15.19:0/961386173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ba4011df0 con 0x56137d8802f0 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2b90ff9640 1 -- 172.21.15.19:0/961386173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b9c0a99d0 con 0x56137d8802f0 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bba7fb740 10 client.18234 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bba7fb740 1 -- 172.21.15.19:0/961386173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56137dccbfe0 con 0x56137d8802f0 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2b90ff9640 1 -- 172.21.15.19:0/961386173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ba4011df0 con 0x56137d8802f0 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2b90ff9640 1 -- 172.21.15.19:0/961386173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2b9c0a6310 con 0x56137d8802f0 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bba7fb740 10 client.18234 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bba7fb740 10 client.18234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bba7fb740 20 client.18234 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56137d83d1c0 0x56137dcce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bba7fb740 4 client.18234 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bba7fb740 1 -- 172.21.15.19:0/961386173 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56137dbb9240 con 0x56137d83d1c0 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.906+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56137d83d1c0 0x56137dcce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.907+0000 7f2b90ff9640 10 client.18234 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.907+0000 7f2bb0dfc640 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56137d83d1c0 0x56137dcce9f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5aa2f61d9a0e2d server_cookie=1945ad03f5ad6abb in_seq=0 out_seq=0 2024-05-09T01:36:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2b90ff9640 1 -- 172.21.15.19:0/961386173 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56137dbb9240 con 0x56137d83d1c0 2024-05-09T01:36:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2b90ff9640 10 client.18234 handle_command_reply: tid=0 2024-05-09T01:36:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 client.18234 shutdown 2024-05-09T01:36:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 -- 172.21.15.19:0/961386173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 msgr2=0x56137dc5bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc5bd20 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0a6100 tx=0x7f2b9c0096c0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 -- 172.21.15.19:0/961386173 shutdown_connections 2024-05-09T01:36:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56137d83d1c0 0x56137dcce9f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56137d8a91b0 0x56137dccaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137d8884d0 0x56137dc5c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2b90ff9640 0 client.18234 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 --2- 172.21.15.19:0/961386173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137d8802f0 0x56137dc5bd20 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 -- 172.21.15.19:0/961386173 >> 172.21.15.19:0/961386173 conn(0x56137d89d300 msgr2=0x56137dc4bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 -- 172.21.15.19:0/961386173 shutdown_connections 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 1 -- 172.21.15.19:0/961386173 wait complete. 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.923+0000 7f2bba7fb740 20 client.18234 trim_cache size 0 max 16384 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:36:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:36:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 -- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b940b27e0 msgr2=0x7f2b940b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b940b27e0 0x7f2b940b4cf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009fd0 tx=0x7f2ba4005f90 comp rx=0 tx=0).stop 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 -- 172.21.15.19:0/3743146384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac103c80 msgr2=0x7f2bac19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac103c80 0x7f2bac19c530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00eb90 tx=0x7f2b9c00c6a0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 -- 172.21.15.19:0/3743146384 shutdown_connections 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b940b27e0 0x7f2b940b4cf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bac109bc0 0x7f2bac1a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bac104ec0 0x7f2bac19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 --2- 172.21.15.19:0/3743146384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac103c80 0x7f2bac19c530 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 -- 172.21.15.19:0/3743146384 >> 172.21.15.19:0/3743146384 conn(0x7f2bac0fee50 msgr2=0x7f2bac0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 -- 172.21.15.19:0/3743146384 shutdown_connections 2024-05-09T01:36:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:00.924+0000 7f2bb3861640 1 -- 172.21.15.19:0/3743146384 wait complete. 2024-05-09T01:36:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:36:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:01 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:36:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:01 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.852 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:36:01.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:36:01.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:01.875 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:01 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:02.031 INFO:tasks.workunit.client.0.smithi019.stdout:437/600: open fd = 3 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:438/600: open fd = 3 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:439/600: open fd = 3 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:440/600: open fd = 3 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.032 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:441/600: open fd = 3 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:442/600: open fd = 3 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:443/600: open fd = 3 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.033 INFO:tasks.workunit.client.0.smithi019.stdout:444/600: open fd = 3 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:445/600: open fd = 3 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:446/600: open fd = 3 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:447/600: open fd = 3 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.034 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:448/600: open fd = 3 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:449/600: open fd = 3 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:450/600: open fd = 3 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.035 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:451/600: open fd = 3 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:452/600: open fd = 3 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:453/600: open fd = 3 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:454/600: open fd = 3 2024-05-09T01:36:02.036 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:455/600: open fd = 3 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:456/600: open fd = 3 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:457/600: open fd = 3 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:458/600: open fd = 3 2024-05-09T01:36:02.037 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:459/600: open fd = 3 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:460/600: open fd = 3 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:461/600: open fd = 3 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.038 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:462/600: open fd = 3 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:463/600: open fd = 3 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:464/600: open fd = 3 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:465/600: open fd = 3 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.039 INFO:tasks.workunit.client.0.smithi019.stdout:466/600: open fd = 3 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:467/600: open fd = 3 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:468/600: open fd = 3 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:469/600: open fd = 3 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:470/600: open fd = 3 2024-05-09T01:36:02.040 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:471/600: open fd = 3 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:472/600: open fd = 3 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:473/600: open fd = 3 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:474/600: open fd = 3 2024-05-09T01:36:02.041 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:475/600: open fd = 3 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:476/600: open fd = 3 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:477/600: open fd = 3 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.042 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:478/600: open fd = 3 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:479/600: open fd = 3 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:480/600: open fd = 3 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:481/600: open fd = 3 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.043 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:482/600: open fd = 3 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:483/600: open fd = 3 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:484/600: open fd = 3 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:485/600: open fd = 3 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.044 INFO:tasks.workunit.client.0.smithi019.stdout:486/600: open fd = 3 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:487/600: open fd = 3 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:488/600: open fd = 3 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:489/600: open fd = 3 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.045 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:490/600: open fd = 3 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:491/600: open fd = 3 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:492/600: open fd = 3 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:493/600: open fd = 3 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.046 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:494/600: open fd = 3 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:495/600: open fd = 3 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:496/600: open fd = 3 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:497/600: open fd = 3 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.047 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.048 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.048 INFO:tasks.workunit.client.0.smithi019.stdout:498/600: open fd = 3 2024-05-09T01:36:02.048 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.048 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:36:02.048 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:36:02.119 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:02.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:02 smithi019 ceph-mon[30158]: pgmap v693: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:36:02.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:36:02.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:36:02.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:02.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:02 smithi196 ceph-mon[35041]: pgmap v693: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:36:02.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:36:02.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:36:02.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:02.747 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:02 smithi059 ceph-mon[34351]: pgmap v693: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-05-09T01:36:02.747 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:36:02.748 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:36:02.748 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:36:03.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.992+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/2729748995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc107df0 msgr2=0x7fd7dc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:03.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.992+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/2729748995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc107df0 0x7fd7dc108200 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40099c0 tx=0x7fd7c4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.992+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/2729748995 shutdown_connections 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.992+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/2729748995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc108740 0x7fd7dc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.992+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/2729748995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7dc1034c0 0x7fd7dc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.992+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/2729748995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc107df0 0x7fd7dc108200 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.992+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/2729748995 >> 172.21.15.19:0/2729748995 conn(0x7fd7dc0fedf0 msgr2=0x7fd7dc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.992+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/2729748995 shutdown_connections 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.992+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/2729748995 wait complete. 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7e0ac0700 1 Processor -- start 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7e0ac0700 1 -- start start 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7e0ac0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc1034c0 0x7fd7dc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7e0ac0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc107df0 0x7fd7dc19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7e0ac0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7dc108740 0x7fd7dc19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7e0ac0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1144c0 con 0x7fd7dc1034c0 2024-05-09T01:36:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7e0ac0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc114340 con 0x7fd7dc107df0 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7e0ac0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc114640 con 0x7fd7dc108740 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc1034c0 0x7fd7dc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc1034c0 0x7fd7dc19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43102/0 (socket says 172.21.15.19:43102) 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7da59c700 1 -- 172.21.15.19:0/3580593464 learned_addr learned my addr 172.21.15.19:0/3580593464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.993+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7dc108740 0x7fd7dc19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc107df0 0x7fd7dc19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7da59c700 1 -- 172.21.15.19:0/3580593464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7dc108740 msgr2=0x7fd7dc19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7da59c700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7dc108740 0x7fd7dc19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7da59c700 1 -- 172.21.15.19:0/3580593464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc107df0 msgr2=0x7fd7dc19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7da59c700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc107df0 0x7fd7dc19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7da59c700 1 -- 172.21.15.19:0/3580593464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7dc19fd00 con 0x7fd7dc1034c0 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7dc108740 0x7fd7dc19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7da59c700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc1034c0 0x7fd7dc19ae30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00ef10 tx=0x7fd7cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7d37fe700 1 -- 172.21.15.19:0/3580593464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc00cad0 con 0x7fd7dc1034c0 2024-05-09T01:36:03.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7d37fe700 1 -- 172.21.15.19:0/3580593464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc019070 con 0x7fd7dc1034c0 2024-05-09T01:36:03.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/3580593464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7dc19ff90 con 0x7fd7dc1034c0 2024-05-09T01:36:03.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7d37fe700 1 -- 172.21.15.19:0/3580593464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0146e0 con 0x7fd7dc1034c0 2024-05-09T01:36:03.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.994+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/3580593464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7dc1a04a0 con 0x7fd7dc1034c0 2024-05-09T01:36:03.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.997+0000 7fd7d37fe700 1 -- 172.21.15.19:0/3580593464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd7cc012050 con 0x7fd7dc1034c0 2024-05-09T01:36:03.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.998+0000 7fd7d37fe700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7c80b2880 0x7fd7c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:03.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.998+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7c80b2880 0x7fd7c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:03.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.998+0000 7fd7d37fe700 1 -- 172.21.15.19:0/3580593464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd7cc0e2320 con 0x7fd7dc1034c0 2024-05-09T01:36:03.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7ef4d6b80 1 Processor -- start 2024-05-09T01:36:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7ef4d6b80 1 -- start start 2024-05-09T01:36:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7c80b2880 0x7fd7c80b4d90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4000c00 tx=0x7fd7c401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05bc4d00 0x55dc05ccd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 0x55dc05ccd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05cc7400 0x55dc05cc7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc05bea0a0 con 0x55dc05bc4d00 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dc05b8f9a0 con 0x55dc05b4a4a0 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc05be6060 con 0x55dc05cc7400 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05bc4d00 0x55dc05ccd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05bc4d00 0x55dc05ccd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43104/0 (socket says 172.21.15.19:43104) 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7da59c700 1 -- 172.21.15.19:0/2134585499 learned_addr learned my addr 172.21.15.19:0/2134585499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 0x55dc05ccd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:03.999+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05cc7400 0x55dc05cc7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7da59c700 1 -- 172.21.15.19:0/2134585499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05cc7400 msgr2=0x55dc05cc7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7da59c700 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05cc7400 0x55dc05cc7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7da59c700 1 -- 172.21.15.19:0/2134585499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 msgr2=0x55dc05ccd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7da59c700 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 0x55dc05ccd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:03.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7da59c700 1 -- 172.21.15.19:0/2134585499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc05b80500 con 0x55dc05bc4d00 2024-05-09T01:36:03.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 0x55dc05ccd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:03.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7da59c700 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05bc4d00 0x55dc05ccd240 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0a9620 tx=0x7fd7cc009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff34c79bc76a5f02 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:03.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7d0ff9700 1 -- 172.21.15.19:0/2134585499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b20c0 con 0x55dc05bc4d00 2024-05-09T01:36:03.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7d0ff9700 1 -- 172.21.15.19:0/2134585499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc0a9430 con 0x55dc05bc4d00 2024-05-09T01:36:03.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.000+0000 7fd7d0ff9700 1 -- 172.21.15.19:0/2134585499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b2780 con 0x55dc05bc4d00 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.001+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/2134585499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05bc4d00 msgr2=0x55dc05ccd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.001+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05bc4d00 0x55dc05ccd240 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0a9620 tx=0x7fd7cc009eb0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.001+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/2134585499 shutdown_connections 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.001+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05cc7400 0x55dc05cc7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.001+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05bc4d00 0x55dc05ccd240 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.001+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/2134585499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 0x55dc05ccd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.001+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/2134585499 >> 172.21.15.19:0/2134585499 conn(0x55dc05b4df70 msgr2=0x55dc05cc2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.001+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/2134585499 shutdown_connections 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.001+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/2134585499 wait complete. 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7ef4d6b80 1 -- start start 2024-05-09T01:36:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7ef4d6b80 1 Processor -- start 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05cd15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 0x55dc05cd1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc05cc8160 con 0x55dc05b4a4a0 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dc05b84b20 con 0x55dc05cc7400 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc05cd4020 con 0x55dc05bc4d00 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7ef4d6b80 10 client.0 _ll_register_callbacks cb 0x55dc05a5b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43112/0 (socket says 172.21.15.19:43112) 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7da59c700 1 -- 172.21.15.19:0/551414170 learned_addr learned my addr 172.21.15.19:0/551414170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.002+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05cd15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7da59c700 1 -- 172.21.15.19:0/551414170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 msgr2=0x55dc05cd15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7da59c700 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05cd15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7da59c700 1 -- 172.21.15.19:0/551414170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 msgr2=0x55dc05cd1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7da59c700 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 0x55dc05cd1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7da59c700 1 -- 172.21.15.19:0/551414170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc05a65490 con 0x55dc05b4a4a0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7da59c700 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd1080 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0dfe50 tx=0x7fd7cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531a5e3583d5f7ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/551414170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0aa020 con 0x55dc05b4a4a0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/551414170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd7cc0a5050 con 0x55dc05b4a4a0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/551414170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7cc0a9430 con 0x55dc05b4a4a0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/551414170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7cc0b2780 con 0x55dc05b4a4a0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7ef4d6b80 10 client.18246 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/551414170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc05cdea60 con 0x55dc05b4a4a0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.003+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/551414170 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc0a9430 con 0x55dc05b4a4a0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.004+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/551414170 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd7cc0a53b0 con 0x55dc05b4a4a0 2024-05-09T01:36:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.004+0000 7fd7ef4d6b80 10 client.18246 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.004+0000 7fd7ef4d6b80 10 client.18246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.004+0000 7fd7ef4d6b80 20 client.18246 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.004+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55dc05ce0f30 0x55dc05ce3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.004+0000 7fd7ef4d6b80 4 client.18246 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.004+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/551414170 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc05bf7a80 con 0x55dc05ce0f30 2024-05-09T01:36:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.004+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55dc05ce0f30 0x55dc05ce3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.005+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55dc05ce0f30 0x55dc05ce3350 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98399d647c34543d server_cookie=2d9eb450662602dd in_seq=0 out_seq=0 2024-05-09T01:36:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.005+0000 7fd7c1ffb700 10 client.18246 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/551414170 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc05bf7a80 con 0x55dc05ce0f30 2024-05-09T01:36:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7c1ffb700 10 client.18246 handle_command_reply: tid=0 2024-05-09T01:36:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 client.18246 shutdown 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/551414170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 msgr2=0x55dc05cd1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd1080 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0dfe50 tx=0x7fd7cc00c950 comp rx=0 tx=0).stop 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/551414170 shutdown_connections 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55dc05ce0f30 0x55dc05ce3350 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 0x55dc05cd1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd1080 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/551414170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05cd15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/551414170 >> 172.21.15.19:0/551414170 conn(0x55dc05b4df70 msgr2=0x55dc05cbbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7c1ffb700 0 client.18246 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/551414170 shutdown_connections 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/551414170 wait complete. 2024-05-09T01:36:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.006+0000 7fd7ef4d6b80 20 client.18246 trim_cache size 0 max 16384 2024-05-09T01:36:04.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7ef4d6b80 1 Processor -- start 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7ef4d6b80 1 -- start start 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05cd1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 0x55dc05cbbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1aee20 con 0x55dc05b4a4a0 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1b0960 con 0x55dc05cc7400 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1af4f0 con 0x55dc05bc4d00 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43116/0 (socket says 172.21.15.19:43116) 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7da59c700 1 -- 172.21.15.19:0/1770000727 learned_addr learned my addr 172.21.15.19:0/1770000727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 0x55dc05cbbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05cd1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7da59c700 1 -- 172.21.15.19:0/1770000727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 msgr2=0x55dc05cd1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:04.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7da59c700 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05cd1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7da59c700 1 -- 172.21.15.19:0/1770000727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 msgr2=0x55dc05cbbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:04.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7da59c700 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 0x55dc05cbbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7da59c700 1 -- 172.21.15.19:0/1770000727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc05cbe9d0 con 0x55dc05b4a4a0 2024-05-09T01:36:04.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.027+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 0x55dc05cbbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:04.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.028+0000 7fd7da59c700 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd03a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0a98b0 tx=0x7fd7cc06f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e28bb602fc3249 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:04.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.028+0000 7fd7c3fff700 1 -- 172.21.15.19:0/1770000727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0c2070 con 0x55dc05b4a4a0 2024-05-09T01:36:04.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.028+0000 7fd7c3fff700 1 -- 172.21.15.19:0/1770000727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc087030 con 0x55dc05b4a4a0 2024-05-09T01:36:04.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.028+0000 7fd7c3fff700 1 -- 172.21.15.19:0/1770000727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc06f250 con 0x55dc05b4a4a0 2024-05-09T01:36:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/1770000727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 msgr2=0x55dc05cd03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd03a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0a98b0 tx=0x7fd7cc06f4e0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/1770000727 shutdown_connections 2024-05-09T01:36:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05cc7400 0x55dc05cbbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05b4a4a0 0x55dc05cd03a0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/1770000727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05cd1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/1770000727 >> 172.21.15.19:0/1770000727 conn(0x55dc05b4df70 msgr2=0x55dc05ccd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/1770000727 shutdown_connections 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/1770000727 wait complete. 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 -- start start 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 Processor -- start 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 0x55dc05ce2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05ce3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05cc7400 0x55dc05ce39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1b0290 con 0x55dc05cc7400 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1afbc0 con 0x55dc05b4a4a0 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.029+0000 7fd7ef4d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1acdc0 con 0x55dc05bc4d00 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7ef4d6b80 10 client.0 _ll_register_callbacks cb 0x55dc05cc2880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05cc7400 0x55dc05ce39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05cc7400 0x55dc05ce39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43132/0 (socket says 172.21.15.19:43132) 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7dad9d700 1 -- 172.21.15.19:0/689429862 learned_addr learned my addr 172.21.15.19:0/689429862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7d9d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05ce3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7da59c700 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 0x55dc05ce2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7dad9d700 1 -- 172.21.15.19:0/689429862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 msgr2=0x55dc05ce3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05ce3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7dad9d700 1 -- 172.21.15.19:0/689429862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 msgr2=0x55dc05ce2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 0x55dc05ce2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7dad9d700 1 -- 172.21.15.19:0/689429862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc05d49520 con 0x55dc05cc7400 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7d9d9b700 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05ce3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05cc7400 0x55dc05ce39c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4004f00 tx=0x7fd7d400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44752996be2755d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/689429862 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d400ce40 con 0x55dc05cc7400 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/689429862 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd7d4010640 con 0x55dc05cc7400 2024-05-09T01:36:04.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.030+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/689429862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7cc087030 con 0x55dc05cc7400 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/689429862 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7d4010840 con 0x55dc05cc7400 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7ef4d6b80 10 client.18252 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/689429862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc05d4a7f0 con 0x55dc05cc7400 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/689429862 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc087030 con 0x55dc05cc7400 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/689429862 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd7d4025b90 con 0x55dc05cc7400 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7ef4d6b80 10 client.18252 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7ef4d6b80 10 client.18252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7ef4d6b80 20 client.18252 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55dc05d4af80 0x55dc05d4d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7ef4d6b80 4 client.18252 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/689429862 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc05bf1b20 con 0x55dc05d4af80 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.031+0000 7fd7da59c700 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55dc05d4af80 0x55dc05d4d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:04.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7c1ffb700 10 client.18252 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7da59c700 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55dc05d4af80 0x55dc05d4d3a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f21f51fe06cb86 server_cookie=256898d256704c5f in_seq=0 out_seq=0 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7c1ffb700 1 -- 172.21.15.19:0/689429862 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc05bf1b20 con 0x55dc05d4af80 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7c1ffb700 10 client.18252 handle_command_reply: tid=0 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7ef4d6b80 1 client.18252 shutdown 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/689429862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05cc7400 msgr2=0x55dc05ce39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05cc7400 0x55dc05ce39c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4004f00 tx=0x7fd7d400c980 comp rx=0 tx=0).stop 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/689429862 shutdown_connections 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55dc05d4af80 0x55dc05d4d3a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc05cc7400 0x55dc05ce39c0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc05b4a4a0 0x55dc05ce2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7ef4d6b80 1 --2- 172.21.15.19:0/689429862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc05bc4d00 0x55dc05ce3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7c1ffb700 0 client.18252 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.032+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/689429862 >> 172.21.15.19:0/689429862 conn(0x55dc05b4df70 msgr2=0x55dc05ccd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/689429862 shutdown_connections 2024-05-09T01:36:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7ef4d6b80 1 -- 172.21.15.19:0/689429862 wait complete. 2024-05-09T01:36:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7ef4d6b80 20 client.18252 trim_cache size 0 max 16384 2024-05-09T01:36:04.035 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:36:04.035 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:36:04.035 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:36:04.035 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:36:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7c80b2880 msgr2=0x7fd7c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7c80b2880 0x7fd7c80b4d90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4000c00 tx=0x7fd7c401a040 comp rx=0 tx=0).stop 2024-05-09T01:36:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc1034c0 msgr2=0x7fd7dc19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc1034c0 0x7fd7dc19ae30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00ef10 tx=0x7fd7cc00c3a0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/3580593464 shutdown_connections 2024-05-09T01:36:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7dc108740 0x7fd7dc19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd7c80b2880 0x7fd7c80b4d90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc1034c0 0x7fd7dc19ae30 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 --2- 172.21.15.19:0/3580593464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc107df0 0x7fd7dc19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/3580593464 >> 172.21.15.19:0/3580593464 conn(0x7fd7dc0fedf0 msgr2=0x7fd7dc101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/3580593464 shutdown_connections 2024-05-09T01:36:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:04.033+0000 7fd7e0ac0700 1 -- 172.21.15.19:0/3580593464 wait complete. 2024-05-09T01:36:04.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:04 smithi196 ceph-mon[35041]: pgmap v694: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-09T01:36:04.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:04 smithi019 ceph-mon[30158]: pgmap v694: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-09T01:36:04.810 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:36:04.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:36:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:04 smithi059 ceph-mon[34351]: pgmap v694: 97 pgs: 97 active+clean; 2.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-05-09T01:36:05.105 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:06.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:06 smithi019 ceph-mon[30158]: pgmap v695: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:36:06.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:06 smithi196 ceph-mon[35041]: pgmap v695: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:36:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:06 smithi059 ceph-mon[34351]: pgmap v695: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:36:06.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.855+0000 7fb01ef54700 1 -- 172.21.15.19:0/481112065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 msgr2=0x7fb0181038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:06.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.855+0000 7fb01ef54700 1 --2- 172.21.15.19:0/481112065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 0x7fb0181038d0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb0080099c0 tx=0x7fb008009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:06.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.856+0000 7fb01ef54700 1 -- 172.21.15.19:0/481112065 shutdown_connections 2024-05-09T01:36:06.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.856+0000 7fb01ef54700 1 --2- 172.21.15.19:0/481112065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb018108740 0x7fb0181100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:06.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.856+0000 7fb01ef54700 1 --2- 172.21.15.19:0/481112065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 0x7fb0181038d0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:06.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.856+0000 7fb01ef54700 1 --2- 172.21.15.19:0/481112065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb018107df0 0x7fb018108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:06.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.856+0000 7fb01ef54700 1 -- 172.21.15.19:0/481112065 >> 172.21.15.19:0/481112065 conn(0x7fb0180fedc0 msgr2=0x7fb0181011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:06.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.856+0000 7fb01ef54700 1 -- 172.21.15.19:0/481112065 shutdown_connections 2024-05-09T01:36:06.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.856+0000 7fb01ef54700 1 -- 172.21.15.19:0/481112065 wait complete. 2024-05-09T01:36:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.856+0000 7fb01ef54700 1 Processor -- start 2024-05-09T01:36:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.856+0000 7fb01ef54700 1 -- start start 2024-05-09T01:36:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ef54700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 0x7fb0180ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ef54700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb018107df0 0x7fb0180ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ef54700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb018108740 0x7fb0180ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ef54700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0181133c0 con 0x7fb0181034c0 2024-05-09T01:36:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ef54700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb018113240 con 0x7fb018108740 2024-05-09T01:36:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ef54700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb018113540 con 0x7fb018107df0 2024-05-09T01:36:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ccf0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 0x7fb0180ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ccf0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 0x7fb0180ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43158/0 (socket says 172.21.15.19:43158) 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ccf0700 1 -- 172.21.15.19:0/805375367 learned_addr learned my addr 172.21.15.19:0/805375367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb017fff700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb018107df0 0x7fb0180ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01d4f1700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb018108740 0x7fb0180ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ccf0700 1 -- 172.21.15.19:0/805375367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb018107df0 msgr2=0x7fb0180ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ccf0700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb018107df0 0x7fb0180ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ccf0700 1 -- 172.21.15.19:0/805375367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb018108740 msgr2=0x7fb0180ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ccf0700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb018108740 0x7fb0180ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01ccf0700 1 -- 172.21.15.19:0/805375367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0181a91c0 con 0x7fb0181034c0 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.857+0000 7fb01d4f1700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb018108740 0x7fb0180ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.858+0000 7fb01ccf0700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 0x7fb0180ff2a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb008009990 tx=0x7fb0080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.858+0000 7fb015ffb700 1 -- 172.21.15.19:0/805375367 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008027070 con 0x7fb0181034c0 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.858+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0181a9450 con 0x7fb0181034c0 2024-05-09T01:36:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.858+0000 7fb015ffb700 1 -- 172.21.15.19:0/805375367 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb008004190 con 0x7fb0181034c0 2024-05-09T01:36:06.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.858+0000 7fb015ffb700 1 -- 172.21.15.19:0/805375367 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00800f460 con 0x7fb0181034c0 2024-05-09T01:36:06.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.858+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0181a97f0 con 0x7fb0181034c0 2024-05-09T01:36:06.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.859+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0180512c0 con 0x7fb0181034c0 2024-05-09T01:36:06.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.861+0000 7fb015ffb700 1 -- 172.21.15.19:0/805375367 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb00802a070 con 0x7fb0181034c0 2024-05-09T01:36:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.861+0000 7fb015ffb700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0040b2880 0x7fb0040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.862+0000 7fb017fff700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0040b2880 0x7fb0040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.862+0000 7fb015ffb700 1 -- 172.21.15.19:0/805375367 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb0080e36d0 con 0x7fb0181034c0 2024-05-09T01:36:06.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.862+0000 7fb017fff700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0040b2880 0x7fb0040b4d90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb000005fd0 tx=0x7fb00000a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:06.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:06.865+0000 7fb015ffb700 1 -- 172.21.15.19:0/805375367 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb008031080 con 0x7fb0181034c0 2024-05-09T01:36:07.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.129+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb01802e5f0 con 0x7fb0181034c0 2024-05-09T01:36:07.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.129+0000 7fb015ffb700 1 -- 172.21.15.19:0/805375367 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fb008013070 con 0x7fb0181034c0 2024-05-09T01:36:07.126 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:36:07.127 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:36:07.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0040b2880 msgr2=0x7fb0040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:07.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0040b2880 0x7fb0040b4d90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb000005fd0 tx=0x7fb00000a560 comp rx=0 tx=0).stop 2024-05-09T01:36:07.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 msgr2=0x7fb0180ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:07.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 0x7fb0180ff2a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb008009990 tx=0x7fb0080047b0 comp rx=0 tx=0).stop 2024-05-09T01:36:07.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 shutdown_connections 2024-05-09T01:36:07.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb018108740 0x7fb0180ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb0040b2880 0x7fb0040b4d90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0181034c0 0x7fb0180ff2a0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 --2- 172.21.15.19:0/805375367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb018107df0 0x7fb0180ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 >> 172.21.15.19:0/805375367 conn(0x7fb0180fedc0 msgr2=0x7fb01806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 shutdown_connections 2024-05-09T01:36:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:07.132+0000 7fb01ef54700 1 -- 172.21.15.19:0/805375367 wait complete. 2024-05-09T01:36:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:36:07.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:07 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/805375367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:36:07.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:07 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/805375367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:36:07.810 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:36:07.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:07 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/805375367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:36:08.066 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:08.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:08 smithi019 ceph-mon[30158]: pgmap v696: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:08 smithi196 ceph-mon[35041]: pgmap v696: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:08 smithi059 ceph-mon[34351]: pgmap v696: 97 pgs: 97 active+clean; 2.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.761+0000 7f9ac3a26700 1 -- 172.21.15.19:0/334575300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc108740 msgr2=0x7f9abc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.761+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/334575300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc108740 0x7f9abc1100a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40099c0 tx=0x7f9ab4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.761+0000 7f9ac3a26700 1 -- 172.21.15.19:0/334575300 shutdown_connections 2024-05-09T01:36:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.762+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/334575300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc108740 0x7f9abc1100a0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.762+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/334575300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc1034c0 0x7f9abc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.762+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/334575300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9abc107df0 0x7f9abc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.762+0000 7f9ac3a26700 1 -- 172.21.15.19:0/334575300 >> 172.21.15.19:0/334575300 conn(0x7f9abc0fedf0 msgr2=0x7f9abc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.762+0000 7f9ac3a26700 1 -- 172.21.15.19:0/334575300 shutdown_connections 2024-05-09T01:36:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.762+0000 7f9ac3a26700 1 -- 172.21.15.19:0/334575300 wait complete. 2024-05-09T01:36:09.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac3a26700 1 Processor -- start 2024-05-09T01:36:09.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac3a26700 1 -- start start 2024-05-09T01:36:09.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac3a26700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc1034c0 0x7f9abc19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac3a26700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc107df0 0x7f9abc19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac3a26700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9abc108740 0x7f9abc199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac3a26700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc113370 con 0x7f9abc1034c0 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac3a26700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1131f0 con 0x7f9abc107df0 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac3a26700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1134f0 con 0x7f9abc108740 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac17c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc1034c0 0x7f9abc19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac17c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc1034c0 0x7f9abc19f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40622/0 (socket says 172.21.15.19:40622) 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1715724930 learned_addr learned my addr 172.21.15.19:0/1715724930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.763+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9abc108740 0x7f9abc199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ac0fc1700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc107df0 0x7f9abc19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1715724930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9abc108740 msgr2=0x7f9abc199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9abc108740 0x7f9abc199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1715724930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc107df0 msgr2=0x7f9abc19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc107df0 0x7f9abc19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1715724930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc19a1d0 con 0x7f9abc1034c0 2024-05-09T01:36:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc1034c0 0x7f9abc19f8b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80060b0 tx=0x7f9ab8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ab27fc700 1 -- 172.21.15.19:0/1715724930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab801a070 con 0x7f9abc1034c0 2024-05-09T01:36:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ac3a26700 1 -- 172.21.15.19:0/1715724930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc19a4c0 con 0x7f9abc1034c0 2024-05-09T01:36:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ab27fc700 1 -- 172.21.15.19:0/1715724930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab800de20 con 0x7f9abc1034c0 2024-05-09T01:36:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ab27fc700 1 -- 172.21.15.19:0/1715724930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8014720 con 0x7f9abc1034c0 2024-05-09T01:36:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.764+0000 7f9ac3a26700 1 -- 172.21.15.19:0/1715724930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc0759f0 con 0x7f9abc1034c0 2024-05-09T01:36:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.767+0000 7f9ab27fc700 1 -- 172.21.15.19:0/1715724930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9ab801d070 con 0x7f9abc1034c0 2024-05-09T01:36:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.768+0000 7f9ab27fc700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9aa80b2880 0x7f9aa80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.768+0000 7f9ac0fc1700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9aa80b2880 0x7f9aa80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.768+0000 7f9ab27fc700 1 -- 172.21.15.19:0/1715724930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9ab80e1b80 con 0x7f9abc1034c0 2024-05-09T01:36:09.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.768+0000 7f9ac0fc1700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9aa80b2880 0x7f9aa80b4d90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9aac005950 tx=0x7f9aac0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.768+0000 7f9ad243cb80 1 Processor -- start 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.768+0000 7f9ad243cb80 1 -- start start 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d3086b160 0x564d309044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d307814a0 0x564d30904a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d308fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564d30820ff0 con 0x564d3086b160 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564d307bf7f0 con 0x564d307814a0 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564d3081cfb0 con 0x564d308fe6a0 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac17c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d3086b160 0x564d309044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac17c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d3086b160 0x564d309044f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40626/0 (socket says 172.21.15.19:40626) 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1920080116 learned_addr learned my addr 172.21.15.19:0/1920080116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac0fc1700 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d307814a0 0x564d30904a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d308fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1920080116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 msgr2=0x564d308fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d308fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1920080116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d307814a0 msgr2=0x564d30904a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d307814a0 0x564d30904a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1920080116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d307b7370 con 0x564d3086b160 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d308fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d3086b160 0x564d309044f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80a5200 tx=0x7f9ab80a9080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3604c53b8e4f3068 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.769+0000 7f9aa77fe700 1 -- 172.21.15.19:0/1920080116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80ad530 con 0x564d3086b160 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.770+0000 7f9aa77fe700 1 -- 172.21.15.19:0/1920080116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab80b1fe0 con 0x564d3086b160 2024-05-09T01:36:09.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.770+0000 7f9aa77fe700 1 -- 172.21.15.19:0/1920080116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80b1b50 con 0x564d3086b160 2024-05-09T01:36:09.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.770+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1920080116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d3086b160 msgr2=0x564d309044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:09.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.770+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d3086b160 0x564d309044f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80a5200 tx=0x7f9ab80a9080 comp rx=0 tx=0).stop 2024-05-09T01:36:09.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.771+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1920080116 shutdown_connections 2024-05-09T01:36:09.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.771+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d308fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.771+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d3086b160 0x564d309044f0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.771+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1920080116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d307814a0 0x564d30904a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.771+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1920080116 >> 172.21.15.19:0/1920080116 conn(0x564d307e16a0 msgr2=0x564d308f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.771+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1920080116 shutdown_connections 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.771+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1920080116 wait complete. 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.771+0000 7f9ad243cb80 1 -- start start 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.771+0000 7f9ad243cb80 1 Processor -- start 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d309081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d309086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d30908c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564d3069cba0 con 0x564d307814a0 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564d307db150 con 0x564d3086b160 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564d3090b1d0 con 0x564d308fe6a0 2024-05-09T01:36:09.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ad243cb80 10 client.0 _ll_register_callbacks cb 0x564d306926c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac17c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d309081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac17c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d309081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40636/0 (socket says 172.21.15.19:40636) 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac17c2700 1 -- 172.21.15.19:0/336778785 learned_addr learned my addr 172.21.15.19:0/336778785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac0fc1700 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d309086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d30908c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac17c2700 1 -- 172.21.15.19:0/336778785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 msgr2=0x564d30908c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d30908c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac17c2700 1 -- 172.21.15.19:0/336778785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 msgr2=0x564d309086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d309086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac17c2700 1 -- 172.21.15.19:0/336778785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d3069c490 con 0x564d307814a0 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d30908c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:09.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d309081b0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80a9110 tx=0x7f9ab80094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee8bf64e93e7ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9aa4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.772+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/336778785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80b1b50 con 0x564d307814a0 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/336778785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9ab80b1cf0 con 0x564d307814a0 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9ad243cb80 1 -- 172.21.15.19:0/336778785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab80b1fe0 con 0x564d307814a0 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/336778785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ab80a83e0 con 0x564d307814a0 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9ad243cb80 10 client.18270 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9ad243cb80 1 -- 172.21.15.19:0/336778785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564d30915b00 con 0x564d307814a0 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/336778785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab80b1fe0 con 0x564d307814a0 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/336778785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9ab8004ad0 con 0x564d307814a0 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9ad243cb80 10 client.18270 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9ad243cb80 10 client.18270 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9ad243cb80 20 client.18270 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x564d30917fd0 0x564d3091a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9ad243cb80 4 client.18270 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.773+0000 7f9ad243cb80 1 -- 172.21.15.19:0/336778785 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d3082e9d0 con 0x564d30917fd0 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.774+0000 7f9ac0fc1700 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x564d30917fd0 0x564d3091a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.774+0000 7f9ac0fc1700 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x564d30917fd0 0x564d3091a3f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c280ad7e1206b71 server_cookie=29cfc101e5d27ec4 in_seq=0 out_seq=0 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.774+0000 7f9aa4ff9700 10 client.18270 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/336778785 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d3082e9d0 con 0x564d30917fd0 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9aa4ff9700 10 client.18270 handle_command_reply: tid=0 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 client.18270 shutdown 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 -- 172.21.15.19:0/336778785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 msgr2=0x564d309081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d309081b0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80a9110 tx=0x7f9ab80094d0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 -- 172.21.15.19:0/336778785 shutdown_connections 2024-05-09T01:36:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x564d30917fd0 0x564d3091a3f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d30908c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d309081b0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/336778785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d309086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 -- 172.21.15.19:0/336778785 >> 172.21.15.19:0/336778785 conn(0x564d307e16a0 msgr2=0x564d308f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9aa4ff9700 0 client.18270 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:36:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 -- 172.21.15.19:0/336778785 shutdown_connections 2024-05-09T01:36:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 1 -- 172.21.15.19:0/336778785 wait complete. 2024-05-09T01:36:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.775+0000 7f9ad243cb80 20 client.18270 trim_cache size 0 max 16384 2024-05-09T01:36:09.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.790+0000 7f9ad243cb80 1 Processor -- start 2024-05-09T01:36:09.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.790+0000 7f9ad243cb80 1 -- start start 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d30914270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d30902510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d3091d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1b0fd0 con 0x564d307814a0 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1b2b10 con 0x564d3086b160 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1b16a0 con 0x564d308fe6a0 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac17c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d30914270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac17c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d30914270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40650/0 (socket says 172.21.15.19:40650) 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1224817767 learned_addr learned my addr 172.21.15.19:0/1224817767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d3091d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac0fc1700 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d30902510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1224817767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 msgr2=0x564d3091d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d3091d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1224817767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 msgr2=0x564d30902510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:09.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d30902510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac17c2700 1 -- 172.21.15.19:0/1224817767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d30906740 con 0x564d307814a0 2024-05-09T01:36:09.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac0fc1700 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d30902510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:09.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d3091d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:09.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d30914270 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8015dc0 tx=0x7f9ab8015e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c8ede51fd75f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:09.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.791+0000 7f9aa6ffd700 1 -- 172.21.15.19:0/1224817767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80a8f50 con 0x564d307814a0 2024-05-09T01:36:09.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9aa6ffd700 1 -- 172.21.15.19:0/1224817767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab80c8070 con 0x564d307814a0 2024-05-09T01:36:09.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9aa6ffd700 1 -- 172.21.15.19:0/1224817767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8004ad0 con 0x564d307814a0 2024-05-09T01:36:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1224817767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 msgr2=0x564d30914270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d30914270 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8015dc0 tx=0x7f9ab8015e10 comp rx=0 tx=0).stop 2024-05-09T01:36:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1224817767 shutdown_connections 2024-05-09T01:36:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d308fe6a0 0x564d3091d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d307814a0 0x564d30914270 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1224817767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d30902510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1224817767 >> 172.21.15.19:0/1224817767 conn(0x564d307e16a0 msgr2=0x564d30901e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:09.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1224817767 shutdown_connections 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.792+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1224817767 wait complete. 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ad243cb80 1 -- start start 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ad243cb80 1 Processor -- start 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d307814a0 0x564d30913260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d309137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ad243cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d308fe6a0 0x564d30913ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1b2440 con 0x564d308fe6a0 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1b1d70 con 0x564d3086b160 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ad243cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1aef90 con 0x564d307814a0 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ac1fc3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d308fe6a0 0x564d30913ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ac1fc3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d308fe6a0 0x564d30913ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40652/0 (socket says 172.21.15.19:40652) 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ad243cb80 10 client.0 _ll_register_callbacks cb 0x564d306926c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ac1fc3700 1 -- 172.21.15.19:0/1377325040 learned_addr learned my addr 172.21.15.19:0/1377325040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.793+0000 7f9ac0fc1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d309137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d307814a0 0x564d30913260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ac1fc3700 1 -- 172.21.15.19:0/1377325040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d307814a0 msgr2=0x564d30913260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d307814a0 0x564d30913260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ac1fc3700 1 -- 172.21.15.19:0/1377325040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 msgr2=0x564d309137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d309137a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ac1fc3700 1 -- 172.21.15.19:0/1377325040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d30980510 con 0x564d308fe6a0 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ac1fc3700 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d308fe6a0 0x564d30913ce0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098e0 tx=0x7f9ab40036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2f68e01c61b09c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9aa4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/1377325040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4027060 con 0x564d308fe6a0 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1377325040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab80c8070 con 0x564d308fe6a0 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/1377325040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9ab40052b0 con 0x564d308fe6a0 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/1377325040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ab4009590 con 0x564d308fe6a0 2024-05-09T01:36:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ad243cb80 10 client.18276 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.794+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1377325040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564d30981580 con 0x564d308fe6a0 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.795+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/1377325040 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab80c8070 con 0x564d308fe6a0 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.795+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/1377325040 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9ab4021070 con 0x564d308fe6a0 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.795+0000 7f9ad243cb80 10 client.18276 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.795+0000 7f9ad243cb80 10 client.18276 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.795+0000 7f9ad243cb80 20 client.18276 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.795+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x564d30981c90 0x564d309840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.795+0000 7f9ad243cb80 4 client.18276 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.795+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1377325040 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d30828a70 con 0x564d30981c90 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.795+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x564d30981c90 0x564d309840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9aa4ff9700 10 client.18276 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ac17c2700 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x564d30981c90 0x564d309840b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cc5ed24c990951bc server_cookie=5252c486a67efd6b in_seq=0 out_seq=0 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9aa4ff9700 1 -- 172.21.15.19:0/1377325040 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d30828a70 con 0x564d30981c90 2024-05-09T01:36:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9aa4ff9700 10 client.18276 handle_command_reply: tid=0 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 client.18276 shutdown 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1377325040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d308fe6a0 msgr2=0x564d30913ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d308fe6a0 0x564d30913ce0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098e0 tx=0x7f9ab40036a0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1377325040 shutdown_connections 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x564d30981c90 0x564d309840b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564d308fe6a0 0x564d30913ce0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d307814a0 0x564d30913260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 --2- 172.21.15.19:0/1377325040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d3086b160 0x564d309137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1377325040 >> 172.21.15.19:0/1377325040 conn(0x564d307e16a0 msgr2=0x564d309031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9aa4ff9700 0 client.18276 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1377325040 shutdown_connections 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 1 -- 172.21.15.19:0/1377325040 wait complete. 2024-05-09T01:36:09.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.796+0000 7f9ad243cb80 20 client.18276 trim_cache size 0 max 16384 2024-05-09T01:36:09.799 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 -- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9aa80b2880 msgr2=0x7f9aa80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9aa80b2880 0x7f9aa80b4d90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9aac005950 tx=0x7f9aac0058e0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 -- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc1034c0 msgr2=0x7f9abc19f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc1034c0 0x7f9abc19f8b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80060b0 tx=0x7f9ab8007f90 comp rx=0 tx=0).stop 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 -- 172.21.15.19:0/1715724930 shutdown_connections 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9abc108740 0x7f9abc199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9aa80b2880 0x7f9aa80b4d90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9abc1034c0 0x7f9abc19f8b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 --2- 172.21.15.19:0/1715724930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc107df0 0x7f9abc19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 -- 172.21.15.19:0/1715724930 >> 172.21.15.19:0/1715724930 conn(0x7f9abc0fedf0 msgr2=0x7f9abc107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 -- 172.21.15.19:0/1715724930 shutdown_connections 2024-05-09T01:36:09.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:09.797+0000 7f9ac3a26700 1 -- 172.21.15.19:0/1715724930 wait complete. 2024-05-09T01:36:10.550 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:36:10.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:10 smithi196 ceph-mon[35041]: pgmap v697: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:36:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:10.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:10 smithi059 ceph-mon[34351]: pgmap v697: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:36:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:10 smithi019 ceph-mon[30158]: pgmap v697: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:36:12.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.428+0000 7fda34adf700 1 -- 172.21.15.19:0/849823320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30107df0 msgr2=0x7fda30108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.428+0000 7fda34adf700 1 --2- 172.21.15.19:0/849823320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30107df0 0x7fda30108200 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fda240099c0 tx=0x7fda24009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.428+0000 7fda34adf700 1 -- 172.21.15.19:0/849823320 shutdown_connections 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.428+0000 7fda34adf700 1 --2- 172.21.15.19:0/849823320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda30108740 0x7fda301100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.428+0000 7fda34adf700 1 --2- 172.21.15.19:0/849823320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda301034c0 0x7fda301038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.428+0000 7fda34adf700 1 --2- 172.21.15.19:0/849823320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30107df0 0x7fda30108200 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.428+0000 7fda34adf700 1 -- 172.21.15.19:0/849823320 >> 172.21.15.19:0/849823320 conn(0x7fda300fedf0 msgr2=0x7fda30101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.428+0000 7fda34adf700 1 -- 172.21.15.19:0/849823320 shutdown_connections 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.428+0000 7fda34adf700 1 -- 172.21.15.19:0/849823320 wait complete. 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.429+0000 7fda34adf700 1 Processor -- start 2024-05-09T01:36:12.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.429+0000 7fda34adf700 1 -- start start 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.429+0000 7fda34adf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda301034c0 0x7fda3019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.429+0000 7fda34adf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda30107df0 0x7fda3019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.429+0000 7fda34adf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30108740 0x7fda3019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.429+0000 7fda34adf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda30113350 con 0x7fda30108740 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.429+0000 7fda34adf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda301131d0 con 0x7fda30107df0 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.429+0000 7fda34adf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda301134d0 con 0x7fda301034c0 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30108740 0x7fda3019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30108740 0x7fda3019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40670/0 (socket says 172.21.15.19:40670) 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2ed9d700 1 -- 172.21.15.19:0/490089212 learned_addr learned my addr 172.21.15.19:0/490089212 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2e59c700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda301034c0 0x7fda3019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda30107df0 0x7fda3019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2ed9d700 1 -- 172.21.15.19:0/490089212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda301034c0 msgr2=0x7fda3019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2ed9d700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda301034c0 0x7fda3019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2ed9d700 1 -- 172.21.15.19:0/490089212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda30107df0 msgr2=0x7fda3019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2ed9d700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda30107df0 0x7fda3019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2ed9d700 1 -- 172.21.15.19:0/490089212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3019fd20 con 0x7fda30108740 2024-05-09T01:36:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda2ed9d700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30108740 0x7fda3019f680 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fda28007ee0 tx=0x7fda2800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda1f7fe700 1 -- 172.21.15.19:0/490089212 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28004bb0 con 0x7fda30108740 2024-05-09T01:36:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda1f7fe700 1 -- 172.21.15.19:0/490089212 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda28004d50 con 0x7fda30108740 2024-05-09T01:36:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda34adf700 1 -- 172.21.15.19:0/490089212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda301a0010 con 0x7fda30108740 2024-05-09T01:36:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.430+0000 7fda34adf700 1 -- 172.21.15.19:0/490089212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda301a7930 con 0x7fda30108740 2024-05-09T01:36:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.431+0000 7fda1f7fe700 1 -- 172.21.15.19:0/490089212 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28005920 con 0x7fda30108740 2024-05-09T01:36:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.433+0000 7fda1f7fe700 1 -- 172.21.15.19:0/490089212 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fda2801a020 con 0x7fda30108740 2024-05-09T01:36:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.434+0000 7fda434f5b80 1 Processor -- start 2024-05-09T01:36:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.434+0000 7fda434f5b80 1 -- start start 2024-05-09T01:36:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda1f7fe700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda180b2880 0x7fda180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e757051d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e7570522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 0x55e757056810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda180b2880 0x7fda180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda434f5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e756f6fff0 con 0x55e756ed04a0 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda434f5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e756f0e7f0 con 0x55e757056400 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda434f5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e756f6bfb0 con 0x55e756fba160 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda1f7fe700 1 -- 172.21.15.19:0/490089212 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fda280e18d0 con 0x7fda30108740 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e757051d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e7570522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e757051d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55816/0 (socket says 172.21.15.19:55816) 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2e59c700 1 -- 172.21.15.19:0/1126520817 learned_addr learned my addr 172.21.15.19:0/1126520817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2ed9d700 1 --2- 172.21.15.19:0/1126520817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 0x55e757056810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2e59c700 1 -- 172.21.15.19:0/1126520817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 msgr2=0x55e757056810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2e59c700 1 --2- 172.21.15.19:0/1126520817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 0x55e757056810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2e59c700 1 -- 172.21.15.19:0/1126520817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 msgr2=0x55e7570522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2e59c700 1 --2- 172.21.15.19:0/1126520817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e7570522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.435+0000 7fda2e59c700 1 -- 172.21.15.19:0/1126520817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e756f06370 con 0x55e756fba160 2024-05-09T01:36:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.436+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda180b2880 0x7fda180b4d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fda24006010 tx=0x7fda24003a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.436+0000 7fda2e59c700 1 --2- 172.21.15.19:0/1126520817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e757051d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda2000acb0 tx=0x7fda2000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fec68d033fa66c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.440+0000 7fda1cff9700 1 -- 172.21.15.19:0/1126520817 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2001d070 con 0x55e756fba160 2024-05-09T01:36:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.440+0000 7fda1cff9700 1 -- 172.21.15.19:0/1126520817 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2000fdd0 con 0x55e756fba160 2024-05-09T01:36:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.440+0000 7fda1cff9700 1 -- 172.21.15.19:0/1126520817 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200107f0 con 0x55e756fba160 2024-05-09T01:36:12.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.440+0000 7fda434f5b80 1 -- 172.21.15.19:0/1126520817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 msgr2=0x55e757051d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:12.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.440+0000 7fda434f5b80 1 --2- 172.21.15.19:0/1126520817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e757051d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda2000acb0 tx=0x7fda2000f3c0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.441+0000 7fda434f5b80 1 -- 172.21.15.19:0/1126520817 shutdown_connections 2024-05-09T01:36:12.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.441+0000 7fda434f5b80 1 --2- 172.21.15.19:0/1126520817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 0x55e757056810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.441+0000 7fda434f5b80 1 --2- 172.21.15.19:0/1126520817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e757051d50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.441+0000 7fda434f5b80 1 --2- 172.21.15.19:0/1126520817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e7570522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.441+0000 7fda434f5b80 1 -- 172.21.15.19:0/1126520817 >> 172.21.15.19:0/1126520817 conn(0x55e756f306a0 msgr2=0x55e757048400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:12.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.441+0000 7fda434f5b80 1 -- 172.21.15.19:0/1126520817 shutdown_connections 2024-05-09T01:36:12.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.441+0000 7fda434f5b80 1 -- 172.21.15.19:0/1126520817 wait complete. 2024-05-09T01:36:12.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.441+0000 7fda434f5b80 1 -- start start 2024-05-09T01:36:12.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.441+0000 7fda434f5b80 1 Processor -- start 2024-05-09T01:36:12.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e75705b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e75705bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 0x55e75705c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda434f5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e756debba0 con 0x55e756ed04a0 2024-05-09T01:36:12.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda434f5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e756f2a150 con 0x55e757056400 2024-05-09T01:36:12.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda434f5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e7570571b0 con 0x55e756fba160 2024-05-09T01:36:12.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda434f5b80 10 client.0 _ll_register_callbacks cb 0x55e756de16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:12.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e75705b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e75705b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40680/0 (socket says 172.21.15.19:40680) 2024-05-09T01:36:12.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2e59c700 1 -- 172.21.15.19:0/3369146893 learned_addr learned my addr 172.21.15.19:0/3369146893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:12.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2ed9d700 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 0x55e75705c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e75705bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2e59c700 1 -- 172.21.15.19:0/3369146893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 msgr2=0x55e75705bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:12.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2e59c700 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e75705bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2e59c700 1 -- 172.21.15.19:0/3369146893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 msgr2=0x55e75705c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:12.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2e59c700 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 0x55e75705c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2e59c700 1 -- 172.21.15.19:0/3369146893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e756deb490 con 0x55e756ed04a0 2024-05-09T01:36:12.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda2e59c700 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e75705b5e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fda20019fd0 tx=0x7fda20005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e4ff66dae5e9f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:12.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:12.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda15ffb700 1 -- 172.21.15.19:0/3369146893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2001d070 con 0x55e756ed04a0 2024-05-09T01:36:12.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda15ffb700 1 -- 172.21.15.19:0/3369146893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fda2000fbd0 con 0x55e756ed04a0 2024-05-09T01:36:12.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.442+0000 7fda434f5b80 1 -- 172.21.15.19:0/3369146893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda2000fdd0 con 0x55e756ed04a0 2024-05-09T01:36:12.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda15ffb700 1 -- 172.21.15.19:0/3369146893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda20018650 con 0x55e756ed04a0 2024-05-09T01:36:12.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda434f5b80 10 client.18288 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:12.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda434f5b80 1 -- 172.21.15.19:0/3369146893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e757068ec0 con 0x55e756ed04a0 2024-05-09T01:36:12.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda15ffb700 1 -- 172.21.15.19:0/3369146893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2000fdd0 con 0x55e756ed04a0 2024-05-09T01:36:12.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda15ffb700 1 -- 172.21.15.19:0/3369146893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fda20022440 con 0x55e756ed04a0 2024-05-09T01:36:12.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda434f5b80 10 client.18288 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:12.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda434f5b80 10 client.18288 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:36:12.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda434f5b80 20 client.18288 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:12.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda434f5b80 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e75706b4a0 0x55e75706d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda434f5b80 4 client.18288 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:12.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda434f5b80 1 -- 172.21.15.19:0/3369146893 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e756f7d9d0 con 0x55e75706b4a0 2024-05-09T01:36:12.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.443+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e75706b4a0 0x55e75706d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.444+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e75706b4a0 0x55e75706d8c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e6d1f0f52700c4b8 server_cookie=c445cbb0736b8c56 in_seq=0 out_seq=0 2024-05-09T01:36:12.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.444+0000 7fda15ffb700 10 client.18288 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:36:12.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda15ffb700 1 -- 172.21.15.19:0/3369146893 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e756f7d9d0 con 0x55e75706b4a0 2024-05-09T01:36:12.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda15ffb700 10 client.18288 handle_command_reply: tid=0 2024-05-09T01:36:12.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 client.18288 shutdown 2024-05-09T01:36:12.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 -- 172.21.15.19:0/3369146893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 msgr2=0x55e75705b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:12.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e75705b5e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fda20019fd0 tx=0x7fda20005710 comp rx=0 tx=0).stop 2024-05-09T01:36:12.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 -- 172.21.15.19:0/3369146893 shutdown_connections 2024-05-09T01:36:12.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e75706b4a0 0x55e75706d8c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e757056400 0x55e75705c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e75705b5e0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda15ffb700 0 client.18288 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:36:12.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 --2- 172.21.15.19:0/3369146893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756fba160 0x55e75705bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 -- 172.21.15.19:0/3369146893 >> 172.21.15.19:0/3369146893 conn(0x55e756f306a0 msgr2=0x55e7570481c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:12.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 -- 172.21.15.19:0/3369146893 shutdown_connections 2024-05-09T01:36:12.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 1 -- 172.21.15.19:0/3369146893 wait complete. 2024-05-09T01:36:12.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.445+0000 7fda434f5b80 20 client.18288 trim_cache size 0 max 16384 2024-05-09T01:36:12.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.460+0000 7fda434f5b80 1 Processor -- start 2024-05-09T01:36:12.457 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:12 smithi019 ceph-mon[30158]: pgmap v698: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.460+0000 7fda434f5b80 1 -- start start 2024-05-09T01:36:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e757067790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 0x55e75705bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e757056400 0x55e75705a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda434f5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda301acae0 con 0x55e756ed04a0 2024-05-09T01:36:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda434f5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda301ae620 con 0x55e756fba160 2024-05-09T01:36:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda434f5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda301ad1b0 con 0x55e757056400 2024-05-09T01:36:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e757067790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e757067790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40684/0 (socket says 172.21.15.19:40684) 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2e59c700 1 -- 172.21.15.19:0/2783142235 learned_addr learned my addr 172.21.15.19:0/2783142235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/2783142235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 0x55e75705bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2ed9d700 1 --2- 172.21.15.19:0/2783142235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e757056400 0x55e75705a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2e59c700 1 -- 172.21.15.19:0/2783142235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e757056400 msgr2=0x55e75705a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2e59c700 1 --2- 172.21.15.19:0/2783142235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e757056400 0x55e75705a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2e59c700 1 -- 172.21.15.19:0/2783142235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 msgr2=0x55e75705bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2e59c700 1 --2- 172.21.15.19:0/2783142235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 0x55e75705bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2e59c700 1 -- 172.21.15.19:0/2783142235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e757059b50 con 0x55e756ed04a0 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda2e59c700 1 --2- 172.21.15.19:0/2783142235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e757067790 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fda2000a420 tx=0x7fda20005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27f94d6620d8dca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.461+0000 7fda17fff700 1 -- 172.21.15.19:0/2783142235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2001d070 con 0x55e756ed04a0 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda17fff700 1 -- 172.21.15.19:0/2783142235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2001e040 con 0x55e756ed04a0 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda17fff700 1 -- 172.21.15.19:0/2783142235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2000fbd0 con 0x55e756ed04a0 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda434f5b80 1 -- 172.21.15.19:0/2783142235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 msgr2=0x55e757067790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:12.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda434f5b80 1 --2- 172.21.15.19:0/2783142235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e757067790 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fda2000a420 tx=0x7fda20005430 comp rx=0 tx=0).stop 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda434f5b80 1 -- 172.21.15.19:0/2783142235 shutdown_connections 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda434f5b80 1 --2- 172.21.15.19:0/2783142235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e757056400 0x55e75705a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda434f5b80 1 --2- 172.21.15.19:0/2783142235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e756ed04a0 0x55e757067790 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda434f5b80 1 --2- 172.21.15.19:0/2783142235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 0x55e75705bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda434f5b80 1 -- 172.21.15.19:0/2783142235 >> 172.21.15.19:0/2783142235 conn(0x55e756f306a0 msgr2=0x55e757059da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.462+0000 7fda434f5b80 1 -- 172.21.15.19:0/2783142235 shutdown_connections 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 1 -- 172.21.15.19:0/2783142235 wait complete. 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 1 -- start start 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 1 Processor -- start 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756ed04a0 0x55e75704f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 0x55e75704ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e757056400 0x55e757050450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda301adf50 con 0x55e757056400 2024-05-09T01:36:12.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda301ad880 con 0x55e756fba160 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda301aaa80 con 0x55e756ed04a0 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda434f5b80 10 client.0 _ll_register_callbacks cb 0x55e756de16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.463+0000 7fda2ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e757056400 0x55e757050450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e757056400 0x55e757050450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40692/0 (socket says 172.21.15.19:40692) 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2ed9d700 1 -- 172.21.15.19:0/4085679185 learned_addr learned my addr 172.21.15.19:0/4085679185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 0x55e75704ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2e59c700 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756ed04a0 0x55e75704f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2dd9b700 1 -- 172.21.15.19:0/4085679185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756ed04a0 msgr2=0x55e75704f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756ed04a0 0x55e75704f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2dd9b700 1 -- 172.21.15.19:0/4085679185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e757056400 msgr2=0x55e757050450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e757056400 0x55e757050450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2dd9b700 1 -- 172.21.15.19:0/4085679185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e75706f8f0 con 0x55e756fba160 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2ed9d700 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e757056400 0x55e757050450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda2dd9b700 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 0x55e75704ff10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda2400b580 tx=0x7fda2401a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b447ee37ebb44a4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda15ffb700 1 -- 172.21.15.19:0/4085679185 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240056a0 con 0x55e756fba160 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda434f5b80 1 -- 172.21.15.19:0/4085679185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda2001e040 con 0x55e756fba160 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.464+0000 7fda15ffb700 1 -- 172.21.15.19:0/4085679185 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fda24035c70 con 0x55e756fba160 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda15ffb700 1 -- 172.21.15.19:0/4085679185 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda2400bbf0 con 0x55e756fba160 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda434f5b80 10 client.27617 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda434f5b80 1 -- 172.21.15.19:0/4085679185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e757070bb0 con 0x55e756fba160 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda15ffb700 1 -- 172.21.15.19:0/4085679185 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2001e040 con 0x55e756fba160 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda15ffb700 1 -- 172.21.15.19:0/4085679185 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fda2400f460 con 0x55e756fba160 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda434f5b80 10 client.27617 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda434f5b80 10 client.27617 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda434f5b80 20 client.27617 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda434f5b80 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e7570da1d0 0x55e7570dc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda434f5b80 4 client.27617 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:12.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.465+0000 7fda434f5b80 1 -- 172.21.15.19:0/4085679185 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e756f77a70 con 0x55e7570da1d0 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda2e59c700 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e7570da1d0 0x55e7570dc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda2e59c700 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e7570da1d0 0x55e7570dc5f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de72fd7724f4dfb6 server_cookie=47141c745660e509 in_seq=0 out_seq=0 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda15ffb700 10 client.27617 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda15ffb700 1 -- 172.21.15.19:0/4085679185 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e756f77a70 con 0x55e7570da1d0 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda15ffb700 10 client.27617 handle_command_reply: tid=0 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda434f5b80 1 client.27617 shutdown 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda434f5b80 1 -- 172.21.15.19:0/4085679185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 msgr2=0x55e75704ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda434f5b80 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 0x55e75704ff10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda2400b580 tx=0x7fda2401a050 comp rx=0 tx=0).stop 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda434f5b80 1 -- 172.21.15.19:0/4085679185 shutdown_connections 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda434f5b80 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e7570da1d0 0x55e7570dc5f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda434f5b80 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e757056400 0x55e757050450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda434f5b80 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e756ed04a0 0x55e75704f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda15ffb700 0 client.27617 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:36:12.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda434f5b80 1 --2- 172.21.15.19:0/4085679185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e756fba160 0x55e75704ff10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.466+0000 7fda434f5b80 1 -- 172.21.15.19:0/4085679185 >> 172.21.15.19:0/4085679185 conn(0x55e756f306a0 msgr2=0x55e75705bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:12.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda434f5b80 1 -- 172.21.15.19:0/4085679185 shutdown_connections 2024-05-09T01:36:12.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda434f5b80 1 -- 172.21.15.19:0/4085679185 wait complete. 2024-05-09T01:36:12.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda434f5b80 20 client.27617 trim_cache size 0 max 16384 2024-05-09T01:36:12.470 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:12.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 -- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda180b2880 msgr2=0x7fda180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:12.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda180b2880 0x7fda180b4d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fda24006010 tx=0x7fda24003a60 comp rx=0 tx=0).stop 2024-05-09T01:36:12.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 -- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30108740 msgr2=0x7fda3019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:12.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30108740 0x7fda3019f680 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fda28007ee0 tx=0x7fda2800d3b0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 -- 172.21.15.19:0/490089212 shutdown_connections 2024-05-09T01:36:12.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fda180b2880 0x7fda180b4d90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda30108740 0x7fda3019f680 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda301034c0 0x7fda3019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 --2- 172.21.15.19:0/490089212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda30107df0 0x7fda3019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:12.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 -- 172.21.15.19:0/490089212 >> 172.21.15.19:0/490089212 conn(0x7fda300fedf0 msgr2=0x7fda30075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:12.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 -- 172.21.15.19:0/490089212 shutdown_connections 2024-05-09T01:36:12.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:12.467+0000 7fda34adf700 1 -- 172.21.15.19:0/490089212 wait complete. 2024-05-09T01:36:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:12 smithi196 ceph-mon[35041]: pgmap v698: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:12 smithi059 ceph-mon[34351]: pgmap v698: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:13.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:36:13.415 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.031+0000 7f2b5bf86640 1 -- 172.21.15.19:0/413539586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 msgr2=0x7f2b5410aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.031+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/413539586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 0x7f2b5410aea0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009ca0 tx=0x7f2b4c02a8c0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.031+0000 7f2b5bf86640 1 -- 172.21.15.19:0/413539586 shutdown_connections 2024-05-09T01:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.031+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/413539586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 0x7f2b5410aea0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.031+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/413539586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b54103d80 0x7f2b54104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.031+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/413539586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b54102b40 0x7f2b54103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.031+0000 7f2b5bf86640 1 -- 172.21.15.19:0/413539586 >> 172.21.15.19:0/413539586 conn(0x7f2b540fdd30 msgr2=0x7f2b541001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.031+0000 7f2b5bf86640 1 -- 172.21.15.19:0/413539586 shutdown_connections 2024-05-09T01:36:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.031+0000 7f2b5bf86640 1 -- 172.21.15.19:0/413539586 wait complete. 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5bf86640 1 Processor -- start 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5bf86640 1 -- start start 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5bf86640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b54102b40 0x7f2b540759d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5bf86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b54103d80 0x7f2b5419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5bf86640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 0x7f2b541a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5bf86640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5410e670 con 0x7f2b54108a80 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5bf86640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5410e4f0 con 0x7f2b54102b40 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5bf86640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5410e7f0 con 0x7f2b54103d80 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5a523640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 0x7f2b541a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5a523640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 0x7f2b541a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40720/0 (socket says 172.21.15.19:40720) 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5a523640 1 -- 172.21.15.19:0/4032210652 learned_addr learned my addr 172.21.15.19:0/4032210652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b59d22640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b54102b40 0x7f2b540759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b59521640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b54103d80 0x7f2b5419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5a523640 1 -- 172.21.15.19:0/4032210652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b54103d80 msgr2=0x7f2b5419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5a523640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b54103d80 0x7f2b5419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5a523640 1 -- 172.21.15.19:0/4032210652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b54102b40 msgr2=0x7f2b540759d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5a523640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b54102b40 0x7f2b540759d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.032+0000 7f2b5a523640 1 -- 172.21.15.19:0/4032210652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b541a1e30 con 0x7f2b54108a80 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.033+0000 7f2b5a523640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 0x7f2b541a1610 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0097b0 tx=0x7f2b4c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.033+0000 7f2b4affd640 1 -- 172.21.15.19:0/4032210652 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c004240 con 0x7f2b54108a80 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.033+0000 7f2b5bf86640 1 -- 172.21.15.19:0/4032210652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b541a20c0 con 0x7f2b54108a80 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.033+0000 7f2b4affd640 1 -- 172.21.15.19:0/4032210652 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b4c02f380 con 0x7f2b54108a80 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.033+0000 7f2b4affd640 1 -- 172.21.15.19:0/4032210652 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c02fc30 con 0x7f2b54108a80 2024-05-09T01:36:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.033+0000 7f2b5bf86640 1 -- 172.21.15.19:0/4032210652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b541a2660 con 0x7f2b54108a80 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b62e41740 1 Processor -- start 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b4affd640 1 -- 172.21.15.19:0/4032210652 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2b4c0043e0 con 0x7f2b54108a80 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b62e41740 1 -- start start 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f69500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435b9c2f0 0x55a435f69a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435ba44d0 0x55a435f70980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b62e41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a435ecd7c0 con 0x55a435bc51b0 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b62e41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a435e690e0 con 0x55a435b9c2f0 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b62e41740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a435ec9780 con 0x55a435ba44d0 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b59521640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f69500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.036+0000 7f2b59521640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f69500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40728/0 (socket says 172.21.15.19:40728) 2024-05-09T01:36:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b59521640 1 -- 172.21.15.19:0/3975989326 learned_addr learned my addr 172.21.15.19:0/3975989326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b59d22640 1 --2- 172.21.15.19:0/3975989326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435b9c2f0 0x55a435f69a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b59521640 1 --2- 172.21.15.19:0/3975989326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435ba44d0 0x55a435f70980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b4affd640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b400b27e0 0x7f2b400b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b59521640 1 -- 172.21.15.19:0/3975989326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435ba44d0 msgr2=0x55a435f70980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b59521640 1 --2- 172.21.15.19:0/3975989326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435ba44d0 0x55a435f70980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b59521640 1 -- 172.21.15.19:0/3975989326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435b9c2f0 msgr2=0x55a435f69a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b59521640 1 --2- 172.21.15.19:0/3975989326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435b9c2f0 0x55a435f69a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b59521640 1 -- 172.21.15.19:0/3975989326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a435e927c0 con 0x55a435bc51b0 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b5a523640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b400b27e0 0x7f2b400b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b59521640 1 --2- 172.21.15.19:0/3975989326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f69500 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400bf90 tx=0x7f2b4400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f44a874cc0d449b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b4affd640 1 -- 172.21.15.19:0/4032210652 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2b4c1039b0 con 0x7f2b54108a80 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b3ffff640 1 -- 172.21.15.19:0/3975989326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44019030 con 0x55a435bc51b0 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b3ffff640 1 -- 172.21.15.19:0/3975989326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b440054d0 con 0x55a435bc51b0 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.037+0000 7f2b3ffff640 1 -- 172.21.15.19:0/3975989326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44004de0 con 0x55a435bc51b0 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b62e41740 1 -- 172.21.15.19:0/3975989326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 msgr2=0x55a435f69500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b62e41740 1 --2- 172.21.15.19:0/3975989326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f69500 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400bf90 tx=0x7f2b4400b610 comp rx=0 tx=0).stop 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b5a523640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b400b27e0 0x7f2b400b4cf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0c8190 tx=0x7f2b4c095740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b62e41740 1 -- 172.21.15.19:0/3975989326 shutdown_connections 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b62e41740 1 --2- 172.21.15.19:0/3975989326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435ba44d0 0x55a435f70980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b62e41740 1 --2- 172.21.15.19:0/3975989326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435b9c2f0 0x55a435f69a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b62e41740 1 --2- 172.21.15.19:0/3975989326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f69500 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b62e41740 1 -- 172.21.15.19:0/3975989326 >> 172.21.15.19:0/3975989326 conn(0x55a435bb9300 msgr2=0x55a435f64e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b62e41740 1 -- 172.21.15.19:0/3975989326 shutdown_connections 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.038+0000 7f2b62e41740 1 -- 172.21.15.19:0/3975989326 wait complete. 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b62e41740 1 -- start start 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b62e41740 1 Processor -- start 2024-05-09T01:36:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f73200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f73740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f7a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b62e41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a435f713d0 con 0x55a435bc51b0 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b62e41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a435e549c0 con 0x55a435ba44d0 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b62e41740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a435f71550 con 0x55a435b9c2f0 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b62e41740 10 client.0 _ll_register_callbacks cb 0x55a435e81020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b59521640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f7a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b59521640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f7a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40740/0 (socket says 172.21.15.19:40740) 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b59521640 1 -- 172.21.15.19:0/868838520 learned_addr learned my addr 172.21.15.19:0/868838520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b59d22640 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f73740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.039+0000 7f2b59521640 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f73200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b59d22640 1 -- 172.21.15.19:0/868838520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 msgr2=0x55a435f73200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b59d22640 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f73200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b59d22640 1 -- 172.21.15.19:0/868838520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 msgr2=0x55a435f7a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b59d22640 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f7a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b59d22640 1 -- 172.21.15.19:0/868838520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a435e7b1f0 con 0x55a435ba44d0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b59521640 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f7a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b59d22640 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f73740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55a435f6a8e0 tx=0x7f2b5000bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6af1c70ea5cbb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/868838520 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50003df0 con 0x55a435ba44d0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b62e41740 1 -- 172.21.15.19:0/868838520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b440054d0 con 0x55a435ba44d0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.040+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/868838520 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2b50011440 con 0x55a435ba44d0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/868838520 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b50011660 con 0x55a435ba44d0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b62e41740 10 client.27629 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b62e41740 1 -- 172.21.15.19:0/868838520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a435c30790 con 0x55a435ba44d0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/868838520 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b440054d0 con 0x55a435ba44d0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/868838520 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2b50024950 con 0x55a435ba44d0 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b62e41740 10 client.27629 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b62e41740 10 client.27629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b62e41740 20 client.27629 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b62e41740 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a435b83d80 0x55a435f80c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b62e41740 4 client.27629 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b62e41740 1 -- 172.21.15.19:0/868838520 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a435edb3d0 con 0x55a435b83d80 2024-05-09T01:36:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.041+0000 7f2b59521640 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a435b83d80 0x55a435f80c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.042+0000 7f2b59521640 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a435b83d80 0x55a435f80c30 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2603afdc600f13e2 server_cookie=b7706f828f92cc93 in_seq=0 out_seq=0 2024-05-09T01:36:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.042+0000 7f2b3d7fa640 10 client.27629 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/868838520 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a435edb3d0 con 0x55a435b83d80 2024-05-09T01:36:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b3d7fa640 10 client.27629 handle_command_reply: tid=0 2024-05-09T01:36:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 client.27629 shutdown 2024-05-09T01:36:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 -- 172.21.15.19:0/868838520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 msgr2=0x55a435f73740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f73740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55a435f6a8e0 tx=0x7f2b5000bed0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 -- 172.21.15.19:0/868838520 shutdown_connections 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a435b83d80 0x55a435f80c30 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435bc51b0 0x55a435f7a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f73740 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b3d7fa640 0 client.27629 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 --2- 172.21.15.19:0/868838520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f73200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 -- 172.21.15.19:0/868838520 >> 172.21.15.19:0/868838520 conn(0x55a435bb9300 msgr2=0x55a435f67da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 -- 172.21.15.19:0/868838520 shutdown_connections 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 1 -- 172.21.15.19:0/868838520 wait complete. 2024-05-09T01:36:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.043+0000 7f2b62e41740 20 client.27629 trim_cache size 0 max 16384 2024-05-09T01:36:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.058+0000 7f2b62e41740 1 Processor -- start 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.058+0000 7f2b62e41740 1 -- start start 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f6d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f6db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f7a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b62e41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b541ab450 con 0x55a435b83d80 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b62e41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b541acf90 con 0x55a435ba44d0 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b62e41740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b541abb20 con 0x55a435b9c2f0 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59521640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f6d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59521640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f6d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40750/0 (socket says 172.21.15.19:40750) 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59521640 1 -- 172.21.15.19:0/2709225239 learned_addr learned my addr 172.21.15.19:0/2709225239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59d22640 1 --2- 172.21.15.19:0/2709225239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f6db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59521640 1 -- 172.21.15.19:0/2709225239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 msgr2=0x55a435f6db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59521640 1 --2- 172.21.15.19:0/2709225239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f6db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59521640 1 -- 172.21.15.19:0/2709225239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 msgr2=0x55a435f7a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59521640 1 --2- 172.21.15.19:0/2709225239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f7a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59521640 1 -- 172.21.15.19:0/2709225239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a435ca75f0 con 0x55a435b83d80 2024-05-09T01:36:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b59521640 1 --2- 172.21.15.19:0/2709225239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f6d620 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400bea0 tx=0x7f2b4400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f77cf2de7d9126 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b3f7fe640 1 -- 172.21.15.19:0/2709225239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44019070 con 0x55a435b83d80 2024-05-09T01:36:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b3f7fe640 1 -- 172.21.15.19:0/2709225239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b4401a040 con 0x55a435b83d80 2024-05-09T01:36:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.059+0000 7f2b3f7fe640 1 -- 172.21.15.19:0/2709225239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b440114c0 con 0x55a435b83d80 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.060+0000 7f2b62e41740 1 -- 172.21.15.19:0/2709225239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 msgr2=0x55a435f6d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.060+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2709225239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f6d620 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400bea0 tx=0x7f2b4400b850 comp rx=0 tx=0).stop 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.060+0000 7f2b62e41740 1 -- 172.21.15.19:0/2709225239 shutdown_connections 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.060+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2709225239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f7a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.060+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2709225239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f6db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.060+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2709225239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f6d620 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.060+0000 7f2b62e41740 1 -- 172.21.15.19:0/2709225239 >> 172.21.15.19:0/2709225239 conn(0x55a435bb9300 msgr2=0x55a435f71920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.060+0000 7f2b62e41740 1 -- 172.21.15.19:0/2709225239 shutdown_connections 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.060+0000 7f2b62e41740 1 -- 172.21.15.19:0/2709225239 wait complete. 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b62e41740 1 -- start start 2024-05-09T01:36:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b62e41740 1 Processor -- start 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f7ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b62e41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f77d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b62e41740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b541ac8c0 con 0x55a435b83d80 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b62e41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b541ac1f0 con 0x55a435ba44d0 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b62e41740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b541a9430 con 0x55a435b9c2f0 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b62e41740 10 client.0 _ll_register_callbacks cb 0x7f2b500042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b59521640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f7ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b59521640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f7ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40764/0 (socket says 172.21.15.19:40764) 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b59521640 1 -- 172.21.15.19:0/2843855999 learned_addr learned my addr 172.21.15.19:0/2843855999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b59d22640 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b59521640 1 -- 172.21.15.19:0/2843855999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 msgr2=0x55a435f777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b59521640 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b59521640 1 -- 172.21.15.19:0/2843855999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 msgr2=0x55a435f77d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b59521640 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f77d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.061+0000 7f2b59521640 1 -- 172.21.15.19:0/2843855999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a435e91ed0 con 0x55a435b83d80 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b59521640 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f7ac60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2b44004730 tx=0x7f2b440053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bcb53f454d3f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/2843855999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44019030 con 0x55a435b83d80 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/2843855999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2b44029040 con 0x55a435b83d80 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b62e41740 1 -- 172.21.15.19:0/2843855999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b4401a040 con 0x55a435b83d80 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/2843855999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b44004960 con 0x55a435b83d80 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b62e41740 10 client.18312 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b62e41740 1 -- 172.21.15.19:0/2843855999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a435fe7f00 con 0x55a435b83d80 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/2843855999 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b4401a040 con 0x55a435b83d80 2024-05-09T01:36:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/2843855999 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2b4402c020 con 0x55a435b83d80 2024-05-09T01:36:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b62e41740 10 client.18312 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b62e41740 10 client.18312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b62e41740 20 client.18312 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a435bc51b0 0x55a435fea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b62e41740 4 client.18312 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:36:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.062+0000 7f2b62e41740 1 -- 172.21.15.19:0/2843855999 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a435ed5240 con 0x55a435bc51b0 2024-05-09T01:36:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.063+0000 7f2b59d22640 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a435bc51b0 0x55a435fea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.063+0000 7f2b59d22640 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a435bc51b0 0x55a435fea6c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b422d70be620136f server_cookie=bcfdf51815d731a2 in_seq=0 out_seq=0 2024-05-09T01:36:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.063+0000 7f2b3d7fa640 10 client.18312 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b3d7fa640 1 -- 172.21.15.19:0/2843855999 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a435ed5240 con 0x55a435bc51b0 2024-05-09T01:36:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b3d7fa640 10 client.18312 handle_command_reply: tid=0 2024-05-09T01:36:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 client.18312 shutdown 2024-05-09T01:36:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 -- 172.21.15.19:0/2843855999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 msgr2=0x55a435f7ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f7ac60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2b44004730 tx=0x7f2b440053f0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 -- 172.21.15.19:0/2843855999 shutdown_connections 2024-05-09T01:36:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55a435bc51b0 0x55a435fea6c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a435ba44d0 0x55a435f77d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a435b9c2f0 0x55a435f777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b3d7fa640 0 client.18312 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 --2- 172.21.15.19:0/2843855999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a435b83d80 0x55a435f7ac60 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 -- 172.21.15.19:0/2843855999 >> 172.21.15.19:0/2843855999 conn(0x55a435bb9300 msgr2=0x55a435f73110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 -- 172.21.15.19:0/2843855999 shutdown_connections 2024-05-09T01:36:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 1 -- 172.21.15.19:0/2843855999 wait complete. 2024-05-09T01:36:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.080+0000 7f2b62e41740 20 client.18312 trim_cache size 0 max 16384 2024-05-09T01:36:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:36:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:36:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:36:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 -- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b400b27e0 msgr2=0x7f2b400b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b400b27e0 0x7f2b400b4cf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0c8190 tx=0x7f2b4c095740 comp rx=0 tx=0).stop 2024-05-09T01:36:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 -- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 msgr2=0x7f2b541a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 0x7f2b541a1610 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0097b0 tx=0x7f2b4c003c20 comp rx=0 tx=0).stop 2024-05-09T01:36:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 -- 172.21.15.19:0/4032210652 shutdown_connections 2024-05-09T01:36:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2b400b27e0 0x7f2b400b4cf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b54108a80 0x7f2b541a1610 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b54103d80 0x7f2b5419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 --2- 172.21.15.19:0/4032210652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b54102b40 0x7f2b540759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 -- 172.21.15.19:0/4032210652 >> 172.21.15.19:0/4032210652 conn(0x7f2b540fdd30 msgr2=0x7f2b54104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 -- 172.21.15.19:0/4032210652 shutdown_connections 2024-05-09T01:36:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:14.081+0000 7f2b5bf86640 1 -- 172.21.15.19:0/4032210652 wait complete. 2024-05-09T01:36:14.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:14 smithi019 ceph-mon[30158]: pgmap v699: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:36:14.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:14 smithi196 ceph-mon[35041]: pgmap v699: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:36:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:14 smithi059 ceph-mon[34351]: pgmap v699: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:36:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.039+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/172430837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4107df0 msgr2=0x7fb1e4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.039+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/172430837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4107df0 0x7fb1e4108200 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80099c0 tx=0x7fb1d8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.040+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/172430837 shutdown_connections 2024-05-09T01:36:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.040+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/172430837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e4108740 0x7fb1e41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.040+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/172430837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1e41034c0 0x7fb1e41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.040+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/172430837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4107df0 0x7fb1e4108200 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.040+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/172430837 >> 172.21.15.19:0/172430837 conn(0x7fb1e40fedf0 msgr2=0x7fb1e4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.040+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/172430837 shutdown_connections 2024-05-09T01:36:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.040+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/172430837 wait complete. 2024-05-09T01:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e8bfe700 1 Processor -- start 2024-05-09T01:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e8bfe700 1 -- start start 2024-05-09T01:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e8bfe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e41034c0 0x7fb1e40ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e8bfe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1e4107df0 0x7fb1e40ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e8bfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4108740 0x7fb1e40ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e8bfe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4113350 con 0x7fb1e4108740 2024-05-09T01:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e8bfe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41131d0 con 0x7fb1e4107df0 2024-05-09T01:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e8bfe700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41134d0 con 0x7fb1e41034c0 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4108740 0x7fb1e40ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4108740 0x7fb1e40ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40776/0 (socket says 172.21.15.19:40776) 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e2d9d700 1 -- 172.21.15.19:0/842111963 learned_addr learned my addr 172.21.15.19:0/842111963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1e4107df0 0x7fb1e40ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e259c700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e41034c0 0x7fb1e40ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e2d9d700 1 -- 172.21.15.19:0/842111963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e41034c0 msgr2=0x7fb1e40ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e2d9d700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e41034c0 0x7fb1e40ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e2d9d700 1 -- 172.21.15.19:0/842111963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1e4107df0 msgr2=0x7fb1e40ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e2d9d700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1e4107df0 0x7fb1e40ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e2d9d700 1 -- 172.21.15.19:0/842111963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e41a9120 con 0x7fb1e4108740 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.041+0000 7fb1e259c700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e41034c0 0x7fb1e40ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.042+0000 7fb1e2d9d700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4108740 0x7fb1e40ffd50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00d980 tx=0x7fb1dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.042+0000 7fb1d37fe700 1 -- 172.21.15.19:0/842111963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc017070 con 0x7fb1e4108740 2024-05-09T01:36:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.042+0000 7fb1d37fe700 1 -- 172.21.15.19:0/842111963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1dc005020 con 0x7fb1e4108740 2024-05-09T01:36:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.042+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/842111963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e41a9410 con 0x7fb1e4108740 2024-05-09T01:36:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.042+0000 7fb1d37fe700 1 -- 172.21.15.19:0/842111963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0099b0 con 0x7fb1e4108740 2024-05-09T01:36:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.042+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/842111963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e41a98d0 con 0x7fb1e4108740 2024-05-09T01:36:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.045+0000 7fb1d37fe700 1 -- 172.21.15.19:0/842111963 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb1dc009b70 con 0x7fb1e4108740 2024-05-09T01:36:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.045+0000 7fb1d37fe700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1cc0b2880 0x7fb1cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.045+0000 7fb1e259c700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1cc0b2880 0x7fb1cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.045+0000 7fb1d37fe700 1 -- 172.21.15.19:0/842111963 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb1dc0e1fe0 con 0x7fb1e4108740 2024-05-09T01:36:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.046+0000 7fb1f7614b80 1 Processor -- start 2024-05-09T01:36:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.046+0000 7fb1e259c700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1cc0b2880 0x7fb1cc0b4d90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e300 tx=0x7fb1d4009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.046+0000 7fb1f7614b80 1 -- start start 2024-05-09T01:36:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.046+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7fc160 0x559b0f8954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.046+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7124a0 0x559b0f895a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.046+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f88fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.046+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559b0f7b1ff0 con 0x559b0f7124a0 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.046+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559b0f7507f0 con 0x559b0f7fc160 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.046+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b0f7adfb0 con 0x559b0f88f6a0 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7124a0 0x559b0f895a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7124a0 0x559b0f895a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40790/0 (socket says 172.21.15.19:40790) 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2441425683 learned_addr learned my addr 172.21.15.19:0/2441425683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e259c700 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7fc160 0x559b0f8954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e2d9d700 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f88fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2441425683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 msgr2=0x559b0f88fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f88fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2441425683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7fc160 msgr2=0x559b0f8954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7fc160 0x559b0f8954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2441425683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0f748370 con 0x559b0f7124a0 2024-05-09T01:36:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e259c700 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7fc160 0x559b0f8954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e2d9d700 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f88fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7124a0 0x559b0f895a30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80099c0 tx=0x7fb1d8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3228b7e855e1d975 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1d0ff9700 1 -- 172.21.15.19:0/2441425683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8003b90 con 0x559b0f7124a0 2024-05-09T01:36:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1d0ff9700 1 -- 172.21.15.19:0/2441425683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d8003d30 con 0x559b0f7124a0 2024-05-09T01:36:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.047+0000 7fb1d0ff9700 1 -- 172.21.15.19:0/2441425683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800bdc0 con 0x559b0f7124a0 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.048+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2441425683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7124a0 msgr2=0x559b0f895a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.048+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7124a0 0x559b0f895a30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80099c0 tx=0x7fb1d8009710 comp rx=0 tx=0).stop 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.048+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2441425683 shutdown_connections 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.048+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f88fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.048+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7fc160 0x559b0f8954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.048+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2441425683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7124a0 0x559b0f895a30 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.048+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2441425683 >> 172.21.15.19:0/2441425683 conn(0x559b0f7726a0 msgr2=0x559b0f88a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.048+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2441425683 shutdown_connections 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.048+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2441425683 wait complete. 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1f7614b80 1 -- start start 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1f7614b80 1 Processor -- start 2024-05-09T01:36:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f899200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f899740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f899c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559b0f62dba0 con 0x559b0f7fc160 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559b0f76c150 con 0x559b0f7124a0 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b0f8904e0 con 0x559b0f88f6a0 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1f7614b80 10 client.0 _ll_register_callbacks cb 0x559b0f6236c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f899740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f899740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40798/0 (socket says 172.21.15.19:40798) 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.049+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2710963674 learned_addr learned my addr 172.21.15.19:0/2710963674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1e259c700 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f899200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1e2d9d700 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f899c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2710963674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 msgr2=0x559b0f899c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f899c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2710963674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 msgr2=0x559b0f899200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f899200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2710963674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0f62d490 con 0x559b0f7fc160 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f899740 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80098e0 tx=0x7fb1d8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e11d4573101b47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2710963674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8030020 con 0x559b0f7fc160 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2710963674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb1d800b990 con 0x559b0f7fc160 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2710963674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d8003d30 con 0x559b0f7fc160 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2710963674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb1d803b050 con 0x559b0f7fc160 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1f7614b80 10 client.18324 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.050+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2710963674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b0f8a6b20 con 0x559b0f7fc160 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.051+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2710963674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d8003d30 con 0x559b0f7fc160 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.051+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2710963674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb1d802cc20 con 0x559b0f7fc160 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.051+0000 7fb1f7614b80 10 client.18324 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.051+0000 7fb1f7614b80 10 client.18324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:36:15.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.051+0000 7fb1f7614b80 20 client.18324 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:15.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.051+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x559b0f8a8ff0 0x559b0f8ab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.051+0000 7fb1f7614b80 4 client.18324 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:15.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.051+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2710963674 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b0f7bf9d0 con 0x559b0f8a8ff0 2024-05-09T01:36:15.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.051+0000 7fb1e259c700 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x559b0f8a8ff0 0x559b0f8ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.052+0000 7fb1c9ffb700 10 client.18324 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:36:15.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.052+0000 7fb1e259c700 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x559b0f8a8ff0 0x559b0f8ab410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b94467868ec28b9b server_cookie=8ce17aef9b1873c8 in_seq=0 out_seq=0 2024-05-09T01:36:15.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.052+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2710963674 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b0f7bf9d0 con 0x559b0f8a8ff0 2024-05-09T01:36:15.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.052+0000 7fb1c9ffb700 10 client.18324 handle_command_reply: tid=0 2024-05-09T01:36:15.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 client.18324 shutdown 2024-05-09T01:36:15.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2710963674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 msgr2=0x559b0f899740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f899740 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80098e0 tx=0x7fb1d8009310 comp rx=0 tx=0).stop 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2710963674 shutdown_connections 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f899c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x559b0f8a8ff0 0x559b0f8ab410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f899200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2710963674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f899740 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2710963674 >> 172.21.15.19:0/2710963674 conn(0x559b0f7726a0 msgr2=0x559b0f88a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1c9ffb700 0 client.18324 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2710963674 shutdown_connections 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2710963674 wait complete. 2024-05-09T01:36:15.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.053+0000 7fb1f7614b80 20 client.18324 trim_cache size 0 max 16384 2024-05-09T01:36:15.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1f7614b80 1 Processor -- start 2024-05-09T01:36:15.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1f7614b80 1 -- start start 2024-05-09T01:36:15.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f895860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f8ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41b0db0 con 0x559b0f7fc160 2024-05-09T01:36:15.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41b28f0 con 0x559b0f7124a0 2024-05-09T01:36:15.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41b1480 con 0x559b0f88f6a0 2024-05-09T01:36:15.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8a52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40814/0 (socket says 172.21.15.19:40814) 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.068+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/550163948 learned_addr learned my addr 172.21.15.19:0/550163948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1e2d9d700 1 --2- 172.21.15.19:0/550163948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f8ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1e259c700 1 --2- 172.21.15.19:0/550163948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f895860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/550163948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 msgr2=0x559b0f8ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/550163948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f8ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/550163948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 msgr2=0x559b0f895860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/550163948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f895860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/550163948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0f883f60 con 0x559b0f7fc160 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/550163948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8a52b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800f620 tx=0x7fb1d8005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c77872144b7deb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1cbfff700 1 -- 172.21.15.19:0/550163948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8030040 con 0x559b0f7fc160 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1cbfff700 1 -- 172.21.15.19:0/550163948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d800f8a0 con 0x559b0f7fc160 2024-05-09T01:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.069+0000 7fb1cbfff700 1 -- 172.21.15.19:0/550163948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d802c460 con 0x559b0f7fc160 2024-05-09T01:36:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 -- 172.21.15.19:0/550163948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 msgr2=0x559b0f8a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/550163948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8a52b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800f620 tx=0x7fb1d8005ac0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 -- 172.21.15.19:0/550163948 shutdown_connections 2024-05-09T01:36:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/550163948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f8ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/550163948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f895860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/550163948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8a52b0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 -- 172.21.15.19:0/550163948 >> 172.21.15.19:0/550163948 conn(0x559b0f7726a0 msgr2=0x559b0f897af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 -- 172.21.15.19:0/550163948 shutdown_connections 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 -- 172.21.15.19:0/550163948 wait complete. 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 -- start start 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.070+0000 7fb1f7614b80 1 Processor -- start 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f8ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1f7614b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f911530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41b2220 con 0x559b0f7fc160 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41b1b50 con 0x559b0f7124a0 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1f7614b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41aed50 con 0x559b0f88f6a0 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1f7614b80 10 client.0 _ll_register_callbacks cb 0x559b0f6236c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8ab540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40822/0 (socket says 172.21.15.19:40822) 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2946476311 learned_addr learned my addr 172.21.15.19:0/2946476311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e259c700 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f8ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e2d9d700 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f911530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2946476311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 msgr2=0x559b0f911530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f911530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2946476311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 msgr2=0x559b0f8ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f8ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.071+0000 7fb1e1d9b700 1 -- 172.21.15.19:0/2946476311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0f8abda0 con 0x559b0f7fc160 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1e1d9b700 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8ab540 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800b970 tx=0x7fb1d8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a359a3981a61dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2946476311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d804c060 con 0x559b0f7fc160 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2946476311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb1d800f6a0 con 0x559b0f7fc160 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2946476311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d800f8a0 con 0x559b0f7fc160 2024-05-09T01:36:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2946476311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb1d800bd40 con 0x559b0f7fc160 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1f7614b80 10 client.18330 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2946476311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b0f912a60 con 0x559b0f7fc160 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2946476311 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d800f8a0 con 0x559b0f7fc160 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2946476311 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb1d802cb20 con 0x559b0f7fc160 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1f7614b80 10 client.18330 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1f7614b80 10 client.18330 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1f7614b80 20 client.18330 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x559b0f912e60 0x559b0f915280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1f7614b80 4 client.18330 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.072+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2946476311 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b0f7b9a70 con 0x559b0f912e60 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.073+0000 7fb1e259c700 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x559b0f912e60 0x559b0f915280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.073+0000 7fb1e259c700 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x559b0f912e60 0x559b0f915280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d2c9251bdc3a7e0 server_cookie=d8f5a5fb6af51c3c in_seq=0 out_seq=0 2024-05-09T01:36:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.073+0000 7fb1c9ffb700 10 client.18330 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1c9ffb700 1 -- 172.21.15.19:0/2946476311 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b0f7b9a70 con 0x559b0f912e60 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1c9ffb700 10 client.18330 handle_command_reply: tid=0 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 client.18330 shutdown 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2946476311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 msgr2=0x559b0f8ab540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8ab540 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800b970 tx=0x7fb1d8009310 comp rx=0 tx=0).stop 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2946476311 shutdown_connections 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0f88f6a0 0x559b0f911530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x559b0f912e60 0x559b0f915280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b0f7124a0 0x559b0f8ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 --2- 172.21.15.19:0/2946476311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b0f7fc160 0x559b0f8ab540 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1c9ffb700 0 client.18330 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2946476311 >> 172.21.15.19:0/2946476311 conn(0x559b0f7726a0 msgr2=0x559b0f8942f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2946476311 shutdown_connections 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 1 -- 172.21.15.19:0/2946476311 wait complete. 2024-05-09T01:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1f7614b80 20 client.18330 trim_cache size 0 max 16384 2024-05-09T01:36:15.076 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1cc0b2880 msgr2=0x7fb1cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1cc0b2880 0x7fb1cc0b4d90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e300 tx=0x7fb1d4009040 comp rx=0 tx=0).stop 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4108740 msgr2=0x7fb1e40ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.074+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4108740 0x7fb1e40ffd50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00d980 tx=0x7fb1dc00dc90 comp rx=0 tx=0).stop 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.075+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/842111963 shutdown_connections 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.075+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb1cc0b2880 0x7fb1cc0b4d90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.075+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e4108740 0x7fb1e40ffd50 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.075+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e41034c0 0x7fb1e40ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.075+0000 7fb1e8bfe700 1 --2- 172.21.15.19:0/842111963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1e4107df0 0x7fb1e40ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.075+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/842111963 >> 172.21.15.19:0/842111963 conn(0x7fb1e40fedf0 msgr2=0x7fb1e4075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.075+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/842111963 shutdown_connections 2024-05-09T01:36:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:15.075+0000 7fb1e8bfe700 1 -- 172.21.15.19:0/842111963 wait complete. 2024-05-09T01:36:15.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:36:16.050 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:16.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:16 smithi196 ceph-mon[35041]: pgmap v700: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:16 smithi059 ceph-mon[34351]: pgmap v700: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:16.854 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:16 smithi019 ceph-mon[30158]: pgmap v700: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.681+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2664824867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0077790 msgr2=0x7f03d0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.681+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2664824867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0077790 0x7f03d0110640 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009990 tx=0x7f03c0009ca0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.681+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2664824867 shutdown_connections 2024-05-09T01:36:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.681+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2664824867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0110b80 0x7f03d0112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.681+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2664824867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0076e40 0x7f03d0077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.681+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2664824867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0077790 0x7f03d0110640 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.681+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2664824867 >> 172.21.15.19:0/2664824867 conn(0x7f03d00ffea0 msgr2=0x7f03d01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.681+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2664824867 shutdown_connections 2024-05-09T01:36:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.681+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2664824867 wait complete. 2024-05-09T01:36:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03d5f2a700 1 Processor -- start 2024-05-09T01:36:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03d5f2a700 1 -- start start 2024-05-09T01:36:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03d5f2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0076e40 0x7f03d019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03d5f2a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0077790 0x7f03d019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03d5f2a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0110b80 0x7f03d01a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03d5f2a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01166f0 con 0x7f03d0076e40 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03d5f2a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0116570 con 0x7f03d0077790 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03d5f2a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0116870 con 0x7f03d0110b80 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03cf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0076e40 0x7f03d019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03cf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0076e40 0x7f03d019d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40834/0 (socket says 172.21.15.19:40834) 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.682+0000 7f03cf7fe700 1 -- 172.21.15.19:0/2053113784 learned_addr learned my addr 172.21.15.19:0/2053113784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03cffff700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0110b80 0x7f03d01a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03ceffd700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0077790 0x7f03d019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03cf7fe700 1 -- 172.21.15.19:0/2053113784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0110b80 msgr2=0x7f03d01a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0110b80 0x7f03d01a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03cf7fe700 1 -- 172.21.15.19:0/2053113784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0077790 msgr2=0x7f03d019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0077790 0x7f03d019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03cf7fe700 1 -- 172.21.15.19:0/2053113784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d01a1ec0 con 0x7f03d0076e40 2024-05-09T01:36:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03ceffd700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0077790 0x7f03d019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0076e40 0x7f03d019d0a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f03b800b530 tx=0x7f03b800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03ccff9700 1 -- 172.21.15.19:0/2053113784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8015020 con 0x7f03d0076e40 2024-05-09T01:36:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2053113784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d01a21b0 con 0x7f03d0076e40 2024-05-09T01:36:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03ccff9700 1 -- 172.21.15.19:0/2053113784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b8011ce0 con 0x7f03d0076e40 2024-05-09T01:36:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03ccff9700 1 -- 172.21.15.19:0/2053113784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8010e40 con 0x7f03d0076e40 2024-05-09T01:36:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.683+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2053113784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d01a9b80 con 0x7f03d0076e40 2024-05-09T01:36:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.686+0000 7f03ccff9700 1 -- 172.21.15.19:0/2053113784 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f03b8018020 con 0x7f03d0076e40 2024-05-09T01:36:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03ccff9700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03bc0b2880 0x7f03bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03ccff9700 1 -- 172.21.15.19:0/2053113784 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03b80e1ae0 con 0x7f03d0076e40 2024-05-09T01:36:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03ceffd700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03bc0b2880 0x7f03bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03e4940b80 1 Processor -- start 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03e4940b80 1 -- start start 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03ceffd700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03bc0b2880 0x7f03bc0b4d90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009750 tx=0x7f03c001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 0x5599f742fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a44a0 0x5599f742ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f7429c10 0x5599f742a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03e4940b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599f73440a0 con 0x5599f72a44a0 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03e4940b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5599f72e99a0 con 0x5599f731ed00 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.687+0000 7f03e4940b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5599f7340060 con 0x5599f7429c10 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a44a0 0x5599f742ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03cf7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 0x5599f742fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a44a0 0x5599f742ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40846/0 (socket says 172.21.15.19:40846) 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03ceffd700 1 -- 172.21.15.19:0/4225200962 learned_addr learned my addr 172.21.15.19:0/4225200962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03cffff700 1 --2- 172.21.15.19:0/4225200962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f7429c10 0x5599f742a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03ceffd700 1 -- 172.21.15.19:0/4225200962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f7429c10 msgr2=0x5599f742a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03ceffd700 1 --2- 172.21.15.19:0/4225200962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f7429c10 0x5599f742a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03ceffd700 1 -- 172.21.15.19:0/4225200962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 msgr2=0x5599f742fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03ceffd700 1 --2- 172.21.15.19:0/4225200962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 0x5599f742fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03ceffd700 1 -- 172.21.15.19:0/4225200962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599f72da500 con 0x5599f72a44a0 2024-05-09T01:36:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03ceffd700 1 --2- 172.21.15.19:0/4225200962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a44a0 0x5599f742ff90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f03c0015000 tx=0x7f03c001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b122e3880e9996f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03c5ffb700 1 -- 172.21.15.19:0/4225200962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000fa70 con 0x5599f72a44a0 2024-05-09T01:36:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03c5ffb700 1 -- 172.21.15.19:0/4225200962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c0031070 con 0x5599f72a44a0 2024-05-09T01:36:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.688+0000 7f03c5ffb700 1 -- 172.21.15.19:0/4225200962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0018400 con 0x5599f72a44a0 2024-05-09T01:36:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.689+0000 7f03e4940b80 1 -- 172.21.15.19:0/4225200962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a44a0 msgr2=0x5599f742ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.689+0000 7f03e4940b80 1 --2- 172.21.15.19:0/4225200962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a44a0 0x5599f742ff90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f03c0015000 tx=0x7f03c001d000 comp rx=0 tx=0).stop 2024-05-09T01:36:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.689+0000 7f03e4940b80 1 -- 172.21.15.19:0/4225200962 shutdown_connections 2024-05-09T01:36:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.689+0000 7f03e4940b80 1 --2- 172.21.15.19:0/4225200962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f7429c10 0x5599f742a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.689+0000 7f03e4940b80 1 --2- 172.21.15.19:0/4225200962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 0x5599f742fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.689+0000 7f03e4940b80 1 --2- 172.21.15.19:0/4225200962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a44a0 0x5599f742ff90 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.689+0000 7f03e4940b80 1 -- 172.21.15.19:0/4225200962 >> 172.21.15.19:0/4225200962 conn(0x5599f72a7f70 msgr2=0x5599f741c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.689+0000 7f03e4940b80 1 -- 172.21.15.19:0/4225200962 shutdown_connections 2024-05-09T01:36:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.689+0000 7f03e4940b80 1 -- 172.21.15.19:0/4225200962 wait complete. 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.690+0000 7f03e4940b80 1 -- start start 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.690+0000 7f03e4940b80 1 Processor -- start 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.690+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 0x5599f7433800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.690+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f7433d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.690+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f7429c10 0x5599f7434280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.690+0000 7f03e4940b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599f742a910 con 0x5599f731ed00 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.690+0000 7f03e4940b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5599f72deb20 con 0x5599f7429c10 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03e4940b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5599f743fa10 con 0x5599f72a44a0 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f7433d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03e4940b80 10 client.0 _ll_register_callbacks cb 0x5599f71b56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f7433d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40856/0 (socket says 172.21.15.19:40856) 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03ceffd700 1 -- 172.21.15.19:0/2524453307 learned_addr learned my addr 172.21.15.19:0/2524453307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 0x5599f7433800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03cffff700 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f7429c10 0x5599f7434280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03ceffd700 1 -- 172.21.15.19:0/2524453307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 msgr2=0x5599f7433800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03ceffd700 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 0x5599f7433800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03ceffd700 1 -- 172.21.15.19:0/2524453307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f7429c10 msgr2=0x5599f7434280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03ceffd700 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f7429c10 0x5599f7434280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03ceffd700 1 -- 172.21.15.19:0/2524453307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599f71bf490 con 0x5599f731ed00 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03ceffd700 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f7433d40 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f03c0006010 tx=0x7f03c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed60edf437cc6a28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03aeffd700 1 -- 172.21.15.19:0/2524453307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000fd40 con 0x5599f731ed00 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03aeffd700 1 -- 172.21.15.19:0/2524453307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03c0018400 con 0x5599f731ed00 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.691+0000 7f03e4940b80 1 -- 172.21.15.19:0/2524453307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c0031070 con 0x5599f731ed00 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03aeffd700 1 -- 172.21.15.19:0/2524453307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03c0043070 con 0x5599f731ed00 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03e4940b80 10 client.18342 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03e4940b80 1 -- 172.21.15.19:0/2524453307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599f7441310 con 0x5599f731ed00 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03aeffd700 1 -- 172.21.15.19:0/2524453307 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c0031070 con 0x5599f731ed00 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03aeffd700 1 -- 172.21.15.19:0/2524453307 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f03c002c510 con 0x5599f731ed00 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03e4940b80 10 client.18342 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03e4940b80 10 client.18342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03e4940b80 20 client.18342 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03e4940b80 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5599f74437e0 0x5599f7445c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03e4940b80 4 client.18342 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03e4940b80 1 -- 172.21.15.19:0/2524453307 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599f7351a80 con 0x5599f74437e0 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.692+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5599f74437e0 0x5599f7445c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.693+0000 7f03aeffd700 10 client.18342 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.693+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5599f74437e0 0x5599f7445c00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29864ad3b91c8987 server_cookie=7df9745810a6b48c in_seq=0 out_seq=0 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03aeffd700 1 -- 172.21.15.19:0/2524453307 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599f7351a80 con 0x5599f74437e0 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03aeffd700 10 client.18342 handle_command_reply: tid=0 2024-05-09T01:36:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 client.18342 shutdown 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 -- 172.21.15.19:0/2524453307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 msgr2=0x5599f7433d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f7433d40 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f03c0006010 tx=0x7f03c000b8d0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 -- 172.21.15.19:0/2524453307 shutdown_connections 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5599f74437e0 0x5599f7445c00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f7429c10 0x5599f7434280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 0x5599f7433800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 --2- 172.21.15.19:0/2524453307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f7433d40 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 -- 172.21.15.19:0/2524453307 >> 172.21.15.19:0/2524453307 conn(0x5599f72a7f70 msgr2=0x5599f7415ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03aeffd700 0 client.18342 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:36:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 -- 172.21.15.19:0/2524453307 shutdown_connections 2024-05-09T01:36:17.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 1 -- 172.21.15.19:0/2524453307 wait complete. 2024-05-09T01:36:17.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.694+0000 7f03e4940b80 20 client.18342 trim_cache size 0 max 16384 2024-05-09T01:36:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.709+0000 7f03e4940b80 1 Processor -- start 2024-05-09T01:36:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.709+0000 7f03e4940b80 1 -- start start 2024-05-09T01:36:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a7f70 0x5599f7416a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 0x5599f743f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 0x5599f7448810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03e4940b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01aca40 con 0x5599f72a7f70 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03e4940b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01ae580 con 0x5599f731ed00 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03e4940b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01ad110 con 0x5599f72a44a0 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a7f70 0x5599f7416a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a7f70 0x5599f7416a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40858/0 (socket says 172.21.15.19:40858) 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cf7fe700 1 -- 172.21.15.19:0/670802976 learned_addr learned my addr 172.21.15.19:0/670802976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cf7fe700 1 -- 172.21.15.19:0/670802976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 msgr2=0x5599f743f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03ceffd700 1 --2- 172.21.15.19:0/670802976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 0x5599f743f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cffff700 1 --2- 172.21.15.19:0/670802976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 0x5599f7448810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/670802976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 0x5599f743f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cf7fe700 1 -- 172.21.15.19:0/670802976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 msgr2=0x5599f7448810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/670802976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 0x5599f7448810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cf7fe700 1 -- 172.21.15.19:0/670802976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599f74189d0 con 0x5599f72a7f70 2024-05-09T01:36:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.710+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/670802976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a7f70 0x5599f7416a20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f03b80a9070 tx=0x7f03b801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b604a120ff2437 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03c57fa700 1 -- 172.21.15.19:0/670802976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b80ad490 con 0x5599f72a7f70 2024-05-09T01:36:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03c57fa700 1 -- 172.21.15.19:0/670802976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b80b1f40 con 0x5599f72a7f70 2024-05-09T01:36:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03c57fa700 1 -- 172.21.15.19:0/670802976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b80b1d10 con 0x5599f72a7f70 2024-05-09T01:36:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03e4940b80 1 -- 172.21.15.19:0/670802976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a7f70 msgr2=0x5599f7416a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03e4940b80 1 --2- 172.21.15.19:0/670802976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a7f70 0x5599f7416a20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f03b80a9070 tx=0x7f03b801c040 comp rx=0 tx=0).stop 2024-05-09T01:36:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03e4940b80 1 -- 172.21.15.19:0/670802976 shutdown_connections 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03e4940b80 1 --2- 172.21.15.19:0/670802976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f731ed00 0x5599f7448810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03e4940b80 1 --2- 172.21.15.19:0/670802976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f72a7f70 0x5599f7416a20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03e4940b80 1 --2- 172.21.15.19:0/670802976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a44a0 0x5599f743f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03e4940b80 1 -- 172.21.15.19:0/670802976 >> 172.21.15.19:0/670802976 conn(0x5599f741c520 msgr2=0x5599f742e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03e4940b80 1 -- 172.21.15.19:0/670802976 shutdown_connections 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.711+0000 7f03e4940b80 1 -- 172.21.15.19:0/670802976 wait complete. 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.712+0000 7f03e4940b80 1 -- start start 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.712+0000 7f03e4940b80 1 Processor -- start 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.712+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f72a44a0 0x5599f7448140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.712+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a7f70 0x5599f74486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.712+0000 7f03e4940b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f74a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.712+0000 7f03e4940b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01adeb0 con 0x5599f731ed00 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.712+0000 7f03e4940b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01ad7e0 con 0x5599f72a44a0 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.712+0000 7f03e4940b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0108060 con 0x5599f72a7f70 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f74a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03e4940b80 10 client.0 _ll_register_callbacks cb 0x5599f742f500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f74a4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40872/0 (socket says 172.21.15.19:40872) 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cffff700 1 -- 172.21.15.19:0/3265527815 learned_addr learned my addr 172.21.15.19:0/3265527815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f72a44a0 0x5599f7448140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03ceffd700 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a7f70 0x5599f74486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cffff700 1 -- 172.21.15.19:0/3265527815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a7f70 msgr2=0x5599f74486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:17.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cffff700 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a7f70 0x5599f74486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cffff700 1 -- 172.21.15.19:0/3265527815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f72a44a0 msgr2=0x5599f7448140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cffff700 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f72a44a0 0x5599f7448140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cffff700 1 -- 172.21.15.19:0/3265527815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599f7415cc0 con 0x5599f731ed00 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03cffff700 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f74a4f20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f03c8015dc0 tx=0x7f03c80135b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1613d0f48a068 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03aeffd700 1 -- 172.21.15.19:0/3265527815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c801b070 con 0x5599f731ed00 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03aeffd700 1 -- 172.21.15.19:0/3265527815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f03c8013ca0 con 0x5599f731ed00 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03e4940b80 1 -- 172.21.15.19:0/3265527815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b80b1f40 con 0x5599f731ed00 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03aeffd700 1 -- 172.21.15.19:0/3265527815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03c8013ea0 con 0x5599f731ed00 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03e4940b80 10 client.18348 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.713+0000 7f03e4940b80 1 -- 172.21.15.19:0/3265527815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599f74ad8b0 con 0x5599f731ed00 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.714+0000 7f03aeffd700 1 -- 172.21.15.19:0/3265527815 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b80b1f40 con 0x5599f731ed00 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.714+0000 7f03aeffd700 1 -- 172.21.15.19:0/3265527815 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f03c8017430 con 0x5599f731ed00 2024-05-09T01:36:17.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.714+0000 7f03e4940b80 10 client.18348 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:17.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.714+0000 7f03e4940b80 10 client.18348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:36:17.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.714+0000 7f03e4940b80 20 client.18348 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:17.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.714+0000 7f03e4940b80 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5599f7429c10 0x5599f74b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:17.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.714+0000 7f03e4940b80 4 client.18348 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:17.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.714+0000 7f03e4940b80 1 -- 172.21.15.19:0/3265527815 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599f734bb20 con 0x5599f7429c10 2024-05-09T01:36:17.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.714+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5599f7429c10 0x5599f74b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:17.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03cf7fe700 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5599f7429c10 0x5599f74b0030 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60a032cc684ebe53 server_cookie=8edda124b588a023 in_seq=0 out_seq=0 2024-05-09T01:36:17.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03aeffd700 10 client.18348 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:36:17.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03aeffd700 1 -- 172.21.15.19:0/3265527815 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599f734bb20 con 0x5599f7429c10 2024-05-09T01:36:17.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03aeffd700 10 client.18348 handle_command_reply: tid=0 2024-05-09T01:36:17.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 client.18348 shutdown 2024-05-09T01:36:17.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 -- 172.21.15.19:0/3265527815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 msgr2=0x5599f74a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:17.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f74a4f20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f03c8015dc0 tx=0x7f03c80135b0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 -- 172.21.15.19:0/3265527815 shutdown_connections 2024-05-09T01:36:17.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5599f7429c10 0x5599f74b0030 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599f731ed00 0x5599f74a4f20 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5599f72a44a0 0x5599f7448140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 --2- 172.21.15.19:0/3265527815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599f72a7f70 0x5599f74486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 -- 172.21.15.19:0/3265527815 >> 172.21.15.19:0/3265527815 conn(0x5599f741c520 msgr2=0x5599f742e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:17.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03aeffd700 0 client.18348 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:36:17.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 -- 172.21.15.19:0/3265527815 shutdown_connections 2024-05-09T01:36:17.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.715+0000 7f03e4940b80 1 -- 172.21.15.19:0/3265527815 wait complete. 2024-05-09T01:36:17.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03e4940b80 20 client.18348 trim_cache size 0 max 16384 2024-05-09T01:36:17.717 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03bc0b2880 msgr2=0x7f03bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03bc0b2880 0x7f03bc0b4d90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009750 tx=0x7f03c001a040 comp rx=0 tx=0).stop 2024-05-09T01:36:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0076e40 msgr2=0x7f03d019d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0076e40 0x7f03d019d0a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f03b800b530 tx=0x7f03b800b840 comp rx=0 tx=0).stop 2024-05-09T01:36:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2053113784 shutdown_connections 2024-05-09T01:36:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03d0110b80 0x7f03d01a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f03bc0b2880 0x7f03bc0b4d90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03d0076e40 0x7f03d019d0a0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 --2- 172.21.15.19:0/2053113784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d0077790 0x7f03d019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:17.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2053113784 >> 172.21.15.19:0/2053113784 conn(0x7f03d00ffea0 msgr2=0x7f03d0101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:17.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2053113784 shutdown_connections 2024-05-09T01:36:17.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:17.716+0000 7f03d5f2a700 1 -- 172.21.15.19:0/2053113784 wait complete. 2024-05-09T01:36:18.392 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:36:18.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:18 smithi019 ceph-mon[30158]: pgmap v701: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:36:18.646 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:18.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:18 smithi196 ceph-mon[35041]: pgmap v701: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:36:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:18 smithi059 ceph-mon[34351]: pgmap v701: 97 pgs: 97 active+clean; 2.9 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:36:20.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:19 smithi059 ceph-mon[34351]: pgmap v702: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:20.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:19 smithi019 ceph-mon[30158]: pgmap v702: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:20.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:19 smithi196 ceph-mon[35041]: pgmap v702: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.466+0000 7f42768b6700 1 -- 172.21.15.19:0/4163914591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42701045e0 msgr2=0x7f42701049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.466+0000 7f42768b6700 1 --2- 172.21.15.19:0/4163914591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42701045e0 0x7f42701049f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f42600099c0 tx=0x7f4260009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.466+0000 7f42768b6700 1 -- 172.21.15.19:0/4163914591 shutdown_connections 2024-05-09T01:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.466+0000 7f42768b6700 1 --2- 172.21.15.19:0/4163914591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270109860 0x7f42701111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.466+0000 7f42768b6700 1 --2- 172.21.15.19:0/4163914591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42701045e0 0x7f42701049f0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.466+0000 7f42768b6700 1 --2- 172.21.15.19:0/4163914591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4270108f10 0x7f4270109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.466+0000 7f42768b6700 1 -- 172.21.15.19:0/4163914591 >> 172.21.15.19:0/4163914591 conn(0x7f42700ffed0 msgr2=0x7f4270102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.467+0000 7f42768b6700 1 -- 172.21.15.19:0/4163914591 shutdown_connections 2024-05-09T01:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.467+0000 7f42768b6700 1 -- 172.21.15.19:0/4163914591 wait complete. 2024-05-09T01:36:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.467+0000 7f42768b6700 1 Processor -- start 2024-05-09T01:36:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.467+0000 7f42768b6700 1 -- start start 2024-05-09T01:36:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f42768b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42701045e0 0x7f427019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f42768b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4270108f10 0x7f427019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f42768b6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270109860 0x7f427019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f42768b6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4270114490 con 0x7f4270108f10 2024-05-09T01:36:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f42768b6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4270114310 con 0x7f42701045e0 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f42768b6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4270114610 con 0x7f4270109860 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4267fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4270108f10 0x7f427019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4267fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4270108f10 0x7f427019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35164/0 (socket says 172.21.15.19:35164) 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4267fff700 1 -- 172.21.15.19:0/1698820997 learned_addr learned my addr 172.21.15.19:0/1698820997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f426ffff700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42701045e0 0x7f427019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4274e53700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270109860 0x7f427019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4267fff700 1 -- 172.21.15.19:0/1698820997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270109860 msgr2=0x7f427019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4267fff700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270109860 0x7f427019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4267fff700 1 -- 172.21.15.19:0/1698820997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42701045e0 msgr2=0x7f427019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4267fff700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42701045e0 0x7f427019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4267fff700 1 -- 172.21.15.19:0/1698820997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427019fc60 con 0x7f4270108f10 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.468+0000 7f4267fff700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4270108f10 0x7f427019b3e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f425800d980 tx=0x7f425800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.469+0000 7f426dffb700 1 -- 172.21.15.19:0/1698820997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258017070 con 0x7f4270108f10 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.469+0000 7f426dffb700 1 -- 172.21.15.19:0/1698820997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4258009de0 con 0x7f4270108f10 2024-05-09T01:36:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.469+0000 7f42768b6700 1 -- 172.21.15.19:0/1698820997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427019ff50 con 0x7f4270108f10 2024-05-09T01:36:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.469+0000 7f426dffb700 1 -- 172.21.15.19:0/1698820997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425800f3c0 con 0x7f4270108f10 2024-05-09T01:36:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.469+0000 7f42768b6700 1 -- 172.21.15.19:0/1698820997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701a0490 con 0x7f4270108f10 2024-05-09T01:36:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.471+0000 7f426dffb700 1 -- 172.21.15.19:0/1698820997 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4258009930 con 0x7f4270108f10 2024-05-09T01:36:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.472+0000 7f426dffb700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f425c0b2880 0x7f425c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.472+0000 7f426ffff700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f425c0b2880 0x7f425c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.472+0000 7f426dffb700 1 -- 172.21.15.19:0/1698820997 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f42580e1de0 con 0x7f4270108f10 2024-05-09T01:36:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.472+0000 7f42852ccb80 1 Processor -- start 2024-05-09T01:36:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.472+0000 7f42852ccb80 1 -- start start 2024-05-09T01:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f426ffff700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f425c0b2880 0x7f425c0b4d90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4260009990 tx=0x7f4260000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 0x55e7f953a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f953aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f9534ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f42852ccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e7f9456ff0 con 0x55e7f93b74a0 2024-05-09T01:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f42852ccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e7f93f57f0 con 0x55e7f95346a0 2024-05-09T01:36:20.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f42852ccb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e7f9452fb0 con 0x55e7f94a1160 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4267fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f953aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4267fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f953aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35178/0 (socket says 172.21.15.19:35178) 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4267fff700 1 -- 172.21.15.19:0/3885736212 learned_addr learned my addr 172.21.15.19:0/3885736212 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f426ffff700 1 --2- 172.21.15.19:0/3885736212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 0x55e7f953a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4274e53700 1 --2- 172.21.15.19:0/3885736212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f9534ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4267fff700 1 -- 172.21.15.19:0/3885736212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 msgr2=0x55e7f953a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4267fff700 1 --2- 172.21.15.19:0/3885736212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 0x55e7f953a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4267fff700 1 -- 172.21.15.19:0/3885736212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 msgr2=0x55e7f9534ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4267fff700 1 --2- 172.21.15.19:0/3885736212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f9534ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4267fff700 1 -- 172.21.15.19:0/3885736212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f93ed370 con 0x55e7f93b74a0 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.473+0000 7f4267fff700 1 --2- 172.21.15.19:0/3885736212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f953aa30 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f42580a9080 tx=0x7f42580a8f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d77745d453bed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.474+0000 7f42667fc700 1 -- 172.21.15.19:0/3885736212 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258020070 con 0x55e7f93b74a0 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.474+0000 7f42667fc700 1 -- 172.21.15.19:0/3885736212 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42580ad730 con 0x55e7f93b74a0 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.474+0000 7f42667fc700 1 -- 172.21.15.19:0/3885736212 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42580a4cf0 con 0x55e7f93b74a0 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.474+0000 7f42852ccb80 1 -- 172.21.15.19:0/3885736212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 msgr2=0x55e7f953aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.474+0000 7f42852ccb80 1 --2- 172.21.15.19:0/3885736212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f953aa30 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f42580a9080 tx=0x7f42580a8f60 comp rx=0 tx=0).stop 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.475+0000 7f42852ccb80 1 -- 172.21.15.19:0/3885736212 shutdown_connections 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.475+0000 7f42852ccb80 1 --2- 172.21.15.19:0/3885736212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f9534ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.475+0000 7f42852ccb80 1 --2- 172.21.15.19:0/3885736212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 0x55e7f953a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.475+0000 7f42852ccb80 1 --2- 172.21.15.19:0/3885736212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f953aa30 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.475+0000 7f42852ccb80 1 -- 172.21.15.19:0/3885736212 >> 172.21.15.19:0/3885736212 conn(0x55e7f94176a0 msgr2=0x55e7f952f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.475+0000 7f42852ccb80 1 -- 172.21.15.19:0/3885736212 shutdown_connections 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.475+0000 7f42852ccb80 1 -- 172.21.15.19:0/3885736212 wait complete. 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.475+0000 7f42852ccb80 1 -- start start 2024-05-09T01:36:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.475+0000 7f42852ccb80 1 Processor -- start 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f93b74a0 0x55e7f953e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f94a1160 0x55e7f953e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f953ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f42852ccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e7f92d2ba0 con 0x55e7f94a1160 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f42852ccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e7f9411150 con 0x55e7f95346a0 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f42852ccb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e7f95354e0 con 0x55e7f93b74a0 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f42852ccb80 10 client.0 _ll_register_callbacks cb 0x55e7f92c86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4267fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f94a1160 0x55e7f953e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4267fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f94a1160 0x55e7f953e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35182/0 (socket says 172.21.15.19:35182) 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4267fff700 1 -- 172.21.15.19:0/737623660 learned_addr learned my addr 172.21.15.19:0/737623660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f426ffff700 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f93b74a0 0x55e7f953e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4274e53700 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f953ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4267fff700 1 -- 172.21.15.19:0/737623660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f93b74a0 msgr2=0x55e7f953e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4267fff700 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f93b74a0 0x55e7f953e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4267fff700 1 -- 172.21.15.19:0/737623660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 msgr2=0x55e7f953ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4267fff700 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f953ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4267fff700 1 -- 172.21.15.19:0/737623660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7f92d2490 con 0x55e7f94a1160 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f4267fff700 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f94a1160 0x55e7f953e7a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f42580a5400 tx=0x7f42580b2260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4aa2ec48f5a1c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.476+0000 7f424f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f424f7fe700 1 -- 172.21.15.19:0/737623660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258020070 con 0x55e7f94a1160 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f42852ccb80 1 -- 172.21.15.19:0/737623660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42580ad730 con 0x55e7f94a1160 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f424f7fe700 1 -- 172.21.15.19:0/737623660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f42580b1f50 con 0x55e7f94a1160 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f424f7fe700 1 -- 172.21.15.19:0/737623660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42580a85e0 con 0x55e7f94a1160 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f42852ccb80 10 client.18360 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f42852ccb80 1 -- 172.21.15.19:0/737623660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7f954bbd0 con 0x55e7f94a1160 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f424f7fe700 1 -- 172.21.15.19:0/737623660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42580ad730 con 0x55e7f94a1160 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f424f7fe700 1 -- 172.21.15.19:0/737623660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f42580b0c40 con 0x55e7f94a1160 2024-05-09T01:36:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f42852ccb80 10 client.18360 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f42852ccb80 10 client.18360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f42852ccb80 20 client.18360 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f42852ccb80 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e7f954e0a0 0x55e7f95504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f42852ccb80 4 client.18360 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f42852ccb80 1 -- 172.21.15.19:0/737623660 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7f94649d0 con 0x55e7f954e0a0 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.477+0000 7f426ffff700 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e7f954e0a0 0x55e7f95504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.478+0000 7f424f7fe700 10 client.18360 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.478+0000 7f426ffff700 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e7f954e0a0 0x55e7f95504c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35eb89d71d9f97d server_cookie=2452aa5acd123637 in_seq=0 out_seq=0 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f424f7fe700 1 -- 172.21.15.19:0/737623660 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e7f94649d0 con 0x55e7f954e0a0 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f424f7fe700 10 client.18360 handle_command_reply: tid=0 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 client.18360 shutdown 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 -- 172.21.15.19:0/737623660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f94a1160 msgr2=0x55e7f953e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f94a1160 0x55e7f953e7a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f42580a5400 tx=0x7f42580b2260 comp rx=0 tx=0).stop 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 -- 172.21.15.19:0/737623660 shutdown_connections 2024-05-09T01:36:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e7f954e0a0 0x55e7f95504c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f424f7fe700 0 client.18360 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f953ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f93b74a0 0x55e7f953e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 --2- 172.21.15.19:0/737623660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f94a1160 0x55e7f953e7a0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 -- 172.21.15.19:0/737623660 >> 172.21.15.19:0/737623660 conn(0x55e7f94176a0 msgr2=0x55e7f952f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 -- 172.21.15.19:0/737623660 shutdown_connections 2024-05-09T01:36:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 1 -- 172.21.15.19:0/737623660 wait complete. 2024-05-09T01:36:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.479+0000 7f42852ccb80 20 client.18360 trim_cache size 0 max 16384 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.494+0000 7f42852ccb80 1 Processor -- start 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f42852ccb80 1 -- start start 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9539970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 0x55e7f953eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f9537e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f42852ccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42701aed20 con 0x55e7f93b74a0 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f42852ccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f42701b0860 con 0x55e7f95346a0 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f42852ccb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42701af3f0 con 0x55e7f94a1160 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f426ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9539970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f426ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9539970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35192/0 (socket says 172.21.15.19:35192) 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f426ffff700 1 -- 172.21.15.19:0/3269584043 learned_addr learned my addr 172.21.15.19:0/3269584043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f4274e53700 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f9537e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f4267fff700 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 0x55e7f953eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f426ffff700 1 -- 172.21.15.19:0/3269584043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 msgr2=0x55e7f953eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f426ffff700 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 0x55e7f953eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f426ffff700 1 -- 172.21.15.19:0/3269584043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 msgr2=0x55e7f9537e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f426ffff700 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f9537e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f426ffff700 1 -- 172.21.15.19:0/3269584043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f9540b00 con 0x55e7f93b74a0 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f4274e53700 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f9537e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f4267fff700 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 0x55e7f953eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.495+0000 7f426ffff700 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9539970 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f426000b580 tx=0x7f4260021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a4409cf68d6d86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f4265ffb700 1 -- 172.21.15.19:0/3269584043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42600056e0 con 0x55e7f93b74a0 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f4265ffb700 1 -- 172.21.15.19:0/3269584043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4260039070 con 0x55e7f93b74a0 2024-05-09T01:36:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f4265ffb700 1 -- 172.21.15.19:0/3269584043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426000fc50 con 0x55e7f93b74a0 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f42852ccb80 1 -- 172.21.15.19:0/3269584043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 msgr2=0x55e7f9539970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f42852ccb80 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9539970 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f426000b580 tx=0x7f4260021000 comp rx=0 tx=0).stop 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f42852ccb80 1 -- 172.21.15.19:0/3269584043 shutdown_connections 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f42852ccb80 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f95346a0 0x55e7f9537e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f42852ccb80 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9539970 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f42852ccb80 1 --2- 172.21.15.19:0/3269584043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f94a1160 0x55e7f953eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.496+0000 7f42852ccb80 1 -- 172.21.15.19:0/3269584043 >> 172.21.15.19:0/3269584043 conn(0x55e7f94176a0 msgr2=0x55e7f9539260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.497+0000 7f42852ccb80 1 -- 172.21.15.19:0/3269584043 shutdown_connections 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.497+0000 7f42852ccb80 1 -- 172.21.15.19:0/3269584043 wait complete. 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.497+0000 7f42852ccb80 1 -- start start 2024-05-09T01:36:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.497+0000 7f42852ccb80 1 Processor -- start 2024-05-09T01:36:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.497+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9550b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f94a1160 0x55e7f9551040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f42852ccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f95346a0 0x55e7f95b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f42852ccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42701b0190 con 0x55e7f93b74a0 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f42852ccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f42701afac0 con 0x55e7f94a1160 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f42852ccb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42701accc0 con 0x55e7f95346a0 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f426ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9550b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f426ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9550b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35206/0 (socket says 172.21.15.19:35206) 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f4267fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f94a1160 0x55e7f9551040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f426ffff700 1 -- 172.21.15.19:0/1596748005 learned_addr learned my addr 172.21.15.19:0/1596748005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f42852ccb80 10 client.0 _ll_register_callbacks cb 0x55e7f92c86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f4274e53700 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f95346a0 0x55e7f95b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f426ffff700 1 -- 172.21.15.19:0/1596748005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f95346a0 msgr2=0x55e7f95b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f426ffff700 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f95346a0 0x55e7f95b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f426ffff700 1 -- 172.21.15.19:0/1596748005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f94a1160 msgr2=0x55e7f9551040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f426ffff700 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f94a1160 0x55e7f9551040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f426ffff700 1 -- 172.21.15.19:0/1596748005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7f95b4f00 con 0x55e7f93b74a0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f4274e53700 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f95346a0 0x55e7f95b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f426ffff700 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9550b00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4260009f50 tx=0x7f4260003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195785f81cdb3efb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f424f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f424f7fe700 1 -- 172.21.15.19:0/1596748005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426000fce0 con 0x55e7f93b74a0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f424f7fe700 1 -- 172.21.15.19:0/1596748005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f426000b790 con 0x55e7f93b74a0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.498+0000 7f42852ccb80 1 -- 172.21.15.19:0/1596748005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4260039070 con 0x55e7f93b74a0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f424f7fe700 1 -- 172.21.15.19:0/1596748005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f426000b990 con 0x55e7f93b74a0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f42852ccb80 10 client.18366 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f42852ccb80 1 -- 172.21.15.19:0/1596748005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7f95bc1a0 con 0x55e7f93b74a0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f424f7fe700 1 -- 172.21.15.19:0/1596748005 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4260039070 con 0x55e7f93b74a0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f424f7fe700 1 -- 172.21.15.19:0/1596748005 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4260022070 con 0x55e7f93b74a0 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f42852ccb80 10 client.18366 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f42852ccb80 10 client.18366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f42852ccb80 20 client.18366 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f42852ccb80 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e7f954e0a0 0x55e7f95be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f42852ccb80 4 client.18366 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f42852ccb80 1 -- 172.21.15.19:0/1596748005 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7f945ea70 con 0x55e7f954e0a0 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.499+0000 7f4267fff700 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e7f954e0a0 0x55e7f95be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f4267fff700 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e7f954e0a0 0x55e7f95be7f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e668a5a8274667 server_cookie=4762808e0a4322de in_seq=0 out_seq=0 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f424f7fe700 10 client.18366 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f424f7fe700 1 -- 172.21.15.19:0/1596748005 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7f945ea70 con 0x55e7f954e0a0 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f424f7fe700 10 client.18366 handle_command_reply: tid=0 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 client.18366 shutdown 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 -- 172.21.15.19:0/1596748005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 msgr2=0x55e7f9550b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9550b00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4260009f50 tx=0x7f4260003c10 comp rx=0 tx=0).stop 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 -- 172.21.15.19:0/1596748005 shutdown_connections 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e7f954e0a0 0x55e7f95be7f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7f95346a0 0x55e7f95b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e7f93b74a0 0x55e7f9550b00 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 --2- 172.21.15.19:0/1596748005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7f94a1160 0x55e7f9551040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f424f7fe700 0 client.18366 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:20.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 -- 172.21.15.19:0/1596748005 >> 172.21.15.19:0/1596748005 conn(0x55e7f94176a0 msgr2=0x55e7f9537e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:20.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.500+0000 7f42852ccb80 1 -- 172.21.15.19:0/1596748005 shutdown_connections 2024-05-09T01:36:20.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42852ccb80 1 -- 172.21.15.19:0/1596748005 wait complete. 2024-05-09T01:36:20.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42852ccb80 20 client.18366 trim_cache size 0 max 16384 2024-05-09T01:36:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:20.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 -- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f425c0b2880 msgr2=0x7f425c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f425c0b2880 0x7f425c0b4d90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4260009990 tx=0x7f4260000bc0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 -- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4270108f10 msgr2=0x7f427019b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4270108f10 0x7f427019b3e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f425800d980 tx=0x7f425800dc90 comp rx=0 tx=0).stop 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 -- 172.21.15.19:0/1698820997 shutdown_connections 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270109860 0x7f427019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42701045e0 0x7f427019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f425c0b2880 0x7f425c0b4d90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 --2- 172.21.15.19:0/1698820997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4270108f10 0x7f427019b3e0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 -- 172.21.15.19:0/1698820997 >> 172.21.15.19:0/1698820997 conn(0x7f42700ffed0 msgr2=0x7f42701022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 -- 172.21.15.19:0/1698820997 shutdown_connections 2024-05-09T01:36:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:20.501+0000 7f42768b6700 1 -- 172.21.15.19:0/1698820997 wait complete. 2024-05-09T01:36:22.287 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:36:22.288 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:36:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:22.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:22 smithi019 ceph-mon[30158]: pgmap v703: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:22.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:22 smithi196 ceph-mon[35041]: pgmap v703: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:22 smithi059 ceph-mon[34351]: pgmap v703: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:24.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.228+0000 7f69a43e7700 1 -- 172.21.15.19:0/1998408416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 msgr2=0x7f699c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:24.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.228+0000 7f69a43e7700 1 --2- 172.21.15.19:0/1998408416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 0x7f699c104a00 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f69980099c0 tx=0x7f6998009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.228+0000 7f69a43e7700 1 -- 172.21.15.19:0/1998408416 shutdown_connections 2024-05-09T01:36:24.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.228+0000 7f69a43e7700 1 --2- 172.21.15.19:0/1998408416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f699c109870 0x7f699c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.228+0000 7f69a43e7700 1 --2- 172.21.15.19:0/1998408416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 0x7f699c104a00 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.228+0000 7f69a43e7700 1 --2- 172.21.15.19:0/1998408416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c108f20 0x7f699c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.228+0000 7f69a43e7700 1 -- 172.21.15.19:0/1998408416 >> 172.21.15.19:0/1998408416 conn(0x7f699c100040 msgr2=0x7f699c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:24.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.228+0000 7f69a43e7700 1 -- 172.21.15.19:0/1998408416 shutdown_connections 2024-05-09T01:36:24.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.228+0000 7f69a43e7700 1 -- 172.21.15.19:0/1998408416 wait complete. 2024-05-09T01:36:24.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a43e7700 1 Processor -- start 2024-05-09T01:36:24.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a43e7700 1 -- start start 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a43e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 0x7f699c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a43e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c108f20 0x7f699c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a43e7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f699c109870 0x7f699c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a43e7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1144a0 con 0x7f699c1045f0 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a43e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114320 con 0x7f699c108f20 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a43e7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114620 con 0x7f699c109870 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a2183700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 0x7f699c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a2183700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 0x7f699c19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35220/0 (socket says 172.21.15.19:35220) 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a2183700 1 -- 172.21.15.19:0/3338924129 learned_addr learned my addr 172.21.15.19:0/3338924129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a2984700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f699c109870 0x7f699c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a1982700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c108f20 0x7f699c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a2183700 1 -- 172.21.15.19:0/3338924129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f699c109870 msgr2=0x7f699c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.229+0000 7f69a2183700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f699c109870 0x7f699c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69a2183700 1 -- 172.21.15.19:0/3338924129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c108f20 msgr2=0x7f699c19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69a2183700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c108f20 0x7f699c19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69a2183700 1 -- 172.21.15.19:0/3338924129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c19fd20 con 0x7f699c1045f0 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69a1982700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c108f20 0x7f699c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69a2183700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 0x7f699c19ae50 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6998006010 tx=0x7f69980045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69937fe700 1 -- 172.21.15.19:0/3338924129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998026070 con 0x7f699c1045f0 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69a43e7700 1 -- 172.21.15.19:0/3338924129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c19ffb0 con 0x7f699c1045f0 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69937fe700 1 -- 172.21.15.19:0/3338924129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f699800f460 con 0x7f699c1045f0 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69937fe700 1 -- 172.21.15.19:0/3338924129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998009e60 con 0x7f699c1045f0 2024-05-09T01:36:24.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.230+0000 7f69a43e7700 1 -- 172.21.15.19:0/3338924129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a7930 con 0x7f699c1045f0 2024-05-09T01:36:24.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.234+0000 7f69937fe700 1 -- 172.21.15.19:0/3338924129 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6998029070 con 0x7f699c1045f0 2024-05-09T01:36:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.234+0000 7f69937fe700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f69880b2880 0x7f69880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69a1982700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f69880b2880 0x7f69880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69b2dfdb80 1 Processor -- start 2024-05-09T01:36:24.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69937fe700 1 -- 172.21.15.19:0/3338924129 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f69980e2ad0 con 0x7f699c1045f0 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69b2dfdb80 1 -- start start 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69a1982700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f69880b2880 0x7f69880b4d90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f699c107860 tx=0x7f698c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c40dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 0x55fba2c41370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 0x55fba2c45890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fba2b5f070 con 0x55fba2ba91e0 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fba2afd870 con 0x55fba2c45480 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fba2b5b030 con 0x55fba2abf520 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69a2183700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c40dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69a2183700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c40dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35226/0 (socket says 172.21.15.19:35226) 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69a2183700 1 -- 172.21.15.19:0/1645847255 learned_addr learned my addr 172.21.15.19:0/1645847255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69a1982700 1 --2- 172.21.15.19:0/1645847255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 0x55fba2c41370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.235+0000 7f69a2984700 1 --2- 172.21.15.19:0/1645847255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 0x55fba2c45890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.236+0000 7f69a2183700 1 -- 172.21.15.19:0/1645847255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 msgr2=0x55fba2c41370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.236+0000 7f69a2183700 1 --2- 172.21.15.19:0/1645847255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 0x55fba2c41370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.236+0000 7f69a2183700 1 -- 172.21.15.19:0/1645847255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 msgr2=0x55fba2c45890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.236+0000 7f69a2183700 1 --2- 172.21.15.19:0/1645847255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 0x55fba2c45890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.236+0000 7f69a2183700 1 -- 172.21.15.19:0/1645847255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fba2af53f0 con 0x55fba2ba91e0 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.236+0000 7f69a2183700 1 --2- 172.21.15.19:0/1645847255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c40dd0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f69980a61e0 tx=0x7f69980a6130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a59e0ed9c1738ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.236+0000 7f6990d47700 1 -- 172.21.15.19:0/1645847255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69980aa160 con 0x55fba2ba91e0 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.236+0000 7f6990d47700 1 -- 172.21.15.19:0/1645847255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69980ae510 con 0x55fba2ba91e0 2024-05-09T01:36:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.236+0000 7f6990d47700 1 -- 172.21.15.19:0/1645847255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69980aa600 con 0x55fba2ba91e0 2024-05-09T01:36:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.237+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/1645847255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 msgr2=0x55fba2c40dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.237+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/1645847255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c40dd0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f69980a61e0 tx=0x7f69980a6130 comp rx=0 tx=0).stop 2024-05-09T01:36:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.237+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/1645847255 shutdown_connections 2024-05-09T01:36:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.237+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/1645847255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 0x55fba2c45890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.237+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/1645847255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c40dd0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.237+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/1645847255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 0x55fba2c41370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.237+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/1645847255 >> 172.21.15.19:0/1645847255 conn(0x55fba2b1f720 msgr2=0x55fba2c37480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.237+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/1645847255 shutdown_connections 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.237+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/1645847255 wait complete. 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69b2dfdb80 1 -- start start 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69b2dfdb80 1 Processor -- start 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2abf520 0x55fba2c500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2c45480 0x55fba2c4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fba29dac20 con 0x55fba2ba91e0 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fba2b191d0 con 0x55fba2abf520 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fba2c45ff0 con 0x55fba2c45480 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69b2dfdb80 10 client.0 _ll_register_callbacks cb 0x55fba29d0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69a2183700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2abf520 0x55fba2c500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69a1982700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69a2183700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2abf520 0x55fba2c500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:46908/0 (socket says 172.21.15.19:46908) 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69a2984700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2c45480 0x55fba2c4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69a1982700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35238/0 (socket says 172.21.15.19:35238) 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.238+0000 7f69a2183700 1 -- 172.21.15.19:0/2429383925 learned_addr learned my addr 172.21.15.19:0/2429383925 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69a1982700 1 -- 172.21.15.19:0/2429383925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2c45480 msgr2=0x55fba2c4a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69a1982700 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2c45480 0x55fba2c4a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69a1982700 1 -- 172.21.15.19:0/2429383925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2abf520 msgr2=0x55fba2c500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69a1982700 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2abf520 0x55fba2c500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69a1982700 1 -- 172.21.15.19:0/2429383925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fba29da510 con 0x55fba2ba91e0 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69a2183700 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2abf520 0x55fba2c500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69a1982700 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c505f0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f698c009810 tx=0x7f698c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d655fd1675611859 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f6985ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f6985ffb700 1 -- 172.21.15.19:0/2429383925 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c025070 con 0x55fba2ba91e0 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f6985ffb700 1 -- 172.21.15.19:0/2429383925 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f698c012450 con 0x55fba2ba91e0 2024-05-09T01:36:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/2429383925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69980ae510 con 0x55fba2ba91e0 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f6985ffb700 1 -- 172.21.15.19:0/2429383925 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f698c031050 con 0x55fba2ba91e0 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69b2dfdb80 10 client.18378 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.239+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/2429383925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fba2c57d30 con 0x55fba2ba91e0 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.240+0000 7f6985ffb700 1 -- 172.21.15.19:0/2429383925 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69980ae510 con 0x55fba2ba91e0 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.240+0000 7f6985ffb700 1 -- 172.21.15.19:0/2429383925 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f698c0215d0 con 0x55fba2ba91e0 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.240+0000 7f69b2dfdb80 10 client.18378 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.240+0000 7f69b2dfdb80 10 client.18378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.240+0000 7f69b2dfdb80 20 client.18378 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.240+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fba2c5a310 0x55fba2c5c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.240+0000 7f69b2dfdb80 4 client.18378 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.240+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/2429383925 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fba2b6ca50 con 0x55fba2c5a310 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.240+0000 7f69a2183700 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fba2c5a310 0x55fba2c5c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.241+0000 7f69a2183700 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fba2c5a310 0x55fba2c5c770 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f13a50e29243128c server_cookie=dc009b53e7abd13 in_seq=0 out_seq=0 2024-05-09T01:36:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.241+0000 7f6985ffb700 10 client.18378 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.241+0000 7f6985ffb700 1 -- 172.21.15.19:0/2429383925 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fba2b6ca50 con 0x55fba2c5a310 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.241+0000 7f6985ffb700 10 client.18378 handle_command_reply: tid=0 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 client.18378 shutdown 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/2429383925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 msgr2=0x55fba2c505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c505f0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f698c009810 tx=0x7f698c0047c0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/2429383925 shutdown_connections 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fba2c5a310 0x55fba2c5c770 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2c45480 0x55fba2c4a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f6985ffb700 0 client.18378 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2abf520 0x55fba2c500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/2429383925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2ba91e0 0x55fba2c505f0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/2429383925 >> 172.21.15.19:0/2429383925 conn(0x55fba2b1f720 msgr2=0x55fba2c37240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/2429383925 shutdown_connections 2024-05-09T01:36:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/2429383925 wait complete. 2024-05-09T01:36:24.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.242+0000 7f69b2dfdb80 20 client.18378 trim_cache size 0 max 16384 2024-05-09T01:36:24.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69b2dfdb80 1 Processor -- start 2024-05-09T01:36:24.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69b2dfdb80 1 -- start start 2024-05-09T01:36:24.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 0x55fba2c3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2ba91e0 0x55fba2c61f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2c45480 0x55fba2c62470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1aca70 con 0x55fba2c45480 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ae5b0 con 0x55fba2ba91e0 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ad140 con 0x55fba2abf520 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a2984700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2c45480 0x55fba2c62470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a2984700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2c45480 0x55fba2c62470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35248/0 (socket says 172.21.15.19:35248) 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a2984700 1 -- 172.21.15.19:0/253925933 learned_addr learned my addr 172.21.15.19:0/253925933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a2183700 1 --2- 172.21.15.19:0/253925933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 0x55fba2c3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a1982700 1 --2- 172.21.15.19:0/253925933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2ba91e0 0x55fba2c61f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a2984700 1 -- 172.21.15.19:0/253925933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 msgr2=0x55fba2c3c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a2984700 1 --2- 172.21.15.19:0/253925933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 0x55fba2c3c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a2984700 1 -- 172.21.15.19:0/253925933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2ba91e0 msgr2=0x55fba2c61f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:24.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a2984700 1 --2- 172.21.15.19:0/253925933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2ba91e0 0x55fba2c61f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.263+0000 7f69a2984700 1 -- 172.21.15.19:0/253925933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fba2c45280 con 0x55fba2c45480 2024-05-09T01:36:24.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.264+0000 7f69a2984700 1 --2- 172.21.15.19:0/253925933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2c45480 0x55fba2c62470 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f6994009fc0 tx=0x7f6994007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0ee771dcbeed72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:24.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.264+0000 7f6987fff700 1 -- 172.21.15.19:0/253925933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994004bb0 con 0x55fba2c45480 2024-05-09T01:36:24.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.264+0000 7f6987fff700 1 -- 172.21.15.19:0/253925933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f699401b070 con 0x55fba2c45480 2024-05-09T01:36:24.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.264+0000 7f6987fff700 1 -- 172.21.15.19:0/253925933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994005280 con 0x55fba2c45480 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/253925933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2c45480 msgr2=0x55fba2c62470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/253925933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2c45480 0x55fba2c62470 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f6994009fc0 tx=0x7f6994007500 comp rx=0 tx=0).stop 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/253925933 shutdown_connections 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/253925933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2c45480 0x55fba2c62470 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/253925933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2abf520 0x55fba2c3c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/253925933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2ba91e0 0x55fba2c61f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/253925933 >> 172.21.15.19:0/253925933 conn(0x55fba2b1f720 msgr2=0x55fba2c44b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/253925933 shutdown_connections 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/253925933 wait complete. 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 -- start start 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 Processor -- start 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2abf520 0x55fba2c56840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2ba91e0 0x55fba2c54bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 0x55fba2c55110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1adee0 con 0x55fba2abf520 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ad810 con 0x55fba2c45480 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.265+0000 7f69b2dfdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1aaa10 con 0x55fba2ba91e0 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69b2dfdb80 10 client.0 _ll_register_callbacks cb 0x55fba29d0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2183700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2abf520 0x55fba2c56840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2183700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2abf520 0x55fba2c56840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35250/0 (socket says 172.21.15.19:35250) 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2183700 1 -- 172.21.15.19:0/3950650316 learned_addr learned my addr 172.21.15.19:0/3950650316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a1982700 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2ba91e0 0x55fba2c54bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2984700 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 0x55fba2c55110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2183700 1 -- 172.21.15.19:0/3950650316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2ba91e0 msgr2=0x55fba2c54bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2183700 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2ba91e0 0x55fba2c54bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2183700 1 -- 172.21.15.19:0/3950650316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 msgr2=0x55fba2c55110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2183700 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 0x55fba2c55110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2183700 1 -- 172.21.15.19:0/3950650316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fba2c55680 con 0x55fba2abf520 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2984700 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 0x55fba2c55110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a1982700 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2ba91e0 0x55fba2c54bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69a2183700 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2abf520 0x55fba2c56840 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f69980a61e0 tx=0x7f69980096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773340d8af018c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f6985ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f6985ffb700 1 -- 172.21.15.19:0/3950650316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69980ae510 con 0x55fba2abf520 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f6985ffb700 1 -- 172.21.15.19:0/3950650316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f699800f610 con 0x55fba2abf520 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/3950650316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f699401b070 con 0x55fba2abf520 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.266+0000 7f6985ffb700 1 -- 172.21.15.19:0/3950650316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f69980aa3c0 con 0x55fba2abf520 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f69b2dfdb80 10 client.18384 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/3950650316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fba2cc58b0 con 0x55fba2abf520 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f6985ffb700 1 -- 172.21.15.19:0/3950650316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f699401b070 con 0x55fba2abf520 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f6985ffb700 1 -- 172.21.15.19:0/3950650316 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f69980a93c0 con 0x55fba2abf520 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f69b2dfdb80 10 client.18384 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:24.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f69b2dfdb80 10 client.18384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:24.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f69b2dfdb80 20 client.18384 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:24.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fba2cc5dd0 0x55fba2cc81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:24.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f69b2dfdb80 4 client.18384 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:36:24.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/3950650316 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fba2b66af0 con 0x55fba2cc5dd0 2024-05-09T01:36:24.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.267+0000 7f69a1982700 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fba2cc5dd0 0x55fba2cc81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:24.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.268+0000 7f6985ffb700 10 client.18384 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:24.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.268+0000 7f69a1982700 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fba2cc5dd0 0x55fba2cc81f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d58e947d31369053 server_cookie=f9706f39236e651f in_seq=0 out_seq=0 2024-05-09T01:36:24.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f6985ffb700 1 -- 172.21.15.19:0/3950650316 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fba2b66af0 con 0x55fba2cc5dd0 2024-05-09T01:36:24.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f6985ffb700 10 client.18384 handle_command_reply: tid=0 2024-05-09T01:36:24.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 client.18384 shutdown 2024-05-09T01:36:24.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/3950650316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2abf520 msgr2=0x55fba2c56840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2abf520 0x55fba2c56840 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f69980a61e0 tx=0x7f69980096f0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/3950650316 shutdown_connections 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55fba2cc5dd0 0x55fba2cc81f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fba2c45480 0x55fba2c55110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba2abf520 0x55fba2c56840 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 --2- 172.21.15.19:0/3950650316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fba2ba91e0 0x55fba2c54bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f6985ffb700 0 client.18384 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/3950650316 >> 172.21.15.19:0/3950650316 conn(0x55fba2b1f720 msgr2=0x55fba2c44b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.269+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/3950650316 shutdown_connections 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69b2dfdb80 1 -- 172.21.15.19:0/3950650316 wait complete. 2024-05-09T01:36:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69b2dfdb80 20 client.18384 trim_cache size 0 max 16384 2024-05-09T01:36:24.271 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:36:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:36:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "84f745e2-cc5d-4013-ad25-9bcda922c740", 2024-05-09T01:36:24.271 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:36:24.271 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 -- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f69880b2880 msgr2=0x7f69880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f69880b2880 0x7f69880b4d90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f699c107860 tx=0x7f698c009260 comp rx=0 tx=0).stop 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 -- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 msgr2=0x7f699c19ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 0x7f699c19ae50 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6998006010 tx=0x7f69980045e0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 -- 172.21.15.19:0/3338924129 shutdown_connections 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f699c109870 0x7f699c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f69880b2880 0x7f69880b4d90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c1045f0 0x7f699c19ae50 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 --2- 172.21.15.19:0/3338924129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c108f20 0x7f699c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 -- 172.21.15.19:0/3338924129 >> 172.21.15.19:0/3338924129 conn(0x7f699c100040 msgr2=0x7f699c1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 -- 172.21.15.19:0/3338924129 shutdown_connections 2024-05-09T01:36:24.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:24.270+0000 7f69a43e7700 1 -- 172.21.15.19:0/3338924129 wait complete. 2024-05-09T01:36:24.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:24 smithi019 ceph-mon[30158]: pgmap v704: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:36:24.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:24 smithi196 ceph-mon[35041]: pgmap v704: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:36:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:24 smithi059 ceph-mon[34351]: pgmap v704: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:36:24.949 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:36:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:25.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:25 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:36:25.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:25 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:25.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:25 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:36:25.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:25 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:25.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:25 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:36:25.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:25 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:25.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:25 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:36:25.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:25 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:25.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:25 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:36:25.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:25 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:25.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:25 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:36:25.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:25 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:26.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:26 smithi019 ceph-mon[30158]: pgmap v705: 97 pgs: 97 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:26.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:26 smithi196 ceph-mon[35041]: pgmap v705: 97 pgs: 97 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:26 smithi059 ceph-mon[34351]: pgmap v705: 97 pgs: 97 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:26.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.856+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/1796475150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb781034c0 msgr2=0x7fdb781038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:26.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.856+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/1796475150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb781034c0 0x7fdb781038d0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fdb740099c0 tx=0x7fdb74009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.856+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/1796475150 shutdown_connections 2024-05-09T01:36:26.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.856+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/1796475150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb78108740 0x7fdb781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.856+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/1796475150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb781034c0 0x7fdb781038d0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.856+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/1796475150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb78107df0 0x7fdb78108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.856+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/1796475150 >> 172.21.15.19:0/1796475150 conn(0x7fdb780fedf0 msgr2=0x7fdb78101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:26.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.856+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/1796475150 shutdown_connections 2024-05-09T01:36:26.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.856+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/1796475150 wait complete. 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7f2fa700 1 Processor -- start 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7f2fa700 1 -- start start 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7f2fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb781034c0 0x7fdb7819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7f2fa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb78107df0 0x7fdb7819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7f2fa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb78108740 0x7fdb7819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7f2fa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78113390 con 0x7fdb78108740 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7f2fa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78113210 con 0x7fdb781034c0 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7f2fa700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78113510 con 0x7fdb78107df0 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7c895700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb78107df0 0x7fdb7819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7c895700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb78107df0 0x7fdb7819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51564/0 (socket says 172.21.15.19:51564) 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7c895700 1 -- 172.21.15.19:0/2142090713 learned_addr learned my addr 172.21.15.19:0/2142090713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7d897700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb78108740 0x7fdb7819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.857+0000 7fdb7d096700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb781034c0 0x7fdb7819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb7d897700 1 -- 172.21.15.19:0/2142090713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb78107df0 msgr2=0x7fdb7819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb7d897700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb78107df0 0x7fdb7819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb7d897700 1 -- 172.21.15.19:0/2142090713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb781034c0 msgr2=0x7fdb7819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb7d897700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb781034c0 0x7fdb7819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb7d897700 1 -- 172.21.15.19:0/2142090713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7819fd00 con 0x7fdb78108740 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb7d897700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb78108740 0x7fdb7819f660 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fdb70007f10 tx=0x7fdb7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb6e7fc700 1 -- 172.21.15.19:0/2142090713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70004db0 con 0x7fdb78108740 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/2142090713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7819fff0 con 0x7fdb78108740 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb6e7fc700 1 -- 172.21.15.19:0/2142090713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb70009e20 con 0x7fdb78108740 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb6e7fc700 1 -- 172.21.15.19:0/2142090713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70015400 con 0x7fdb78108740 2024-05-09T01:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.858+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/2142090713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb781a7910 con 0x7fdb78108740 2024-05-09T01:36:26.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.861+0000 7fdb6e7fc700 1 -- 172.21.15.19:0/2142090713 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdb70005260 con 0x7fdb78108740 2024-05-09T01:36:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.862+0000 7fdb6e7fc700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb640b2880 0x7fdb640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.862+0000 7fdb7d096700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb640b2880 0x7fdb640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.862+0000 7fdb8dd10b80 1 Processor -- start 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.862+0000 7fdb6e7fc700 1 -- 172.21.15.19:0/2142090713 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdb700e1b20 con 0x7fdb78108740 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.862+0000 7fdb8dd10b80 1 -- start start 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.862+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878ede44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878eddead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55878ed00ff0 con 0x55878ec614a0 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55878ec9f7f0 con 0x55878ed4b160 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55878ecfcfb0 con 0x55878edde6a0 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7d096700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb640b2880 0x7fdb640b4d90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009990 tx=0x7fdb740036e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7c895700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7c895700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35280/0 (socket says 172.21.15.19:35280) 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7c895700 1 -- 172.21.15.19:0/1054586963 learned_addr learned my addr 172.21.15.19:0/1054586963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7d096700 1 --2- 172.21.15.19:0/1054586963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878ede44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7d897700 1 --2- 172.21.15.19:0/1054586963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878eddead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7c895700 1 -- 172.21.15.19:0/1054586963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 msgr2=0x55878eddead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7c895700 1 --2- 172.21.15.19:0/1054586963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878eddead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7c895700 1 -- 172.21.15.19:0/1054586963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 msgr2=0x55878ede44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7c895700 1 --2- 172.21.15.19:0/1054586963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878ede44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7c895700 1 -- 172.21.15.19:0/1054586963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878ec97370 con 0x55878ec614a0 2024-05-09T01:36:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb7c895700 1 --2- 172.21.15.19:0/1054586963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede4a30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800bde0 tx=0x7fdb680106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5eb803678b1aaa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.863+0000 7fdb637fe700 1 -- 172.21.15.19:0/1054586963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68010e10 con 0x55878ec614a0 2024-05-09T01:36:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.864+0000 7fdb637fe700 1 -- 172.21.15.19:0/1054586963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb680118f0 con 0x55878ec614a0 2024-05-09T01:36:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.864+0000 7fdb637fe700 1 -- 172.21.15.19:0/1054586963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68019790 con 0x55878ec614a0 2024-05-09T01:36:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.864+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/1054586963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 msgr2=0x55878ede4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.864+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/1054586963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede4a30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800bde0 tx=0x7fdb680106b0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.864+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/1054586963 shutdown_connections 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.864+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/1054586963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878eddead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.864+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/1054586963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878ede44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.864+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/1054586963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede4a30 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.865+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/1054586963 >> 172.21.15.19:0/1054586963 conn(0x55878ecc16a0 msgr2=0x55878edd9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.865+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/1054586963 shutdown_connections 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.865+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/1054586963 wait complete. 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.865+0000 7fdb8dd10b80 1 -- start start 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.865+0000 7fdb8dd10b80 1 Processor -- start 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878ede8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878ede8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55878eb7cba0 con 0x55878ec614a0 2024-05-09T01:36:26.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55878ecbb150 con 0x55878ed4b160 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55878edeb1d0 con 0x55878edde6a0 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb8dd10b80 10 client.0 _ll_register_callbacks cb 0x55878eb726c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7d096700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7d096700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35282/0 (socket says 172.21.15.19:35282) 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7d096700 1 -- 172.21.15.19:0/964404723 learned_addr learned my addr 172.21.15.19:0/964404723 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7c895700 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878ede8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7d897700 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878ede8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7d096700 1 -- 172.21.15.19:0/964404723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 msgr2=0x55878ede8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7d096700 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878ede8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7d096700 1 -- 172.21.15.19:0/964404723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 msgr2=0x55878ede8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7d096700 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878ede8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7d096700 1 -- 172.21.15.19:0/964404723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878eb7c490 con 0x55878ec614a0 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.866+0000 7fdb7c895700 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878ede8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.867+0000 7fdb7d096700 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede8200 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdb74015000 tx=0x7fdb74009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6147f6abf209c2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.867+0000 7fdb60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.867+0000 7fdb60ff9700 1 -- 172.21.15.19:0/964404723 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400bda0 con 0x55878ec614a0 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.867+0000 7fdb60ff9700 1 -- 172.21.15.19:0/964404723 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdb7400f460 con 0x55878ec614a0 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.867+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/964404723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb680118f0 con 0x55878ec614a0 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.867+0000 7fdb60ff9700 1 -- 172.21.15.19:0/964404723 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdb7400f660 con 0x55878ec614a0 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.867+0000 7fdb8dd10b80 10 client.18396 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.867+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/964404723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55878edf5be0 con 0x55878ec614a0 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.867+0000 7fdb60ff9700 1 -- 172.21.15.19:0/964404723 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb680118f0 con 0x55878ec614a0 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb60ff9700 1 -- 172.21.15.19:0/964404723 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdb74022070 con 0x55878ec614a0 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb8dd10b80 10 client.18396 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb8dd10b80 10 client.18396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb8dd10b80 20 client.18396 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:26.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55878edf8220 0x55878edfa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb8dd10b80 4 client.18396 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:26.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/964404723 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55878ed0e9d0 con 0x55878edf8220 2024-05-09T01:36:26.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb7c895700 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55878edf8220 0x55878edfa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb60ff9700 10 client.18396 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:26.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.868+0000 7fdb7c895700 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55878edf8220 0x55878edfa640 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c3c1e184cdc920 server_cookie=9d8fe367acf8613a in_seq=0 out_seq=0 2024-05-09T01:36:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb60ff9700 1 -- 172.21.15.19:0/964404723 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55878ed0e9d0 con 0x55878edf8220 2024-05-09T01:36:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb60ff9700 10 client.18396 handle_command_reply: tid=0 2024-05-09T01:36:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb8dd10b80 1 client.18396 shutdown 2024-05-09T01:36:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/964404723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 msgr2=0x55878ede8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede8200 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdb74015000 tx=0x7fdb74009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/964404723 shutdown_connections 2024-05-09T01:36:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55878edf8220 0x55878edfa640 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878ede8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ec614a0 0x55878ede8200 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb60ff9700 0 client.18396 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/964404723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878ede8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.869+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/964404723 >> 172.21.15.19:0/964404723 conn(0x55878ecc16a0 msgr2=0x55878edd91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.870+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/964404723 shutdown_connections 2024-05-09T01:36:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.870+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/964404723 wait complete. 2024-05-09T01:36:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.870+0000 7fdb8dd10b80 20 client.18396 trim_cache size 0 max 16384 2024-05-09T01:36:26.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb8dd10b80 1 Processor -- start 2024-05-09T01:36:26.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb8dd10b80 1 -- start start 2024-05-09T01:36:26.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878ec614a0 0x55878ede8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878eddd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878edde6a0 0x55878edddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781aee00 con 0x55878edde6a0 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781b0940 con 0x55878ed4b160 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781af4d0 con 0x55878ec614a0 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb7d897700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878edde6a0 0x55878edddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb7d897700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878edde6a0 0x55878edddbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35284/0 (socket says 172.21.15.19:35284) 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb7d897700 1 -- 172.21.15.19:0/4064700260 learned_addr learned my addr 172.21.15.19:0/4064700260 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.891+0000 7fdb7c895700 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878eddd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb7d096700 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878ec614a0 0x55878ede8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb7d897700 1 -- 172.21.15.19:0/4064700260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878ec614a0 msgr2=0x55878ede8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb7d897700 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878ec614a0 0x55878ede8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb7d897700 1 -- 172.21.15.19:0/4064700260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 msgr2=0x55878eddd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb7d897700 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878eddd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb7d897700 1 -- 172.21.15.19:0/4064700260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878edeb000 con 0x55878edde6a0 2024-05-09T01:36:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb7c895700 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878eddd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb7d897700 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878edde6a0 0x55878edddbd0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fdb700a6330 tx=0x7fdb70005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42616cab0ac7bf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb62ffd700 1 -- 172.21.15.19:0/4064700260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb700c3070 con 0x55878edde6a0 2024-05-09T01:36:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb62ffd700 1 -- 172.21.15.19:0/4064700260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb700ad470 con 0x55878edde6a0 2024-05-09T01:36:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.892+0000 7fdb62ffd700 1 -- 172.21.15.19:0/4064700260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb700b1a90 con 0x55878edde6a0 2024-05-09T01:36:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.893+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/4064700260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878edde6a0 msgr2=0x55878edddbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.893+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878edde6a0 0x55878edddbd0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fdb700a6330 tx=0x7fdb70005600 comp rx=0 tx=0).stop 2024-05-09T01:36:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.893+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/4064700260 shutdown_connections 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.893+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878edde6a0 0x55878edddbd0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.893+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878ec614a0 0x55878ede8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.893+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/4064700260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ed4b160 0x55878eddd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.893+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/4064700260 >> 172.21.15.19:0/4064700260 conn(0x55878ecc16a0 msgr2=0x55878ede6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.893+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/4064700260 shutdown_connections 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.893+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/4064700260 wait complete. 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb8dd10b80 1 -- start start 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb8dd10b80 1 Processor -- start 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ec614a0 0x55878edf0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ed4b160 0x55878edf0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb8dd10b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878eded2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781b0270 con 0x55878ed4b160 2024-05-09T01:36:26.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781afba0 con 0x55878ec614a0 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb8dd10b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781acda0 con 0x55878edde6a0 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb8dd10b80 10 client.0 _ll_register_callbacks cb 0x55878eb726c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb7c895700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ed4b160 0x55878edf0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb7d096700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ec614a0 0x55878edf0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb7c895700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ed4b160 0x55878edf0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35294/0 (socket says 172.21.15.19:35294) 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb7d096700 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ec614a0 0x55878edf0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47004/0 (socket says 172.21.15.19:47004) 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.894+0000 7fdb7c895700 1 -- 172.21.15.19:0/535701604 learned_addr learned my addr 172.21.15.19:0/535701604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb7d897700 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878eded2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb7c895700 1 -- 172.21.15.19:0/535701604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 msgr2=0x55878eded2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb7c895700 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878eded2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb7c895700 1 -- 172.21.15.19:0/535701604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ec614a0 msgr2=0x55878edf0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb7c895700 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ec614a0 0x55878edf0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb7c895700 1 -- 172.21.15.19:0/535701604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878eddad00 con 0x55878ed4b160 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb7d897700 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878eded2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb7d096700 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ec614a0 0x55878edf0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb7c895700 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ed4b160 0x55878edf0840 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fdb68000c00 tx=0x7fdb6800b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83606536cdaa778e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb60ff9700 1 -- 172.21.15.19:0/535701604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68005600 con 0x55878ed4b160 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb60ff9700 1 -- 172.21.15.19:0/535701604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdb680046d0 con 0x55878ed4b160 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/535701604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb700ad470 con 0x55878ed4b160 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb60ff9700 1 -- 172.21.15.19:0/535701604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdb6803a070 con 0x55878ed4b160 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb8dd10b80 10 client.18402 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.895+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/535701604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55878ee62fb0 con 0x55878ed4b160 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.896+0000 7fdb60ff9700 1 -- 172.21.15.19:0/535701604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb700ad470 con 0x55878ed4b160 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.896+0000 7fdb60ff9700 1 -- 172.21.15.19:0/535701604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdb68011850 con 0x55878ed4b160 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.896+0000 7fdb8dd10b80 10 client.18402 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.896+0000 7fdb8dd10b80 10 client.18402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.896+0000 7fdb8dd10b80 20 client.18402 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.896+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55878ee63430 0x55878ee65850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:26.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.896+0000 7fdb8dd10b80 4 client.18402 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.896+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/535701604 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55878ed08a70 con 0x55878ee63430 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.896+0000 7fdb7d096700 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55878ee63430 0x55878ee65850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb60ff9700 10 client.18402 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb7d096700 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55878ee63430 0x55878ee65850 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c37450bf04f46d30 server_cookie=54aa8329ba69c596 in_seq=0 out_seq=0 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb60ff9700 1 -- 172.21.15.19:0/535701604 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55878ed08a70 con 0x55878ee63430 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb60ff9700 10 client.18402 handle_command_reply: tid=0 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 client.18402 shutdown 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/535701604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ed4b160 msgr2=0x55878edf0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ed4b160 0x55878edf0840 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fdb68000c00 tx=0x7fdb6800b510 comp rx=0 tx=0).stop 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/535701604 shutdown_connections 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55878ee63430 0x55878ee65850 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55878edde6a0 0x55878eded2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55878ec614a0 0x55878edf0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 --2- 172.21.15.19:0/535701604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55878ed4b160 0x55878edf0840 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/535701604 >> 172.21.15.19:0/535701604 conn(0x55878ecc16a0 msgr2=0x55878ede1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb60ff9700 0 client.18402 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/535701604 shutdown_connections 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.897+0000 7fdb8dd10b80 1 -- 172.21.15.19:0/535701604 wait complete. 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb8dd10b80 20 client.18402 trim_cache size 0 max 16384 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:36:26.899 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:36:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb640b2880 msgr2=0x7fdb640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb640b2880 0x7fdb640b4d90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009990 tx=0x7fdb740036e0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb78108740 msgr2=0x7fdb7819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb78108740 0x7fdb7819f660 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fdb70007f10 tx=0x7fdb7000d3b0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/2142090713 shutdown_connections 2024-05-09T01:36:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb640b2880 0x7fdb640b4d90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb78108740 0x7fdb7819f660 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb781034c0 0x7fdb7819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 --2- 172.21.15.19:0/2142090713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb78107df0 0x7fdb7819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/2142090713 >> 172.21.15.19:0/2142090713 conn(0x7fdb780fedf0 msgr2=0x7fdb78075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/2142090713 shutdown_connections 2024-05-09T01:36:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:26.898+0000 7fdb7f2fa700 1 -- 172.21.15.19:0/2142090713 wait complete. 2024-05-09T01:36:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 -- 172.21.15.19:0/3428654523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28104eb0 msgr2=0x7f9f281052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/3428654523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28104eb0 0x7f9f281052c0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009900 tx=0x7f9f1c02a410 comp rx=0 tx=0).stop 2024-05-09T01:36:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 -- 172.21.15.19:0/3428654523 shutdown_connections 2024-05-09T01:36:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/3428654523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28109bb0 0x7f9f2810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/3428654523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28104eb0 0x7f9f281052c0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/3428654523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28103c70 0x7f9f28104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 -- 172.21.15.19:0/3428654523 >> 172.21.15.19:0/3428654523 conn(0x7f9f280fee40 msgr2=0x7f9f28101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 -- 172.21.15.19:0/3428654523 shutdown_connections 2024-05-09T01:36:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 -- 172.21.15.19:0/3428654523 wait complete. 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.190+0000 7f9f2d17d640 1 Processor -- start 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2d17d640 1 -- start start 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2d17d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103c70 0x7f9f2819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2d17d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28104eb0 0x7f9f2819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2d17d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28109bb0 0x7f9f281a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2d17d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2810f790 con 0x7f9f28109bb0 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2d17d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2810f610 con 0x7f9f28103c70 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2d17d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2810f910 con 0x7f9f28104eb0 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28109bb0 0x7f9f281a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28109bb0 0x7f9f281a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35308/0 (socket says 172.21.15.19:35308) 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28104eb0 0x7f9f2819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f2759e640 1 -- 172.21.15.19:0/2652602775 learned_addr learned my addr 172.21.15.19:0/2652602775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.191+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103c70 0x7f9f2819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f2759e640 1 -- 172.21.15.19:0/2652602775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28104eb0 msgr2=0x7f9f2819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f2759e640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28104eb0 0x7f9f2819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f2759e640 1 -- 172.21.15.19:0/2652602775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103c70 msgr2=0x7f9f2819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f2759e640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103c70 0x7f9f2819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f2759e640 1 -- 172.21.15.19:0/2652602775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f281a1eb0 con 0x7f9f28109bb0 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103c70 0x7f9f2819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f2759e640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28109bb0 0x7f9f281a1840 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000eaa0 tx=0x7f9f2000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f0ffff640 1 -- 172.21.15.19:0/2652602775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000cdb0 con 0x7f9f28109bb0 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f2d17d640 1 -- 172.21.15.19:0/2652602775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f281a21a0 con 0x7f9f28109bb0 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f0ffff640 1 -- 172.21.15.19:0/2652602775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f20005020 con 0x7f9f28109bb0 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f0ffff640 1 -- 172.21.15.19:0/2652602775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f200116d0 con 0x7f9f28109bb0 2024-05-09T01:36:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.192+0000 7f9f2d17d640 1 -- 172.21.15.19:0/2652602775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f281a26a0 con 0x7f9f28109bb0 2024-05-09T01:36:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f34032740 1 Processor -- start 2024-05-09T01:36:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f0ffff640 1 -- 172.21.15.19:0/2652602775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9f200051d0 con 0x7f9f28109bb0 2024-05-09T01:36:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f34032740 1 -- start start 2024-05-09T01:36:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f34032740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f221b0 0x55e9422ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f34032740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f34032740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941f014d0 0x55e9422d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f34032740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e94222a7c0 con 0x55e941ef92f0 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f34032740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e9421c60e0 con 0x55e941f014d0 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f34032740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e942226780 con 0x55e941f221b0 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422cae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35322/0 (socket says 172.21.15.19:35322) 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f26d9d640 1 -- 172.21.15.19:0/1224165176 learned_addr learned my addr 172.21.15.19:0/1224165176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f2659c640 1 --2- 172.21.15.19:0/1224165176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f221b0 0x55e9422ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f2659c640 1 --2- 172.21.15.19:0/1224165176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941f014d0 0x55e9422d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f26d9d640 1 -- 172.21.15.19:0/1224165176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f221b0 msgr2=0x55e9422ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/1224165176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f221b0 0x55e9422ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.195+0000 7f9f26d9d640 1 -- 172.21.15.19:0/1224165176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941f014d0 msgr2=0x55e9422d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.196+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/1224165176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941f014d0 0x55e9422d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.196+0000 7f9f0ffff640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9f100b27e0 0x7f9f100b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.196+0000 7f9f26d9d640 1 -- 172.21.15.19:0/1224165176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9421ef7c0 con 0x55e941ef92f0 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.196+0000 7f9f2759e640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9f100b27e0 0x7f9f100b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.196+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/1224165176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422cae90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009cb0 tx=0x7f9f180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3923b681cab8057d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.196+0000 7f9f0ffff640 1 -- 172.21.15.19:0/2652602775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9f200e2430 con 0x7f9f28109bb0 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.196+0000 7f9f0d7fa640 1 -- 172.21.15.19:0/1224165176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004020 con 0x55e941ef92f0 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.196+0000 7f9f0d7fa640 1 -- 172.21.15.19:0/1224165176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f180041c0 con 0x55e941ef92f0 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.196+0000 7f9f0d7fa640 1 -- 172.21.15.19:0/1224165176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18012780 con 0x55e941ef92f0 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 -- 172.21.15.19:0/1224165176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 msgr2=0x55e9422cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f2759e640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9f100b27e0 0x7f9f100b4cf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9f200a99c0 tx=0x7f9f2006f020 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 --2- 172.21.15.19:0/1224165176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422cae90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009cb0 tx=0x7f9f180117b0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 -- 172.21.15.19:0/1224165176 shutdown_connections 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 --2- 172.21.15.19:0/1224165176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941f014d0 0x55e9422d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 --2- 172.21.15.19:0/1224165176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422cae90 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 --2- 172.21.15.19:0/1224165176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f221b0 0x55e9422ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 -- 172.21.15.19:0/1224165176 >> 172.21.15.19:0/1224165176 conn(0x55e941f16300 msgr2=0x55e9422c1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 -- 172.21.15.19:0/1224165176 shutdown_connections 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 -- 172.21.15.19:0/1224165176 wait complete. 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 -- start start 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.197+0000 7f9f34032740 1 Processor -- start 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f34032740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941ef92f0 0x55e9422d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f34032740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f014d0 0x55e9422d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f34032740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f221b0 0x55e9422dba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f34032740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e9422d27e0 con 0x55e941f221b0 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f34032740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e9421b19c0 con 0x55e941ef92f0 2024-05-09T01:36:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f34032740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9422d2960 con 0x55e941f014d0 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f34032740 10 client.0 _ll_register_callbacks cb 0x55e9421de020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f221b0 0x55e9422dba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f221b0 0x55e9422dba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35330/0 (socket says 172.21.15.19:35330) 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 -- 172.21.15.19:0/1927022394 learned_addr learned my addr 172.21.15.19:0/1927022394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941ef92f0 0x55e9422d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f014d0 0x55e9422d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 -- 172.21.15.19:0/1927022394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f014d0 msgr2=0x55e9422d4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f014d0 0x55e9422d4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 -- 172.21.15.19:0/1927022394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941ef92f0 msgr2=0x55e9422d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941ef92f0 0x55e9422d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 -- 172.21.15.19:0/1927022394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9421d81f0 con 0x55e941f221b0 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f2659c640 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f221b0 0x55e9422dba80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c031500 tx=0x7f9f1c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7008163d05b95fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.198+0000 7f9f0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f0aff5640 1 -- 172.21.15.19:0/1927022394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c035040 con 0x55e941f221b0 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f0aff5640 1 -- 172.21.15.19:0/1927022394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9f1c004250 con 0x55e941f221b0 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f34032740 1 -- 172.21.15.19:0/1927022394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f180041c0 con 0x55e941f221b0 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f0aff5640 1 -- 172.21.15.19:0/1927022394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9f1c004450 con 0x55e941f221b0 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f34032740 10 client.18414 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f34032740 1 -- 172.21.15.19:0/1927022394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e941f8d790 con 0x55e941f221b0 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f0aff5640 1 -- 172.21.15.19:0/1927022394 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f180041c0 con 0x55e941f221b0 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f0aff5640 1 -- 172.21.15.19:0/1927022394 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9f1c03d450 con 0x55e941f221b0 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f34032740 10 client.18414 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f34032740 10 client.18414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f34032740 20 client.18414 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f34032740 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e941ee0d80 0x55e9422e1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f34032740 4 client.18414 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.199+0000 7f9f34032740 1 -- 172.21.15.19:0/1927022394 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9422383d0 con 0x55e941ee0d80 2024-05-09T01:36:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.200+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e941ee0d80 0x55e9422e1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.200+0000 7f9f0aff5640 10 client.18414 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.200+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e941ee0d80 0x55e9422e1f00 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e70396019f921c server_cookie=191820c919c0b07e in_seq=0 out_seq=0 2024-05-09T01:36:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f0aff5640 1 -- 172.21.15.19:0/1927022394 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9422383d0 con 0x55e941ee0d80 2024-05-09T01:36:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f0aff5640 10 client.18414 handle_command_reply: tid=0 2024-05-09T01:36:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 client.18414 shutdown 2024-05-09T01:36:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 -- 172.21.15.19:0/1927022394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f221b0 msgr2=0x55e9422dba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f221b0 0x55e9422dba80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c031500 tx=0x7f9f1c009260 comp rx=0 tx=0).stop 2024-05-09T01:36:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 -- 172.21.15.19:0/1927022394 shutdown_connections 2024-05-09T01:36:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e941ee0d80 0x55e9422e1f00 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f221b0 0x55e9422dba80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f014d0 0x55e9422d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f0aff5640 0 client.18414 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 --2- 172.21.15.19:0/1927022394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941ef92f0 0x55e9422d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 -- 172.21.15.19:0/1927022394 >> 172.21.15.19:0/1927022394 conn(0x55e941f16300 msgr2=0x55e9422c0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 -- 172.21.15.19:0/1927022394 shutdown_connections 2024-05-09T01:36:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 1 -- 172.21.15.19:0/1927022394 wait complete. 2024-05-09T01:36:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.201+0000 7f9f34032740 20 client.18414 trim_cache size 0 max 16384 2024-05-09T01:36:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f34032740 1 Processor -- start 2024-05-09T01:36:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f34032740 1 -- start start 2024-05-09T01:36:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f34032740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422c5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f34032740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f014d0 0x55e9422dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f34032740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941f221b0 0x55e9422dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f34032740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281a9250 con 0x55e941ef92f0 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f34032740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281aad90 con 0x55e941f221b0 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f34032740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281a9920 con 0x55e941f014d0 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f2659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f2659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35338/0 (socket says 172.21.15.19:35338) 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f2659c640 1 -- 172.21.15.19:0/1253605874 learned_addr learned my addr 172.21.15.19:0/1253605874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f2659c640 1 -- 172.21.15.19:0/1253605874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f014d0 msgr2=0x55e9422dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f2659c640 1 --2- 172.21.15.19:0/1253605874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f014d0 0x55e9422dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f2659c640 1 -- 172.21.15.19:0/1253605874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941f221b0 msgr2=0x55e9422dbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f2659c640 1 --2- 172.21.15.19:0/1253605874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941f221b0 0x55e9422dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f2659c640 1 -- 172.21.15.19:0/1253605874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9420045f0 con 0x55e941ef92f0 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.217+0000 7f9f2659c640 1 --2- 172.21.15.19:0/1253605874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422c5420 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c031a90 tx=0x7f9f1c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394839568438948f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f0cff9640 1 -- 172.21.15.19:0/1253605874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c035040 con 0x55e941ef92f0 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f0cff9640 1 -- 172.21.15.19:0/1253605874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f1c004250 con 0x55e941ef92f0 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f0cff9640 1 -- 172.21.15.19:0/1253605874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c0035f0 con 0x55e941ef92f0 2024-05-09T01:36:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f34032740 1 -- 172.21.15.19:0/1253605874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 msgr2=0x55e9422c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f34032740 1 --2- 172.21.15.19:0/1253605874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422c5420 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c031a90 tx=0x7f9f1c000bc0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f34032740 1 -- 172.21.15.19:0/1253605874 shutdown_connections 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f34032740 1 --2- 172.21.15.19:0/1253605874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941f221b0 0x55e9422dbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f34032740 1 --2- 172.21.15.19:0/1253605874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f014d0 0x55e9422dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f34032740 1 --2- 172.21.15.19:0/1253605874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941ef92f0 0x55e9422c5420 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f34032740 1 -- 172.21.15.19:0/1253605874 >> 172.21.15.19:0/1253605874 conn(0x55e941f16300 msgr2=0x55e9422c4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f34032740 1 -- 172.21.15.19:0/1253605874 shutdown_connections 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.218+0000 7f9f34032740 1 -- 172.21.15.19:0/1253605874 wait complete. 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f34032740 1 -- start start 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f34032740 1 Processor -- start 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f34032740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941ef92f0 0x55e9422d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f34032740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f014d0 0x55e9422d9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f34032740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f221b0 0x55e9422dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f34032740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281aa6c0 con 0x55e941f014d0 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f34032740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281a9ff0 con 0x55e941ef92f0 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f34032740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281a7230 con 0x55e941f221b0 2024-05-09T01:36:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f34032740 10 client.0 _ll_register_callbacks cb 0x7f9f1c032c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f014d0 0x55e9422d9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f014d0 0x55e9422d9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35354/0 (socket says 172.21.15.19:35354) 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f26d9d640 1 -- 172.21.15.19:0/38005499 learned_addr learned my addr 172.21.15.19:0/38005499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f26d9d640 1 -- 172.21.15.19:0/38005499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f221b0 msgr2=0x55e9422dab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f221b0 0x55e9422dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f26d9d640 1 -- 172.21.15.19:0/38005499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941ef92f0 msgr2=0x55e9422d96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941ef92f0 0x55e9422d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.219+0000 7f9f26d9d640 1 -- 172.21.15.19:0/38005499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9421eeed0 con 0x55e941f014d0 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f26d9d640 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f014d0 0x55e9422d9c30 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9f18011cb0 tx=0x7f9f18009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934910b876afa174 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f0aff5640 1 -- 172.21.15.19:0/38005499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18029070 con 0x55e941f014d0 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f0aff5640 1 -- 172.21.15.19:0/38005499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9f180035f0 con 0x55e941f014d0 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f34032740 1 -- 172.21.15.19:0/38005499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f1c004250 con 0x55e941f014d0 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f0aff5640 1 -- 172.21.15.19:0/38005499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9f18003810 con 0x55e941f014d0 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f34032740 10 client.18420 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f34032740 1 -- 172.21.15.19:0/38005499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9422dc210 con 0x55e941f014d0 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f0aff5640 1 -- 172.21.15.19:0/38005499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f1c004250 con 0x55e941f014d0 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f0aff5640 1 -- 172.21.15.19:0/38005499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9f1802a4e0 con 0x55e941f014d0 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f34032740 10 client.18420 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f34032740 10 client.18420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f34032740 20 client.18420 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f34032740 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e941eb61c0 0x55e94234c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f34032740 4 client.18420 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.220+0000 7f9f34032740 1 -- 172.21.15.19:0/38005499 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e942232240 con 0x55e941eb61c0 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.221+0000 7f9f2659c640 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e941eb61c0 0x55e94234c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.221+0000 7f9f0aff5640 10 client.18420 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.221+0000 7f9f2659c640 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e941eb61c0 0x55e94234c510 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deec92ab7f735ba server_cookie=8e7bc7c2070261c6 in_seq=0 out_seq=0 2024-05-09T01:36:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f0aff5640 1 -- 172.21.15.19:0/38005499 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e942232240 con 0x55e941eb61c0 2024-05-09T01:36:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f0aff5640 10 client.18420 handle_command_reply: tid=0 2024-05-09T01:36:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 client.18420 shutdown 2024-05-09T01:36:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 -- 172.21.15.19:0/38005499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f014d0 msgr2=0x55e9422d9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f014d0 0x55e9422d9c30 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9f18011cb0 tx=0x7f9f18009f30 comp rx=0 tx=0).stop 2024-05-09T01:36:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 -- 172.21.15.19:0/38005499 shutdown_connections 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e941eb61c0 0x55e94234c510 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e941f221b0 0x55e9422dab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e941f014d0 0x55e9422d9c30 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f0aff5640 0 client.18420 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 --2- 172.21.15.19:0/38005499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e941ef92f0 0x55e9422d96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 -- 172.21.15.19:0/38005499 >> 172.21.15.19:0/38005499 conn(0x55e941f16300 msgr2=0x55e9422e66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 -- 172.21.15.19:0/38005499 shutdown_connections 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 1 -- 172.21.15.19:0/38005499 wait complete. 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.237+0000 7f9f34032740 20 client.18420 trim_cache size 0 max 16384 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:36:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:36:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 -- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9f100b27e0 msgr2=0x7f9f100b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9f100b27e0 0x7f9f100b4cf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9f200a99c0 tx=0x7f9f2006f020 comp rx=0 tx=0).stop 2024-05-09T01:36:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 -- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28109bb0 msgr2=0x7f9f281a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28109bb0 0x7f9f281a1840 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000eaa0 tx=0x7f9f2000ef60 comp rx=0 tx=0).stop 2024-05-09T01:36:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 -- 172.21.15.19:0/2652602775 shutdown_connections 2024-05-09T01:36:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9f100b27e0 0x7f9f100b4cf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f28109bb0 0x7f9f281a1840 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28104eb0 0x7f9f2819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 --2- 172.21.15.19:0/2652602775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103c70 0x7f9f2819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 -- 172.21.15.19:0/2652602775 >> 172.21.15.19:0/2652602775 conn(0x7f9f280fee40 msgr2=0x7f9f28105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 -- 172.21.15.19:0/2652602775 shutdown_connections 2024-05-09T01:36:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:27.238+0000 7f9f2d17d640 1 -- 172.21.15.19:0/2652602775 wait complete. 2024-05-09T01:36:27.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:27 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:36:27.583 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:36:27.584 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:36:27.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:27 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:36:27.834 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:27.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:27 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:36:28.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:28 smithi019 ceph-mon[30158]: pgmap v706: 97 pgs: 97 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:28.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:28 smithi196 ceph-mon[35041]: pgmap v706: 97 pgs: 97 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:28 smithi059 ceph-mon[34351]: pgmap v706: 97 pgs: 97 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.548+0000 7f74cfa0e700 1 -- 172.21.15.19:0/774692192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8108760 msgr2=0x7f74c81100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.548+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/774692192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8108760 0x7f74c81100c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f74c000b0d0 tx=0x7f74c000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.548+0000 7f74cfa0e700 1 -- 172.21.15.19:0/774692192 shutdown_connections 2024-05-09T01:36:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.548+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/774692192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8108760 0x7f74c81100c0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.548+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/774692192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74c81034e0 0x7f74c81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.548+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/774692192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c8107e10 0x7f74c8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.548+0000 7f74cfa0e700 1 -- 172.21.15.19:0/774692192 >> 172.21.15.19:0/774692192 conn(0x7f74c80fef30 msgr2=0x7f74c8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.548+0000 7f74cfa0e700 1 -- 172.21.15.19:0/774692192 shutdown_connections 2024-05-09T01:36:29.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.548+0000 7f74cfa0e700 1 -- 172.21.15.19:0/774692192 wait complete. 2024-05-09T01:36:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74cfa0e700 1 Processor -- start 2024-05-09T01:36:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74cfa0e700 1 -- start start 2024-05-09T01:36:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74cfa0e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74c81034e0 0x7f74c819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74cfa0e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8107e10 0x7f74c819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74cfa0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c8108760 0x7f74c819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74cfa0e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8113390 con 0x7f74c8107e10 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74cfa0e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8113210 con 0x7f74c8108760 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74cfa0e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8113510 con 0x7f74c81034e0 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74ccfa9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8107e10 0x7f74c819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74ccfa9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8107e10 0x7f74c819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35376/0 (socket says 172.21.15.19:35376) 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74ccfa9700 1 -- 172.21.15.19:0/1202826921 learned_addr learned my addr 172.21.15.19:0/1202826921 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.549+0000 7f74cd7aa700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74c81034e0 0x7f74c819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74cdfab700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c8108760 0x7f74c819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74ccfa9700 1 -- 172.21.15.19:0/1202826921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74c81034e0 msgr2=0x7f74c819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74ccfa9700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74c81034e0 0x7f74c819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74ccfa9700 1 -- 172.21.15.19:0/1202826921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c8108760 msgr2=0x7f74c819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74ccfa9700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c8108760 0x7f74c819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74ccfa9700 1 -- 172.21.15.19:0/1202826921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c819fb30 con 0x7f74c8107e10 2024-05-09T01:36:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74cdfab700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c8108760 0x7f74c819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74ccfa9700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8107e10 0x7f74c819b2b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c9e0 tx=0x7f74b800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74be7fc700 1 -- 172.21.15.19:0/1202826921 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8007aa0 con 0x7f74c8107e10 2024-05-09T01:36:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74be7fc700 1 -- 172.21.15.19:0/1202826921 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b8007c40 con 0x7f74c8107e10 2024-05-09T01:36:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c819fe20 con 0x7f74c8107e10 2024-05-09T01:36:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74be7fc700 1 -- 172.21.15.19:0/1202826921 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8018660 con 0x7f74c8107e10 2024-05-09T01:36:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.550+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c81a77f0 con 0x7f74c8107e10 2024-05-09T01:36:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.551+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c80512c0 con 0x7f74c8107e10 2024-05-09T01:36:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.553+0000 7f74be7fc700 1 -- 172.21.15.19:0/1202826921 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f74b8018800 con 0x7f74c8107e10 2024-05-09T01:36:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.554+0000 7f74be7fc700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f74b40b2900 0x7f74b40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.554+0000 7f74cd7aa700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f74b40b2900 0x7f74b40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.554+0000 7f74be7fc700 1 -- 172.21.15.19:0/1202826921 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f74b80e23e0 con 0x7f74c8107e10 2024-05-09T01:36:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.554+0000 7f74cd7aa700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f74b40b2900 0x7f74b40b4e10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f74c400ac50 tx=0x7f74c400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.557+0000 7f74be7fc700 1 -- 172.21.15.19:0/1202826921 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74b8014030 con 0x7f74c8107e10 2024-05-09T01:36:29.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.820+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74c819c070 con 0x7f74c8107e10 2024-05-09T01:36:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.821+0000 7f74be7fc700 1 -- 172.21.15.19:0/1202826921 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f74b80a5070 con 0x7f74c8107e10 2024-05-09T01:36:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:36:29.817 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:36:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f74b40b2900 msgr2=0x7f74b40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f74b40b2900 0x7f74b40b4e10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f74c400ac50 tx=0x7f74c400a380 comp rx=0 tx=0).stop 2024-05-09T01:36:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8107e10 msgr2=0x7f74c819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8107e10 0x7f74c819b2b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c9e0 tx=0x7f74b800ccf0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 shutdown_connections 2024-05-09T01:36:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c8108760 0x7f74c819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74c81034e0 0x7f74c819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f74b40b2900 0x7f74b40b4e10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 --2- 172.21.15.19:0/1202826921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c8107e10 0x7f74c819b2b0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 >> 172.21.15.19:0/1202826921 conn(0x7f74c80fef30 msgr2=0x7f74c806c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 shutdown_connections 2024-05-09T01:36:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:29.823+0000 7f74cfa0e700 1 -- 172.21.15.19:0/1202826921 wait complete. 2024-05-09T01:36:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:36:30.483 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:36:30.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:30 smithi196 ceph-mon[35041]: pgmap v707: 97 pgs: 97 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-05-09T01:36:30.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:30 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1202826921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:36:30.738 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:30 smithi059 ceph-mon[34351]: pgmap v707: 97 pgs: 97 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-05-09T01:36:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:30 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1202826921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:36:30.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:30 smithi019 ceph-mon[30158]: pgmap v707: 97 pgs: 97 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-05-09T01:36:30.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:30 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1202826921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.652+0000 7f584da98700 1 -- 172.21.15.19:0/2532506351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 msgr2=0x7f58481100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.652+0000 7f584da98700 1 --2- 172.21.15.19:0/2532506351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 0x7f58481100c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f58400099c0 tx=0x7f5840009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.652+0000 7f584da98700 1 -- 172.21.15.19:0/2532506351 shutdown_connections 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.652+0000 7f584da98700 1 --2- 172.21.15.19:0/2532506351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 0x7f58481100c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.652+0000 7f584da98700 1 --2- 172.21.15.19:0/2532506351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58481034e0 0x7f58481038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.652+0000 7f584da98700 1 --2- 172.21.15.19:0/2532506351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5848107e10 0x7f5848108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.652+0000 7f584da98700 1 -- 172.21.15.19:0/2532506351 >> 172.21.15.19:0/2532506351 conn(0x7f58480fef30 msgr2=0x7f5848101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.652+0000 7f584da98700 1 -- 172.21.15.19:0/2532506351 shutdown_connections 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.652+0000 7f584da98700 1 -- 172.21.15.19:0/2532506351 wait complete. 2024-05-09T01:36:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.653+0000 7f584da98700 1 Processor -- start 2024-05-09T01:36:32.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.653+0000 7f584da98700 1 -- start start 2024-05-09T01:36:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f584da98700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58481034e0 0x7f5848199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f584da98700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5848107e10 0x7f584819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f584da98700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 0x7f584819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f584da98700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5848113390 con 0x7f5848108760 2024-05-09T01:36:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f584da98700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5848113210 con 0x7f58481034e0 2024-05-09T01:36:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f584da98700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5848113510 con 0x7f5848107e10 2024-05-09T01:36:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5847fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 0x7f584819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5847fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 0x7f584819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35504/0 (socket says 172.21.15.19:35504) 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5847fff700 1 -- 172.21.15.19:0/2188469789 learned_addr learned my addr 172.21.15.19:0/2188469789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f58477fe700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58481034e0 0x7f5848199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5846ffd700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5848107e10 0x7f584819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5847fff700 1 -- 172.21.15.19:0/2188469789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5848107e10 msgr2=0x7f584819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5847fff700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5848107e10 0x7f584819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5847fff700 1 -- 172.21.15.19:0/2188469789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58481034e0 msgr2=0x7f5848199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5847fff700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58481034e0 0x7f5848199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5847fff700 1 -- 172.21.15.19:0/2188469789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584819ec50 con 0x7f5848108760 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5847fff700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 0x7f584819e5b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5840005910 tx=0x7f5840009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.654+0000 7f5844ff9700 1 -- 172.21.15.19:0/2188469789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840027070 con 0x7f5848108760 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.655+0000 7f5844ff9700 1 -- 172.21.15.19:0/2188469789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5840003f20 con 0x7f5848108760 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.655+0000 7f584da98700 1 -- 172.21.15.19:0/2188469789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584819eee0 con 0x7f5848108760 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.655+0000 7f5844ff9700 1 -- 172.21.15.19:0/2188469789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840022c50 con 0x7f5848108760 2024-05-09T01:36:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.655+0000 7f584da98700 1 -- 172.21.15.19:0/2188469789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584819f3c0 con 0x7f5848108760 2024-05-09T01:36:32.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.657+0000 7f5844ff9700 1 -- 172.21.15.19:0/2188469789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f584002a070 con 0x7f5848108760 2024-05-09T01:36:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.658+0000 7f5844ff9700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f58300b2880 0x7f58300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.658+0000 7f58477fe700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f58300b2880 0x7f58300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.658+0000 7f5844ff9700 1 -- 172.21.15.19:0/2188469789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f58400e2e90 con 0x7f5848108760 2024-05-09T01:36:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.658+0000 7f585c4aeb80 1 Processor -- start 2024-05-09T01:36:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.658+0000 7f585c4aeb80 1 -- start start 2024-05-09T01:36:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f58477fe700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f58300b2880 0x7f58300b4d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f58380060b0 tx=0x7f583800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:32.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:32 smithi019 ceph-mon[30158]: pgmap v708: 97 pgs: 97 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 0x563005bce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005a4b4a0 0x563005bcea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 0x563005bc8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563005aeaff0 con 0x563005a4b4a0 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563005a897f0 con 0x563005bc86a0 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563005ae6fb0 con 0x563005b35160 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f5846ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005a4b4a0 0x563005bcea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f5846ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005a4b4a0 0x563005bcea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35516/0 (socket says 172.21.15.19:35516) 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f5846ffd700 1 -- 172.21.15.19:0/1691142070 learned_addr learned my addr 172.21.15.19:0/1691142070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f58477fe700 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 0x563005bce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f5846ffd700 1 -- 172.21.15.19:0/1691142070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 msgr2=0x563005bce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f5846ffd700 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 0x563005bce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.660+0000 7f5847fff700 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 0x563005bc8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.659+0000 7f5846ffd700 1 -- 172.21.15.19:0/1691142070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 msgr2=0x563005bc8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.660+0000 7f5846ffd700 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 0x563005bc8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.660+0000 7f5846ffd700 1 -- 172.21.15.19:0/1691142070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563005a81370 con 0x563005a4b4a0 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.660+0000 7f5847fff700 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 0x563005bc8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.660+0000 7f5846ffd700 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005a4b4a0 0x563005bcea30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f583c00fe10 tx=0x7f583c00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b7a71b2399d620 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.660+0000 7f582dffb700 1 -- 172.21.15.19:0/1691142070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c01d070 con 0x563005a4b4a0 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.660+0000 7f582dffb700 1 -- 172.21.15.19:0/1691142070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f583c018410 con 0x563005a4b4a0 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.660+0000 7f582dffb700 1 -- 172.21.15.19:0/1691142070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c017570 con 0x563005a4b4a0 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.661+0000 7f585c4aeb80 1 -- 172.21.15.19:0/1691142070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005a4b4a0 msgr2=0x563005bcea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.661+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005a4b4a0 0x563005bcea30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f583c00fe10 tx=0x7f583c00b5a0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.661+0000 7f585c4aeb80 1 -- 172.21.15.19:0/1691142070 shutdown_connections 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.661+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 0x563005bc8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.661+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 0x563005bce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.661+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/1691142070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005a4b4a0 0x563005bcea30 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.661+0000 7f585c4aeb80 1 -- 172.21.15.19:0/1691142070 >> 172.21.15.19:0/1691142070 conn(0x563005aab6a0 msgr2=0x563005bc3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.661+0000 7f585c4aeb80 1 -- 172.21.15.19:0/1691142070 shutdown_connections 2024-05-09T01:36:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.661+0000 7f585c4aeb80 1 -- 172.21.15.19:0/1691142070 wait complete. 2024-05-09T01:36:32.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.662+0000 7f585c4aeb80 1 -- start start 2024-05-09T01:36:32.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.662+0000 7f585c4aeb80 1 Processor -- start 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.662+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 0x563005bd21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.662+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005bd2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.662+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005bc86a0 0x563005bd2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563005966ba0 con 0x563005b35160 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563005aa5150 con 0x563005a4b4a0 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563005bd51d0 con 0x563005bc86a0 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5846ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005bd2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5846ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005bd2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35518/0 (socket says 172.21.15.19:35518) 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5846ffd700 1 -- 172.21.15.19:0/217864209 learned_addr learned my addr 172.21.15.19:0/217864209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f585c4aeb80 10 client.0 _ll_register_callbacks cb 0x56300595c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f58477fe700 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 0x563005bd21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5847fff700 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005bc86a0 0x563005bd2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5846ffd700 1 -- 172.21.15.19:0/217864209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005bc86a0 msgr2=0x563005bd2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5846ffd700 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005bc86a0 0x563005bd2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5846ffd700 1 -- 172.21.15.19:0/217864209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 msgr2=0x563005bd21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5846ffd700 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 0x563005bd21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5846ffd700 1 -- 172.21.15.19:0/217864209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563005966490 con 0x563005b35160 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5847fff700 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005bc86a0 0x563005bd2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5846ffd700 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005bd2710 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f583c018870 tx=0x7f583c0188a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498b49b788fd8794 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5826ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5826ffd700 1 -- 172.21.15.19:0/217864209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c024070 con 0x563005b35160 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f5826ffd700 1 -- 172.21.15.19:0/217864209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f583c005250 con 0x563005b35160 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.663+0000 7f585c4aeb80 1 -- 172.21.15.19:0/217864209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f583c018410 con 0x563005b35160 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f5826ffd700 1 -- 172.21.15.19:0/217864209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f583c030050 con 0x563005b35160 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f585c4aeb80 10 client.18438 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f585c4aeb80 1 -- 172.21.15.19:0/217864209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563005bdfb10 con 0x563005b35160 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f5826ffd700 1 -- 172.21.15.19:0/217864209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f583c018410 con 0x563005b35160 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f5826ffd700 1 -- 172.21.15.19:0/217864209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f583c020580 con 0x563005b35160 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f585c4aeb80 10 client.18438 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f585c4aeb80 10 client.18438 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f585c4aeb80 20 client.18438 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563005be2040 0x563005be4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f585c4aeb80 4 client.18438 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.664+0000 7f585c4aeb80 1 -- 172.21.15.19:0/217864209 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563005af89d0 con 0x563005be2040 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.665+0000 7f58477fe700 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563005be2040 0x563005be4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.665+0000 7f5826ffd700 10 client.18438 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.665+0000 7f58477fe700 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563005be2040 0x563005be4460 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ba56478403a2959 server_cookie=8ffc0296942e223d in_seq=0 out_seq=0 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f5826ffd700 1 -- 172.21.15.19:0/217864209 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563005af89d0 con 0x563005be2040 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f5826ffd700 10 client.18438 handle_command_reply: tid=0 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f585c4aeb80 1 client.18438 shutdown 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f585c4aeb80 1 -- 172.21.15.19:0/217864209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 msgr2=0x563005bd2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005bd2710 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f583c018870 tx=0x7f583c0188a0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f585c4aeb80 1 -- 172.21.15.19:0/217864209 shutdown_connections 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563005be2040 0x563005be4460 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f5826ffd700 0 client.18438 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:36:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005bc86a0 0x563005bd2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 0x563005bd21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/217864209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005bd2710 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.666+0000 7f585c4aeb80 1 -- 172.21.15.19:0/217864209 >> 172.21.15.19:0/217864209 conn(0x563005aab6a0 msgr2=0x563005bc31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:32.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.667+0000 7f585c4aeb80 1 -- 172.21.15.19:0/217864209 shutdown_connections 2024-05-09T01:36:32.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.667+0000 7f585c4aeb80 1 -- 172.21.15.19:0/217864209 wait complete. 2024-05-09T01:36:32.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.667+0000 7f585c4aeb80 20 client.18438 trim_cache size 0 max 16384 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f585c4aeb80 1 Processor -- start 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f585c4aeb80 1 -- start start 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 0x563005bd4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 0x563005bcd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005bc86a0 0x563005bde370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58481ac960 con 0x563005bc86a0 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58481ae4a0 con 0x563005a4b4a0 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58481ad030 con 0x563005b35160 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f5847fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005bc86a0 0x563005bde370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f5847fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005bc86a0 0x563005bde370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35532/0 (socket says 172.21.15.19:35532) 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f5847fff700 1 -- 172.21.15.19:0/3322100946 learned_addr learned my addr 172.21.15.19:0/3322100946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f58477fe700 1 --2- 172.21.15.19:0/3322100946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 0x563005bd4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.682+0000 7f5846ffd700 1 --2- 172.21.15.19:0/3322100946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 0x563005bcd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.683+0000 7f5847fff700 1 -- 172.21.15.19:0/3322100946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 msgr2=0x563005bcd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.683+0000 7f5847fff700 1 --2- 172.21.15.19:0/3322100946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 0x563005bcd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.683+0000 7f5847fff700 1 -- 172.21.15.19:0/3322100946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 msgr2=0x563005bd4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.683+0000 7f5847fff700 1 --2- 172.21.15.19:0/3322100946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 0x563005bd4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.683+0000 7f5847fff700 1 -- 172.21.15.19:0/3322100946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563005bd14f0 con 0x563005bc86a0 2024-05-09T01:36:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.683+0000 7f5847fff700 1 --2- 172.21.15.19:0/3322100946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005bc86a0 0x563005bde370 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5840009990 tx=0x7f58400aa2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40a7e6ac9bbbfbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.683+0000 7f582d7fa700 1 -- 172.21.15.19:0/3322100946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840022470 con 0x563005bc86a0 2024-05-09T01:36:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.683+0000 7f582d7fa700 1 -- 172.21.15.19:0/3322100946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58400b2e60 con 0x563005bc86a0 2024-05-09T01:36:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.683+0000 7f582d7fa700 1 -- 172.21.15.19:0/3322100946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840092340 con 0x563005bc86a0 2024-05-09T01:36:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3322100946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005bc86a0 msgr2=0x563005bde370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3322100946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005bc86a0 0x563005bde370 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5840009990 tx=0x7f58400aa2e0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3322100946 shutdown_connections 2024-05-09T01:36:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3322100946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005bc86a0 0x563005bde370 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3322100946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005a4b4a0 0x563005bd4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3322100946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005b35160 0x563005bcd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3322100946 >> 172.21.15.19:0/3322100946 conn(0x563005aab6a0 msgr2=0x563005bcd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3322100946 shutdown_connections 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3322100946 wait complete. 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.684+0000 7f585c4aeb80 1 -- start start 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f585c4aeb80 1 Processor -- start 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005a4b4a0 0x563005be7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005be7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f585c4aeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 0x563005be4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58481addd0 con 0x563005b35160 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58481ad700 con 0x563005bc86a0 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f585c4aeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58481aa900 con 0x563005a4b4a0 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f585c4aeb80 10 client.0 _ll_register_callbacks cb 0x56300595c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:32.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f5846ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005be7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f5846ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005be7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35544/0 (socket says 172.21.15.19:35544) 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f5846ffd700 1 -- 172.21.15.19:0/3879369625 learned_addr learned my addr 172.21.15.19:0/3879369625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f5847fff700 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 0x563005be4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f58477fe700 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005a4b4a0 0x563005be7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f5846ffd700 1 -- 172.21.15.19:0/3879369625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005a4b4a0 msgr2=0x563005be7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f5846ffd700 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005a4b4a0 0x563005be7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f5846ffd700 1 -- 172.21.15.19:0/3879369625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 msgr2=0x563005be4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f5846ffd700 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 0x563005be4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.685+0000 7f5846ffd700 1 -- 172.21.15.19:0/3879369625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563005be52d0 con 0x563005b35160 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f5846ffd700 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005be7b10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x563005bbe3d0 tx=0x7f583c005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121f37fa36ab7081 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f5826ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f5826ffd700 1 -- 172.21.15.19:0/3879369625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c039060 con 0x563005b35160 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3879369625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58400b2e60 con 0x563005b35160 2024-05-09T01:36:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f5826ffd700 1 -- 172.21.15.19:0/3879369625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f583c020b40 con 0x563005b35160 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f5826ffd700 1 -- 172.21.15.19:0/3879369625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f583c024040 con 0x563005b35160 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f585c4aeb80 10 client.18444 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3879369625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563005c4c070 con 0x563005b35160 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f5826ffd700 1 -- 172.21.15.19:0/3879369625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58400b2e60 con 0x563005b35160 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.686+0000 7f5826ffd700 1 -- 172.21.15.19:0/3879369625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f583c004710 con 0x563005b35160 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.687+0000 7f585c4aeb80 10 client.18444 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.687+0000 7f585c4aeb80 10 client.18444 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.687+0000 7f585c4aeb80 20 client.18444 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.687+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563005c4c720 0x563005c4eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.687+0000 7f585c4aeb80 4 client.18444 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.687+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3879369625 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563005af2a70 con 0x563005c4c720 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.687+0000 7f58477fe700 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563005c4c720 0x563005c4eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.687+0000 7f5826ffd700 10 client.18444 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.687+0000 7f58477fe700 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563005c4c720 0x563005c4eb40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da22ecc76d45f869 server_cookie=273f4b37bbc9d019 in_seq=0 out_seq=0 2024-05-09T01:36:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f5826ffd700 1 -- 172.21.15.19:0/3879369625 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563005af2a70 con 0x563005c4c720 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f5826ffd700 10 client.18444 handle_command_reply: tid=0 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 client.18444 shutdown 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3879369625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 msgr2=0x563005be7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005be7b10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x563005bbe3d0 tx=0x7f583c005c60 comp rx=0 tx=0).stop 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3879369625 shutdown_connections 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x563005c4c720 0x563005c4eb40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563005bc86a0 0x563005be4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563005a4b4a0 0x563005be7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 --2- 172.21.15.19:0/3879369625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563005b35160 0x563005be7b10 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3879369625 >> 172.21.15.19:0/3879369625 conn(0x563005aab6a0 msgr2=0x563005bcbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f5826ffd700 0 client.18444 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3879369625 shutdown_connections 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 1 -- 172.21.15.19:0/3879369625 wait complete. 2024-05-09T01:36:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.688+0000 7f585c4aeb80 20 client.18444 trim_cache size 0 max 16384 2024-05-09T01:36:32.689 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:32.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 -- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f58300b2880 msgr2=0x7f58300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:32.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f58300b2880 0x7f58300b4d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f58380060b0 tx=0x7f583800c040 comp rx=0 tx=0).stop 2024-05-09T01:36:32.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 -- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 msgr2=0x7f584819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:32.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 0x7f584819e5b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5840005910 tx=0x7f5840009580 comp rx=0 tx=0).stop 2024-05-09T01:36:32.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 -- 172.21.15.19:0/2188469789 shutdown_connections 2024-05-09T01:36:32.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f58300b2880 0x7f58300b4d90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5848108760 0x7f584819e5b0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58481034e0 0x7f5848199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 --2- 172.21.15.19:0/2188469789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5848107e10 0x7f584819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:32.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 -- 172.21.15.19:0/2188469789 >> 172.21.15.19:0/2188469789 conn(0x7f58480fef30 msgr2=0x7f5848107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:32.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 -- 172.21.15.19:0/2188469789 shutdown_connections 2024-05-09T01:36:32.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:32.689+0000 7f584da98700 1 -- 172.21.15.19:0/2188469789 wait complete. 2024-05-09T01:36:32.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:32 smithi196 ceph-mon[35041]: pgmap v708: 97 pgs: 97 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:36:32.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:32 smithi059 ceph-mon[34351]: pgmap v708: 97 pgs: 97 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:36:33.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:36:33.604 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:34.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:34 smithi019 ceph-mon[30158]: pgmap v709: 97 pgs: 97 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:36:34.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:34 smithi196 ceph-mon[35041]: pgmap v709: 97 pgs: 97 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:36:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:34 smithi059 ceph-mon[34351]: pgmap v709: 97 pgs: 97 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-05-09T01:36:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.243+0000 7f2f96132700 1 -- 172.21.15.19:0/2492861104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90107df0 msgr2=0x7f2f90108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.243+0000 7f2f96132700 1 --2- 172.21.15.19:0/2492861104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90107df0 0x7f2f90108200 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2f780099c0 tx=0x7f2f78009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.244+0000 7f2f96132700 1 -- 172.21.15.19:0/2492861104 shutdown_connections 2024-05-09T01:36:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.244+0000 7f2f96132700 1 --2- 172.21.15.19:0/2492861104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f90108740 0x7f2f901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.244+0000 7f2f96132700 1 --2- 172.21.15.19:0/2492861104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f901034c0 0x7f2f901038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.244+0000 7f2f96132700 1 --2- 172.21.15.19:0/2492861104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90107df0 0x7f2f90108200 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.244+0000 7f2f96132700 1 -- 172.21.15.19:0/2492861104 >> 172.21.15.19:0/2492861104 conn(0x7f2f900fedf0 msgr2=0x7f2f90101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.244+0000 7f2f96132700 1 -- 172.21.15.19:0/2492861104 shutdown_connections 2024-05-09T01:36:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.244+0000 7f2f96132700 1 -- 172.21.15.19:0/2492861104 wait complete. 2024-05-09T01:36:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.244+0000 7f2f96132700 1 Processor -- start 2024-05-09T01:36:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.244+0000 7f2f96132700 1 -- start start 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f96132700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f901034c0 0x7f2f9019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f96132700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f90107df0 0x7f2f9019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f96132700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90108740 0x7f2f9019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f96132700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901133c0 con 0x7f2f90108740 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f96132700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90113240 con 0x7f2f901034c0 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f96132700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90113540 con 0x7f2f90107df0 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f8ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90108740 0x7f2f9019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f86ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f90107df0 0x7f2f9019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f8ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90108740 0x7f2f9019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35560/0 (socket says 172.21.15.19:35560) 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f8ffff700 1 -- 172.21.15.19:0/2976732266 learned_addr learned my addr 172.21.15.19:0/2976732266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f901034c0 0x7f2f9019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f8ffff700 1 -- 172.21.15.19:0/2976732266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f90107df0 msgr2=0x7f2f9019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f90107df0 0x7f2f9019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f8ffff700 1 -- 172.21.15.19:0/2976732266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f901034c0 msgr2=0x7f2f9019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f901034c0 0x7f2f9019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.245+0000 7f2f8ffff700 1 -- 172.21.15.19:0/2976732266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9019fd70 con 0x7f2f90108740 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.246+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f901034c0 0x7f2f9019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.246+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90108740 0x7f2f9019f6d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800ada0 tx=0x7f2f8800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.246+0000 7f2f8d7fa700 1 -- 172.21.15.19:0/2976732266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88017020 con 0x7f2f90108740 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.246+0000 7f2f8d7fa700 1 -- 172.21.15.19:0/2976732266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f8800ce30 con 0x7f2f90108740 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.246+0000 7f2f96132700 1 -- 172.21.15.19:0/2976732266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f901a0060 con 0x7f2f90108740 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.246+0000 7f2f8d7fa700 1 -- 172.21.15.19:0/2976732266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88005350 con 0x7f2f90108740 2024-05-09T01:36:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.246+0000 7f2f96132700 1 -- 172.21.15.19:0/2976732266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f901a7980 con 0x7f2f90108740 2024-05-09T01:36:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.249+0000 7f2f8d7fa700 1 -- 172.21.15.19:0/2976732266 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2f88005510 con 0x7f2f90108740 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.249+0000 7f2f8d7fa700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f7c0b2880 0x7f2f7c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f7c0b2880 0x7f2f7c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2f8d7fa700 1 -- 172.21.15.19:0/2976732266 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2f880e19c0 con 0x7f2f90108740 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2fa4b48b80 1 Processor -- start 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2fa4b48b80 1 -- start start 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e94c160 0x55e38e9e3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f7c0b2880 0x7f2f7c0b4d90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f800099d0 tx=0x7f2f80008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e8624a0 0x55e38e9e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e9e8400 0x55e38e9e8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e38e901ff0 con 0x55e38e9e8400 2024-05-09T01:36:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e38e8a07f0 con 0x55e38e94c160 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e38e8fdfb0 con 0x55e38e8624a0 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2f8ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e9e8400 0x55e38e9e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2f8ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e9e8400 0x55e38e9e8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35570/0 (socket says 172.21.15.19:35570) 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2f8ffff700 1 -- 172.21.15.19:0/1431528760 learned_addr learned my addr 172.21.15.19:0/1431528760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e8624a0 0x55e38e9e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.250+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e94c160 0x55e38e9e3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f8ffff700 1 -- 172.21.15.19:0/1431528760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e8624a0 msgr2=0x55e38e9e42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e8624a0 0x55e38e9e42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f8ffff700 1 -- 172.21.15.19:0/1431528760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e94c160 msgr2=0x55e38e9e3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e94c160 0x55e38e9e3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f8ffff700 1 -- 172.21.15.19:0/1431528760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e38e898370 con 0x55e38e9e8400 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e8624a0 0x55e38e9e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e94c160 0x55e38e9e3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e9e8400 0x55e38e9e8810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2f880a62c0 tx=0x7f2f8800a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929d42cc5aa6568c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f85ffb700 1 -- 172.21.15.19:0/1431528760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8806e070 con 0x55e38e9e8400 2024-05-09T01:36:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f85ffb700 1 -- 172.21.15.19:0/1431528760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f880a9050 con 0x55e38e9e8400 2024-05-09T01:36:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.251+0000 7f2f85ffb700 1 -- 172.21.15.19:0/1431528760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f880a94b0 con 0x55e38e9e8400 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.252+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/1431528760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e9e8400 msgr2=0x55e38e9e8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.252+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e9e8400 0x55e38e9e8810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2f880a62c0 tx=0x7f2f8800a1f0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.252+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/1431528760 shutdown_connections 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.252+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e9e8400 0x55e38e9e8810 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.252+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e94c160 0x55e38e9e3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.252+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/1431528760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e8624a0 0x55e38e9e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.252+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/1431528760 >> 172.21.15.19:0/1431528760 conn(0x55e38e8c26a0 msgr2=0x55e38e9da400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.252+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/1431528760 shutdown_connections 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.252+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/1431528760 wait complete. 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2fa4b48b80 1 -- start start 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2fa4b48b80 1 Processor -- start 2024-05-09T01:36:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9ed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e38e77dba0 con 0x55e38e94c160 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e38e8bc150 con 0x55e38e8624a0 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e38e9e91b0 con 0x55e38e9e8400 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2fa4b48b80 10 client.0 _ll_register_callbacks cb 0x55e38e7736c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2f86ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2f86ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9edbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35582/0 (socket says 172.21.15.19:35582) 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3001375676 learned_addr learned my addr 172.21.15.19:0/3001375676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.253+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3001375676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 msgr2=0x55e38e9ee100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9ee100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3001375676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 msgr2=0x55e38e9ed680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9ed680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3001375676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e38e77d490 con 0x55e38e94c160 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9edbc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009990 tx=0x7f2f780052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fafb651d3176e54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f6effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f6effd700 1 -- 172.21.15.19:0/3001375676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7800b990 con 0x55e38e94c160 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f6effd700 1 -- 172.21.15.19:0/3001375676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2f7800bb30 con 0x55e38e94c160 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3001375676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f880a9050 con 0x55e38e94c160 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2f6effd700 1 -- 172.21.15.19:0/3001375676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f7800bd30 con 0x55e38e94c160 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2fa4b48b80 10 client.18456 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.254+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3001375676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e38e9faff0 con 0x55e38e94c160 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.255+0000 7f2f6effd700 1 -- 172.21.15.19:0/3001375676 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f880a9050 con 0x55e38e94c160 2024-05-09T01:36:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.255+0000 7f2f6effd700 1 -- 172.21.15.19:0/3001375676 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2f7800f460 con 0x55e38e94c160 2024-05-09T01:36:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.255+0000 7f2fa4b48b80 10 client.18456 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.255+0000 7f2fa4b48b80 10 client.18456 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:36:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.255+0000 7f2fa4b48b80 20 client.18456 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.255+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e38e9fd680 0x55e38e9ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.255+0000 7f2fa4b48b80 4 client.18456 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.255+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3001375676 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e38e90f9d0 con 0x55e38e9fd680 2024-05-09T01:36:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.255+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e38e9fd680 0x55e38e9ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.256+0000 7f2f6effd700 10 client.18456 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:36:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.256+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e38e9fd680 0x55e38e9ffaa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e08efca814a2b90a server_cookie=d13a7c146869c288 in_seq=0 out_seq=0 2024-05-09T01:36:35.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2f6effd700 1 -- 172.21.15.19:0/3001375676 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e38e90f9d0 con 0x55e38e9fd680 2024-05-09T01:36:35.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2f6effd700 10 client.18456 handle_command_reply: tid=0 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 client.18456 shutdown 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3001375676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 msgr2=0x55e38e9edbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9edbc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009990 tx=0x7f2f780052e0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3001375676 shutdown_connections 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e38e9fd680 0x55e38e9ffaa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9ee100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3001375676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9edbc0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3001375676 >> 172.21.15.19:0/3001375676 conn(0x55e38e8c26a0 msgr2=0x55e38e9da1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2f6effd700 0 client.18456 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3001375676 shutdown_connections 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3001375676 wait complete. 2024-05-09T01:36:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.257+0000 7f2fa4b48b80 20 client.18456 trim_cache size 0 max 16384 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2fa4b48b80 1 Processor -- start 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2fa4b48b80 1 -- start start 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9edf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9ec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9f9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ac970 con 0x55e38e94c160 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ae4b0 con 0x55e38e8624a0 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ad040 con 0x55e38e9e8400 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f86ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f86ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35594/0 (socket says 172.21.15.19:35594) 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f86ffd700 1 -- 172.21.15.19:0/388920064 learned_addr learned my addr 172.21.15.19:0/388920064 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f86ffd700 1 -- 172.21.15.19:0/388920064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 msgr2=0x55e38e9f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f86ffd700 1 -- 172.21.15.19:0/388920064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 msgr2=0x55e38e9edf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9edf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f86ffd700 1 -- 172.21.15.19:0/388920064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e38ea00000 con 0x55e38e94c160 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.273+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9edf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.274+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9ec680 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800fa20 tx=0x7f2f78005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cf7d1464fe1c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.274+0000 7f2f857fa700 1 -- 172.21.15.19:0/388920064 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f780058e0 con 0x55e38e94c160 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.274+0000 7f2f857fa700 1 -- 172.21.15.19:0/388920064 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f78028070 con 0x55e38e94c160 2024-05-09T01:36:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.274+0000 7f2f857fa700 1 -- 172.21.15.19:0/388920064 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78023470 con 0x55e38e94c160 2024-05-09T01:36:35.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.274+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/388920064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 msgr2=0x55e38e9ec680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:35.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.274+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9ec680 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800fa20 tx=0x7f2f78005480 comp rx=0 tx=0).stop 2024-05-09T01:36:35.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.275+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/388920064 shutdown_connections 2024-05-09T01:36:35.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.275+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.275+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9edf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.275+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/388920064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9ec680 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.275+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/388920064 >> 172.21.15.19:0/388920064 conn(0x55e38e8c26a0 msgr2=0x55e38e9ebf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.275+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/388920064 shutdown_connections 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.275+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/388920064 wait complete. 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.275+0000 7f2fa4b48b80 1 -- start start 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.275+0000 7f2fa4b48b80 1 Processor -- start 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2fa4b48b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901adde0 con 0x55e38e94c160 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ad710 con 0x55e38e8624a0 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2fa4b48b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901aa930 con 0x55e38e9e8400 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2fa4b48b80 10 client.0 _ll_register_callbacks cb 0x55e38e7736c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f86ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f86ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35606/0 (socket says 172.21.15.19:35606) 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3100888558 learned_addr learned my addr 172.21.15.19:0/3100888558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:35.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f8ffff700 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3100888558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 msgr2=0x55e38e9e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9e6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3100888558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 msgr2=0x55e38e9f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3100888558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e38e9e6770 con 0x55e38e94c160 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9f5cc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2f78039040 tx=0x7f2f7800fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5b67ce4e8b1b51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f6effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f6effd700 1 -- 172.21.15.19:0/3100888558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7800b810 con 0x55e38e94c160 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2f6effd700 1 -- 172.21.15.19:0/3100888558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2f7800b9b0 con 0x55e38e94c160 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.276+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3100888558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f78028070 con 0x55e38e94c160 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2f6effd700 1 -- 172.21.15.19:0/3100888558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f78023470 con 0x55e38e94c160 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2fa4b48b80 10 client.18462 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3100888558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e38ea04850 con 0x55e38e94c160 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2f6effd700 1 -- 172.21.15.19:0/3100888558 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f78028070 con 0x55e38e94c160 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2f6effd700 1 -- 172.21.15.19:0/3100888558 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2f780407a0 con 0x55e38e94c160 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2fa4b48b80 10 client.18462 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2fa4b48b80 10 client.18462 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2fa4b48b80 20 client.18462 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e38ea67e30 0x55e38ea6a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2fa4b48b80 4 client.18462 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.277+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3100888558 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e38e9e8210 con 0x55e38ea67e30 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.278+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e38ea67e30 0x55e38ea6a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.278+0000 7f2f6effd700 10 client.18462 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.278+0000 7f2f8f7fe700 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e38ea67e30 0x55e38ea6a250 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea15057debe6d2ab server_cookie=ca46ecd243b34aad in_seq=0 out_seq=0 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.278+0000 7f2f6effd700 1 -- 172.21.15.19:0/3100888558 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e38e9e8210 con 0x55e38ea67e30 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.278+0000 7f2f6effd700 10 client.18462 handle_command_reply: tid=0 2024-05-09T01:36:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.278+0000 7f2fa4b48b80 1 client.18462 shutdown 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.278+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3100888558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 msgr2=0x55e38e9f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.278+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9f5cc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2f78039040 tx=0x7f2f7800fe00 comp rx=0 tx=0).stop 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3100888558 shutdown_connections 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x55e38ea67e30 0x55e38ea6a250 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e38e9e8400 0x55e38e9e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e38e8624a0 0x55e38e9f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2fa4b48b80 1 --2- 172.21.15.19:0/3100888558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e38e94c160 0x55e38e9f5cc0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2f6effd700 0 client.18462 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3100888558 >> 172.21.15.19:0/3100888558 conn(0x55e38e8c26a0 msgr2=0x55e38e9de690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3100888558 shutdown_connections 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2fa4b48b80 1 -- 172.21.15.19:0/3100888558 wait complete. 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2fa4b48b80 20 client.18462 trim_cache size 0 max 16384 2024-05-09T01:36:35.280 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2f96132700 1 -- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f7c0b2880 msgr2=0x7f2f7c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2f96132700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f7c0b2880 0x7f2f7c0b4d90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f800099d0 tx=0x7f2f80008040 comp rx=0 tx=0).stop 2024-05-09T01:36:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2f96132700 1 -- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90108740 msgr2=0x7f2f9019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.279+0000 7f2f96132700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90108740 0x7f2f9019f6d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800ada0 tx=0x7f2f8800c330 comp rx=0 tx=0).stop 2024-05-09T01:36:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.280+0000 7f2f96132700 1 -- 172.21.15.19:0/2976732266 shutdown_connections 2024-05-09T01:36:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.280+0000 7f2f96132700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f7c0b2880 0x7f2f7c0b4d90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.280+0000 7f2f96132700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90108740 0x7f2f9019f6d0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.280+0000 7f2f96132700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f901034c0 0x7f2f9019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.280+0000 7f2f96132700 1 --2- 172.21.15.19:0/2976732266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f90107df0 0x7f2f9019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.280+0000 7f2f96132700 1 -- 172.21.15.19:0/2976732266 >> 172.21.15.19:0/2976732266 conn(0x7f2f900fedf0 msgr2=0x7f2f90075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.280+0000 7f2f96132700 1 -- 172.21.15.19:0/2976732266 shutdown_connections 2024-05-09T01:36:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:35.280+0000 7f2f96132700 1 -- 172.21.15.19:0/2976732266 wait complete. 2024-05-09T01:36:35.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:36:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:36.592 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:36 smithi019 ceph-mon[30158]: pgmap v710: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:36:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:36 smithi059 ceph-mon[34351]: pgmap v710: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:36:36.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:36 smithi196 ceph-mon[35041]: pgmap v710: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-05-09T01:36:37.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.903+0000 7f91fe0fb700 1 -- 172.21.15.19:0/4242106144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 msgr2=0x7f91f8109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.903+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/4242106144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 0x7f91f8109340 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0099c0 tx=0x7f91ec009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.903+0000 7f91fe0fb700 1 -- 172.21.15.19:0/4242106144 shutdown_connections 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.903+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/4242106144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f8109880 0x7f91f81111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.903+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/4242106144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91f8104650 0x7f91f8104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.903+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/4242106144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 0x7f91f8109340 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.903+0000 7f91fe0fb700 1 -- 172.21.15.19:0/4242106144 >> 172.21.15.19:0/4242106144 conn(0x7f91f8079f00 msgr2=0x7f91f807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.903+0000 7f91fe0fb700 1 -- 172.21.15.19:0/4242106144 shutdown_connections 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.903+0000 7f91fe0fb700 1 -- 172.21.15.19:0/4242106144 wait complete. 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.904+0000 7f91fe0fb700 1 Processor -- start 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.904+0000 7f91fe0fb700 1 -- start start 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91fe0fb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f8104650 0x7f91f8075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91fe0fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 0x7f91f8076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91fe0fb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91f8109880 0x7f91f8076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91fe0fb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91f81144b0 con 0x7f91f8108f30 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91fe0fb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91f8114330 con 0x7f91f8104650 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91fe0fb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91f8114630 con 0x7f91f8109880 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 0x7f91f8076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 0x7f91f8076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35630/0 (socket says 172.21.15.19:35630) 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f6ffd700 1 -- 172.21.15.19:0/834448127 learned_addr learned my addr 172.21.15.19:0/834448127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f7fff700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91f8109880 0x7f91f8076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f77fe700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f8104650 0x7f91f8075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f6ffd700 1 -- 172.21.15.19:0/834448127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91f8109880 msgr2=0x7f91f8076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f6ffd700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91f8109880 0x7f91f8076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f6ffd700 1 -- 172.21.15.19:0/834448127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f8104650 msgr2=0x7f91f8075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f6ffd700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f8104650 0x7f91f8075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f6ffd700 1 -- 172.21.15.19:0/834448127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f81a9170 con 0x7f91f8108f30 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f7fff700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91f8109880 0x7f91f8076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.905+0000 7f91f6ffd700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 0x7f91f8076000 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009f50 tx=0x7f91ec0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:37.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.906+0000 7f91f4ff9700 1 -- 172.21.15.19:0/834448127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec027070 con 0x7f91f8108f30 2024-05-09T01:36:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.906+0000 7f91f4ff9700 1 -- 172.21.15.19:0/834448127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91ec004140 con 0x7f91f8108f30 2024-05-09T01:36:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.906+0000 7f91f4ff9700 1 -- 172.21.15.19:0/834448127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec00f460 con 0x7f91f8108f30 2024-05-09T01:36:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.906+0000 7f91fe0fb700 1 -- 172.21.15.19:0/834448127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f81a9400 con 0x7f91f8108f30 2024-05-09T01:36:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.906+0000 7f91fe0fb700 1 -- 172.21.15.19:0/834448127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f81a9800 con 0x7f91f8108f30 2024-05-09T01:36:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.908+0000 7f91f4ff9700 1 -- 172.21.15.19:0/834448127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f91ec02a070 con 0x7f91f8108f30 2024-05-09T01:36:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.909+0000 7f91f4ff9700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f91e00b2880 0x7f91e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.909+0000 7f91f4ff9700 1 -- 172.21.15.19:0/834448127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f91ec0e31c0 con 0x7f91f8108f30 2024-05-09T01:36:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.909+0000 7f91f77fe700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f91e00b2880 0x7f91e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f920cb11b80 1 Processor -- start 2024-05-09T01:36:37.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f920cb11b80 1 -- start start 2024-05-09T01:36:37.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f91f77fe700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f91e00b2880 0x7f91e00b4d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f91e8005950 tx=0x7f91e8005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:37.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 0x563c027d4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c027ce400 0x563c027ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f920cb11b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563c026f10a0 con 0x563c026cbd00 2024-05-09T01:36:37.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f920cb11b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563c026969a0 con 0x563c026514a0 2024-05-09T01:36:37.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f920cb11b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c026ed060 con 0x563c027ce400 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f91f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f91f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027d4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35638/0 (socket says 172.21.15.19:35638) 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f91f77fe700 1 -- 172.21.15.19:0/4102618874 learned_addr learned my addr 172.21.15.19:0/4102618874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f91f6ffd700 1 --2- 172.21.15.19:0/4102618874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 0x563c027d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f91f7fff700 1 --2- 172.21.15.19:0/4102618874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c027ce400 0x563c027ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.910+0000 7f91f77fe700 1 -- 172.21.15.19:0/4102618874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c027ce400 msgr2=0x563c027ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.911+0000 7f91f77fe700 1 --2- 172.21.15.19:0/4102618874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c027ce400 0x563c027ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.911+0000 7f91f77fe700 1 -- 172.21.15.19:0/4102618874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 msgr2=0x563c027d4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.911+0000 7f91f77fe700 1 --2- 172.21.15.19:0/4102618874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 0x563c027d4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.911+0000 7f91f77fe700 1 -- 172.21.15.19:0/4102618874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c02687500 con 0x563c026cbd00 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.911+0000 7f91f77fe700 1 --2- 172.21.15.19:0/4102618874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027d4240 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f91e800ec60 tx=0x7f91e8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb4eab9d66cdc17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.911+0000 7f91ddffb700 1 -- 172.21.15.19:0/4102618874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8021070 con 0x563c026cbd00 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.911+0000 7f91ddffb700 1 -- 172.21.15.19:0/4102618874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e8010cd0 con 0x563c026cbd00 2024-05-09T01:36:37.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.911+0000 7f91ddffb700 1 -- 172.21.15.19:0/4102618874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e800f620 con 0x563c026cbd00 2024-05-09T01:36:37.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.912+0000 7f920cb11b80 1 -- 172.21.15.19:0/4102618874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 msgr2=0x563c027d4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:37.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.912+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4102618874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027d4240 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f91e800ec60 tx=0x7f91e8011000 comp rx=0 tx=0).stop 2024-05-09T01:36:37.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.912+0000 7f920cb11b80 1 -- 172.21.15.19:0/4102618874 shutdown_connections 2024-05-09T01:36:37.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.912+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4102618874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c027ce400 0x563c027ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.912+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4102618874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027d4240 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.912+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4102618874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 0x563c027d4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.912+0000 7f920cb11b80 1 -- 172.21.15.19:0/4102618874 >> 172.21.15.19:0/4102618874 conn(0x563c02654f70 msgr2=0x563c027c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.912+0000 7f920cb11b80 1 -- 172.21.15.19:0/4102618874 shutdown_connections 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.912+0000 7f920cb11b80 1 -- 172.21.15.19:0/4102618874 wait complete. 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f920cb11b80 1 -- start start 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f920cb11b80 1 Processor -- start 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 0x563c027d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026cbd00 0x563c027d8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027d89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f920cb11b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563c027cf160 con 0x563c027ce400 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f920cb11b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563c0268bb20 con 0x563c026514a0 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f920cb11b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c027db020 con 0x563c026cbd00 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f920cb11b80 10 client.0 _ll_register_callbacks cb 0x563c025626c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f91f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027d89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f91f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027d89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35648/0 (socket says 172.21.15.19:35648) 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f91f7fff700 1 -- 172.21.15.19:0/1810236746 learned_addr learned my addr 172.21.15.19:0/1810236746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f91f77fe700 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 0x563c027d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.913+0000 7f91f6ffd700 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026cbd00 0x563c027d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91f7fff700 1 -- 172.21.15.19:0/1810236746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026cbd00 msgr2=0x563c027d8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91f7fff700 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026cbd00 0x563c027d8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91f7fff700 1 -- 172.21.15.19:0/1810236746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 msgr2=0x563c027d7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91f7fff700 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 0x563c027d7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91f7fff700 1 -- 172.21.15.19:0/1810236746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0256c490 con 0x563c027ce400 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91f7fff700 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027d89c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f91f0010b00 tx=0x7f91f0010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63114b687c97f392 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91d6ffd700 1 -- 172.21.15.19:0/1810236746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f001a020 con 0x563c027ce400 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91d6ffd700 1 -- 172.21.15.19:0/1810236746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f91f000e850 con 0x563c027ce400 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f920cb11b80 1 -- 172.21.15.19:0/1810236746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e8010cd0 con 0x563c027ce400 2024-05-09T01:36:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f91d6ffd700 1 -- 172.21.15.19:0/1810236746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91f0025050 con 0x563c027ce400 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f920cb11b80 10 client.18474 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.914+0000 7f920cb11b80 1 -- 172.21.15.19:0/1810236746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c027e5970 con 0x563c027ce400 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.915+0000 7f91d6ffd700 1 -- 172.21.15.19:0/1810236746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e8010cd0 con 0x563c027ce400 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.915+0000 7f91d6ffd700 1 -- 172.21.15.19:0/1810236746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f91f0015bf0 con 0x563c027ce400 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.915+0000 7f920cb11b80 10 client.18474 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.915+0000 7f920cb11b80 10 client.18474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.915+0000 7f920cb11b80 20 client.18474 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.915+0000 7f920cb11b80 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x563c027e7fb0 0x563c027ea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.915+0000 7f920cb11b80 4 client.18474 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.915+0000 7f920cb11b80 1 -- 172.21.15.19:0/1810236746 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c026fea80 con 0x563c027e7fb0 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.915+0000 7f91f77fe700 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x563c027e7fb0 0x563c027ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.916+0000 7f91f77fe700 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x563c027e7fb0 0x563c027ea460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56ef188a7aad574e server_cookie=720d1d182daf6e2b in_seq=0 out_seq=0 2024-05-09T01:36:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.916+0000 7f91d6ffd700 10 client.18474 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:36:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.916+0000 7f91d6ffd700 1 -- 172.21.15.19:0/1810236746 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563c026fea80 con 0x563c027e7fb0 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.916+0000 7f91d6ffd700 10 client.18474 handle_command_reply: tid=0 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 client.18474 shutdown 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 -- 172.21.15.19:0/1810236746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 msgr2=0x563c027d89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027d89c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f91f0010b00 tx=0x7f91f0010e10 comp rx=0 tx=0).stop 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 -- 172.21.15.19:0/1810236746 shutdown_connections 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027d89c0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x563c027e7fb0 0x563c027ea460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026514a0 0x563c027d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 --2- 172.21.15.19:0/1810236746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026cbd00 0x563c027d8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 -- 172.21.15.19:0/1810236746 >> 172.21.15.19:0/1810236746 conn(0x563c02654f70 msgr2=0x563c027c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f91d6ffd700 0 client.18474 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 -- 172.21.15.19:0/1810236746 shutdown_connections 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 1 -- 172.21.15.19:0/1810236746 wait complete. 2024-05-09T01:36:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.917+0000 7f920cb11b80 20 client.18474 trim_cache size 0 max 16384 2024-05-09T01:36:37.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.932+0000 7f920cb11b80 1 Processor -- start 2024-05-09T01:36:37.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.932+0000 7f920cb11b80 1 -- start start 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 0x563c027d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026cbd00 0x563c027ea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f920cb11b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91f81b0d80 con 0x563c027ce400 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f920cb11b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91f81b28c0 con 0x563c026cbd00 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f920cb11b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91f81b1450 con 0x563c026514a0 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35656/0 (socket says 172.21.15.19:35656) 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f7fff700 1 -- 172.21.15.19:0/4150209543 learned_addr learned my addr 172.21.15.19:0/4150209543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f6ffd700 1 --2- 172.21.15.19:0/4150209543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026cbd00 0x563c027ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f77fe700 1 --2- 172.21.15.19:0/4150209543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 0x563c027d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f7fff700 1 -- 172.21.15.19:0/4150209543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 msgr2=0x563c027d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f7fff700 1 --2- 172.21.15.19:0/4150209543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 0x563c027d3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f7fff700 1 -- 172.21.15.19:0/4150209543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026cbd00 msgr2=0x563c027ea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f7fff700 1 --2- 172.21.15.19:0/4150209543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026cbd00 0x563c027ea630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f7fff700 1 -- 172.21.15.19:0/4150209543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c027c9cf0 con 0x563c027ce400 2024-05-09T01:36:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91f7fff700 1 --2- 172.21.15.19:0/4150209543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027c3700 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f91f000e5a0 tx=0x7f91f00166d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c77a7c815a862f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.933+0000 7f91dd7fa700 1 -- 172.21.15.19:0/4150209543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f001a070 con 0x563c027ce400 2024-05-09T01:36:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f91dd7fa700 1 -- 172.21.15.19:0/4150209543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f002e070 con 0x563c027ce400 2024-05-09T01:36:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f91dd7fa700 1 -- 172.21.15.19:0/4150209543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f000eae0 con 0x563c027ce400 2024-05-09T01:36:37.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f920cb11b80 1 -- 172.21.15.19:0/4150209543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 msgr2=0x563c027c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4150209543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027c3700 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f91f000e5a0 tx=0x7f91f00166d0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f920cb11b80 1 -- 172.21.15.19:0/4150209543 shutdown_connections 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4150209543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c027ce400 0x563c027c3700 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4150209543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 0x563c027d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4150209543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c026cbd00 0x563c027ea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f920cb11b80 1 -- 172.21.15.19:0/4150209543 >> 172.21.15.19:0/4150209543 conn(0x563c02654f70 msgr2=0x563c027c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f920cb11b80 1 -- 172.21.15.19:0/4150209543 shutdown_connections 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.934+0000 7f920cb11b80 1 -- 172.21.15.19:0/4150209543 wait complete. 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.935+0000 7f920cb11b80 1 -- start start 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.935+0000 7f920cb11b80 1 Processor -- start 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.935+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 0x563c027e0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.935+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027e0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.935+0000 7f920cb11b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c027ce400 0x563c027d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.935+0000 7f920cb11b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91f81b21f0 con 0x563c026cbd00 2024-05-09T01:36:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.935+0000 7f920cb11b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91f81b1b20 con 0x563c027ce400 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.935+0000 7f920cb11b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91f81aed20 con 0x563c026514a0 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.935+0000 7f920cb11b80 10 client.0 _ll_register_callbacks cb 0x563c027cedc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027e0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027e0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35666/0 (socket says 172.21.15.19:35666) 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f6ffd700 1 -- 172.21.15.19:0/4017027150 learned_addr learned my addr 172.21.15.19:0/4017027150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f77fe700 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 0x563c027e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f7fff700 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c027ce400 0x563c027d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f6ffd700 1 -- 172.21.15.19:0/4017027150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 msgr2=0x563c027e0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f6ffd700 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 0x563c027e0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f6ffd700 1 -- 172.21.15.19:0/4017027150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c027ce400 msgr2=0x563c027d4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f6ffd700 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c027ce400 0x563c027d4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f6ffd700 1 -- 172.21.15.19:0/4017027150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c027d5e80 con 0x563c026cbd00 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91f6ffd700 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027e0640 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0e1010 tx=0x7f91ec0b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7bd98aa621d2bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91d6ffd700 1 -- 172.21.15.19:0/4017027150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0a9fc0 con 0x563c026cbd00 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f920cb11b80 1 -- 172.21.15.19:0/4017027150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91f002e070 con 0x563c026cbd00 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91d6ffd700 1 -- 172.21.15.19:0/4017027150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f91ec0042f0 con 0x563c026cbd00 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f91d6ffd700 1 -- 172.21.15.19:0/4017027150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91ec0aa360 con 0x563c026cbd00 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f920cb11b80 10 client.18480 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.936+0000 7f920cb11b80 1 -- 172.21.15.19:0/4017027150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c02851cb0 con 0x563c026cbd00 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.937+0000 7f91d6ffd700 1 -- 172.21.15.19:0/4017027150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f002e070 con 0x563c026cbd00 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.937+0000 7f91d6ffd700 1 -- 172.21.15.19:0/4017027150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f91ec070080 con 0x563c026cbd00 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.937+0000 7f920cb11b80 10 client.18480 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.937+0000 7f920cb11b80 10 client.18480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.937+0000 7f920cb11b80 20 client.18480 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.937+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x563c028523f0 0x563c02854810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.937+0000 7f920cb11b80 4 client.18480 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.937+0000 7f920cb11b80 1 -- 172.21.15.19:0/4017027150 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c026f8b20 con 0x563c028523f0 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.937+0000 7f91f77fe700 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x563c028523f0 0x563c02854810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f91f77fe700 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x563c028523f0 0x563c02854810 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a06c7ee5e452261 server_cookie=90ca7808f5f5adaa in_seq=0 out_seq=0 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f91d6ffd700 10 client.18480 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f91d6ffd700 1 -- 172.21.15.19:0/4017027150 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c026f8b20 con 0x563c028523f0 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f91d6ffd700 10 client.18480 handle_command_reply: tid=0 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 client.18480 shutdown 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 -- 172.21.15.19:0/4017027150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 msgr2=0x563c027e0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027e0640 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0e1010 tx=0x7f91ec0b3040 comp rx=0 tx=0).stop 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 -- 172.21.15.19:0/4017027150 shutdown_connections 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x563c028523f0 0x563c02854810 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c027ce400 0x563c027d4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c026514a0 0x563c027e0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 --2- 172.21.15.19:0/4017027150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c026cbd00 0x563c027e0640 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 -- 172.21.15.19:0/4017027150 >> 172.21.15.19:0/4017027150 conn(0x563c02654f70 msgr2=0x563c027d2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f91d6ffd700 0 client.18480 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:36:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 -- 172.21.15.19:0/4017027150 shutdown_connections 2024-05-09T01:36:37.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 1 -- 172.21.15.19:0/4017027150 wait complete. 2024-05-09T01:36:37.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.938+0000 7f920cb11b80 20 client.18480 trim_cache size 0 max 16384 2024-05-09T01:36:37.940 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:37.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 -- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f91e00b2880 msgr2=0x7f91e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:37.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f91e00b2880 0x7f91e00b4d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f91e8005950 tx=0x7f91e8005eb0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 -- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 msgr2=0x7f91f8076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 0x7f91f8076000 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009f50 tx=0x7f91ec0037c0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 -- 172.21.15.19:0/834448127 shutdown_connections 2024-05-09T01:36:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91f8109880 0x7f91f8076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f8104650 0x7f91f8075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f91e00b2880 0x7f91e00b4d90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 --2- 172.21.15.19:0/834448127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f8108f30 0x7f91f8076000 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 -- 172.21.15.19:0/834448127 >> 172.21.15.19:0/834448127 conn(0x7f91f8079f00 msgr2=0x7f91f8102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 -- 172.21.15.19:0/834448127 shutdown_connections 2024-05-09T01:36:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:37.939+0000 7f91fe0fb700 1 -- 172.21.15.19:0/834448127 wait complete. 2024-05-09T01:36:38.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:38 smithi019 ceph-mon[30158]: pgmap v711: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:36:38.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:36:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:38 smithi059 ceph-mon[34351]: pgmap v711: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:36:38.909 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:38.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:38 smithi196 ceph-mon[35041]: pgmap v711: 97 pgs: 97 active+clean; 3.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-05-09T01:36:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 -- 172.21.15.19:0/630455464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 msgr2=0x7fc54410be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 --2- 172.21.15.19:0/630455464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 0x7fc54410be50 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009860 tx=0x7fc53c02a340 comp rx=0 tx=0).stop 2024-05-09T01:36:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 -- 172.21.15.19:0/630455464 shutdown_connections 2024-05-09T01:36:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 --2- 172.21.15.19:0/630455464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 0x7fc54410be50 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 --2- 172.21.15.19:0/630455464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5441070d0 0x7fc5441094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 --2- 172.21.15.19:0/630455464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc544104770 0x7fc544106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 -- 172.21.15.19:0/630455464 >> 172.21.15.19:0/630455464 conn(0x7fc5440fdd90 msgr2=0x7fc544100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 -- 172.21.15.19:0/630455464 shutdown_connections 2024-05-09T01:36:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 -- 172.21.15.19:0/630455464 wait complete. 2024-05-09T01:36:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.347+0000 7fc549e61640 1 Processor -- start 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc549e61640 1 -- start start 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc549e61640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc544104770 0x7fc544103bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc549e61640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5441070d0 0x7fc544102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc549e61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 0x7fc5441027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc549e61640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc54410f7b0 con 0x7fc544109a30 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc549e61640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc54410f630 con 0x7fc544104770 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc549e61640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc54410f930 con 0x7fc5441070d0 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc543fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 0x7fc5441027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc543fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 0x7fc5441027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54678/0 (socket says 172.21.15.19:54678) 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc543fff640 1 -- 172.21.15.19:0/1489667818 learned_addr learned my addr 172.21.15.19:0/1489667818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc5437fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc544104770 0x7fc544103bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc542ffd640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5441070d0 0x7fc544102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc543fff640 1 -- 172.21.15.19:0/1489667818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5441070d0 msgr2=0x7fc544102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc543fff640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5441070d0 0x7fc544102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc543fff640 1 -- 172.21.15.19:0/1489667818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc544104770 msgr2=0x7fc544103bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc543fff640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc544104770 0x7fc544103bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc543fff640 1 -- 172.21.15.19:0/1489667818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc544102ed0 con 0x7fc544109a30 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc542ffd640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5441070d0 0x7fc544102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc5437fe640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc544104770 0x7fc544103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.348+0000 7fc543fff640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 0x7fc5441027a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc53c02a860 tx=0x7fc53c032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.349+0000 7fc540ff9640 1 -- 172.21.15.19:0/1489667818 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c037040 con 0x7fc544109a30 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.349+0000 7fc540ff9640 1 -- 172.21.15.19:0/1489667818 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53c041070 con 0x7fc544109a30 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.349+0000 7fc549e61640 1 -- 172.21.15.19:0/1489667818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5441a4410 con 0x7fc544109a30 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.349+0000 7fc540ff9640 1 -- 172.21.15.19:0/1489667818 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c003850 con 0x7fc544109a30 2024-05-09T01:36:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.349+0000 7fc549e61640 1 -- 172.21.15.19:0/1489667818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5441a4810 con 0x7fc544109a30 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.351+0000 7fc550df7740 1 Processor -- start 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc540ff9640 1 -- 172.21.15.19:0/1489667818 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc53c004050 con 0x7fc544109a30 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc550df7740 1 -- start start 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc550df7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4551b0 0x55eaad7f9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc550df7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 0x55eaad7f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc550df7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad4344d0 0x55eaad800980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc550df7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eaad75d7c0 con 0x55eaad4551b0 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc550df7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eaad6f90e0 con 0x55eaad4344d0 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc550df7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55eaad759780 con 0x55eaad42c2f0 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc542ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4551b0 0x55eaad7f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc542ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4551b0 0x55eaad7f9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54692/0 (socket says 172.21.15.19:54692) 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc542ffd640 1 -- 172.21.15.19:0/2253324845 learned_addr learned my addr 172.21.15.19:0/2253324845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc5437fe640 1 --2- 172.21.15.19:0/2253324845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 0x55eaad7f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.352+0000 7fc542ffd640 1 --2- 172.21.15.19:0/2253324845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad4344d0 0x55eaad800980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc540ff9640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc52c0b2860 0x7fc52c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc542ffd640 1 -- 172.21.15.19:0/2253324845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 msgr2=0x55eaad7f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc542ffd640 1 --2- 172.21.15.19:0/2253324845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 0x55eaad7f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc542ffd640 1 -- 172.21.15.19:0/2253324845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad4344d0 msgr2=0x55eaad800980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc542ffd640 1 --2- 172.21.15.19:0/2253324845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad4344d0 0x55eaad800980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc542ffd640 1 -- 172.21.15.19:0/2253324845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaad7227c0 con 0x55eaad4551b0 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc540ff9640 1 -- 172.21.15.19:0/1489667818 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc53c102120 con 0x7fc544109a30 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc543fff640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc52c0b2860 0x7fc52c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc542ffd640 1 --2- 172.21.15.19:0/2253324845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4551b0 0x55eaad7f9500 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc538015990 tx=0x7fc538009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bc5f433b3638f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc529ffb640 1 -- 172.21.15.19:0/2253324845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538013680 con 0x55eaad4551b0 2024-05-09T01:36:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.353+0000 7fc529ffb640 1 -- 172.21.15.19:0/2253324845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc538013cd0 con 0x55eaad4551b0 2024-05-09T01:36:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.354+0000 7fc529ffb640 1 -- 172.21.15.19:0/2253324845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53800dad0 con 0x55eaad4551b0 2024-05-09T01:36:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.354+0000 7fc543fff640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc52c0b2860 0x7fc52c0b4d70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0c96b0 tx=0x7fc53c02a340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.355+0000 7fc550df7740 1 -- 172.21.15.19:0/2253324845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4551b0 msgr2=0x55eaad7f9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.355+0000 7fc550df7740 1 --2- 172.21.15.19:0/2253324845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4551b0 0x55eaad7f9500 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc538015990 tx=0x7fc538009480 comp rx=0 tx=0).stop 2024-05-09T01:36:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.355+0000 7fc550df7740 1 -- 172.21.15.19:0/2253324845 shutdown_connections 2024-05-09T01:36:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.355+0000 7fc550df7740 1 --2- 172.21.15.19:0/2253324845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad4344d0 0x55eaad800980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.355+0000 7fc550df7740 1 --2- 172.21.15.19:0/2253324845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 0x55eaad7f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.355+0000 7fc550df7740 1 --2- 172.21.15.19:0/2253324845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4551b0 0x55eaad7f9500 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.355+0000 7fc550df7740 1 -- 172.21.15.19:0/2253324845 >> 172.21.15.19:0/2253324845 conn(0x55eaad449300 msgr2=0x55eaad7f4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.355+0000 7fc550df7740 1 -- 172.21.15.19:0/2253324845 shutdown_connections 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.355+0000 7fc550df7740 1 -- 172.21.15.19:0/2253324845 wait complete. 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.356+0000 7fc550df7740 1 -- start start 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.356+0000 7fc550df7740 1 Processor -- start 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.356+0000 7fc550df7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 0x55eaad8031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.356+0000 7fc550df7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad803720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.356+0000 7fc550df7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad4551b0 0x55eaad80a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.356+0000 7fc550df7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eaad801370 con 0x55eaad4344d0 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.356+0000 7fc550df7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eaad6e49c0 con 0x55eaad42c2f0 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.356+0000 7fc550df7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55eaad8014f0 con 0x55eaad4551b0 2024-05-09T01:36:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.356+0000 7fc550df7740 10 client.0 _ll_register_callbacks cb 0x55eaad711020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc5437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad803720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc542ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 0x55eaad8031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc5437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad803720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54698/0 (socket says 172.21.15.19:54698) 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc5437fe640 1 -- 172.21.15.19:0/487609905 learned_addr learned my addr 172.21.15.19:0/487609905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc542ffd640 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad4551b0 0x55eaad80a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc5437fe640 1 -- 172.21.15.19:0/487609905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad4551b0 msgr2=0x55eaad80a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc5437fe640 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad4551b0 0x55eaad80a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc5437fe640 1 -- 172.21.15.19:0/487609905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 msgr2=0x55eaad8031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc5437fe640 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 0x55eaad8031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc5437fe640 1 -- 172.21.15.19:0/487609905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaad70b1f0 con 0x55eaad4344d0 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc5437fe640 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad803720 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fc53400d410 tx=0x7fc53400d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f1b79ca6d83be0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc522ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc522ffd640 1 -- 172.21.15.19:0/487609905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534003e40 con 0x55eaad4344d0 2024-05-09T01:36:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.357+0000 7fc550df7740 1 -- 172.21.15.19:0/487609905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc538013cd0 con 0x55eaad4344d0 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc522ffd640 1 -- 172.21.15.19:0/487609905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc534004050 con 0x55eaad4344d0 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc522ffd640 1 -- 172.21.15.19:0/487609905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc534004250 con 0x55eaad4344d0 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc550df7740 10 client.18492 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc550df7740 1 -- 172.21.15.19:0/487609905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eaad4c0790 con 0x55eaad4344d0 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc522ffd640 1 -- 172.21.15.19:0/487609905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc538013cd0 con 0x55eaad4344d0 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc522ffd640 1 -- 172.21.15.19:0/487609905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc53400b040 con 0x55eaad4344d0 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc550df7740 10 client.18492 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc550df7740 10 client.18492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc550df7740 20 client.18492 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc550df7740 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55eaad413d80 0x55eaad810c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc550df7740 4 client.18492 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.358+0000 7fc550df7740 1 -- 172.21.15.19:0/487609905 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaad76b3d0 con 0x55eaad413d80 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.359+0000 7fc542ffd640 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55eaad413d80 0x55eaad810c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.359+0000 7fc542ffd640 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55eaad413d80 0x55eaad810c40 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4052f1628980768d server_cookie=7afcd39a6d5f008f in_seq=0 out_seq=0 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.359+0000 7fc522ffd640 10 client.18492 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc522ffd640 1 -- 172.21.15.19:0/487609905 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eaad76b3d0 con 0x55eaad413d80 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc522ffd640 10 client.18492 handle_command_reply: tid=0 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 client.18492 shutdown 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 -- 172.21.15.19:0/487609905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 msgr2=0x55eaad803720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad803720 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fc53400d410 tx=0x7fc53400d8d0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 -- 172.21.15.19:0/487609905 shutdown_connections 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55eaad413d80 0x55eaad810c40 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad4551b0 0x55eaad80a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad803720 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc522ffd640 0 client.18492 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 --2- 172.21.15.19:0/487609905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 0x55eaad8031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 -- 172.21.15.19:0/487609905 >> 172.21.15.19:0/487609905 conn(0x55eaad449300 msgr2=0x55eaad7f7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 -- 172.21.15.19:0/487609905 shutdown_connections 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 1 -- 172.21.15.19:0/487609905 wait complete. 2024-05-09T01:36:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.360+0000 7fc550df7740 20 client.18492 trim_cache size 0 max 16384 2024-05-09T01:36:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.376+0000 7fc550df7740 1 Processor -- start 2024-05-09T01:36:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.376+0000 7fc550df7740 1 -- start start 2024-05-09T01:36:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.376+0000 7fc550df7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad413d80 0x55eaad7f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.376+0000 7fc550df7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 0x55eaad7f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.376+0000 7fc550df7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad80a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.376+0000 7fc550df7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441ad630 con 0x55eaad4344d0 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.376+0000 7fc550df7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441af170 con 0x55eaad42c2f0 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.376+0000 7fc550df7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441add00 con 0x55eaad413d80 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc542ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad80a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5437fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 0x55eaad7f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc542ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad80a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54700/0 (socket says 172.21.15.19:54700) 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc542ffd640 1 -- 172.21.15.19:0/2279244666 learned_addr learned my addr 172.21.15.19:0/2279244666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc542ffd640 1 --2- 172.21.15.19:0/2279244666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad413d80 0x55eaad7f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5437fe640 1 -- 172.21.15.19:0/2279244666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad413d80 msgr2=0x55eaad7f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5437fe640 1 --2- 172.21.15.19:0/2279244666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad413d80 0x55eaad7f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5437fe640 1 -- 172.21.15.19:0/2279244666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 msgr2=0x55eaad80a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5437fe640 1 --2- 172.21.15.19:0/2279244666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad80a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5437fe640 1 -- 172.21.15.19:0/2279244666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaad5375f0 con 0x55eaad42c2f0 2024-05-09T01:36:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc542ffd640 1 --2- 172.21.15.19:0/2279244666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad80a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5437fe640 1 --2- 172.21.15.19:0/2279244666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 0x55eaad7f4d90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55eaad7fa400 tx=0x7fc53400b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d890cf42b3f3c20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5297fa640 1 -- 172.21.15.19:0/2279244666 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534003c40 con 0x55eaad42c2f0 2024-05-09T01:36:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5297fa640 1 -- 172.21.15.19:0/2279244666 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53401d070 con 0x55eaad42c2f0 2024-05-09T01:36:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.377+0000 7fc5297fa640 1 -- 172.21.15.19:0/2279244666 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5340186e0 con 0x55eaad42c2f0 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.378+0000 7fc550df7740 1 -- 172.21.15.19:0/2279244666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 msgr2=0x55eaad7f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.378+0000 7fc550df7740 1 --2- 172.21.15.19:0/2279244666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 0x55eaad7f4d90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55eaad7fa400 tx=0x7fc53400b5d0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.379+0000 7fc550df7740 1 -- 172.21.15.19:0/2279244666 shutdown_connections 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.379+0000 7fc550df7740 1 --2- 172.21.15.19:0/2279244666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad80a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.379+0000 7fc550df7740 1 --2- 172.21.15.19:0/2279244666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad42c2f0 0x55eaad7f4d90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.379+0000 7fc550df7740 1 --2- 172.21.15.19:0/2279244666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad413d80 0x55eaad7f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.379+0000 7fc550df7740 1 -- 172.21.15.19:0/2279244666 >> 172.21.15.19:0/2279244666 conn(0x55eaad449300 msgr2=0x55eaad7fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.379+0000 7fc550df7740 1 -- 172.21.15.19:0/2279244666 shutdown_connections 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.379+0000 7fc550df7740 1 -- 172.21.15.19:0/2279244666 wait complete. 2024-05-09T01:36:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.379+0000 7fc550df7740 1 -- start start 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.379+0000 7fc550df7740 1 Processor -- start 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc550df7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad413d80 0x55eaad807d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc550df7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 0x55eaad8082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc550df7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad876fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc550df7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441aeaa0 con 0x55eaad4344d0 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc550df7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441ae3d0 con 0x55eaad413d80 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc550df7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441ab610 con 0x55eaad42c2f0 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc550df7740 10 client.0 _ll_register_callbacks cb 0x7fc53400e500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad876fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad876fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54714/0 (socket says 172.21.15.19:54714) 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 -- 172.21.15.19:0/3512781568 learned_addr learned my addr 172.21.15.19:0/3512781568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc5437fe640 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 0x55eaad8082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad413d80 0x55eaad807d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 -- 172.21.15.19:0/3512781568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 msgr2=0x55eaad8082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 0x55eaad8082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 -- 172.21.15.19:0/3512781568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad413d80 msgr2=0x55eaad807d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad413d80 0x55eaad807d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 -- 172.21.15.19:0/3512781568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaad721ed0 con 0x55eaad4344d0 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc5437fe640 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 0x55eaad8082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc542ffd640 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad876fe0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc538015990 tx=0x7fc538009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ea9557003b5a57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.380+0000 7fc522ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc522ffd640 1 -- 172.21.15.19:0/3512781568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53802f720 con 0x55eaad4344d0 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc550df7740 1 -- 172.21.15.19:0/3512781568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc53401d070 con 0x55eaad4344d0 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc522ffd640 1 -- 172.21.15.19:0/3512781568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc53802a040 con 0x55eaad4344d0 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc522ffd640 1 -- 172.21.15.19:0/3512781568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc53802f8c0 con 0x55eaad4344d0 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc550df7740 10 client.18498 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc550df7740 1 -- 172.21.15.19:0/3512781568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eaad8784e0 con 0x55eaad4344d0 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc522ffd640 1 -- 172.21.15.19:0/3512781568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53401d070 con 0x55eaad4344d0 2024-05-09T01:36:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc522ffd640 1 -- 172.21.15.19:0/3512781568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc538004a50 con 0x55eaad4344d0 2024-05-09T01:36:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc550df7740 10 client.18498 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc550df7740 10 client.18498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc550df7740 20 client.18498 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc550df7740 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55eaad4551b0 0x55eaad87ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc550df7740 4 client.18498 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:36:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.381+0000 7fc550df7740 1 -- 172.21.15.19:0/3512781568 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eaad765240 con 0x55eaad4551b0 2024-05-09T01:36:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.382+0000 7fc5437fe640 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55eaad4551b0 0x55eaad87ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.382+0000 7fc5437fe640 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55eaad4551b0 0x55eaad87ace0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1754c3371e65ec7b server_cookie=60a5f6535d07585f in_seq=0 out_seq=0 2024-05-09T01:36:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.382+0000 7fc522ffd640 10 client.18498 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.397+0000 7fc522ffd640 1 -- 172.21.15.19:0/3512781568 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eaad765240 con 0x55eaad4551b0 2024-05-09T01:36:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.397+0000 7fc522ffd640 10 client.18498 handle_command_reply: tid=0 2024-05-09T01:36:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 client.18498 shutdown 2024-05-09T01:36:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 -- 172.21.15.19:0/3512781568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 msgr2=0x55eaad876fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad876fe0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc538015990 tx=0x7fc538009480 comp rx=0 tx=0).stop 2024-05-09T01:36:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 -- 172.21.15.19:0/3512781568 shutdown_connections 2024-05-09T01:36:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55eaad4551b0 0x55eaad87ace0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eaad4344d0 0x55eaad876fe0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaad42c2f0 0x55eaad8082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc522ffd640 0 client.18498 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 --2- 172.21.15.19:0/3512781568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eaad413d80 0x55eaad807d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 -- 172.21.15.19:0/3512781568 >> 172.21.15.19:0/3512781568 conn(0x55eaad449300 msgr2=0x55eaad7f4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 -- 172.21.15.19:0/3512781568 shutdown_connections 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 1 -- 172.21.15.19:0/3512781568 wait complete. 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.398+0000 7fc550df7740 20 client.18498 trim_cache size 0 max 16384 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:36:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:36:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:36:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 -- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc52c0b2860 msgr2=0x7fc52c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc52c0b2860 0x7fc52c0b4d70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0c96b0 tx=0x7fc53c02a340 comp rx=0 tx=0).stop 2024-05-09T01:36:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 -- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 msgr2=0x7fc5441027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 0x7fc5441027a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc53c02a860 tx=0x7fc53c032e10 comp rx=0 tx=0).stop 2024-05-09T01:36:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 -- 172.21.15.19:0/1489667818 shutdown_connections 2024-05-09T01:36:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fc52c0b2860 0x7fc52c0b4d70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc544109a30 0x7fc5441027a0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5441070d0 0x7fc544102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 --2- 172.21.15.19:0/1489667818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc544104770 0x7fc544103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 -- 172.21.15.19:0/1489667818 >> 172.21.15.19:0/1489667818 conn(0x7fc5440fdd90 msgr2=0x7fc544105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 -- 172.21.15.19:0/1489667818 shutdown_connections 2024-05-09T01:36:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:40.399+0000 7fc549e61640 1 -- 172.21.15.19:0/1489667818 wait complete. 2024-05-09T01:36:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.619+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/743912438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 msgr2=0x7fbb181038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.620+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/743912438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 0x7fbb181038d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fbb080099c0 tx=0x7fbb08009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.620+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/743912438 shutdown_connections 2024-05-09T01:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.620+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/743912438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb18108740 0x7fbb181100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.620+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/743912438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 0x7fbb181038d0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.620+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/743912438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18107df0 0x7fbb18108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.620+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/743912438 >> 172.21.15.19:0/743912438 conn(0x7fbb180fedc0 msgr2=0x7fbb181011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.620+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/743912438 shutdown_connections 2024-05-09T01:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.620+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/743912438 wait complete. 2024-05-09T01:36:40.617 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:40 smithi019 ceph-mon[30158]: pgmap v712: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:36:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.620+0000 7fbb1dfaa700 1 Processor -- start 2024-05-09T01:36:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb1dfaa700 1 -- start start 2024-05-09T01:36:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb1dfaa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 0x7fbb180ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb1dfaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18107df0 0x7fbb180ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb1dfaa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb18108740 0x7fbb180ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb1dfaa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18113390 con 0x7fbb181034c0 2024-05-09T01:36:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb1dfaa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18113210 con 0x7fbb18107df0 2024-05-09T01:36:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb1dfaa700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18113510 con 0x7fbb18108740 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 0x7fbb180ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 0x7fbb180ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54734/0 (socket says 172.21.15.19:54734) 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb177fe700 1 -- 172.21.15.19:0/3959593614 learned_addr learned my addr 172.21.15.19:0/3959593614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18107df0 0x7fbb180ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb17fff700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb18108740 0x7fbb180ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb177fe700 1 -- 172.21.15.19:0/3959593614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb18108740 msgr2=0x7fbb180ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb177fe700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb18108740 0x7fbb180ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb177fe700 1 -- 172.21.15.19:0/3959593614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18107df0 msgr2=0x7fbb180ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb177fe700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18107df0 0x7fbb180ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb177fe700 1 -- 172.21.15.19:0/3959593614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb181a9100 con 0x7fbb181034c0 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.621+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18107df0 0x7fbb180ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.622+0000 7fbb177fe700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 0x7fbb180ff2a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800b580 tx=0x7fbb080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.622+0000 7fbb14ff9700 1 -- 172.21.15.19:0/3959593614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08027070 con 0x7fbb181034c0 2024-05-09T01:36:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.622+0000 7fbb14ff9700 1 -- 172.21.15.19:0/3959593614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb08004020 con 0x7fbb181034c0 2024-05-09T01:36:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.622+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/3959593614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb181a9390 con 0x7fbb181034c0 2024-05-09T01:36:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.622+0000 7fbb14ff9700 1 -- 172.21.15.19:0/3959593614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08022e30 con 0x7fbb181034c0 2024-05-09T01:36:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.622+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/3959593614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb181a97e0 con 0x7fbb181034c0 2024-05-09T01:36:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.625+0000 7fbb14ff9700 1 -- 172.21.15.19:0/3959593614 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbb0800f460 con 0x7fbb181034c0 2024-05-09T01:36:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.626+0000 7fbb14ff9700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb000b2880 0x7fbb000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.626+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb000b2880 0x7fbb000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.626+0000 7fbb14ff9700 1 -- 172.21.15.19:0/3959593614 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbb080e2e70 con 0x7fbb181034c0 2024-05-09T01:36:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.626+0000 7fbb2c9c0b80 1 Processor -- start 2024-05-09T01:36:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.626+0000 7fbb2c9c0b80 1 -- start start 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb000b2880 0x7fbb000b4d90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005fd0 tx=0x7fbb0c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270f1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270f1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 0x55fa270eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fa2700e0a0 con 0x55fa26fe8d00 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa26fb39a0 con 0x55fa26f6e4a0 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa2700a060 con 0x55fa270eb400 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb177fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54746/0 (socket says 172.21.15.19:54746) 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb177fe700 1 -- 172.21.15.19:0/1832718031 learned_addr learned my addr 172.21.15.19:0/1832718031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb17fff700 1 --2- 172.21.15.19:0/1832718031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 0x55fa270eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/1832718031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb177fe700 1 -- 172.21.15.19:0/1832718031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 msgr2=0x55fa270eb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb177fe700 1 --2- 172.21.15.19:0/1832718031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 0x55fa270eb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb177fe700 1 -- 172.21.15.19:0/1832718031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 msgr2=0x55fa270f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb177fe700 1 --2- 172.21.15.19:0/1832718031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb177fe700 1 -- 172.21.15.19:0/1832718031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa26fa4500 con 0x55fa26fe8d00 2024-05-09T01:36:40.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.627+0000 7fbb177fe700 1 --2- 172.21.15.19:0/1832718031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270f1240 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbb080aa200 tx=0x7fbb08009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d5f8a5fa178a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.628+0000 7fbafdffb700 1 -- 172.21.15.19:0/1832718031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb080aa500 con 0x55fa26fe8d00 2024-05-09T01:36:40.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.628+0000 7fbafdffb700 1 -- 172.21.15.19:0/1832718031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb080ae8b0 con 0x55fa26fe8d00 2024-05-09T01:36:40.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.628+0000 7fbafdffb700 1 -- 172.21.15.19:0/1832718031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb080b2e40 con 0x55fa26fe8d00 2024-05-09T01:36:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.628+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/1832718031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 msgr2=0x55fa270f1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.628+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/1832718031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270f1240 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbb080aa200 tx=0x7fbb08009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.629+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/1832718031 shutdown_connections 2024-05-09T01:36:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.629+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/1832718031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 0x55fa270eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.629+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/1832718031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270f1240 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.629+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/1832718031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.629+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/1832718031 >> 172.21.15.19:0/1832718031 conn(0x55fa26f71f70 msgr2=0x55fa270e6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.629+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/1832718031 shutdown_connections 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.629+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/1832718031 wait complete. 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.629+0000 7fbb2c9c0b80 1 -- start start 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.629+0000 7fbb2c9c0b80 1 Processor -- start 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270faab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270faff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 0x55fa270f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fa2700e0a0 con 0x55fa26fe8d00 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa26fa8b20 con 0x55fa26f6e4a0 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa270ebfb0 con 0x55fa270eb400 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb2c9c0b80 10 client.0 _ll_register_callbacks cb 0x55fa26e7f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb16ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb16ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270faff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54756/0 (socket says 172.21.15.19:54756) 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb16ffd700 1 -- 172.21.15.19:0/3855143855 learned_addr learned my addr 172.21.15.19:0/3855143855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb177fe700 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb17fff700 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 0x55fa270f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb16ffd700 1 -- 172.21.15.19:0/3855143855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 msgr2=0x55fa270f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 0x55fa270f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb16ffd700 1 -- 172.21.15.19:0/3855143855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 msgr2=0x55fa270faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270faab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb16ffd700 1 -- 172.21.15.19:0/3855143855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa26e89490 con 0x55fa26fe8d00 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb17fff700 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 0x55fa270f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270faff0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbb18107cd0 tx=0x7fbb0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7d51b1db0a74a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbaf6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.630+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/3855143855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00f8d0 con 0x55fa26fe8d00 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/3855143855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbb0c00fa70 con 0x55fa26fe8d00 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3855143855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb080ae8b0 con 0x55fa26fe8d00 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/3855143855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c00fc70 con 0x55fa26fe8d00 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbb2c9c0b80 10 client.18510 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3855143855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa27102730 con 0x55fa26fe8d00 2024-05-09T01:36:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/3855143855 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb080ae8b0 con 0x55fa26fe8d00 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/3855143855 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb0c031490 con 0x55fa26fe8d00 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbb2c9c0b80 10 client.18510 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbb2c9c0b80 10 client.18510 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbb2c9c0b80 20 client.18510 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55fa27104d70 0x55fa27107190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbb2c9c0b80 4 client.18510 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.631+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3855143855 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa2701ba80 con 0x55fa27104d70 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.632+0000 7fbb177fe700 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55fa27104d70 0x55fa27107190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.632+0000 7fbb177fe700 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55fa27104d70 0x55fa27107190 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=582c0d7b07a33798 server_cookie=7f84a993bf218bd4 in_seq=0 out_seq=0 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.632+0000 7fbaf6ffd700 10 client.18510 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/3855143855 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa2701ba80 con 0x55fa27104d70 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbaf6ffd700 10 client.18510 handle_command_reply: tid=0 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 client.18510 shutdown 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3855143855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 msgr2=0x55fa270faff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270faff0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbb18107cd0 tx=0x7fbb0c004750 comp rx=0 tx=0).stop 2024-05-09T01:36:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3855143855 shutdown_connections 2024-05-09T01:36:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55fa27104d70 0x55fa27107190 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa270eb400 0x55fa270f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3855143855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270faff0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3855143855 >> 172.21.15.19:0/3855143855 conn(0x55fa26f71f70 msgr2=0x55fa270dfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbaf6ffd700 0 client.18510 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:36:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3855143855 shutdown_connections 2024-05-09T01:36:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3855143855 wait complete. 2024-05-09T01:36:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.633+0000 7fbb2c9c0b80 20 client.18510 trim_cache size 0 max 16384 2024-05-09T01:36:40.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.648+0000 7fbb2c9c0b80 1 Processor -- start 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.648+0000 7fbb2c9c0b80 1 -- start start 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270e6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26fe8d00 0x55fa270ef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa270eb400 0x55fa2710c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b0d70 con 0x55fa270eb400 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b28b0 con 0x55fa26f6e4a0 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b1440 con 0x55fa26fe8d00 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb17fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa270eb400 0x55fa2710c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb17fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa270eb400 0x55fa2710c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54760/0 (socket says 172.21.15.19:54760) 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb17fff700 1 -- 172.21.15.19:0/3003602682 learned_addr learned my addr 172.21.15.19:0/3003602682 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb177fe700 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26fe8d00 0x55fa270ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb17fff700 1 -- 172.21.15.19:0/3003602682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26fe8d00 msgr2=0x55fa270ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb17fff700 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26fe8d00 0x55fa270ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb17fff700 1 -- 172.21.15.19:0/3003602682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 msgr2=0x55fa270e6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb17fff700 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270e6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb17fff700 1 -- 172.21.15.19:0/3003602682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa270e6f30 con 0x55fa270eb400 2024-05-09T01:36:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26fe8d00 0x55fa270ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.649+0000 7fbb177fe700 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.650+0000 7fbb17fff700 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa270eb400 0x55fa2710c930 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009de0 tx=0x7fbb100077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cb7d0f7bd26e74 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.650+0000 7fbafd7fa700 1 -- 172.21.15.19:0/3003602682 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1001b020 con 0x55fa270eb400 2024-05-09T01:36:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.650+0000 7fbafd7fa700 1 -- 172.21.15.19:0/3003602682 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb10017920 con 0x55fa270eb400 2024-05-09T01:36:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.650+0000 7fbafd7fa700 1 -- 172.21.15.19:0/3003602682 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100169c0 con 0x55fa270eb400 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.650+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3003602682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa270eb400 msgr2=0x55fa2710c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.650+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa270eb400 0x55fa2710c930 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009de0 tx=0x7fbb100077e0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.651+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3003602682 shutdown_connections 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.651+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa270eb400 0x55fa2710c930 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.651+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa26f6e4a0 0x55fa270e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.651+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/3003602682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26fe8d00 0x55fa270ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.651+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3003602682 >> 172.21.15.19:0/3003602682 conn(0x55fa26f71f70 msgr2=0x55fa270eec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.651+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3003602682 shutdown_connections 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.651+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/3003602682 wait complete. 2024-05-09T01:36:40.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.651+0000 7fbb2c9c0b80 1 -- start start 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.651+0000 7fbb2c9c0b80 1 Processor -- start 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26f6e4a0 0x55fa270fe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270fe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb2c9c0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa270eb400 0x55fa27167520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b21e0 con 0x55fa26fe8d00 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b1b10 con 0x55fa270eb400 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb2c9c0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181aed10 con 0x55fa26f6e4a0 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb16ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb2c9c0b80 10 client.0 _ll_register_callbacks cb 0x55fa26e7f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb16ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270fe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54764/0 (socket says 172.21.15.19:54764) 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb16ffd700 1 -- 172.21.15.19:0/25039590 learned_addr learned my addr 172.21.15.19:0/25039590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb17fff700 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa270eb400 0x55fa27167520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb177fe700 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26f6e4a0 0x55fa270fe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb16ffd700 1 -- 172.21.15.19:0/25039590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26f6e4a0 msgr2=0x55fa270fe3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26f6e4a0 0x55fa270fe3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb16ffd700 1 -- 172.21.15.19:0/25039590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa270eb400 msgr2=0x55fa27167520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa270eb400 0x55fa27167520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb16ffd700 1 -- 172.21.15.19:0/25039590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa27167ab0 con 0x55fa26fe8d00 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb177fe700 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26f6e4a0 0x55fa270fe3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb17fff700 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa270eb400 0x55fa27167520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbb16ffd700 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270fe910 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c031aa0 tx=0x7fbb0c031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2da1e34b0840b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.652+0000 7fbaf6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/25039590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c040060 con 0x55fa26fe8d00 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/25039590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbb0c004850 con 0x55fa26fe8d00 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/25039590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb10017920 con 0x55fa26fe8d00 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/25039590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c0316c0 con 0x55fa26fe8d00 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbb2c9c0b80 10 client.18516 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:40.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/25039590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa2716edb0 con 0x55fa26fe8d00 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/25039590 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb10017920 con 0x55fa26fe8d00 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/25039590 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb0c005590 con 0x55fa26fe8d00 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbb2c9c0b80 10 client.18516 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbb2c9c0b80 10 client.18516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbb2c9c0b80 20 client.18516 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55fa2716f4c0 0x55fa271718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbb2c9c0b80 4 client.18516 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.653+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/25039590 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa27015b20 con 0x55fa2716f4c0 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb177fe700 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55fa2716f4c0 0x55fa271718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb177fe700 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55fa2716f4c0 0x55fa271718e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12db25c817ae98ce server_cookie=4454d99eabe45001 in_seq=0 out_seq=0 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbaf6ffd700 10 client.18516 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbaf6ffd700 1 -- 172.21.15.19:0/25039590 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa27015b20 con 0x55fa2716f4c0 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbaf6ffd700 10 client.18516 handle_command_reply: tid=0 2024-05-09T01:36:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb2c9c0b80 1 client.18516 shutdown 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/25039590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 msgr2=0x55fa270fe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270fe910 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c031aa0 tx=0x7fbb0c031420 comp rx=0 tx=0).stop 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/25039590 shutdown_connections 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x55fa2716f4c0 0x55fa271718e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbaf6ffd700 0 client.18516 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa270eb400 0x55fa27167520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa26f6e4a0 0x55fa270fe3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb2c9c0b80 1 --2- 172.21.15.19:0/25039590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fa26fe8d00 0x55fa270fe910 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.654+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/25039590 >> 172.21.15.19:0/25039590 conn(0x55fa26f71f70 msgr2=0x55fa270f07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/25039590 shutdown_connections 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb2c9c0b80 1 -- 172.21.15.19:0/25039590 wait complete. 2024-05-09T01:36:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb2c9c0b80 20 client.18516 trim_cache size 0 max 16384 2024-05-09T01:36:40.657 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb000b2880 msgr2=0x7fbb000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb000b2880 0x7fbb000b4d90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005fd0 tx=0x7fbb0c005e30 comp rx=0 tx=0).stop 2024-05-09T01:36:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 msgr2=0x7fbb180ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 0x7fbb180ff2a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800b580 tx=0x7fbb080047b0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/3959593614 shutdown_connections 2024-05-09T01:36:40.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb18108740 0x7fbb180ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fbb000b2880 0x7fbb000b4d90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb181034c0 0x7fbb180ff2a0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 --2- 172.21.15.19:0/3959593614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18107df0 0x7fbb180ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/3959593614 >> 172.21.15.19:0/3959593614 conn(0x7fbb180fedc0 msgr2=0x7fbb1806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.655+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/3959593614 shutdown_connections 2024-05-09T01:36:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:40.656+0000 7fbb1dfaa700 1 -- 172.21.15.19:0/3959593614 wait complete. 2024-05-09T01:36:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:40 smithi059 ceph-mon[34351]: pgmap v712: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:36:40.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:40 smithi196 ceph-mon[35041]: pgmap v712: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:36:41.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:36:41.731 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:42 smithi059 ceph-mon[34351]: pgmap v713: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:42.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:42 smithi019 ceph-mon[30158]: pgmap v713: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:42.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:42 smithi196 ceph-mon[35041]: pgmap v713: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:43.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.444+0000 7f8333d87700 1 -- 172.21.15.19:0/2525535868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 msgr2=0x7f832c105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:43.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.445+0000 7f8333d87700 1 --2- 172.21.15.19:0/2525535868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 0x7f832c105900 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f831c00b0a0 tx=0x7f831c00b3b0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.445+0000 7f8333d87700 1 -- 172.21.15.19:0/2525535868 shutdown_connections 2024-05-09T01:36:43.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.445+0000 7f8333d87700 1 --2- 172.21.15.19:0/2525535868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f832c10e920 0x7f832c110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.445+0000 7f8333d87700 1 --2- 172.21.15.19:0/2525535868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 0x7f832c105900 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.445+0000 7f8333d87700 1 --2- 172.21.15.19:0/2525535868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f832c105e40 0x7f832c10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.445+0000 7f8333d87700 1 -- 172.21.15.19:0/2525535868 >> 172.21.15.19:0/2525535868 conn(0x7f832c078e00 msgr2=0x7f832c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.445+0000 7f8333d87700 1 -- 172.21.15.19:0/2525535868 shutdown_connections 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.445+0000 7f8333d87700 1 -- 172.21.15.19:0/2525535868 wait complete. 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.445+0000 7f8333d87700 1 Processor -- start 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8333d87700 1 -- start start 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8333d87700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 0x7f832c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8333d87700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f832c105e40 0x7f832c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8333d87700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f832c10e920 0x7f832c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8333d87700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f832c114470 con 0x7f832c1054f0 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8333d87700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1142f0 con 0x7f832c10e920 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8333d87700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1145f0 con 0x7f832c105e40 2024-05-09T01:36:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8331b23700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 0x7f832c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8331b23700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 0x7f832c19adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54786/0 (socket says 172.21.15.19:54786) 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8331b23700 1 -- 172.21.15.19:0/1549669247 learned_addr learned my addr 172.21.15.19:0/1549669247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8332324700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f832c10e920 0x7f832c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.446+0000 7f8331322700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f832c105e40 0x7f832c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8331b23700 1 -- 172.21.15.19:0/1549669247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f832c105e40 msgr2=0x7f832c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8331b23700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f832c105e40 0x7f832c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8331b23700 1 -- 172.21.15.19:0/1549669247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f832c10e920 msgr2=0x7f832c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8331b23700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f832c10e920 0x7f832c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8331b23700 1 -- 172.21.15.19:0/1549669247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c19fc90 con 0x7f832c1054f0 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8331322700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f832c105e40 0x7f832c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8332324700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f832c10e920 0x7f832c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:36:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8331b23700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 0x7f832c19adc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f831c00b070 tx=0x7f831c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:43.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f832affd700 1 -- 172.21.15.19:0/1549669247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c021070 con 0x7f832c1054f0 2024-05-09T01:36:43.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8333d87700 1 -- 172.21.15.19:0/1549669247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c19ff20 con 0x7f832c1054f0 2024-05-09T01:36:43.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f832affd700 1 -- 172.21.15.19:0/1549669247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f831c007e30 con 0x7f832c1054f0 2024-05-09T01:36:43.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f832affd700 1 -- 172.21.15.19:0/1549669247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c004760 con 0x7f832c1054f0 2024-05-09T01:36:43.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.447+0000 7f8333d87700 1 -- 172.21.15.19:0/1549669247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c1a78a0 con 0x7f832c1054f0 2024-05-09T01:36:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.450+0000 7f832affd700 1 -- 172.21.15.19:0/1549669247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f831c003de0 con 0x7f832c1054f0 2024-05-09T01:36:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.451+0000 7f832affd700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f83140b2900 0x7f83140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.451+0000 7f832affd700 1 -- 172.21.15.19:0/1549669247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f831c00f020 con 0x7f832c1054f0 2024-05-09T01:36:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.451+0000 7f8331322700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f83140b2900 0x7f83140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.451+0000 7f834279db80 1 Processor -- start 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.451+0000 7f834279db80 1 -- start start 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.451+0000 7f8331322700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f83140b2900 0x7f83140b4e10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f832c1107c0 tx=0x7f832000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f834279db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e0a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f834279db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f834279db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 0x560c4e0a7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f834279db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560c4dfc0ff0 con 0x560c4df214a0 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f834279db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560c4df5f7f0 con 0x560c4e00b160 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f834279db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560c4dfbcfb0 con 0x560c4e0a7400 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8331322700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8331322700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54798/0 (socket says 172.21.15.19:54798) 2024-05-09T01:36:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8331322700 1 -- 172.21.15.19:0/3355271286 learned_addr learned my addr 172.21.15.19:0/3355271286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8331b23700 1 --2- 172.21.15.19:0/3355271286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e0a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8332324700 1 --2- 172.21.15.19:0/3355271286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 0x560c4e0a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8331322700 1 -- 172.21.15.19:0/3355271286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 msgr2=0x560c4e0a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8331322700 1 --2- 172.21.15.19:0/3355271286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 0x560c4e0a7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8331322700 1 -- 172.21.15.19:0/3355271286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 msgr2=0x560c4e0a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8331322700 1 --2- 172.21.15.19:0/3355271286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e0a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.452+0000 7f8331322700 1 -- 172.21.15.19:0/3355271286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c4df57370 con 0x560c4df214a0 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.453+0000 7f8331322700 1 --2- 172.21.15.19:0/3355271286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a32f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f8320011790 tx=0x7f8320013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb64f19f52e944a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.453+0000 7f8313fff700 1 -- 172.21.15.19:0/3355271286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832001e620 con 0x560c4df214a0 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.453+0000 7f8313fff700 1 -- 172.21.15.19:0/3355271286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8320010e70 con 0x560c4df214a0 2024-05-09T01:36:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.453+0000 7f8313fff700 1 -- 172.21.15.19:0/3355271286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832001cb60 con 0x560c4df214a0 2024-05-09T01:36:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 -- 172.21.15.19:0/3355271286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 msgr2=0x560c4e0a32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 --2- 172.21.15.19:0/3355271286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a32f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f8320011790 tx=0x7f8320013000 comp rx=0 tx=0).stop 2024-05-09T01:36:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 -- 172.21.15.19:0/3355271286 shutdown_connections 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 --2- 172.21.15.19:0/3355271286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 0x560c4e0a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 --2- 172.21.15.19:0/3355271286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e0a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 --2- 172.21.15.19:0/3355271286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a32f0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 -- 172.21.15.19:0/3355271286 >> 172.21.15.19:0/3355271286 conn(0x560c4df816a0 msgr2=0x560c4e099400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 -- 172.21.15.19:0/3355271286 shutdown_connections 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 -- 172.21.15.19:0/3355271286 wait complete. 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 -- start start 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.454+0000 7f834279db80 1 Processor -- start 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f834279db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 0x560c4e0ac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f834279db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e00b160 0x560c4e0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f834279db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e0a7400 0x560c4e0ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f834279db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560c4de3cba0 con 0x560c4e00b160 2024-05-09T01:36:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f834279db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560c4df7b150 con 0x560c4e0a7400 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f834279db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560c4e0a81b0 con 0x560c4df214a0 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f834279db80 10 client.0 _ll_register_callbacks cb 0x560c4de326c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f8331322700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e00b160 0x560c4e0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f8331322700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e00b160 0x560c4e0acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54810/0 (socket says 172.21.15.19:54810) 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f8331322700 1 -- 172.21.15.19:0/1189105199 learned_addr learned my addr 172.21.15.19:0/1189105199 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f8332324700 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e0a7400 0x560c4e0ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f8331b23700 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 0x560c4e0ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f8331322700 1 -- 172.21.15.19:0/1189105199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 msgr2=0x560c4e0ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f8331322700 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 0x560c4e0ac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f8331322700 1 -- 172.21.15.19:0/1189105199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e0a7400 msgr2=0x560c4e0ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.455+0000 7f8331322700 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e0a7400 0x560c4e0ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f8331322700 1 -- 172.21.15.19:0/1189105199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c4de3c490 con 0x560c4e00b160 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f8331322700 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e00b160 0x560c4e0acbc0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f83200059d0 tx=0x7f8320005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb03a8cd8e96f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f83117fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f83117fa700 1 -- 172.21.15.19:0/1189105199 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832001c7a0 con 0x560c4e00b160 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f83117fa700 1 -- 172.21.15.19:0/1189105199 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f832001c940 con 0x560c4e00b160 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f834279db80 1 -- 172.21.15.19:0/1189105199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8320010e70 con 0x560c4e00b160 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f83117fa700 1 -- 172.21.15.19:0/1189105199 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f832001cb40 con 0x560c4e00b160 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f834279db80 10 client.18528 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f834279db80 1 -- 172.21.15.19:0/1189105199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c4e0b9f10 con 0x560c4e00b160 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.456+0000 7f83117fa700 1 -- 172.21.15.19:0/1189105199 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8320010e70 con 0x560c4e00b160 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f83117fa700 1 -- 172.21.15.19:0/1189105199 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f832001e4b0 con 0x560c4e00b160 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f834279db80 10 client.18528 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f834279db80 10 client.18528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f834279db80 20 client.18528 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f834279db80 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560c4e0bc440 0x560c4e0be860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f834279db80 4 client.18528 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f834279db80 1 -- 172.21.15.19:0/1189105199 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c4dfce9d0 con 0x560c4e0bc440 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f8331b23700 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560c4e0bc440 0x560c4e0be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f83117fa700 10 client.18528 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.457+0000 7f8331b23700 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560c4e0bc440 0x560c4e0be860 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562d339c8a7b5fbb server_cookie=9e54e8fe37c80305 in_seq=0 out_seq=0 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.458+0000 7f83117fa700 1 -- 172.21.15.19:0/1189105199 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c4dfce9d0 con 0x560c4e0bc440 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.458+0000 7f83117fa700 10 client.18528 handle_command_reply: tid=0 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.458+0000 7f834279db80 1 client.18528 shutdown 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.458+0000 7f834279db80 1 -- 172.21.15.19:0/1189105199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e00b160 msgr2=0x560c4e0acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.458+0000 7f834279db80 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e00b160 0x560c4e0acbc0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f83200059d0 tx=0x7f8320005270 comp rx=0 tx=0).stop 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.458+0000 7f834279db80 1 -- 172.21.15.19:0/1189105199 shutdown_connections 2024-05-09T01:36:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.458+0000 7f834279db80 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560c4e0bc440 0x560c4e0be860 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.459+0000 7f834279db80 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e0a7400 0x560c4e0ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.459+0000 7f834279db80 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 0x560c4e0ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.459+0000 7f834279db80 1 --2- 172.21.15.19:0/1189105199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e00b160 0x560c4e0acbc0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.459+0000 7f83117fa700 0 client.18528 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.459+0000 7f834279db80 1 -- 172.21.15.19:0/1189105199 >> 172.21.15.19:0/1189105199 conn(0x560c4df816a0 msgr2=0x560c4e0991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.459+0000 7f834279db80 1 -- 172.21.15.19:0/1189105199 shutdown_connections 2024-05-09T01:36:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.459+0000 7f834279db80 1 -- 172.21.15.19:0/1189105199 wait complete. 2024-05-09T01:36:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.459+0000 7f834279db80 20 client.18528 trim_cache size 0 max 16384 2024-05-09T01:36:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.474+0000 7f834279db80 1 Processor -- start 2024-05-09T01:36:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.474+0000 7f834279db80 1 -- start start 2024-05-09T01:36:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.474+0000 7f834279db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 0x560c4e0acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.474+0000 7f834279db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e09dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.474+0000 7f834279db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e0a7400 0x560c4e0b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.474+0000 7f834279db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1aca60 con 0x560c4e0a7400 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.474+0000 7f834279db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1ae5a0 con 0x560c4e00b160 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.474+0000 7f834279db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1ad130 con 0x560c4df214a0 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8332324700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e0a7400 0x560c4e0b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8332324700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e0a7400 0x560c4e0b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54824/0 (socket says 172.21.15.19:54824) 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8332324700 1 -- 172.21.15.19:0/1080127291 learned_addr learned my addr 172.21.15.19:0/1080127291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8331b23700 1 --2- 172.21.15.19:0/1080127291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 0x560c4e0acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8331322700 1 --2- 172.21.15.19:0/1080127291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e09dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8332324700 1 -- 172.21.15.19:0/1080127291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 msgr2=0x560c4e0acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8332324700 1 --2- 172.21.15.19:0/1080127291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 0x560c4e0acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8332324700 1 -- 172.21.15.19:0/1080127291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 msgr2=0x560c4e09dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8332324700 1 --2- 172.21.15.19:0/1080127291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e09dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8332324700 1 -- 172.21.15.19:0/1080127291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c4e09b180 con 0x560c4e0a7400 2024-05-09T01:36:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f8332324700 1 --2- 172.21.15.19:0/1080127291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e0a7400 0x560c4e0b8400 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f8324004390 tx=0x7f83240046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f175c7839127c053 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f83137fe700 1 -- 172.21.15.19:0/1080127291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324003b60 con 0x560c4e0a7400 2024-05-09T01:36:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f83137fe700 1 -- 172.21.15.19:0/1080127291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8324003d00 con 0x560c4e0a7400 2024-05-09T01:36:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.475+0000 7f83137fe700 1 -- 172.21.15.19:0/1080127291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324010690 con 0x560c4e0a7400 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.476+0000 7f834279db80 1 -- 172.21.15.19:0/1080127291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e0a7400 msgr2=0x560c4e0b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.476+0000 7f834279db80 1 --2- 172.21.15.19:0/1080127291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e0a7400 0x560c4e0b8400 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f8324004390 tx=0x7f83240046a0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.476+0000 7f834279db80 1 -- 172.21.15.19:0/1080127291 shutdown_connections 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.476+0000 7f834279db80 1 --2- 172.21.15.19:0/1080127291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4e0a7400 0x560c4e0b8400 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.476+0000 7f834279db80 1 --2- 172.21.15.19:0/1080127291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4df214a0 0x560c4e0acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.476+0000 7f834279db80 1 --2- 172.21.15.19:0/1080127291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e09dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.476+0000 7f834279db80 1 -- 172.21.15.19:0/1080127291 >> 172.21.15.19:0/1080127291 conn(0x560c4df816a0 msgr2=0x560c4e09d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.476+0000 7f834279db80 1 -- 172.21.15.19:0/1080127291 shutdown_connections 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.476+0000 7f834279db80 1 -- 172.21.15.19:0/1080127291 wait complete. 2024-05-09T01:36:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f834279db80 1 -- start start 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f834279db80 1 Processor -- start 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f834279db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f834279db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f834279db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 0x560c4e0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f834279db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1aded0 con 0x560c4df214a0 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f834279db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1ad800 con 0x560c4e00b160 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f834279db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1aaa00 con 0x560c4e0a7400 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f834279db80 10 client.0 _ll_register_callbacks cb 0x560c4de326c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.477+0000 7f8331b23700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331b23700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54840/0 (socket says 172.21.15.19:54840) 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331b23700 1 -- 172.21.15.19:0/2992783856 learned_addr learned my addr 172.21.15.19:0/2992783856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8332324700 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 0x560c4e0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331322700 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331b23700 1 -- 172.21.15.19:0/2992783856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 msgr2=0x560c4e0a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331b23700 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 0x560c4e0a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331b23700 1 -- 172.21.15.19:0/2992783856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 msgr2=0x560c4e0a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331b23700 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e0a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331b23700 1 -- 172.21.15.19:0/2992783856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c4e0b3780 con 0x560c4df214a0 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8332324700 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 0x560c4e0a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331322700 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e0a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f8331b23700 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a4220 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f831c0aab10 tx=0x7f831c004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df2dc08aeccf872 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f83117fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f83117fa700 1 -- 172.21.15.19:0/2992783856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c0af1c0 con 0x560c4df214a0 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f83117fa700 1 -- 172.21.15.19:0/2992783856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f831c0a7400 con 0x560c4df214a0 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f834279db80 1 -- 172.21.15.19:0/2992783856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8324003d00 con 0x560c4df214a0 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f83117fa700 1 -- 172.21.15.19:0/2992783856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f831c0b3be0 con 0x560c4df214a0 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f834279db80 10 client.18534 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.478+0000 7f834279db80 1 -- 172.21.15.19:0/2992783856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c4e0b4a40 con 0x560c4df214a0 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.479+0000 7f83117fa700 1 -- 172.21.15.19:0/2992783856 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8324003d00 con 0x560c4df214a0 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.479+0000 7f83117fa700 1 -- 172.21.15.19:0/2992783856 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f831c011070 con 0x560c4df214a0 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.479+0000 7f834279db80 10 client.18534 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.479+0000 7f834279db80 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.479+0000 7f834279db80 20 client.18534 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.479+0000 7f834279db80 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560c4e126aa0 0x560c4e128ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.479+0000 7f834279db80 4 client.18534 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.479+0000 7f834279db80 1 -- 172.21.15.19:0/2992783856 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c4dfc8a70 con 0x560c4e126aa0 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.479+0000 7f8331322700 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560c4e126aa0 0x560c4e128ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f8331322700 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560c4e126aa0 0x560c4e128ec0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0544851f962544c server_cookie=a97bca7b5668b6f in_seq=0 out_seq=0 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f83117fa700 10 client.18534 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f83117fa700 1 -- 172.21.15.19:0/2992783856 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c4dfc8a70 con 0x560c4e126aa0 2024-05-09T01:36:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f83117fa700 10 client.18534 handle_command_reply: tid=0 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 client.18534 shutdown 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 -- 172.21.15.19:0/2992783856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 msgr2=0x560c4e0a4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a4220 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f831c0aab10 tx=0x7f831c004200 comp rx=0 tx=0).stop 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 -- 172.21.15.19:0/2992783856 shutdown_connections 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560c4e126aa0 0x560c4e128ec0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c4e0a7400 0x560c4e0a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560c4df214a0 0x560c4e0a4220 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 --2- 172.21.15.19:0/2992783856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560c4e00b160 0x560c4e0a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f83117fa700 0 client.18534 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 -- 172.21.15.19:0/2992783856 >> 172.21.15.19:0/2992783856 conn(0x560c4df816a0 msgr2=0x560c4e0acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 -- 172.21.15.19:0/2992783856 shutdown_connections 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 1 -- 172.21.15.19:0/2992783856 wait complete. 2024-05-09T01:36:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.480+0000 7f834279db80 20 client.18534 trim_cache size 0 max 16384 2024-05-09T01:36:43.484 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:36:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 -- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f83140b2900 msgr2=0x7f83140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f83140b2900 0x7f83140b4e10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f832c1107c0 tx=0x7f832000b410 comp rx=0 tx=0).stop 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 -- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 msgr2=0x7f832c19adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 0x7f832c19adc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f831c00b070 tx=0x7f831c004d00 comp rx=0 tx=0).stop 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 -- 172.21.15.19:0/1549669247 shutdown_connections 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f832c10e920 0x7f832c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f83140b2900 0x7f83140b4e10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832c1054f0 0x7f832c19adc0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 --2- 172.21.15.19:0/1549669247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f832c105e40 0x7f832c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 -- 172.21.15.19:0/1549669247 >> 172.21.15.19:0/1549669247 conn(0x7f832c078e00 msgr2=0x7f832c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 -- 172.21.15.19:0/1549669247 shutdown_connections 2024-05-09T01:36:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:43.481+0000 7f8333d87700 1 -- 172.21.15.19:0/1549669247 wait complete. 2024-05-09T01:36:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:44 smithi019 ceph-mon[30158]: pgmap v714: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:36:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:44 smithi059 ceph-mon[34351]: pgmap v714: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:36:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:44 smithi196 ceph-mon[35041]: pgmap v714: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-05-09T01:36:45.345 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:36:45.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:36:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:46.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:46 smithi019 ceph-mon[30158]: pgmap v715: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:46 smithi059 ceph-mon[34351]: pgmap v715: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:46.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:46 smithi196 ceph-mon[35041]: pgmap v715: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:36:47.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.229+0000 7fe5fb266700 1 -- 172.21.15.19:0/2292046974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106580 msgr2=0x7fe5f4106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:47.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.229+0000 7fe5fb266700 1 --2- 172.21.15.19:0/2292046974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106580 0x7fe5f4106990 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009990 tx=0x7fe5e4009ca0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.229+0000 7fe5fb266700 1 -- 172.21.15.19:0/2292046974 shutdown_connections 2024-05-09T01:36:47.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.229+0000 7fe5fb266700 1 --2- 172.21.15.19:0/2292046974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5f4106ed0 0x7fe5f4113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.229+0000 7fe5fb266700 1 --2- 172.21.15.19:0/2292046974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f410c5c0 0x7fe5f410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.229+0000 7fe5fb266700 1 --2- 172.21.15.19:0/2292046974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106580 0x7fe5f4106990 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.230+0000 7fe5fb266700 1 -- 172.21.15.19:0/2292046974 >> 172.21.15.19:0/2292046974 conn(0x7fe5f4102080 msgr2=0x7fe5f41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:47.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.230+0000 7fe5fb266700 1 -- 172.21.15.19:0/2292046974 shutdown_connections 2024-05-09T01:36:47.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.230+0000 7fe5fb266700 1 -- 172.21.15.19:0/2292046974 wait complete. 2024-05-09T01:36:47.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5fb266700 1 Processor -- start 2024-05-09T01:36:47.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5fb266700 1 -- start start 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5fb266700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5f4106580 0x7fe5f419d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5fb266700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106ed0 0x7fe5f419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5fb266700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f410c5c0 0x7fe5f41a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5fb266700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41166f0 con 0x7fe5f4106ed0 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5fb266700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f4116570 con 0x7fe5f410c5c0 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5fb266700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f4116870 con 0x7fe5f4106580 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5f8801700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106ed0 0x7fe5f419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5f8801700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106ed0 0x7fe5f419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54862/0 (socket says 172.21.15.19:54862) 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5f8801700 1 -- 172.21.15.19:0/1496681076 learned_addr learned my addr 172.21.15.19:0/1496681076 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5f9803700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f410c5c0 0x7fe5f41a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5f9002700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5f4106580 0x7fe5f419d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5f8801700 1 -- 172.21.15.19:0/1496681076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5f4106580 msgr2=0x7fe5f419d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5f8801700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5f4106580 0x7fe5f419d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5f8801700 1 -- 172.21.15.19:0/1496681076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f410c5c0 msgr2=0x7fe5f41a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.231+0000 7fe5f8801700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f410c5c0 0x7fe5f41a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.232+0000 7fe5f8801700 1 -- 172.21.15.19:0/1496681076 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f41a1f30 con 0x7fe5f4106ed0 2024-05-09T01:36:47.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.232+0000 7fe5f9002700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5f4106580 0x7fe5f419d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:47.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.232+0000 7fe5f8801700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106ed0 0x7fe5f419d600 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:47.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.232+0000 7fe5ea7fc700 1 -- 172.21.15.19:0/1496681076 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4026070 con 0x7fe5f4106ed0 2024-05-09T01:36:47.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.232+0000 7fe5fb266700 1 -- 172.21.15.19:0/1496681076 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f41a21c0 con 0x7fe5f4106ed0 2024-05-09T01:36:47.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.232+0000 7fe5ea7fc700 1 -- 172.21.15.19:0/1496681076 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e4021410 con 0x7fe5f4106ed0 2024-05-09T01:36:47.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.232+0000 7fe5ea7fc700 1 -- 172.21.15.19:0/1496681076 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4017650 con 0x7fe5f4106ed0 2024-05-09T01:36:47.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.233+0000 7fe5fb266700 1 -- 172.21.15.19:0/1496681076 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f41a9b40 con 0x7fe5f4106ed0 2024-05-09T01:36:47.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.235+0000 7fe5ea7fc700 1 -- 172.21.15.19:0/1496681076 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe5e4029070 con 0x7fe5f4106ed0 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.236+0000 7fe5ea7fc700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5e00b2880 0x7fe5e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.236+0000 7fe5f9002700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5e00b2880 0x7fe5e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe5ea7fc700 1 -- 172.21.15.19:0/1496681076 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe5e40e2ea0 con 0x7fe5f4106ed0 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe609c7cb80 1 Processor -- start 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe609c7cb80 1 -- start start 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe5f9002700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5e00b2880 0x7fe5e00b4d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe5f41079f0 tx=0x7fe5f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c7952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c795800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c78f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56017c6b2120 con 0x56017c68cd80 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56017c657a20 con 0x56017c612520 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56017c6ae0e0 con 0x56017c78f480 2024-05-09T01:36:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe5f9002700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c7952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe5f9002700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c7952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54870/0 (socket says 172.21.15.19:54870) 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.237+0000 7fe5f9002700 1 -- 172.21.15.19:0/2102196701 learned_addr learned my addr 172.21.15.19:0/2102196701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5f9803700 1 --2- 172.21.15.19:0/2102196701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c78f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5f8801700 1 --2- 172.21.15.19:0/2102196701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c795800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5f9002700 1 -- 172.21.15.19:0/2102196701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 msgr2=0x56017c78f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5f9002700 1 --2- 172.21.15.19:0/2102196701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c78f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5f9002700 1 -- 172.21.15.19:0/2102196701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 msgr2=0x56017c795800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5f9002700 1 --2- 172.21.15.19:0/2102196701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c795800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5f9002700 1 -- 172.21.15.19:0/2102196701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017c648580 con 0x56017c68cd80 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5f9002700 1 --2- 172.21.15.19:0/2102196701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c7952c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0011f60 tx=0x7fe5f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0b7e9c1b1f7fe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5df7fe700 1 -- 172.21.15.19:0/2102196701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0025070 con 0x56017c68cd80 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5df7fe700 1 -- 172.21.15.19:0/2102196701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f0016070 con 0x56017c68cd80 2024-05-09T01:36:47.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.238+0000 7fe5df7fe700 1 -- 172.21.15.19:0/2102196701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0012a00 con 0x56017c68cd80 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.239+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2102196701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 msgr2=0x56017c7952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.239+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2102196701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c7952c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0011f60 tx=0x7fe5f0014000 comp rx=0 tx=0).stop 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.239+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2102196701 shutdown_connections 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.239+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2102196701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c78f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.239+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2102196701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c7952c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.239+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2102196701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c795800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.239+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2102196701 >> 172.21.15.19:0/2102196701 conn(0x56017c615ff0 msgr2=0x56017c78a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.239+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2102196701 shutdown_connections 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.239+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2102196701 wait complete. 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe609c7cb80 1 -- start start 2024-05-09T01:36:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe609c7cb80 1 Processor -- start 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c799070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c68cd80 0x56017c7995b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c78f480 0x56017c799af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56017c7901e0 con 0x56017c78f480 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56017c64cba0 con 0x56017c612520 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56017c79c0a0 con 0x56017c68cd80 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe609c7cb80 10 client.0 _ll_register_callbacks cb 0x56017c523740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.240+0000 7fe5f9803700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c78f480 0x56017c799af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9803700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c78f480 0x56017c799af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54884/0 (socket says 172.21.15.19:54884) 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9803700 1 -- 172.21.15.19:0/2812398202 learned_addr learned my addr 172.21.15.19:0/2812398202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9002700 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c799070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f8801700 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c68cd80 0x56017c7995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9803700 1 -- 172.21.15.19:0/2812398202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c68cd80 msgr2=0x56017c7995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9803700 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c68cd80 0x56017c7995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9803700 1 -- 172.21.15.19:0/2812398202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 msgr2=0x56017c799070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9803700 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c799070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9803700 1 -- 172.21.15.19:0/2812398202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017c52d510 con 0x56017c78f480 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9002700 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c799070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5f9803700 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c78f480 0x56017c799af0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec007f40 tx=0x7fe5ec009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f57cb8a9c8a6ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5dcff9700 1 -- 172.21.15.19:0/2812398202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec01a030 con 0x56017c78f480 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe5dcff9700 1 -- 172.21.15.19:0/2812398202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe5ec013930 con 0x56017c78f480 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.241+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2812398202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f0016070 con 0x56017c78f480 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe5dcff9700 1 -- 172.21.15.19:0/2812398202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5ec025070 con 0x56017c78f480 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe609c7cb80 10 client.18546 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2812398202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56017c7a6aa0 con 0x56017c78f480 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe5dcff9700 1 -- 172.21.15.19:0/2812398202 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f0016070 con 0x56017c78f480 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe5dcff9700 1 -- 172.21.15.19:0/2812398202 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe5ec01f410 con 0x56017c78f480 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe609c7cb80 10 client.18546 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe609c7cb80 10 client.18546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe609c7cb80 20 client.18546 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56017c7a90e0 0x56017c7ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe609c7cb80 4 client.18546 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:47.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.242+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2812398202 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56017c6bfb00 con 0x56017c7a90e0 2024-05-09T01:36:47.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.243+0000 7fe5f9002700 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56017c7a90e0 0x56017c7ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.243+0000 7fe5f9002700 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56017c7a90e0 0x56017c7ab540 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee86f690036b68c server_cookie=5178cb20dcddf4b5 in_seq=0 out_seq=0 2024-05-09T01:36:47.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.243+0000 7fe5dcff9700 10 client.18546 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe5dcff9700 1 -- 172.21.15.19:0/2812398202 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56017c6bfb00 con 0x56017c7a90e0 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe5dcff9700 10 client.18546 handle_command_reply: tid=0 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe609c7cb80 1 client.18546 shutdown 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2812398202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c78f480 msgr2=0x56017c799af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c78f480 0x56017c799af0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec007f40 tx=0x7fe5ec009350 comp rx=0 tx=0).stop 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2812398202 shutdown_connections 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56017c7a90e0 0x56017c7ab540 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c78f480 0x56017c799af0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c799070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/2812398202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c68cd80 0x56017c7995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe5dcff9700 0 client.18546 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:47.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.244+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2812398202 >> 172.21.15.19:0/2812398202 conn(0x56017c615ff0 msgr2=0x56017c784070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:47.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.245+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2812398202 shutdown_connections 2024-05-09T01:36:47.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.245+0000 7fe609c7cb80 1 -- 172.21.15.19:0/2812398202 wait complete. 2024-05-09T01:36:47.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.245+0000 7fe609c7cb80 20 client.18546 trim_cache size 0 max 16384 2024-05-09T01:36:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.265+0000 7fe609c7cb80 1 Processor -- start 2024-05-09T01:36:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.265+0000 7fe609c7cb80 1 -- start start 2024-05-09T01:36:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.265+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c79b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.265+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c784780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.265+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c7a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41b0e20 con 0x56017c68cd80 2024-05-09T01:36:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41b2960 con 0x56017c612520 2024-05-09T01:36:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41b14f0 con 0x56017c78f480 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f8801700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c784780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f8801700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c784780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54896/0 (socket says 172.21.15.19:54896) 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f8801700 1 -- 172.21.15.19:0/3211944640 learned_addr learned my addr 172.21.15.19:0/3211944640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f9002700 1 --2- 172.21.15.19:0/3211944640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c79b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f9803700 1 --2- 172.21.15.19:0/3211944640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c7a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f8801700 1 -- 172.21.15.19:0/3211944640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 msgr2=0x56017c7a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f8801700 1 --2- 172.21.15.19:0/3211944640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c7a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f8801700 1 -- 172.21.15.19:0/3211944640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 msgr2=0x56017c79b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f8801700 1 --2- 172.21.15.19:0/3211944640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c79b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f8801700 1 -- 172.21.15.19:0/3211944640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017c798e40 con 0x56017c68cd80 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5f8801700 1 --2- 172.21.15.19:0/3211944640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c784780 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b2e0 tx=0x7fe5e40a9d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac9b4343588c80f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5deffd700 1 -- 172.21.15.19:0/3211944640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40aa530 con 0x56017c68cd80 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5deffd700 1 -- 172.21.15.19:0/3211944640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e40ae8e0 con 0x56017c68cd80 2024-05-09T01:36:47.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.266+0000 7fe5deffd700 1 -- 172.21.15.19:0/3211944640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40aa9f0 con 0x56017c68cd80 2024-05-09T01:36:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.267+0000 7fe609c7cb80 1 -- 172.21.15.19:0/3211944640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 msgr2=0x56017c784780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.267+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/3211944640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c784780 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b2e0 tx=0x7fe5e40a9d70 comp rx=0 tx=0).stop 2024-05-09T01:36:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.267+0000 7fe609c7cb80 1 -- 172.21.15.19:0/3211944640 shutdown_connections 2024-05-09T01:36:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.267+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/3211944640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c7a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.267+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/3211944640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c612520 0x56017c79b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.267+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/3211944640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c68cd80 0x56017c784780 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.267+0000 7fe609c7cb80 1 -- 172.21.15.19:0/3211944640 >> 172.21.15.19:0/3211944640 conn(0x56017c615ff0 msgr2=0x56017c784070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.267+0000 7fe609c7cb80 1 -- 172.21.15.19:0/3211944640 shutdown_connections 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.267+0000 7fe609c7cb80 1 -- 172.21.15.19:0/3211944640 wait complete. 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe609c7cb80 1 -- start start 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe609c7cb80 1 Processor -- start 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c612520 0x56017c7a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c68cd80 0x56017c7b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe609c7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c7b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41b2290 con 0x56017c612520 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41b1bc0 con 0x56017c68cd80 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe609c7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41aede0 con 0x56017c78f480 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe609c7cb80 10 client.0 _ll_register_callbacks cb 0x56017c523740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.268+0000 7fe5f9002700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c612520 0x56017c7a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f9002700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c612520 0x56017c7a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54904/0 (socket says 172.21.15.19:54904) 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f9002700 1 -- 172.21.15.19:0/4241973179 learned_addr learned my addr 172.21.15.19:0/4241973179 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f8801700 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c68cd80 0x56017c7b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f9803700 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c7b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f9002700 1 -- 172.21.15.19:0/4241973179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 msgr2=0x56017c7b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f9002700 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c7b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f9002700 1 -- 172.21.15.19:0/4241973179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c68cd80 msgr2=0x56017c7b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f9002700 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c68cd80 0x56017c7b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f9002700 1 -- 172.21.15.19:0/4241973179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017c7b0e10 con 0x56017c612520 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5f9002700 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c612520 0x56017c7a15a0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0011f30 tx=0x7fe5f0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c9ed02b8a98dc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5dcff9700 1 -- 172.21.15.19:0/4241973179 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0025070 con 0x56017c612520 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5dcff9700 1 -- 172.21.15.19:0/4241973179 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe5f0003680 con 0x56017c612520 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe609c7cb80 1 -- 172.21.15.19:0/4241973179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5e40ae8e0 con 0x56017c612520 2024-05-09T01:36:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe5dcff9700 1 -- 172.21.15.19:0/4241973179 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5f0005690 con 0x56017c612520 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe609c7cb80 10 client.18552 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.269+0000 7fe609c7cb80 1 -- 172.21.15.19:0/4241973179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56017c813660 con 0x56017c612520 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.270+0000 7fe5dcff9700 1 -- 172.21.15.19:0/4241973179 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e40ae8e0 con 0x56017c612520 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.270+0000 7fe5dcff9700 1 -- 172.21.15.19:0/4241973179 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe5f002b040 con 0x56017c612520 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.270+0000 7fe609c7cb80 10 client.18552 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.270+0000 7fe609c7cb80 10 client.18552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.270+0000 7fe609c7cb80 20 client.18552 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.270+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56017c813a40 0x56017c815e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.270+0000 7fe609c7cb80 4 client.18552 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.270+0000 7fe609c7cb80 1 -- 172.21.15.19:0/4241973179 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56017c6b9ba0 con 0x56017c813a40 2024-05-09T01:36:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.270+0000 7fe5f8801700 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56017c813a40 0x56017c815e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.271+0000 7fe5f8801700 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56017c813a40 0x56017c815e60 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c999210e22311753 server_cookie=8b7435b8471709c6 in_seq=0 out_seq=0 2024-05-09T01:36:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.271+0000 7fe5dcff9700 10 client.18552 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:47.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe5dcff9700 1 -- 172.21.15.19:0/4241973179 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56017c6b9ba0 con 0x56017c813a40 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe5dcff9700 10 client.18552 handle_command_reply: tid=0 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 client.18552 shutdown 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 -- 172.21.15.19:0/4241973179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c612520 msgr2=0x56017c7a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c612520 0x56017c7a15a0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0011f30 tx=0x7fe5f0011b50 comp rx=0 tx=0).stop 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 -- 172.21.15.19:0/4241973179 shutdown_connections 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x56017c813a40 0x56017c815e60 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56017c78f480 0x56017c7b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c612520 0x56017c7a15a0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 --2- 172.21.15.19:0/4241973179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56017c68cd80 0x56017c7b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 -- 172.21.15.19:0/4241973179 >> 172.21.15.19:0/4241973179 conn(0x56017c615ff0 msgr2=0x56017c792c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe5dcff9700 0 client.18552 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 -- 172.21.15.19:0/4241973179 shutdown_connections 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 1 -- 172.21.15.19:0/4241973179 wait complete. 2024-05-09T01:36:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.272+0000 7fe609c7cb80 20 client.18552 trim_cache size 0 max 16384 2024-05-09T01:36:47.273 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:36:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:36:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "cca71913-29de-4e88-8098-94990fc735c3", 2024-05-09T01:36:47.273 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:36:47.273 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:36:47.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 -- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5e00b2880 msgr2=0x7fe5e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5e00b2880 0x7fe5e00b4d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe5f41079f0 tx=0x7fe5f0009260 comp rx=0 tx=0).stop 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 -- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106ed0 msgr2=0x7fe5f419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106ed0 0x7fe5f419d600 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e40047b0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 -- 172.21.15.19:0/1496681076 shutdown_connections 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f410c5c0 0x7fe5f41a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5f4106580 0x7fe5f419d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe5e00b2880 0x7fe5e00b4d90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 --2- 172.21.15.19:0/1496681076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5f4106ed0 0x7fe5f419d600 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 -- 172.21.15.19:0/1496681076 >> 172.21.15.19:0/1496681076 conn(0x7fe5f4102080 msgr2=0x7fe5f41047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 -- 172.21.15.19:0/1496681076 shutdown_connections 2024-05-09T01:36:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:47.273+0000 7fe5fb266700 1 -- 172.21.15.19:0/1496681076 wait complete. 2024-05-09T01:36:48.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:36:48.344 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:36:48.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:48 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:36:48.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:48 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:48.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:48 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:36:48.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:48 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:48.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:48 smithi019 ceph-mon[30158]: pgmap v716: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:48 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:36:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:48 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:48 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:36:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:48 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:48 smithi059 ceph-mon[34351]: pgmap v716: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:48 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:36:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:48 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:48 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:36:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:48 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:36:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:48 smithi196 ceph-mon[35041]: pgmap v716: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:36:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.135+0000 7f36f486e700 1 -- 172.21.15.19:0/2447334250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107e10 msgr2=0x7f36f0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.135+0000 7f36f486e700 1 --2- 172.21.15.19:0/2447334250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107e10 0x7f36f0108220 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f36e40099c0 tx=0x7f36e4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.135+0000 7f36f486e700 1 -- 172.21.15.19:0/2447334250 shutdown_connections 2024-05-09T01:36:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.135+0000 7f36f486e700 1 --2- 172.21.15.19:0/2447334250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0108760 0x7f36f01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.135+0000 7f36f486e700 1 --2- 172.21.15.19:0/2447334250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f01034e0 0x7f36f01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.135+0000 7f36f486e700 1 --2- 172.21.15.19:0/2447334250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107e10 0x7f36f0108220 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.135+0000 7f36f486e700 1 -- 172.21.15.19:0/2447334250 >> 172.21.15.19:0/2447334250 conn(0x7f36f00fef30 msgr2=0x7f36f0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.135+0000 7f36f486e700 1 -- 172.21.15.19:0/2447334250 shutdown_connections 2024-05-09T01:36:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.135+0000 7f36f486e700 1 -- 172.21.15.19:0/2447334250 wait complete. 2024-05-09T01:36:50.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.136+0000 7f36f486e700 1 Processor -- start 2024-05-09T01:36:50.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.136+0000 7f36f486e700 1 -- start start 2024-05-09T01:36:50.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.136+0000 7f36f486e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f01034e0 0x7f36f0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.136+0000 7f36f486e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0107e10 0x7f36f019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.136+0000 7f36f486e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f0108760 0x7f36f019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.136+0000 7f36f486e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0113370 con 0x7f36f01034e0 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.136+0000 7f36f486e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01131f0 con 0x7f36f0108760 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.136+0000 7f36f486e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01134f0 con 0x7f36f0107e10 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.136+0000 7f36ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f01034e0 0x7f36f0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f01034e0 0x7f36f0199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51130/0 (socket says 172.21.15.19:51130) 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36ee59c700 1 -- 172.21.15.19:0/4249825812 learned_addr learned my addr 172.21.15.19:0/4249825812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36eed9d700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f0108760 0x7f36f019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36edd9b700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0107e10 0x7f36f019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36ee59c700 1 -- 172.21.15.19:0/4249825812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0107e10 msgr2=0x7f36f019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36ee59c700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0107e10 0x7f36f019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36ee59c700 1 -- 172.21.15.19:0/4249825812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f0108760 msgr2=0x7f36f019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36ee59c700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f0108760 0x7f36f019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36ee59c700 1 -- 172.21.15.19:0/4249825812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f019ec10 con 0x7f36f01034e0 2024-05-09T01:36:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36ee59c700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f01034e0 0x7f36f0199d40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009970 tx=0x7f36e0009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36df7fe700 1 -- 172.21.15.19:0/4249825812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e001c070 con 0x7f36f01034e0 2024-05-09T01:36:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36f486e700 1 -- 172.21.15.19:0/4249825812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f019ef00 con 0x7f36f01034e0 2024-05-09T01:36:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36df7fe700 1 -- 172.21.15.19:0/4249825812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e000d260 con 0x7f36f01034e0 2024-05-09T01:36:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36df7fe700 1 -- 172.21.15.19:0/4249825812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0017910 con 0x7f36f01034e0 2024-05-09T01:36:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.137+0000 7f36f486e700 1 -- 172.21.15.19:0/4249825812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f006c040 con 0x7f36f01034e0 2024-05-09T01:36:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.140+0000 7f36df7fe700 1 -- 172.21.15.19:0/4249825812 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f36e001f070 con 0x7f36f01034e0 2024-05-09T01:36:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.141+0000 7f36df7fe700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36d80b2880 0x7f36d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.141+0000 7f36edd9b700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36d80b2880 0x7f36d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.141+0000 7f36df7fe700 1 -- 172.21.15.19:0/4249825812 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f36e00e33f0 con 0x7f36f01034e0 2024-05-09T01:36:50.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.141+0000 7f3703284b80 1 Processor -- start 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.141+0000 7f3703284b80 1 -- start start 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36edd9b700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36d80b2880 0x7f36d80b4d90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f36e4006010 tx=0x7f36e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f3703284b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 0x559862f73900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f3703284b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f73e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f3703284b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f6dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f3703284b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559862e8bff0 con 0x559862f6dab0 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f3703284b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559862e2a7f0 con 0x559862dec4a0 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f3703284b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559862e87fb0 con 0x559862ed6160 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36eed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f6dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36eed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f6dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51142/0 (socket says 172.21.15.19:51142) 2024-05-09T01:36:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36eed9d700 1 -- 172.21.15.19:0/1233613761 learned_addr learned my addr 172.21.15.19:0/1233613761 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36edd9b700 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f73e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36ee59c700 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 0x559862f73900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36eed9d700 1 -- 172.21.15.19:0/1233613761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 msgr2=0x559862f73900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36eed9d700 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 0x559862f73900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36eed9d700 1 -- 172.21.15.19:0/1233613761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 msgr2=0x559862f73e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36eed9d700 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f73e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36eed9d700 1 -- 172.21.15.19:0/1233613761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559862e22370 con 0x559862f6dab0 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.142+0000 7f36ee59c700 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 0x559862f73900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.143+0000 7f36eed9d700 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f6dee0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f36e8003fa0 tx=0x7f36e8009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c859a7c84ac935cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.143+0000 7f36dcff9700 1 -- 172.21.15.19:0/1233613761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8004bb0 con 0x559862f6dab0 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.143+0000 7f36dcff9700 1 -- 172.21.15.19:0/1233613761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e8004d50 con 0x559862f6dab0 2024-05-09T01:36:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.143+0000 7f36dcff9700 1 -- 172.21.15.19:0/1233613761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8007d20 con 0x559862f6dab0 2024-05-09T01:36:50.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.144+0000 7f3703284b80 1 -- 172.21.15.19:0/1233613761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 msgr2=0x559862f6dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:50.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.144+0000 7f3703284b80 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f6dee0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f36e8003fa0 tx=0x7f36e8009ba0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.144+0000 7f3703284b80 1 -- 172.21.15.19:0/1233613761 shutdown_connections 2024-05-09T01:36:50.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.144+0000 7f3703284b80 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f6dee0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.144+0000 7f3703284b80 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 0x559862f73900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.144+0000 7f3703284b80 1 --2- 172.21.15.19:0/1233613761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f73e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.144+0000 7f3703284b80 1 -- 172.21.15.19:0/1233613761 >> 172.21.15.19:0/1233613761 conn(0x559862e4c6a0 msgr2=0x559862f64400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.144+0000 7f3703284b80 1 -- 172.21.15.19:0/1233613761 shutdown_connections 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.144+0000 7f3703284b80 1 -- 172.21.15.19:0/1233613761 wait complete. 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f3703284b80 1 -- start start 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f3703284b80 1 Processor -- start 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f3703284b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f3703284b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862ed6160 0x559862f77a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f3703284b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 0x559862f77f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f3703284b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559862d07ba0 con 0x559862ed6160 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f3703284b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559862e46150 con 0x559862dec4a0 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f3703284b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559862f7a5e0 con 0x559862f6dab0 2024-05-09T01:36:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f3703284b80 10 client.0 _ll_register_callbacks cb 0x559862cfd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f36edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862ed6160 0x559862f77a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f36edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862ed6160 0x559862f77a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51158/0 (socket says 172.21.15.19:51158) 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f36edd9b700 1 -- 172.21.15.19:0/49101487 learned_addr learned my addr 172.21.15.19:0/49101487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f36eed9d700 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 0x559862f77f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.145+0000 7f36ee59c700 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36edd9b700 1 -- 172.21.15.19:0/49101487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 msgr2=0x559862f77f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36edd9b700 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 0x559862f77f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36edd9b700 1 -- 172.21.15.19:0/49101487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 msgr2=0x559862f774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36edd9b700 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f774e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36edd9b700 1 -- 172.21.15.19:0/49101487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559862d07490 con 0x559862ed6160 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36eed9d700 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 0x559862f77f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36edd9b700 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862ed6160 0x559862f77a20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f36f0076ab0 tx=0x7f36e401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abe5e6ec55a8695 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36d5ffb700 1 -- 172.21.15.19:0/49101487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e400f460 con 0x559862ed6160 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36d5ffb700 1 -- 172.21.15.19:0/49101487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f36e400f600 con 0x559862ed6160 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f3703284b80 1 -- 172.21.15.19:0/49101487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e8004d50 con 0x559862ed6160 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f36d5ffb700 1 -- 172.21.15.19:0/49101487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36e400f800 con 0x559862ed6160 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f3703284b80 10 client.18564 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.146+0000 7f3703284b80 1 -- 172.21.15.19:0/49101487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559862f84f90 con 0x559862ed6160 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.147+0000 7f36d5ffb700 1 -- 172.21.15.19:0/49101487 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e8004d50 con 0x559862ed6160 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.147+0000 7f36d5ffb700 1 -- 172.21.15.19:0/49101487 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f36e4022070 con 0x559862ed6160 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.147+0000 7f3703284b80 10 client.18564 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.147+0000 7f3703284b80 10 client.18564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.147+0000 7f3703284b80 20 client.18564 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.147+0000 7f3703284b80 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559862f87570 0x559862f899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.147+0000 7f3703284b80 4 client.18564 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.147+0000 7f3703284b80 1 -- 172.21.15.19:0/49101487 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559862e999d0 con 0x559862f87570 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.147+0000 7f36ee59c700 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559862f87570 0x559862f899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.148+0000 7f36d5ffb700 10 client.18564 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.148+0000 7f36ee59c700 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559862f87570 0x559862f899d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72cd2f542ec1915 server_cookie=6c034aaee82cdaa0 in_seq=0 out_seq=0 2024-05-09T01:36:50.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f36d5ffb700 1 -- 172.21.15.19:0/49101487 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559862e999d0 con 0x559862f87570 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f36d5ffb700 10 client.18564 handle_command_reply: tid=0 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 client.18564 shutdown 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 -- 172.21.15.19:0/49101487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862ed6160 msgr2=0x559862f77a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862ed6160 0x559862f77a20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f36f0076ab0 tx=0x7f36e401e000 comp rx=0 tx=0).stop 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 -- 172.21.15.19:0/49101487 shutdown_connections 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559862f87570 0x559862f899d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 0x559862f77f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 --2- 172.21.15.19:0/49101487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862ed6160 0x559862f77a20 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f36d5ffb700 0 client.18564 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 -- 172.21.15.19:0/49101487 >> 172.21.15.19:0/49101487 conn(0x559862e4c6a0 msgr2=0x559862f641c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 -- 172.21.15.19:0/49101487 shutdown_connections 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 1 -- 172.21.15.19:0/49101487 wait complete. 2024-05-09T01:36:50.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.149+0000 7f3703284b80 20 client.18564 trim_cache size 0 max 16384 2024-05-09T01:36:50.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.169+0000 7f3703284b80 1 Processor -- start 2024-05-09T01:36:50.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.169+0000 7f3703284b80 1 -- start start 2024-05-09T01:36:50.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f3703284b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862dec4a0 0x559862f71c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f3703284b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862ed6160 0x559862f77da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f3703284b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 0x559862f834b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f3703284b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01acb20 con 0x559862dec4a0 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f3703284b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ae660 con 0x559862ed6160 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f3703284b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ad1f0 con 0x559862f6dab0 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862dec4a0 0x559862f71c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862dec4a0 0x559862f71c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51164/0 (socket says 172.21.15.19:51164) 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36ee59c700 1 -- 172.21.15.19:0/2323622507 learned_addr learned my addr 172.21.15.19:0/2323622507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36eed9d700 1 --2- 172.21.15.19:0/2323622507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 0x559862f834b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36ee59c700 1 -- 172.21.15.19:0/2323622507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 msgr2=0x559862f834b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36ee59c700 1 --2- 172.21.15.19:0/2323622507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 0x559862f834b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36ee59c700 1 -- 172.21.15.19:0/2323622507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862ed6160 msgr2=0x559862f77da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36ee59c700 1 --2- 172.21.15.19:0/2323622507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862ed6160 0x559862f77da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36ee59c700 1 -- 172.21.15.19:0/2323622507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559862f5df60 con 0x559862dec4a0 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.170+0000 7f36ee59c700 1 --2- 172.21.15.19:0/2323622507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862dec4a0 0x559862f71c40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f36e00aa690 tx=0x7f36e00cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512b5414228de3a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f36d7fff700 1 -- 172.21.15.19:0/2323622507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00aa990 con 0x559862dec4a0 2024-05-09T01:36:50.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f36d7fff700 1 -- 172.21.15.19:0/2323622507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e00aed40 con 0x559862dec4a0 2024-05-09T01:36:50.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f36d7fff700 1 -- 172.21.15.19:0/2323622507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00aae50 con 0x559862dec4a0 2024-05-09T01:36:50.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f3703284b80 1 -- 172.21.15.19:0/2323622507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862dec4a0 msgr2=0x559862f71c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:50.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f3703284b80 1 --2- 172.21.15.19:0/2323622507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862dec4a0 0x559862f71c40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f36e00aa690 tx=0x7f36e00cd040 comp rx=0 tx=0).stop 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f3703284b80 1 -- 172.21.15.19:0/2323622507 shutdown_connections 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f3703284b80 1 --2- 172.21.15.19:0/2323622507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862f6dab0 0x559862f834b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f3703284b80 1 --2- 172.21.15.19:0/2323622507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862dec4a0 0x559862f71c40 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f3703284b80 1 --2- 172.21.15.19:0/2323622507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862ed6160 0x559862f77da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f3703284b80 1 -- 172.21.15.19:0/2323622507 >> 172.21.15.19:0/2323622507 conn(0x559862e4c6a0 msgr2=0x559862f78b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f3703284b80 1 -- 172.21.15.19:0/2323622507 shutdown_connections 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.171+0000 7f3703284b80 1 -- 172.21.15.19:0/2323622507 wait complete. 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f3703284b80 1 -- start start 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f3703284b80 1 Processor -- start 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f3703284b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f7f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f3703284b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 0x559862f7fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f3703284b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f7c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f3703284b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01adf90 con 0x559862f6dab0 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f3703284b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ad8c0 con 0x559862dec4a0 2024-05-09T01:36:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f3703284b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01aaac0 con 0x559862ed6160 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f3703284b80 10 client.0 _ll_register_callbacks cb 0x559862cfd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f36eed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f7c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f36eed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f7c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51180/0 (socket says 172.21.15.19:51180) 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f36eed9d700 1 -- 172.21.15.19:0/4140398410 learned_addr learned my addr 172.21.15.19:0/4140398410 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.172+0000 7f36edd9b700 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 0x559862f7fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36ee59c700 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f7f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36eed9d700 1 -- 172.21.15.19:0/4140398410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 msgr2=0x559862f7fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36eed9d700 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 0x559862f7fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36eed9d700 1 -- 172.21.15.19:0/4140398410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 msgr2=0x559862f7f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36eed9d700 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f7f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36eed9d700 1 -- 172.21.15.19:0/4140398410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559862f7cfa0 con 0x559862f6dab0 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36eed9d700 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f7c800 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f36e8003f40 tx=0x7f36e8008d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659c017fb3de5dd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36d5ffb700 1 -- 172.21.15.19:0/4140398410 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8015690 con 0x559862f6dab0 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36d5ffb700 1 -- 172.21.15.19:0/4140398410 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f36e8015830 con 0x559862f6dab0 2024-05-09T01:36:50.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f3703284b80 1 -- 172.21.15.19:0/4140398410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e00aed40 con 0x559862f6dab0 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f36d5ffb700 1 -- 172.21.15.19:0/4140398410 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36e8015a30 con 0x559862f6dab0 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f3703284b80 10 client.18570 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.173+0000 7f3703284b80 1 -- 172.21.15.19:0/4140398410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559862ff2760 con 0x559862f6dab0 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.174+0000 7f36d5ffb700 1 -- 172.21.15.19:0/4140398410 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e00aed40 con 0x559862f6dab0 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.174+0000 7f36d5ffb700 1 -- 172.21.15.19:0/4140398410 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f36e8004ba0 con 0x559862f6dab0 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.174+0000 7f3703284b80 10 client.18570 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.174+0000 7f3703284b80 10 client.18570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.174+0000 7f3703284b80 20 client.18570 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.174+0000 7f3703284b80 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559862ff2bb0 0x559862ff4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.174+0000 7f3703284b80 4 client.18570 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.174+0000 7f3703284b80 1 -- 172.21.15.19:0/4140398410 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559862e93a70 con 0x559862ff2bb0 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.174+0000 7f36ee59c700 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559862ff2bb0 0x559862ff4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f36ee59c700 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559862ff2bb0 0x559862ff4fd0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be40595bec682a server_cookie=b79890b02428b9a in_seq=0 out_seq=0 2024-05-09T01:36:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f36d5ffb700 10 client.18570 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f36d5ffb700 1 -- 172.21.15.19:0/4140398410 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559862e93a70 con 0x559862ff2bb0 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f36d5ffb700 10 client.18570 handle_command_reply: tid=0 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 client.18570 shutdown 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 -- 172.21.15.19:0/4140398410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 msgr2=0x559862f7c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f7c800 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f36e8003f40 tx=0x7f36e8008d70 comp rx=0 tx=0).stop 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 -- 172.21.15.19:0/4140398410 shutdown_connections 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x559862ff2bb0 0x559862ff4fd0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559862f6dab0 0x559862f7c800 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559862dec4a0 0x559862f7f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 --2- 172.21.15.19:0/4140398410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559862ed6160 0x559862f7fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 -- 172.21.15.19:0/4140398410 >> 172.21.15.19:0/4140398410 conn(0x559862e4c6a0 msgr2=0x559862f719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f36d5ffb700 0 client.18570 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 -- 172.21.15.19:0/4140398410 shutdown_connections 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 1 -- 172.21.15.19:0/4140398410 wait complete. 2024-05-09T01:36:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.175+0000 7f3703284b80 20 client.18570 trim_cache size 0 max 16384 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout: "cca71913-29de-4e88-8098-94990fc735c3": { 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "cca71913-29de-4e88-8098-94990fc735c3", 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:36:50.177 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:36:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 -- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36d80b2880 msgr2=0x7f36d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36d80b2880 0x7f36d80b4d90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f36e4006010 tx=0x7f36e4000bc0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 -- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f01034e0 msgr2=0x7f36f0199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f01034e0 0x7f36f0199d40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009970 tx=0x7f36e0009c80 comp rx=0 tx=0).stop 2024-05-09T01:36:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 -- 172.21.15.19:0/4249825812 shutdown_connections 2024-05-09T01:36:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f0108760 0x7f36f019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f36d80b2880 0x7f36d80b4d90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f01034e0 0x7f36f0199d40 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 --2- 172.21.15.19:0/4249825812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0107e10 0x7f36f019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 -- 172.21.15.19:0/4249825812 >> 172.21.15.19:0/4249825812 conn(0x7f36f00fef30 msgr2=0x7f36f01096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 -- 172.21.15.19:0/4249825812 shutdown_connections 2024-05-09T01:36:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:36:50.176+0000 7f36f486e700 1 -- 172.21.15.19:0/4249825812 wait complete. 2024-05-09T01:36:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:50 smithi059 ceph-mon[34351]: pgmap v717: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:36:50.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:50 smithi019 ceph-mon[30158]: pgmap v717: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:36:50.942 INFO:tasks.cephfs.filesystem:scrub status for tag:cca71913-29de-4e88-8098-94990fc735c3 - {'path': '/', 'tag': 'cca71913-29de-4e88-8098-94990fc735c3', 'options': 'recursive,force'} 2024-05-09T01:36:50.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:50 smithi196 ceph-mon[35041]: pgmap v717: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:36:52.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:52 smithi059 ceph-mon[34351]: pgmap v718: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:36:52.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:52 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:36:52.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:52 smithi019 ceph-mon[30158]: pgmap v718: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:36:52.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:52 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:36:52.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:52 smithi196 ceph-mon[35041]: pgmap v718: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:36:52.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:52 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:36:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.514+0000 7facad29f640 1 -- 172.21.15.19:0/440220428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8103c70 msgr2=0x7faca8104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.514+0000 7facad29f640 1 --2- 172.21.15.19:0/440220428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8103c70 0x7faca8104970 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7faca0009ca0 tx=0x7faca002a8c0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.514+0000 7facad29f640 1 -- 172.21.15.19:0/440220428 shutdown_connections 2024-05-09T01:36:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.514+0000 7facad29f640 1 --2- 172.21.15.19:0/440220428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca8109bb0 0x7faca810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.514+0000 7facad29f640 1 --2- 172.21.15.19:0/440220428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca8104eb0 0x7faca81052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.514+0000 7facad29f640 1 --2- 172.21.15.19:0/440220428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8103c70 0x7faca8104970 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.514+0000 7facad29f640 1 -- 172.21.15.19:0/440220428 >> 172.21.15.19:0/440220428 conn(0x7faca80fee40 msgr2=0x7faca8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.514+0000 7facad29f640 1 -- 172.21.15.19:0/440220428 shutdown_connections 2024-05-09T01:36:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.514+0000 7facad29f640 1 -- 172.21.15.19:0/440220428 wait complete. 2024-05-09T01:36:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.515+0000 7facad29f640 1 Processor -- start 2024-05-09T01:36:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.515+0000 7facad29f640 1 -- start start 2024-05-09T01:36:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.515+0000 7facad29f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca8103c70 0x7faca819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.515+0000 7facad29f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8104eb0 0x7faca819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.515+0000 7facad29f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca8109bb0 0x7faca81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.515+0000 7facad29f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faca810f750 con 0x7faca8104eb0 2024-05-09T01:36:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.515+0000 7facad29f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faca810f5d0 con 0x7faca8109bb0 2024-05-09T01:36:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.515+0000 7facad29f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faca810f8d0 con 0x7faca8103c70 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8104eb0 0x7faca819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8104eb0 0x7faca819a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51200/0 (socket says 172.21.15.19:51200) 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca67fc640 1 -- 172.21.15.19:0/148089326 learned_addr learned my addr 172.21.15.19:0/148089326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca77fe640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca8109bb0 0x7faca81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca6ffd640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca8103c70 0x7faca819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca67fc640 1 -- 172.21.15.19:0/148089326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca8103c70 msgr2=0x7faca819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca67fc640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca8103c70 0x7faca819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca67fc640 1 -- 172.21.15.19:0/148089326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca8109bb0 msgr2=0x7faca81a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca67fc640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca8109bb0 0x7faca81a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca67fc640 1 -- 172.21.15.19:0/148089326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca81a1df0 con 0x7faca8104eb0 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7faca67fc640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8104eb0 0x7faca819a8a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c920 tx=0x7fac9000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7fac9ffff640 1 -- 172.21.15.19:0/148089326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90007c70 con 0x7faca8104eb0 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7fac9ffff640 1 -- 172.21.15.19:0/148089326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac90007e10 con 0x7faca8104eb0 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7facad29f640 1 -- 172.21.15.19:0/148089326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca81a20e0 con 0x7faca8104eb0 2024-05-09T01:36:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7fac9ffff640 1 -- 172.21.15.19:0/148089326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90018820 con 0x7faca8104eb0 2024-05-09T01:36:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.516+0000 7facad29f640 1 -- 172.21.15.19:0/148089326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca81a2630 con 0x7faca8104eb0 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.519+0000 7facb422e740 1 Processor -- start 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.519+0000 7facb422e740 1 -- start start 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.519+0000 7fac9ffff640 1 -- 172.21.15.19:0/148089326 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fac9000f450 con 0x7faca8104eb0 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7facb422e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 0x562b0dff4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7facb422e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dff4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7facb422e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7facb422e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b0df587c0 con 0x562b0dc2f4d0 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7facb422e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562b0def40e0 con 0x562b0dc272f0 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7facb422e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b0df54780 con 0x562b0dc501b0 2024-05-09T01:36:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca6ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dff4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca6ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dff4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:37498/0 (socket says 172.21.15.19:37498) 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca6ffd640 1 -- 172.21.15.19:0/1272680798 learned_addr learned my addr 172.21.15.19:0/1272680798 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca67fc640 1 --2- 172.21.15.19:0/1272680798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51202/0 (socket says 172.21.15.19:51202) 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca77fe640 1 --2- 172.21.15.19:0/1272680798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 0x562b0dff4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca6ffd640 1 -- 172.21.15.19:0/1272680798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 msgr2=0x562b0dff4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca6ffd640 1 --2- 172.21.15.19:0/1272680798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 0x562b0dff4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca6ffd640 1 -- 172.21.15.19:0/1272680798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 msgr2=0x562b0dffb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca6ffd640 1 --2- 172.21.15.19:0/1272680798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.520+0000 7faca6ffd640 1 -- 172.21.15.19:0/1272680798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0df1d7c0 con 0x562b0dc272f0 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.521+0000 7fac9ffff640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac940b27e0 0x7fac940b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.521+0000 7faca6ffd640 1 --2- 172.21.15.19:0/1272680798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dff4a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faca8103540 tx=0x7faca0009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e3b455371c4996 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.521+0000 7faca77fe640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac940b27e0 0x7fac940b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.521+0000 7fac9ffff640 1 -- 172.21.15.19:0/148089326 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fac900e22a0 con 0x7faca8104eb0 2024-05-09T01:36:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.521+0000 7fac9d7fa640 1 -- 172.21.15.19:0/1272680798 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0031710 con 0x562b0dc272f0 2024-05-09T01:36:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.521+0000 7fac9d7fa640 1 -- 172.21.15.19:0/1272680798 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca0036030 con 0x562b0dc272f0 2024-05-09T01:36:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.521+0000 7fac9d7fa640 1 -- 172.21.15.19:0/1272680798 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0004830 con 0x562b0dc272f0 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 -- 172.21.15.19:0/1272680798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 msgr2=0x562b0dff4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 --2- 172.21.15.19:0/1272680798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dff4a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faca8103540 tx=0x7faca0009800 comp rx=0 tx=0).stop 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7faca77fe640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac940b27e0 0x7fac940b4cf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x562b0dff56d0 tx=0x7fac98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 -- 172.21.15.19:0/1272680798 shutdown_connections 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 --2- 172.21.15.19:0/1272680798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 --2- 172.21.15.19:0/1272680798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dff4a80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 --2- 172.21.15.19:0/1272680798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 0x562b0dff4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 -- 172.21.15.19:0/1272680798 >> 172.21.15.19:0/1272680798 conn(0x562b0dc44300 msgr2=0x562b0dfefe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 -- 172.21.15.19:0/1272680798 shutdown_connections 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 -- 172.21.15.19:0/1272680798 wait complete. 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.522+0000 7facb422e740 1 -- start start 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7facb422e740 1 Processor -- start 2024-05-09T01:36:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7facb422e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 0x562b0dffe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7facb422e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7facb422e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc501b0 0x562b0e005550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7facb422e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b0dffc1f0 con 0x562b0dc2f4d0 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7facb422e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562b0dedf9c0 con 0x562b0dc501b0 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7facb422e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b0dffc370 con 0x562b0dc272f0 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7facb422e740 10 client.0 _ll_register_callbacks cb 0x562b0df0c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffe5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51214/0 (socket says 172.21.15.19:51214) 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca67fc640 1 -- 172.21.15.19:0/338026891 learned_addr learned my addr 172.21.15.19:0/338026891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca77fe640 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc501b0 0x562b0e005550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca6ffd640 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 0x562b0dffe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca67fc640 1 -- 172.21.15.19:0/338026891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 msgr2=0x562b0dffe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca67fc640 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 0x562b0dffe0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca67fc640 1 -- 172.21.15.19:0/338026891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc501b0 msgr2=0x562b0e005550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca67fc640 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc501b0 0x562b0e005550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.523+0000 7faca67fc640 1 -- 172.21.15.19:0/338026891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0df061f0 con 0x562b0dc2f4d0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7faca67fc640 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffe5f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c8f0 tx=0x7fac9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c59bf2afec69ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7fac8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7fac8e7fc640 1 -- 172.21.15.19:0/338026891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900a9670 con 0x562b0dc2f4d0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7fac8e7fc640 1 -- 172.21.15.19:0/338026891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fac900a5b30 con 0x562b0dc2f4d0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7facb422e740 1 -- 172.21.15.19:0/338026891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faca0036030 con 0x562b0dc2f4d0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7fac8e7fc640 1 -- 172.21.15.19:0/338026891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac900a9010 con 0x562b0dc2f4d0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7facb422e740 10 client.18582 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7facb422e740 1 -- 172.21.15.19:0/338026891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b0dcbb790 con 0x562b0dc2f4d0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7fac8e7fc640 1 -- 172.21.15.19:0/338026891 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca0036030 con 0x562b0dc2f4d0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7fac8e7fc640 1 -- 172.21.15.19:0/338026891 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac900aabd0 con 0x562b0dc2f4d0 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7facb422e740 10 client.18582 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7facb422e740 10 client.18582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7facb422e740 20 client.18582 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7facb422e740 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562b0dc0ed80 0x562b0e00bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7facb422e740 4 client.18582 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.524+0000 7facb422e740 1 -- 172.21.15.19:0/338026891 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b0df663d0 con 0x562b0dc0ed80 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.525+0000 7faca6ffd640 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562b0dc0ed80 0x562b0e00bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.525+0000 7fac8e7fc640 10 client.18582 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.525+0000 7faca6ffd640 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562b0dc0ed80 0x562b0e00bae0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d554d6af146506e6 server_cookie=b1fd02c36b8f9ac6 in_seq=0 out_seq=0 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.526+0000 7fac8e7fc640 1 -- 172.21.15.19:0/338026891 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b0df663d0 con 0x562b0dc0ed80 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.526+0000 7fac8e7fc640 10 client.18582 handle_command_reply: tid=0 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.526+0000 7facb422e740 1 client.18582 shutdown 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.526+0000 7facb422e740 1 -- 172.21.15.19:0/338026891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 msgr2=0x562b0dffe5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffe5f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c8f0 tx=0x7fac9000cf90 comp rx=0 tx=0).stop 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 1 -- 172.21.15.19:0/338026891 shutdown_connections 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562b0dc0ed80 0x562b0e00bae0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc501b0 0x562b0e005550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0dffe5f0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 1 --2- 172.21.15.19:0/338026891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 0x562b0dffe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7fac8e7fc640 0 client.18582 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 1 -- 172.21.15.19:0/338026891 >> 172.21.15.19:0/338026891 conn(0x562b0dc44300 msgr2=0x562b0dfee700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 1 -- 172.21.15.19:0/338026891 shutdown_connections 2024-05-09T01:36:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 1 -- 172.21.15.19:0/338026891 wait complete. 2024-05-09T01:36:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.527+0000 7facb422e740 20 client.18582 trim_cache size 0 max 16384 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.542+0000 7facb422e740 1 Processor -- start 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.542+0000 7facb422e740 1 -- start start 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.542+0000 7facb422e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dffe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.542+0000 7facb422e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0e0102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.542+0000 7facb422e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 0x562b0e010810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.542+0000 7facb422e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faca81ab340 con 0x562b0dc2f4d0 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.542+0000 7facb422e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faca81ace80 con 0x562b0dc272f0 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.542+0000 7facb422e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faca81aba10 con 0x562b0dc501b0 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0e0102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0e0102d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51226/0 (socket says 172.21.15.19:51226) 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca67fc640 1 -- 172.21.15.19:0/705635550 learned_addr learned my addr 172.21.15.19:0/705635550 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca77fe640 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 0x562b0e010810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca6ffd640 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dffe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca67fc640 1 -- 172.21.15.19:0/705635550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 msgr2=0x562b0e010810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca67fc640 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 0x562b0e010810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca67fc640 1 -- 172.21.15.19:0/705635550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 msgr2=0x562b0dffe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca67fc640 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dffe390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca67fc640 1 -- 172.21.15.19:0/705635550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0dd325f0 con 0x562b0dc2f4d0 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca77fe640 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 0x562b0e010810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca6ffd640 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dffe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7faca67fc640 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0e0102d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fac900a5920 tx=0x7fac90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bd1b390f9b61b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7fac9cff9640 1 -- 172.21.15.19:0/705635550 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900a91f0 con 0x562b0dc2f4d0 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7fac9cff9640 1 -- 172.21.15.19:0/705635550 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac900a93b0 con 0x562b0dc2f4d0 2024-05-09T01:36:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.543+0000 7fac9cff9640 1 -- 172.21.15.19:0/705635550 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900a98a0 con 0x562b0dc2f4d0 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 -- 172.21.15.19:0/705635550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 msgr2=0x562b0e0102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0e0102d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fac900a5920 tx=0x7fac90000bc0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 -- 172.21.15.19:0/705635550 shutdown_connections 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc501b0 0x562b0e010810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc2f4d0 0x562b0e0102d0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 --2- 172.21.15.19:0/705635550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc272f0 0x562b0dffe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 -- 172.21.15.19:0/705635550 >> 172.21.15.19:0/705635550 conn(0x562b0dc44300 msgr2=0x562b0dffdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 -- 172.21.15.19:0/705635550 shutdown_connections 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 -- 172.21.15.19:0/705635550 wait complete. 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.544+0000 7facb422e740 1 -- start start 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7facb422e740 1 Processor -- start 2024-05-09T01:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7facb422e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 0x562b0e001ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7facb422e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc2f4d0 0x562b0e0021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7facb422e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc501b0 0x562b0e003110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7facb422e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faca81ac7b0 con 0x562b0dc501b0 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7facb422e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faca81ac0e0 con 0x562b0dc2f4d0 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7facb422e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faca81a9320 con 0x562b0dc272f0 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7facb422e740 10 client.0 _ll_register_callbacks cb 0x7fac9000fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc501b0 0x562b0e003110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc501b0 0x562b0e003110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51228/0 (socket says 172.21.15.19:51228) 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca77fe640 1 -- 172.21.15.19:0/4002793438 learned_addr learned my addr 172.21.15.19:0/4002793438 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca67fc640 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc2f4d0 0x562b0e0021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca6ffd640 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 0x562b0e001ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca77fe640 1 -- 172.21.15.19:0/4002793438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 msgr2=0x562b0e001ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca77fe640 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 0x562b0e001ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca77fe640 1 -- 172.21.15.19:0/4002793438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc2f4d0 msgr2=0x562b0e0021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:36:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca77fe640 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc2f4d0 0x562b0e0021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca77fe640 1 -- 172.21.15.19:0/4002793438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0df1ced0 con 0x562b0dc501b0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca67fc640 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc2f4d0 0x562b0e0021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7faca77fe640 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc501b0 0x562b0e003110 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fac9801da00 tx=0x7fac98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1192361865f28042 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.545+0000 7fac8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7fac8e7fc640 1 -- 172.21.15.19:0/4002793438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9802d030 con 0x562b0dc501b0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7fac8e7fc640 1 -- 172.21.15.19:0/4002793438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fac98018030 con 0x562b0dc501b0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7facb422e740 1 -- 172.21.15.19:0/4002793438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac900a93b0 con 0x562b0dc501b0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7fac8e7fc640 1 -- 172.21.15.19:0/4002793438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac98005020 con 0x562b0dc501b0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7facb422e740 10 client.18588 fetch_fsmap learned FSMap version 28 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7facb422e740 1 -- 172.21.15.19:0/4002793438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b0e004860 con 0x562b0dc501b0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7fac8e7fc640 1 -- 172.21.15.19:0/4002793438 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac900a93b0 con 0x562b0dc501b0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7fac8e7fc640 1 -- 172.21.15.19:0/4002793438 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac98029700 con 0x562b0dc501b0 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7facb422e740 10 client.18588 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7facb422e740 10 client.18588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:36:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7facb422e740 20 client.18588 populate_metadata read hostname 'smithi019' 2024-05-09T01:36:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7facb422e740 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562b0dbe41c0 0x562b0e0760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:36:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7facb422e740 4 client.18588 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:36:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.546+0000 7facb422e740 1 -- 172.21.15.19:0/4002793438 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b0dff81f0 con 0x562b0dbe41c0 2024-05-09T01:36:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.547+0000 7faca6ffd640 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562b0dbe41c0 0x562b0e0760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:36:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.547+0000 7faca6ffd640 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562b0dbe41c0 0x562b0e0760b0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b108aef59e0cf63f server_cookie=306f0f4454eb2c48 in_seq=0 out_seq=0 2024-05-09T01:36:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.547+0000 7fac8e7fc640 10 client.18588 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7fac8e7fc640 1 -- 172.21.15.19:0/4002793438 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562b0dff81f0 con 0x562b0dbe41c0 2024-05-09T01:36:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7fac8e7fc640 10 client.18588 handle_command_reply: tid=0 2024-05-09T01:36:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 client.18588 shutdown 2024-05-09T01:36:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 -- 172.21.15.19:0/4002793438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc501b0 msgr2=0x562b0e003110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc501b0 0x562b0e003110 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fac9801da00 tx=0x7fac98004730 comp rx=0 tx=0).stop 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 -- 172.21.15.19:0/4002793438 shutdown_connections 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x562b0dbe41c0 0x562b0e0760b0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b0dc501b0 0x562b0e003110 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0dc2f4d0 0x562b0e0021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 --2- 172.21.15.19:0/4002793438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0dc272f0 0x562b0e001ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7fac8e7fc640 0 client.18588 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 -- 172.21.15.19:0/4002793438 >> 172.21.15.19:0/4002793438 conn(0x562b0dc44300 msgr2=0x562b0dffe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 -- 172.21.15.19:0/4002793438 shutdown_connections 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 1 -- 172.21.15.19:0/4002793438 wait complete. 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.563+0000 7facb422e740 20 client.18588 trim_cache size 0 max 16384 2024-05-09T01:36:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:36:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:36:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:36:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 -- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac940b27e0 msgr2=0x7fac940b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac940b27e0 0x7fac940b4cf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x562b0dff56d0 tx=0x7fac98005f90 comp rx=0 tx=0).stop 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 -- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8104eb0 msgr2=0x7faca819a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8104eb0 0x7faca819a8a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c920 tx=0x7fac9000cde0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 -- 172.21.15.19:0/148089326 shutdown_connections 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fac940b27e0 0x7fac940b4cf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca8109bb0 0x7faca81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8104eb0 0x7faca819a8a0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 --2- 172.21.15.19:0/148089326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca8103c70 0x7faca819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 -- 172.21.15.19:0/148089326 >> 172.21.15.19:0/148089326 conn(0x7faca80fee40 msgr2=0x7faca8105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 -- 172.21.15.19:0/148089326 shutdown_connections 2024-05-09T01:36:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:36:53.564+0000 7facad29f640 1 -- 172.21.15.19:0/148089326 wait complete. 2024-05-09T01:36:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:54 smithi059 ceph-mon[34351]: pgmap v719: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:54.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:54 smithi019 ceph-mon[30158]: pgmap v719: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:54.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:54 smithi196 ceph-mon[35041]: pgmap v719: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:36:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:56 smithi059 ceph-mon[34351]: pgmap v720: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:36:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:56 smithi019 ceph-mon[30158]: pgmap v720: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:36:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:56 smithi196 ceph-mon[35041]: pgmap v720: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:36:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:36:58 smithi059 ceph-mon[34351]: pgmap v721: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:36:58.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:36:58 smithi019 ceph-mon[30158]: pgmap v721: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:36:58.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:36:58 smithi196 ceph-mon[35041]: pgmap v721: 97 pgs: 97 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:37:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:00 smithi059 ceph-mon[34351]: pgmap v722: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:37:00.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:00 smithi019 ceph-mon[30158]: pgmap v722: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:37:00.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:00 smithi196 ceph-mon[35041]: pgmap v722: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-05-09T01:37:02.746 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:02 smithi196 ceph-mon[35041]: pgmap v723: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:37:02.746 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:02 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:37:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:02 smithi059 ceph-mon[34351]: pgmap v723: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:37:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:02 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:37:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:02 smithi019 ceph-mon[30158]: pgmap v723: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:37:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:02 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:37:04.090 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:37:04.090 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:37:04.090 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:03 smithi019 ceph-mon[30158]: pgmap v724: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:37:04.090 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:37:04.090 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:37:04.090 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:03 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:37:04.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:37:04.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:37:04.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:03 smithi059 ceph-mon[34351]: pgmap v724: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:37:04.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:37:04.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:37:04.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:03 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:37:04.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:37:04.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:37:04.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:03 smithi196 ceph-mon[35041]: pgmap v724: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:37:04.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:37:04.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:37:04.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:03 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:499/600: open fd = 3 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:500/600: open fd = 3 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:501/600: open fd = 3 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.560 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:502/600: open fd = 3 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:503/600: open fd = 3 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:504/600: open fd = 3 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:505/600: open fd = 3 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.561 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:506/600: open fd = 3 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:507/600: open fd = 3 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:508/600: open fd = 3 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.562 INFO:tasks.workunit.client.0.smithi019.stdout:509/600: open fd = 3 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:510/600: open fd = 3 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:511/600: open fd = 3 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:512/600: open fd = 3 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.563 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:513/600: open fd = 3 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:514/600: open fd = 3 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:515/600: open fd = 3 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:516/600: open fd = 3 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.564 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:517/600: open fd = 3 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:518/600: open fd = 3 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:519/600: open fd = 3 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:520/600: open fd = 3 2024-05-09T01:37:04.565 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:521/600: open fd = 3 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:522/600: open fd = 3 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:523/600: open fd = 3 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.566 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:524/600: open fd = 3 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:525/600: open fd = 3 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:526/600: open fd = 3 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.567 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.568 INFO:tasks.workunit.client.0.smithi019.stdout:527/600: open fd = 3 2024-05-09T01:37:04.568 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.568 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.568 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.568 INFO:tasks.workunit.client.0.smithi019.stdout:528/600: open fd = 3 2024-05-09T01:37:04.568 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.568 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.568 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.569 INFO:tasks.workunit.client.0.smithi019.stdout:529/600: open fd = 3 2024-05-09T01:37:04.569 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.569 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.569 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.569 INFO:tasks.workunit.client.0.smithi019.stdout:530/600: open fd = 3 2024-05-09T01:37:04.569 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.569 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.569 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.570 INFO:tasks.workunit.client.0.smithi019.stdout:531/600: open fd = 3 2024-05-09T01:37:04.570 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.570 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.570 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.570 INFO:tasks.workunit.client.0.smithi019.stdout:532/600: open fd = 3 2024-05-09T01:37:04.570 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.571 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.571 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.571 INFO:tasks.workunit.client.0.smithi019.stdout:533/600: open fd = 3 2024-05-09T01:37:04.571 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.571 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.571 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.571 INFO:tasks.workunit.client.0.smithi019.stdout:534/600: open fd = 3 2024-05-09T01:37:04.572 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.572 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.572 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.572 INFO:tasks.workunit.client.0.smithi019.stdout:535/600: open fd = 3 2024-05-09T01:37:04.572 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.572 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.572 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.572 INFO:tasks.workunit.client.0.smithi019.stdout:536/600: open fd = 3 2024-05-09T01:37:04.573 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.573 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.573 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.573 INFO:tasks.workunit.client.0.smithi019.stdout:537/600: open fd = 3 2024-05-09T01:37:04.573 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.573 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.573 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.573 INFO:tasks.workunit.client.0.smithi019.stdout:538/600: open fd = 3 2024-05-09T01:37:04.574 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.574 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.574 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.574 INFO:tasks.workunit.client.0.smithi019.stdout:539/600: open fd = 3 2024-05-09T01:37:04.574 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.574 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.574 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.575 INFO:tasks.workunit.client.0.smithi019.stdout:540/600: open fd = 3 2024-05-09T01:37:04.575 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.575 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.575 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.575 INFO:tasks.workunit.client.0.smithi019.stdout:541/600: open fd = 3 2024-05-09T01:37:04.575 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.575 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.576 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.576 INFO:tasks.workunit.client.0.smithi019.stdout:542/600: open fd = 3 2024-05-09T01:37:04.576 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.576 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.576 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.576 INFO:tasks.workunit.client.0.smithi019.stdout:543/600: open fd = 3 2024-05-09T01:37:04.576 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.577 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.577 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.577 INFO:tasks.workunit.client.0.smithi019.stdout:544/600: open fd = 3 2024-05-09T01:37:04.577 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.577 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.577 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.577 INFO:tasks.workunit.client.0.smithi019.stdout:545/600: open fd = 3 2024-05-09T01:37:04.577 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.577 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:546/600: open fd = 3 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:547/600: open fd = 3 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:548/600: open fd = 3 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.578 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:549/600: open fd = 3 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:550/600: open fd = 3 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:551/600: open fd = 3 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:552/600: open fd = 3 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.579 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:553/600: open fd = 3 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:554/600: open fd = 3 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:555/600: open fd = 3 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:556/600: open fd = 3 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.580 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:557/600: open fd = 3 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:558/600: open fd = 3 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:559/600: open fd = 3 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.581 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.582 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:04.582 INFO:tasks.workunit.client.0.smithi019.stdout:560/600: open fd = 3 2024-05-09T01:37:04.582 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.582 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:04.582 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.677+0000 7fbaad9ba640 1 -- 172.21.15.19:0/3056830042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076170 msgr2=0x7fbaa8076580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.677+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/3056830042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076170 0x7fbaa8076580 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009ca0 tx=0x7fbaa002a8c0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.677+0000 7fbaad9ba640 1 -- 172.21.15.19:0/3056830042 shutdown_connections 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.677+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/3056830042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa810bc40 0x7fbaa810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.677+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/3056830042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa8076ac0 0x7fbaa810b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.677+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/3056830042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076170 0x7fbaa8076580 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.677+0000 7fbaad9ba640 1 -- 172.21.15.19:0/3056830042 >> 172.21.15.19:0/3056830042 conn(0x7fbaa80fee60 msgr2=0x7fbaa8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.677+0000 7fbaad9ba640 1 -- 172.21.15.19:0/3056830042 shutdown_connections 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.677+0000 7fbaad9ba640 1 -- 172.21.15.19:0/3056830042 wait complete. 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaad9ba640 1 Processor -- start 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaad9ba640 1 -- start start 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaad9ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa8076170 0x7fbaa819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaad9ba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076ac0 0x7fbaa819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaad9ba640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa810bc40 0x7fbaa81a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaad9ba640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa8111960 con 0x7fbaa8076ac0 2024-05-09T01:37:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaad9ba640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81117e0 con 0x7fbaa810bc40 2024-05-09T01:37:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaad9ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa8111ae0 con 0x7fbaa8076170 2024-05-09T01:37:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaa67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076ac0 0x7fbaa819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaa67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076ac0 0x7fbaa819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60336/0 (socket says 172.21.15.19:60336) 2024-05-09T01:37:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.678+0000 7fbaa67fc640 1 -- 172.21.15.19:0/2582118116 learned_addr learned my addr 172.21.15.19:0/2582118116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa8076170 0x7fbaa819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa810bc40 0x7fbaa81a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaa67fc640 1 -- 172.21.15.19:0/2582118116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa8076170 msgr2=0x7fbaa819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa8076170 0x7fbaa819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaa67fc640 1 -- 172.21.15.19:0/2582118116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa810bc40 msgr2=0x7fbaa81a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa810bc40 0x7fbaa81a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaa67fc640 1 -- 172.21.15.19:0/2582118116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa81a40b0 con 0x7fbaa8076ac0 2024-05-09T01:37:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa8076170 0x7fbaa819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076ac0 0x7fbaa819cab0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fba9000d770 tx=0x7fba9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaac9b8640 1 -- 172.21.15.19:0/2582118116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90017070 con 0x7fbaa8076ac0 2024-05-09T01:37:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaac9b8640 1 -- 172.21.15.19:0/2582118116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba900054d0 con 0x7fbaa8076ac0 2024-05-09T01:37:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaac9b8640 1 -- 172.21.15.19:0/2582118116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90004c30 con 0x7fbaa8076ac0 2024-05-09T01:37:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaad9ba640 1 -- 172.21.15.19:0/2582118116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa81a43a0 con 0x7fbaa8076ac0 2024-05-09T01:37:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.679+0000 7fbaad9ba640 1 -- 172.21.15.19:0/2582118116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa81a48f0 con 0x7fbaa8076ac0 2024-05-09T01:37:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbab486a740 1 Processor -- start 2024-05-09T01:37:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbab486a740 1 -- start start 2024-05-09T01:37:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbab486a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbab486a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f57a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbab486a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f5e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbab486a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df20ebb7c0 con 0x55df20bb31b0 2024-05-09T01:37:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbab486a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df20e570e0 con 0x55df20b8a2f0 2024-05-09T01:37:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbab486a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df20eb7780 con 0x55df20b924d0 2024-05-09T01:37:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbaac9b8640 1 -- 172.21.15.19:0/2582118116 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fba90010460 con 0x7fbaa8076ac0 2024-05-09T01:37:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbaa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbaa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f57500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60338/0 (socket says 172.21.15.19:60338) 2024-05-09T01:37:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.682+0000 7fbaa77fe640 1 -- 172.21.15.19:0/4200743916 learned_addr learned my addr 172.21.15.19:0/4200743916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f57a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f5e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaa77fe640 1 -- 172.21.15.19:0/4200743916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 msgr2=0x55df20f5e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f5e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaa77fe640 1 -- 172.21.15.19:0/4200743916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 msgr2=0x55df20f57a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f57a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaa77fe640 1 -- 172.21.15.19:0/4200743916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df20e807c0 con 0x55df20bb31b0 2024-05-09T01:37:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f57a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:37:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f57500 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fba9800b6b0 tx=0x7fba9800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6347ec3500ab1467 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fba9dffb640 1 -- 172.21.15.19:0/4200743916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98003bd0 con 0x55df20bb31b0 2024-05-09T01:37:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fba9dffb640 1 -- 172.21.15.19:0/4200743916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba98015070 con 0x55df20bb31b0 2024-05-09T01:37:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fba9dffb640 1 -- 172.21.15.19:0/4200743916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba980116a0 con 0x55df20bb31b0 2024-05-09T01:37:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.683+0000 7fbaac9b8640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba940b2860 0x7fba940b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba940b2860 0x7fba940b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbaac9b8640 1 -- 172.21.15.19:0/2582118116 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fba900e1cb0 con 0x7fbaa8076ac0 2024-05-09T01:37:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbab486a740 1 -- 172.21.15.19:0/4200743916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 msgr2=0x55df20f57500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbab486a740 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f57500 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fba9800b6b0 tx=0x7fba9800bb70 comp rx=0 tx=0).stop 2024-05-09T01:37:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba940b2860 0x7fba940b4d70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0031a90 tx=0x7fbaa0031a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbab486a740 1 -- 172.21.15.19:0/4200743916 shutdown_connections 2024-05-09T01:37:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbab486a740 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f5e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbab486a740 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f57a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbab486a740 1 --2- 172.21.15.19:0/4200743916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f57500 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbab486a740 1 -- 172.21.15.19:0/4200743916 >> 172.21.15.19:0/4200743916 conn(0x55df20ba7300 msgr2=0x55df20f52e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbab486a740 1 -- 172.21.15.19:0/4200743916 shutdown_connections 2024-05-09T01:37:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.684+0000 7fbab486a740 1 -- 172.21.15.19:0/4200743916 wait complete. 2024-05-09T01:37:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.685+0000 7fbab486a740 1 -- start start 2024-05-09T01:37:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.685+0000 7fbab486a740 1 Processor -- start 2024-05-09T01:37:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.685+0000 7fbab486a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.685+0000 7fbab486a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f61800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.685+0000 7fbab486a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.685+0000 7fbab486a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df20f5f3d0 con 0x55df20bb31b0 2024-05-09T01:37:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.685+0000 7fbab486a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df20e429c0 con 0x55df20b8a2f0 2024-05-09T01:37:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.685+0000 7fbab486a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df20f5f550 con 0x55df20b924d0 2024-05-09T01:37:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbab486a740 10 client.0 _ll_register_callbacks cb 0x55df20e6f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f68760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60354/0 (socket says 172.21.15.19:60354) 2024-05-09T01:37:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa77fe640 1 -- 172.21.15.19:0/1758291622 learned_addr learned my addr 172.21.15.19:0/1758291622 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f61800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa77fe640 1 -- 172.21.15.19:0/1758291622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 msgr2=0x55df20f61800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f61800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa77fe640 1 -- 172.21.15.19:0/1758291622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 msgr2=0x55df20f612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa77fe640 1 -- 172.21.15.19:0/1758291622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df20e691f0 con 0x55df20bb31b0 2024-05-09T01:37:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f61800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f68760 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fba98006010 tx=0x7fba98004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb5cbafa7c9bb18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fba86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fba86ffd640 1 -- 172.21.15.19:0/1758291622 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9801a470 con 0x55df20bb31b0 2024-05-09T01:37:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fba86ffd640 1 -- 172.21.15.19:0/1758291622 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fba9801a610 con 0x55df20bb31b0 2024-05-09T01:37:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbab486a740 1 -- 172.21.15.19:0/1758291622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba98015070 con 0x55df20bb31b0 2024-05-09T01:37:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fba86ffd640 1 -- 172.21.15.19:0/1758291622 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba9801a810 con 0x55df20bb31b0 2024-05-09T01:37:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbab486a740 10 client.18600 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.686+0000 7fbab486a740 1 -- 172.21.15.19:0/1758291622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df20c1e790 con 0x55df20bb31b0 2024-05-09T01:37:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.687+0000 7fba86ffd640 1 -- 172.21.15.19:0/1758291622 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba98015070 con 0x55df20bb31b0 2024-05-09T01:37:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.687+0000 7fba86ffd640 1 -- 172.21.15.19:0/1758291622 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fba98004c30 con 0x55df20bb31b0 2024-05-09T01:37:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.687+0000 7fbab486a740 10 client.18600 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.687+0000 7fbab486a740 10 client.18600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.687+0000 7fbab486a740 20 client.18600 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.687+0000 7fbab486a740 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df20b71d80 0x55df20f6e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.687+0000 7fbab486a740 4 client.18600 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.687+0000 7fbab486a740 1 -- 172.21.15.19:0/1758291622 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df20ec93d0 con 0x55df20b71d80 2024-05-09T01:37:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.687+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df20b71d80 0x55df20f6e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.688+0000 7fba86ffd640 10 client.18600 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.688+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df20b71d80 0x55df20f6e950 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88048a4ca41776ad server_cookie=8badef8861dc71c7 in_seq=0 out_seq=0 2024-05-09T01:37:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.688+0000 7fba86ffd640 1 -- 172.21.15.19:0/1758291622 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df20ec93d0 con 0x55df20b71d80 2024-05-09T01:37:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.688+0000 7fba86ffd640 10 client.18600 handle_command_reply: tid=0 2024-05-09T01:37:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 client.18600 shutdown 2024-05-09T01:37:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 -- 172.21.15.19:0/1758291622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 msgr2=0x55df20f68760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f68760 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fba98006010 tx=0x7fba98004340 comp rx=0 tx=0).stop 2024-05-09T01:37:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 -- 172.21.15.19:0/1758291622 shutdown_connections 2024-05-09T01:37:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df20b71d80 0x55df20f6e950 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20bb31b0 0x55df20f68760 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f61800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 --2- 172.21.15.19:0/1758291622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b8a2f0 0x55df20f612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fba86ffd640 0 client.18600 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 -- 172.21.15.19:0/1758291622 >> 172.21.15.19:0/1758291622 conn(0x55df20ba7300 msgr2=0x55df20f55da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 -- 172.21.15.19:0/1758291622 shutdown_connections 2024-05-09T01:37:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 1 -- 172.21.15.19:0/1758291622 wait complete. 2024-05-09T01:37:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.689+0000 7fbab486a740 20 client.18600 trim_cache size 0 max 16384 2024-05-09T01:37:06.708 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:06 smithi196 ceph-mon[35041]: pgmap v725: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.704+0000 7fbab486a740 1 Processor -- start 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.704+0000 7fbab486a740 1 -- start start 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.704+0000 7fbab486a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 0x55df20f5ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbab486a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b8a2f0 0x55df20f619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbab486a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f68680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbab486a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81ad700 con 0x55df20b8a2f0 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbab486a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81af240 con 0x55df20b71d80 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbab486a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81addd0 con 0x55df20b924d0 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b8a2f0 0x55df20f619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b8a2f0 0x55df20f619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60370/0 (socket says 172.21.15.19:60370) 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa67fc640 1 -- 172.21.15.19:0/2764502105 learned_addr learned my addr 172.21.15.19:0/2764502105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f68680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 0x55df20f5ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa67fc640 1 -- 172.21.15.19:0/2764502105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 msgr2=0x55df20f68680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f68680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa67fc640 1 -- 172.21.15.19:0/2764502105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 msgr2=0x55df20f5ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 0x55df20f5ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa67fc640 1 -- 172.21.15.19:0/2764502105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df20c955f0 con 0x55df20b8a2f0 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f68680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b8a2f0 0x55df20f619a0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fba90009610 tx=0x7fba900a8dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd914440c216625 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fba9d7fa640 1 -- 172.21.15.19:0/2764502105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90020040 con 0x55df20b8a2f0 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fba9d7fa640 1 -- 172.21.15.19:0/2764502105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba900a92b0 con 0x55df20b8a2f0 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.705+0000 7fba9d7fa640 1 -- 172.21.15.19:0/2764502105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba900ad120 con 0x55df20b8a2f0 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 -- 172.21.15.19:0/2764502105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b8a2f0 msgr2=0x55df20f619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b8a2f0 0x55df20f619a0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fba90009610 tx=0x7fba900a8dc0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 -- 172.21.15.19:0/2764502105 shutdown_connections 2024-05-09T01:37:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b924d0 0x55df20f68680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b8a2f0 0x55df20f619a0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 --2- 172.21.15.19:0/2764502105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 0x55df20f5ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 -- 172.21.15.19:0/2764502105 >> 172.21.15.19:0/2764502105 conn(0x55df20ba7300 msgr2=0x55df20f61290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 -- 172.21.15.19:0/2764502105 shutdown_connections 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 -- 172.21.15.19:0/2764502105 wait complete. 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.706+0000 7fbab486a740 1 -- start start 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbab486a740 1 Processor -- start 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbab486a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 0x55df20f65f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbab486a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b8a2f0 0x55df20f66460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbab486a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b924d0 0x55df20fd4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbab486a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81aeb70 con 0x55df20b924d0 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbab486a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81ae4a0 con 0x55df20b71d80 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbab486a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81ab6e0 con 0x55df20b8a2f0 2024-05-09T01:37:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbab486a740 10 client.0 _ll_register_callbacks cb 0x7fbaa0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbaa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b924d0 0x55df20fd4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbaa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b924d0 0x55df20fd4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60382/0 (socket says 172.21.15.19:60382) 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbaa77fe640 1 -- 172.21.15.19:0/2627963029 learned_addr learned my addr 172.21.15.19:0/2627963029 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbaa67fc640 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b8a2f0 0x55df20f66460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 0x55df20f65f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbaa77fe640 1 -- 172.21.15.19:0/2627963029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b8a2f0 msgr2=0x55df20f66460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.707+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b8a2f0 0x55df20f66460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbaa77fe640 1 -- 172.21.15.19:0/2627963029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 msgr2=0x55df20f65f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 0x55df20f65f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbaa77fe640 1 -- 172.21.15.19:0/2627963029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df20e7fed0 con 0x55df20b924d0 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbaa77fe640 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b924d0 0x55df20fd4ca0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fba98011930 tx=0x7fba98011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f613122ce3d7ef5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fba86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fba86ffd640 1 -- 172.21.15.19:0/2627963029 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98035060 con 0x55df20b924d0 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fba86ffd640 1 -- 172.21.15.19:0/2627963029 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fba98011a50 con 0x55df20b924d0 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbab486a740 1 -- 172.21.15.19:0/2627963029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba900a92b0 con 0x55df20b924d0 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fba86ffd640 1 -- 172.21.15.19:0/2627963029 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba98003af0 con 0x55df20b924d0 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbab486a740 10 client.18606 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbab486a740 1 -- 172.21.15.19:0/2627963029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df20fd6250 con 0x55df20b924d0 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fba86ffd640 1 -- 172.21.15.19:0/2627963029 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba900a92b0 con 0x55df20b924d0 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fba86ffd640 1 -- 172.21.15.19:0/2627963029 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fba98004fa0 con 0x55df20b924d0 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbab486a740 10 client.18606 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbab486a740 10 client.18606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.708+0000 7fbab486a740 20 client.18606 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.709+0000 7fbab486a740 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df20bb31b0 0x55df20fd8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.709+0000 7fbab486a740 4 client.18606 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.709+0000 7fbab486a740 1 -- 172.21.15.19:0/2627963029 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df20ec3240 con 0x55df20bb31b0 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.709+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df20bb31b0 0x55df20fd8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.709+0000 7fba86ffd640 10 client.18606 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.709+0000 7fbaa6ffd640 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df20bb31b0 0x55df20fd8be0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ebd93b665637770 server_cookie=d8c7612a305e44a3 in_seq=0 out_seq=0 2024-05-09T01:37:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fba86ffd640 1 -- 172.21.15.19:0/2627963029 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55df20ec3240 con 0x55df20bb31b0 2024-05-09T01:37:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fba86ffd640 10 client.18606 handle_command_reply: tid=0 2024-05-09T01:37:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 client.18606 shutdown 2024-05-09T01:37:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 -- 172.21.15.19:0/2627963029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b924d0 msgr2=0x55df20fd4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b924d0 0x55df20fd4ca0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fba98011930 tx=0x7fba98011630 comp rx=0 tx=0).stop 2024-05-09T01:37:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 -- 172.21.15.19:0/2627963029 shutdown_connections 2024-05-09T01:37:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55df20bb31b0 0x55df20fd8be0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df20b924d0 0x55df20fd4ca0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df20b8a2f0 0x55df20f66460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fba86ffd640 0 client.18606 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 --2- 172.21.15.19:0/2627963029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df20b71d80 0x55df20f65f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 -- 172.21.15.19:0/2627963029 >> 172.21.15.19:0/2627963029 conn(0x55df20ba7300 msgr2=0x55df20f5a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 -- 172.21.15.19:0/2627963029 shutdown_connections 2024-05-09T01:37:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 1 -- 172.21.15.19:0/2627963029 wait complete. 2024-05-09T01:37:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.727+0000 7fbab486a740 20 client.18606 trim_cache size 0 max 16384 2024-05-09T01:37:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:37:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:37:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:37:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 -- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba940b2860 msgr2=0x7fba940b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba940b2860 0x7fba940b4d70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0031a90 tx=0x7fbaa0031a20 comp rx=0 tx=0).stop 2024-05-09T01:37:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 -- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076ac0 msgr2=0x7fbaa819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076ac0 0x7fbaa819cab0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fba9000d770 tx=0x7fba9000dc30 comp rx=0 tx=0).stop 2024-05-09T01:37:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 -- 172.21.15.19:0/2582118116 shutdown_connections 2024-05-09T01:37:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fba940b2860 0x7fba940b4d70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa810bc40 0x7fbaa81a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa8076ac0 0x7fbaa819cab0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 --2- 172.21.15.19:0/2582118116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa8076170 0x7fbaa819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 -- 172.21.15.19:0/2582118116 >> 172.21.15.19:0/2582118116 conn(0x7fbaa80fee60 msgr2=0x7fbaa81076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 -- 172.21.15.19:0/2582118116 shutdown_connections 2024-05-09T01:37:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:06.728+0000 7fbaad9ba640 1 -- 172.21.15.19:0/2582118116 wait complete. 2024-05-09T01:37:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:06 smithi059 ceph-mon[34351]: pgmap v725: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:37:06.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:06 smithi019 ceph-mon[30158]: pgmap v725: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:37:08.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:08 smithi196 ceph-mon[35041]: pgmap v726: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:37:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:08 smithi059 ceph-mon[34351]: pgmap v726: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:37:08.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:08 smithi019 ceph-mon[30158]: pgmap v726: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:37:10.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:10 smithi196 ceph-mon[35041]: pgmap v727: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:37:10.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:10 smithi059 ceph-mon[34351]: pgmap v727: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:37:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:10 smithi019 ceph-mon[30158]: pgmap v727: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-05-09T01:37:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:12 smithi196 ceph-mon[35041]: pgmap v728: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:37:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:12 smithi059 ceph-mon[34351]: pgmap v728: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:37:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:12 smithi019 ceph-mon[30158]: pgmap v728: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:37:14.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:14 smithi196 ceph-mon[35041]: pgmap v729: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:37:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:14 smithi059 ceph-mon[34351]: pgmap v729: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:37:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:14 smithi019 ceph-mon[30158]: pgmap v729: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-05-09T01:37:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:16 smithi059 ceph-mon[34351]: pgmap v730: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:37:16.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:16 smithi019 ceph-mon[30158]: pgmap v730: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:37:16.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:16 smithi196 ceph-mon[35041]: pgmap v730: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:37:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:18 smithi059 ceph-mon[34351]: pgmap v731: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:37:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:18 smithi019 ceph-mon[30158]: pgmap v731: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:37:18.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:18 smithi196 ceph-mon[35041]: pgmap v731: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-05-09T01:37:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.842+0000 7f170b1ff640 1 -- 172.21.15.19:0/1986418195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041018b0 msgr2=0x7f1704101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.842+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1986418195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041018b0 0x7f1704101d20 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009ca0 tx=0x7f16ec02a8c0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.842+0000 7f170b1ff640 1 -- 172.21.15.19:0/1986418195 shutdown_connections 2024-05-09T01:37:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.842+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1986418195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17041023b0 0x7f170410be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.842+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1986418195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041018b0 0x7f1704101d20 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.842+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1986418195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1704105200 0x7f1704105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.842+0000 7f170b1ff640 1 -- 172.21.15.19:0/1986418195 >> 172.21.15.19:0/1986418195 conn(0x7f17040fd4d0 msgr2=0x7f17040ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.842+0000 7f170b1ff640 1 -- 172.21.15.19:0/1986418195 shutdown_connections 2024-05-09T01:37:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.842+0000 7f170b1ff640 1 -- 172.21.15.19:0/1986418195 wait complete. 2024-05-09T01:37:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f170b1ff640 1 Processor -- start 2024-05-09T01:37:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f170b1ff640 1 -- start start 2024-05-09T01:37:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f170b1ff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17041018b0 0x7f170419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f170b1ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041023b0 0x7f170419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f170b1ff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1704105200 0x7f17041a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f170b1ff640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f170410f770 con 0x7f17041023b0 2024-05-09T01:37:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f170b1ff640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f170410f5f0 con 0x7f1704105200 2024-05-09T01:37:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f170b1ff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f170410f8f0 con 0x7f17041018b0 2024-05-09T01:37:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f1703fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041023b0 0x7f170419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f1703fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041023b0 0x7f170419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39290/0 (socket says 172.21.15.19:39290) 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f1703fff640 1 -- 172.21.15.19:0/1435757824 learned_addr learned my addr 172.21.15.19:0/1435757824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.843+0000 7f1708f9b640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17041018b0 0x7f170419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f170979c640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1704105200 0x7f17041a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1703fff640 1 -- 172.21.15.19:0/1435757824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17041018b0 msgr2=0x7f170419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1703fff640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17041018b0 0x7f170419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1703fff640 1 -- 172.21.15.19:0/1435757824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1704105200 msgr2=0x7f17041a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1703fff640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1704105200 0x7f17041a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1703fff640 1 -- 172.21.15.19:0/1435757824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17041a1e00 con 0x7f17041023b0 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f170979c640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1704105200 0x7f17041a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1708f9b640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17041018b0 0x7f170419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1703fff640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041023b0 0x7f170419a8b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f16ec02ade0 tx=0x7f16ec003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1701ffb640 1 -- 172.21.15.19:0/1435757824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0317c0 con 0x7f17041023b0 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f170b1ff640 1 -- 172.21.15.19:0/1435757824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17041a2090 con 0x7f17041023b0 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1701ffb640 1 -- 172.21.15.19:0/1435757824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16ec0044f0 con 0x7f17041023b0 2024-05-09T01:37:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f1701ffb640 1 -- 172.21.15.19:0/1435757824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec02fd10 con 0x7f17041023b0 2024-05-09T01:37:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.844+0000 7f170b1ff640 1 -- 172.21.15.19:0/1435757824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041a25e0 con 0x7f17041023b0 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.847+0000 7f1712186740 1 Processor -- start 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1712186740 1 -- start start 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1712186740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b07051b0 0x5577b0aa9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1712186740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 0x5577b0aa9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1712186740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1701ffb640 1 -- 172.21.15.19:0/1435757824 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f16ec004040 con 0x7f17041023b0 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1712186740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577b0a0d7c0 con 0x5577b07051b0 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1712186740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5577b09a90e0 con 0x5577b06dc2f0 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1712186740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5577b0a09780 con 0x5577b06e44d0 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f170979c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b07051b0 0x5577b0aa9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f170979c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b07051b0 0x5577b0aa9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39298/0 (socket says 172.21.15.19:39298) 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f170979c640 1 -- 172.21.15.19:0/1910586215 learned_addr learned my addr 172.21.15.19:0/1910586215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1708f9b640 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 0x5577b0aa9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f1703fff640 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f170979c640 1 -- 172.21.15.19:0/1910586215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 msgr2=0x5577b0ab0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.848+0000 7f170979c640 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f170979c640 1 -- 172.21.15.19:0/1910586215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 msgr2=0x5577b0aa9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f170979c640 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 0x5577b0aa9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f170979c640 1 -- 172.21.15.19:0/1910586215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b09d27c0 con 0x5577b07051b0 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f1703fff640 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f170979c640 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b07051b0 0x5577b0aa9500 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f16f400bb60 tx=0x7f16f400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9761289a7d318e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f16faffd640 1 -- 172.21.15.19:0/1910586215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400be60 con 0x5577b07051b0 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f1701ffb640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16f00b27e0 0x7f16f00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f16faffd640 1 -- 172.21.15.19:0/1910586215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f4017ad0 con 0x5577b07051b0 2024-05-09T01:37:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f16faffd640 1 -- 172.21.15.19:0/1910586215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400fbc0 con 0x5577b07051b0 2024-05-09T01:37:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f1708f9b640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16f00b27e0 0x7f16f00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.849+0000 7f1701ffb640 1 -- 172.21.15.19:0/1435757824 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f16ec103980 con 0x7f17041023b0 2024-05-09T01:37:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 -- 172.21.15.19:0/1910586215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b07051b0 msgr2=0x5577b0aa9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1708f9b640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16f00b27e0 0x7f16f00b4cf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x5577b0aaa8e0 tx=0x7f16fc00b6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b07051b0 0x5577b0aa9500 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f16f400bb60 tx=0x7f16f400d710 comp rx=0 tx=0).stop 2024-05-09T01:37:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 -- 172.21.15.19:0/1910586215 shutdown_connections 2024-05-09T01:37:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 0x5577b0aa9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 --2- 172.21.15.19:0/1910586215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b07051b0 0x5577b0aa9500 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 -- 172.21.15.19:0/1910586215 >> 172.21.15.19:0/1910586215 conn(0x5577b06f9300 msgr2=0x5577b0aa4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 -- 172.21.15.19:0/1910586215 shutdown_connections 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 -- 172.21.15.19:0/1910586215 wait complete. 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.850+0000 7f1712186740 1 -- start start 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1712186740 1 Processor -- start 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1712186740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1712186740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1712186740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b07051b0 0x5577b0aba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1712186740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577b0ab13d0 con 0x5577b06dc2f0 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1712186740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5577b09949c0 con 0x5577b07051b0 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1712186740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5577b0ab1550 con 0x5577b06e44d0 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1712186740 10 client.0 _ll_register_callbacks cb 0x5577b09c1020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1708f9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1708f9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39308/0 (socket says 172.21.15.19:39308) 2024-05-09T01:37:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1708f9b640 1 -- 172.21.15.19:0/2266139382 learned_addr learned my addr 172.21.15.19:0/2266139382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f170979c640 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b07051b0 0x5577b0aba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.851+0000 7f1703fff640 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1708f9b640 1 -- 172.21.15.19:0/2266139382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 msgr2=0x5577b0ab3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1708f9b640 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1708f9b640 1 -- 172.21.15.19:0/2266139382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b07051b0 msgr2=0x5577b0aba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1708f9b640 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b07051b0 0x5577b0aba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1708f9b640 1 -- 172.21.15.19:0/2266139382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b09bb1f0 con 0x5577b06dc2f0 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f170979c640 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b07051b0 0x5577b0aba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1703fff640 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1708f9b640 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab32c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00baf0 tx=0x7f16fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3eef554ab031c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f16e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f16e3fff640 1 -- 172.21.15.19:0/2266139382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc004220 con 0x5577b06dc2f0 2024-05-09T01:37:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f16e3fff640 1 -- 172.21.15.19:0/2266139382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f16fc0043c0 con 0x5577b06dc2f0 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1712186740 1 -- 172.21.15.19:0/2266139382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f4017ad0 con 0x5577b06dc2f0 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f16e3fff640 1 -- 172.21.15.19:0/2266139382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16fc02d070 con 0x5577b06dc2f0 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1712186740 10 client.18618 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f1712186740 1 -- 172.21.15.19:0/2266139382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577b0770790 con 0x5577b06dc2f0 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.852+0000 7f16e3fff640 1 -- 172.21.15.19:0/2266139382 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f4017ad0 con 0x5577b06dc2f0 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f16e3fff640 1 -- 172.21.15.19:0/2266139382 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f16fc003ae0 con 0x5577b06dc2f0 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f1712186740 10 client.18618 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f1712186740 10 client.18618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f1712186740 20 client.18618 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f1712186740 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5577b06c3d80 0x5577b0ac0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f1712186740 4 client.18618 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f1712186740 1 -- 172.21.15.19:0/2266139382 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577b0a1b3d0 con 0x5577b06c3d80 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f1703fff640 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5577b06c3d80 0x5577b0ac0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f1703fff640 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5577b06c3d80 0x5577b0ac0c70 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc762fd134a579b0 server_cookie=630e65ed8053ad01 in_seq=0 out_seq=0 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.853+0000 7f16e3fff640 10 client.18618 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.854+0000 7f16e3fff640 1 -- 172.21.15.19:0/2266139382 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5577b0a1b3d0 con 0x5577b06c3d80 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.854+0000 7f16e3fff640 10 client.18618 handle_command_reply: tid=0 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.854+0000 7f1712186740 1 client.18618 shutdown 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.854+0000 7f1712186740 1 -- 172.21.15.19:0/2266139382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 msgr2=0x5577b0ab32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.854+0000 7f1712186740 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab32c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00baf0 tx=0x7f16fc0046c0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f1712186740 1 -- 172.21.15.19:0/2266139382 shutdown_connections 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f1712186740 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5577b06c3d80 0x5577b0ac0c70 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f1712186740 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b07051b0 0x5577b0aba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f1712186740 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0ab3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f16e3fff640 0 client.18618 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f1712186740 1 --2- 172.21.15.19:0/2266139382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab32c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f1712186740 1 -- 172.21.15.19:0/2266139382 >> 172.21.15.19:0/2266139382 conn(0x5577b06f9300 msgr2=0x5577b0aa3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f1712186740 1 -- 172.21.15.19:0/2266139382 shutdown_connections 2024-05-09T01:37:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f1712186740 1 -- 172.21.15.19:0/2266139382 wait complete. 2024-05-09T01:37:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.855+0000 7f1712186740 20 client.18618 trim_cache size 0 max 16384 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1712186740 1 Processor -- start 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1712186740 1 -- start start 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1712186740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06f9300 0x5577b0aa86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1712186740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 0x5577b0aba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1712186740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0abac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1712186740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17041ab460 con 0x5577b06f9300 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1712186740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17041acfa0 con 0x5577b06dc2f0 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1712186740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17041abb30 con 0x5577b06e44d0 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1708f9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06f9300 0x5577b0aa86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1708f9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06f9300 0x5577b0aa86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39318/0 (socket says 172.21.15.19:39318) 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1708f9b640 1 -- 172.21.15.19:0/3453470496 learned_addr learned my addr 172.21.15.19:0/3453470496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1703fff640 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 0x5577b0aba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f170979c640 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0abac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1708f9b640 1 -- 172.21.15.19:0/3453470496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 msgr2=0x5577b0abac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1708f9b640 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0abac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1708f9b640 1 -- 172.21.15.19:0/3453470496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 msgr2=0x5577b0aba6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1708f9b640 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 0x5577b0aba6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f1708f9b640 1 -- 172.21.15.19:0/3453470496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b07e75f0 con 0x5577b06f9300 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.873+0000 7f170979c640 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0abac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.874+0000 7f1708f9b640 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06f9300 0x5577b0aa86c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0275d0 tx=0x7f16fc027600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5e4af5e9a05a7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.874+0000 7f16fa7fc640 1 -- 172.21.15.19:0/3453470496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc004020 con 0x5577b06f9300 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.874+0000 7f16fa7fc640 1 -- 172.21.15.19:0/3453470496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc027cd0 con 0x5577b06f9300 2024-05-09T01:37:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.874+0000 7f16fa7fc640 1 -- 172.21.15.19:0/3453470496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc005590 con 0x5577b06f9300 2024-05-09T01:37:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 -- 172.21.15.19:0/3453470496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06f9300 msgr2=0x5577b0aa86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06f9300 0x5577b0aa86c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0275d0 tx=0x7f16fc027600 comp rx=0 tx=0).stop 2024-05-09T01:37:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 -- 172.21.15.19:0/3453470496 shutdown_connections 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06e44d0 0x5577b0abac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06dc2f0 0x5577b0aba6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 --2- 172.21.15.19:0/3453470496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06f9300 0x5577b0aa86c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 -- 172.21.15.19:0/3453470496 >> 172.21.15.19:0/3453470496 conn(0x5577b0aaded0 msgr2=0x5577b0ab3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 -- 172.21.15.19:0/3453470496 shutdown_connections 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 -- 172.21.15.19:0/3453470496 wait complete. 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 -- start start 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.875+0000 7f1712186740 1 Processor -- start 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1712186740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1712186740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06e44d0 0x5577b0ab8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1712186740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06f9300 0x5577b0ab9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1712186740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17041ac8d0 con 0x5577b06dc2f0 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1712186740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17041ac200 con 0x5577b06e44d0 2024-05-09T01:37:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1712186740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17041a9440 con 0x5577b06f9300 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1712186740 10 client.0 _ll_register_callbacks cb 0x7f16fc00b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1708f9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1708f9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39334/0 (socket says 172.21.15.19:39334) 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1708f9b640 1 -- 172.21.15.19:0/3289695748 learned_addr learned my addr 172.21.15.19:0/3289695748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1703fff640 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06e44d0 0x5577b0ab8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1708f9b640 1 -- 172.21.15.19:0/3289695748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06f9300 msgr2=0x5577b0ab9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1708f9b640 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06f9300 0x5577b0ab9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1708f9b640 1 -- 172.21.15.19:0/3289695748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06e44d0 msgr2=0x5577b0ab8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1708f9b640 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06e44d0 0x5577b0ab8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1708f9b640 1 -- 172.21.15.19:0/3289695748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b09d1ed0 con 0x5577b06dc2f0 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f1708f9b640 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab85b0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0059f0 tx=0x7f16fc027c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f066205c401878f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f16e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.876+0000 7f16e3fff640 1 -- 172.21.15.19:0/3289695748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0056a0 con 0x5577b06dc2f0 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f16e3fff640 1 -- 172.21.15.19:0/3289695748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f16fc003f50 con 0x5577b06dc2f0 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f1712186740 1 -- 172.21.15.19:0/3289695748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16fc027cd0 con 0x5577b06dc2f0 2024-05-09T01:37:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f16e3fff640 1 -- 172.21.15.19:0/3289695748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16fc04e070 con 0x5577b06dc2f0 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f1712186740 10 client.18624 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f1712186740 1 -- 172.21.15.19:0/3289695748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577b0abb010 con 0x5577b06dc2f0 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f16e3fff640 1 -- 172.21.15.19:0/3289695748 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc027cd0 con 0x5577b06dc2f0 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f16e3fff640 1 -- 172.21.15.19:0/3289695748 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f16fc01f6f0 con 0x5577b06dc2f0 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f1712186740 10 client.18624 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f1712186740 10 client.18624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f1712186740 20 client.18624 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f1712186740 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5577b07051b0 0x5577b0b2b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f1712186740 4 client.18624 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.877+0000 7f1712186740 1 -- 172.21.15.19:0/3289695748 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577b0a15240 con 0x5577b07051b0 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.878+0000 7f1703fff640 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5577b07051b0 0x5577b0b2b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.878+0000 7f16e3fff640 10 client.18624 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.878+0000 7f1703fff640 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5577b07051b0 0x5577b0b2b730 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7314a88ad6399778 server_cookie=d1a993ce61ad1ba in_seq=0 out_seq=0 2024-05-09T01:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f16e3fff640 1 -- 172.21.15.19:0/3289695748 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5577b0a15240 con 0x5577b07051b0 2024-05-09T01:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f16e3fff640 10 client.18624 handle_command_reply: tid=0 2024-05-09T01:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 client.18624 shutdown 2024-05-09T01:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 -- 172.21.15.19:0/3289695748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 msgr2=0x5577b0ab85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab85b0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0059f0 tx=0x7f16fc027c20 comp rx=0 tx=0).stop 2024-05-09T01:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 -- 172.21.15.19:0/3289695748 shutdown_connections 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x5577b07051b0 0x5577b0b2b730 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577b06f9300 0x5577b0ab9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577b06e44d0 0x5577b0ab8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f16e3fff640 0 client.18624 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 --2- 172.21.15.19:0/3289695748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b06dc2f0 0x5577b0ab85b0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 -- 172.21.15.19:0/3289695748 >> 172.21.15.19:0/3289695748 conn(0x5577b0aaded0 msgr2=0x5577b0ac5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 -- 172.21.15.19:0/3289695748 shutdown_connections 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 1 -- 172.21.15.19:0/3289695748 wait complete. 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.895+0000 7f1712186740 20 client.18624 trim_cache size 0 max 16384 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 -- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16f00b27e0 msgr2=0x7f16f00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16f00b27e0 0x7f16f00b4cf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x5577b0aaa8e0 tx=0x7f16fc00b6f0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 -- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041023b0 msgr2=0x7f170419a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041023b0 0x7f170419a8b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f16ec02ade0 tx=0x7f16ec003ba0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 -- 172.21.15.19:0/1435757824 shutdown_connections 2024-05-09T01:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16f00b27e0 0x7f16f00b4cf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1704105200 0x7f17041a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17041023b0 0x7f170419a8b0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 --2- 172.21.15.19:0/1435757824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17041018b0 0x7f170419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 -- 172.21.15.19:0/1435757824 >> 172.21.15.19:0/1435757824 conn(0x7f17040fd4d0 msgr2=0x7f1704105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 -- 172.21.15.19:0/1435757824 shutdown_connections 2024-05-09T01:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:19.896+0000 7f170b1ff640 1 -- 172.21.15.19:0/1435757824 wait complete. 2024-05-09T01:37:20.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:20 smithi059 ceph-mon[34351]: pgmap v732: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:37:20.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:20 smithi019 ceph-mon[30158]: pgmap v732: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:37:20.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:37:20.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:20 smithi196 ceph-mon[35041]: pgmap v732: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-05-09T01:37:21.211 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:37:22.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:22 smithi019 ceph-mon[30158]: pgmap v733: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-05-09T01:37:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:22 smithi059 ceph-mon[34351]: pgmap v733: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-05-09T01:37:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.924+0000 7f4412ce6700 1 -- 172.21.15.19:0/574290580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 msgr2=0x7f440c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.924+0000 7f4412ce6700 1 --2- 172.21.15.19:0/574290580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 0x7f440c1100a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f440400b0d0 tx=0x7f440400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.924+0000 7f4412ce6700 1 -- 172.21.15.19:0/574290580 shutdown_connections 2024-05-09T01:37:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.924+0000 7f4412ce6700 1 --2- 172.21.15.19:0/574290580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 0x7f440c1100a0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.924+0000 7f4412ce6700 1 --2- 172.21.15.19:0/574290580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440c1034c0 0x7f440c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.924+0000 7f4412ce6700 1 --2- 172.21.15.19:0/574290580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f440c107df0 0x7f440c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.924+0000 7f4412ce6700 1 -- 172.21.15.19:0/574290580 >> 172.21.15.19:0/574290580 conn(0x7f440c0fedf0 msgr2=0x7f440c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.924+0000 7f4412ce6700 1 -- 172.21.15.19:0/574290580 shutdown_connections 2024-05-09T01:37:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.924+0000 7f4412ce6700 1 -- 172.21.15.19:0/574290580 wait complete. 2024-05-09T01:37:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.925+0000 7f4412ce6700 1 Processor -- start 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.925+0000 7f4412ce6700 1 -- start start 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4412ce6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440c1034c0 0x7f440c19f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4412ce6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f440c107df0 0x7f440c19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4412ce6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 0x7f440c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4412ce6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f440c113370 con 0x7f440c108740 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4412ce6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1131f0 con 0x7f440c107df0 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4412ce6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1134f0 con 0x7f440c1034c0 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4411283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 0x7f440c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4411283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 0x7f440c199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39366/0 (socket says 172.21.15.19:39366) 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4411283700 1 -- 172.21.15.19:0/2112108639 learned_addr learned my addr 172.21.15.19:0/2112108639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f440bfff700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f440c107df0 0x7f440c19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4410a82700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440c1034c0 0x7f440c19f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4411283700 1 -- 172.21.15.19:0/2112108639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440c1034c0 msgr2=0x7f440c19f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4411283700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440c1034c0 0x7f440c19f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4411283700 1 -- 172.21.15.19:0/2112108639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f440c107df0 msgr2=0x7f440c19fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4411283700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f440c107df0 0x7f440c19fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4411283700 1 -- 172.21.15.19:0/2112108639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c19a260 con 0x7f440c108740 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4410a82700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440c1034c0 0x7f440c19f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.926+0000 7f4411283700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 0x7f440c199b30 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f44040095c0 tx=0x7f4404007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.927+0000 7f4409ffb700 1 -- 172.21.15.19:0/2112108639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440401f070 con 0x7f440c108740 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.927+0000 7f4412ce6700 1 -- 172.21.15.19:0/2112108639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440c19a4f0 con 0x7f440c108740 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.927+0000 7f4409ffb700 1 -- 172.21.15.19:0/2112108639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4404003c00 con 0x7f440c108740 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.927+0000 7f4409ffb700 1 -- 172.21.15.19:0/2112108639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440401b6a0 con 0x7f440c108740 2024-05-09T01:37:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.927+0000 7f4412ce6700 1 -- 172.21.15.19:0/2112108639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440c0759f0 con 0x7f440c108740 2024-05-09T01:37:22.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.930+0000 7f4409ffb700 1 -- 172.21.15.19:0/2112108639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4404022070 con 0x7f440c108740 2024-05-09T01:37:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.931+0000 7f44216fcb80 1 Processor -- start 2024-05-09T01:37:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.931+0000 7f4409ffb700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f43fc0b2880 0x7f43fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.931+0000 7f44216fcb80 1 -- start start 2024-05-09T01:37:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.931+0000 7f440bfff700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f43fc0b2880 0x7f43fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.931+0000 7f4409ffb700 1 -- 172.21.15.19:0/2112108639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f440400f020 con 0x7f440c108740 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.931+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 0x55f24a622d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a6232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a627400 0x55f24a627810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f44216fcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f24a540ff0 con 0x55f24a4a14a0 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f44216fcb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f24a4df7f0 con 0x55f24a58b160 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f44216fcb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f24a53cfb0 con 0x55f24a627400 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f440bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a6232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f440bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a6232f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39368/0 (socket says 172.21.15.19:39368) 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f440bfff700 1 -- 172.21.15.19:0/596614572 learned_addr learned my addr 172.21.15.19:0/596614572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f4410a82700 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 0x55f24a622d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f440bfff700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f43fc0b2880 0x7f43fc0b4d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f440c19b260 tx=0x7f43f8007f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f4411283700 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a627400 0x55f24a627810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f4410a82700 1 -- 172.21.15.19:0/596614572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a627400 msgr2=0x55f24a627810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f4410a82700 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a627400 0x55f24a627810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f4410a82700 1 -- 172.21.15.19:0/596614572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 msgr2=0x55f24a6232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f4410a82700 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a6232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f4410a82700 1 -- 172.21.15.19:0/596614572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f24a4d7370 con 0x55f24a58b160 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f440bfff700 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a6232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f4411283700 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a627400 0x55f24a627810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.932+0000 7f4410a82700 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 0x55f24a622d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4400009cc0 tx=0x7f440000f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7722f0a55166fd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.933+0000 7f43f6ffd700 1 -- 172.21.15.19:0/596614572 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440000fbb0 con 0x55f24a58b160 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.933+0000 7f43f6ffd700 1 -- 172.21.15.19:0/596614572 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440000fd50 con 0x55f24a58b160 2024-05-09T01:37:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.933+0000 7f43f6ffd700 1 -- 172.21.15.19:0/596614572 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400011440 con 0x55f24a58b160 2024-05-09T01:37:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 -- 172.21.15.19:0/596614572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 msgr2=0x55f24a622d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 0x55f24a622d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4400009cc0 tx=0x7f440000f360 comp rx=0 tx=0).stop 2024-05-09T01:37:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 -- 172.21.15.19:0/596614572 shutdown_connections 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a627400 0x55f24a627810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 0x55f24a622d50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 --2- 172.21.15.19:0/596614572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a6232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 -- 172.21.15.19:0/596614572 >> 172.21.15.19:0/596614572 conn(0x55f24a5016a0 msgr2=0x55f24a619400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 -- 172.21.15.19:0/596614572 shutdown_connections 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 -- 172.21.15.19:0/596614572 wait complete. 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 -- start start 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.934+0000 7f44216fcb80 1 Processor -- start 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a62c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 0x55f24a62cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 0x55f24a62d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f44216fcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f24a3bcba0 con 0x55f24a4a14a0 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f44216fcb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f24a4fb150 con 0x55f24a627400 2024-05-09T01:37:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f44216fcb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f24a6281b0 con 0x55f24a58b160 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f44216fcb80 10 client.0 _ll_register_callbacks cb 0x55f24a3b26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f4410a82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a62c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f4410a82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a62c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39374/0 (socket says 172.21.15.19:39374) 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f4410a82700 1 -- 172.21.15.19:0/3415901972 learned_addr learned my addr 172.21.15.19:0/3415901972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f440bfff700 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 0x55f24a62cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f4411283700 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 0x55f24a62d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f4410a82700 1 -- 172.21.15.19:0/3415901972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 msgr2=0x55f24a62cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f4410a82700 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 0x55f24a62cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f4410a82700 1 -- 172.21.15.19:0/3415901972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 msgr2=0x55f24a62d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f4410a82700 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 0x55f24a62d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.935+0000 7f4410a82700 1 -- 172.21.15.19:0/3415901972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f24a3bc490 con 0x55f24a4a14a0 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.936+0000 7f4410a82700 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a62c630 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4400005330 tx=0x7f440000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42484d7a28b625a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.936+0000 7f43ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.936+0000 7f43ebfff700 1 -- 172.21.15.19:0/3415901972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400004960 con 0x55f24a4a14a0 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.936+0000 7f43ebfff700 1 -- 172.21.15.19:0/3415901972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f4400004b00 con 0x55f24a4a14a0 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.936+0000 7f44216fcb80 1 -- 172.21.15.19:0/3415901972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440000fd50 con 0x55f24a4a14a0 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.936+0000 7f43ebfff700 1 -- 172.21.15.19:0/3415901972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4400004d00 con 0x55f24a4a14a0 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.936+0000 7f44216fcb80 10 client.18636 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.936+0000 7f44216fcb80 1 -- 172.21.15.19:0/3415901972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f24a639ec0 con 0x55f24a4a14a0 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.936+0000 7f43ebfff700 1 -- 172.21.15.19:0/3415901972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440000fd50 con 0x55f24a4a14a0 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f43ebfff700 1 -- 172.21.15.19:0/3415901972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4400030420 con 0x55f24a4a14a0 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f44216fcb80 10 client.18636 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f44216fcb80 10 client.18636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f44216fcb80 20 client.18636 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f24a63c390 0x55f24a63e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f44216fcb80 4 client.18636 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f44216fcb80 1 -- 172.21.15.19:0/3415901972 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f24a54e9d0 con 0x55f24a63c390 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f440bfff700 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f24a63c390 0x55f24a63e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f43ebfff700 10 client.18636 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.937+0000 7f440bfff700 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f24a63c390 0x55f24a63e7b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e418e61eacf7dc6c server_cookie=5f5e77c5e89a59a8 in_seq=0 out_seq=0 2024-05-09T01:37:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f43ebfff700 1 -- 172.21.15.19:0/3415901972 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f24a54e9d0 con 0x55f24a63c390 2024-05-09T01:37:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f43ebfff700 10 client.18636 handle_command_reply: tid=0 2024-05-09T01:37:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f44216fcb80 1 client.18636 shutdown 2024-05-09T01:37:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f44216fcb80 1 -- 172.21.15.19:0/3415901972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 msgr2=0x55f24a62c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a62c630 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4400005330 tx=0x7f440000fc90 comp rx=0 tx=0).stop 2024-05-09T01:37:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f44216fcb80 1 -- 172.21.15.19:0/3415901972 shutdown_connections 2024-05-09T01:37:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f24a63c390 0x55f24a63e7b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 0x55f24a62d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a62c630 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3415901972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 0x55f24a62cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.938+0000 7f44216fcb80 1 -- 172.21.15.19:0/3415901972 >> 172.21.15.19:0/3415901972 conn(0x55f24a5016a0 msgr2=0x55f24a6191c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.939+0000 7f43ebfff700 0 client.18636 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.939+0000 7f44216fcb80 1 -- 172.21.15.19:0/3415901972 shutdown_connections 2024-05-09T01:37:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.939+0000 7f44216fcb80 1 -- 172.21.15.19:0/3415901972 wait complete. 2024-05-09T01:37:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.939+0000 7f44216fcb80 20 client.18636 trim_cache size 0 max 16384 2024-05-09T01:37:22.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:22 smithi196 ceph-mon[35041]: pgmap v733: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-05-09T01:37:22.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f44216fcb80 1 Processor -- start 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f44216fcb80 1 -- start start 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a626c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 0x55f24a638400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 0x55f24a643e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f44216fcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1aca90 con 0x55f24a4a14a0 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f44216fcb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1ae5d0 con 0x55f24a627400 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f44216fcb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1ad160 con 0x55f24a58b160 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f4410a82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a626c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f4410a82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a626c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39380/0 (socket says 172.21.15.19:39380) 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f4410a82700 1 -- 172.21.15.19:0/3925709269 learned_addr learned my addr 172.21.15.19:0/3925709269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f440bfff700 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 0x55f24a638400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f4411283700 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 0x55f24a643e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f4410a82700 1 -- 172.21.15.19:0/3925709269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 msgr2=0x55f24a638400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f4410a82700 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 0x55f24a638400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f4410a82700 1 -- 172.21.15.19:0/3925709269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 msgr2=0x55f24a643e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f4410a82700 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 0x55f24a643e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f4410a82700 1 -- 172.21.15.19:0/3925709269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f24a612f60 con 0x55f24a4a14a0 2024-05-09T01:37:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.960+0000 7f440bfff700 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 0x55f24a638400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.961+0000 7f4410a82700 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a626c60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4400030840 tx=0x7f440000f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c07dd028066ad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.961+0000 7f43f67fc700 1 -- 172.21.15.19:0/3925709269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400005580 con 0x55f24a4a14a0 2024-05-09T01:37:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.961+0000 7f43f67fc700 1 -- 172.21.15.19:0/3925709269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4400035020 con 0x55f24a4a14a0 2024-05-09T01:37:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.961+0000 7f43f67fc700 1 -- 172.21.15.19:0/3925709269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400004960 con 0x55f24a4a14a0 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 -- 172.21.15.19:0/3925709269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 msgr2=0x55f24a626c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a626c60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4400030840 tx=0x7f440000f360 comp rx=0 tx=0).stop 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 -- 172.21.15.19:0/3925709269 shutdown_connections 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a627400 0x55f24a643e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a4a14a0 0x55f24a626c60 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 --2- 172.21.15.19:0/3925709269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a58b160 0x55f24a638400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 -- 172.21.15.19:0/3925709269 >> 172.21.15.19:0/3925709269 conn(0x55f24a5016a0 msgr2=0x55f24a61dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 -- 172.21.15.19:0/3925709269 shutdown_connections 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 -- 172.21.15.19:0/3925709269 wait complete. 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 -- start start 2024-05-09T01:37:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.962+0000 7f44216fcb80 1 Processor -- start 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a4a14a0 0x55f24a6295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 0x55f24a629b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f44216fcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a627400 0x55f24a62a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f44216fcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1adf00 con 0x55f24a627400 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f44216fcb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1ad830 con 0x55f24a58b160 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f44216fcb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1aaa30 con 0x55f24a4a14a0 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f44216fcb80 10 client.0 _ll_register_callbacks cb 0x55f24a3b26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4411283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a627400 0x55f24a62a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4411283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a627400 0x55f24a62a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39386/0 (socket says 172.21.15.19:39386) 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4411283700 1 -- 172.21.15.19:0/2055331295 learned_addr learned my addr 172.21.15.19:0/2055331295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f440bfff700 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 0x55f24a629b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4410a82700 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a4a14a0 0x55f24a6295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4411283700 1 -- 172.21.15.19:0/2055331295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a4a14a0 msgr2=0x55f24a6295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:22.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4411283700 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a4a14a0 0x55f24a6295d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4411283700 1 -- 172.21.15.19:0/2055331295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 msgr2=0x55f24a629b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4411283700 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 0x55f24a629b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4411283700 1 -- 172.21.15.19:0/2055331295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f24a6a5a40 con 0x55f24a627400 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f4411283700 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a627400 0x55f24a62a050 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f440400d010 tx=0x7f44040aaa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0920a0f0582dc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.963+0000 7f43ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f43ebfff700 1 -- 172.21.15.19:0/2055331295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040c4060 con 0x55f24a627400 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f43ebfff700 1 -- 172.21.15.19:0/2055331295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f44040ab010 con 0x55f24a627400 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f44216fcb80 1 -- 172.21.15.19:0/2055331295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4400035020 con 0x55f24a627400 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f43ebfff700 1 -- 172.21.15.19:0/2055331295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44040b32b0 con 0x55f24a627400 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f44216fcb80 10 client.18642 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f44216fcb80 1 -- 172.21.15.19:0/2055331295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f24a6a6cf0 con 0x55f24a627400 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f43ebfff700 1 -- 172.21.15.19:0/2055331295 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4400035020 con 0x55f24a627400 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f43ebfff700 1 -- 172.21.15.19:0/2055331295 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4404011070 con 0x55f24a627400 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f44216fcb80 10 client.18642 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:22.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f44216fcb80 10 client.18642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f44216fcb80 20 client.18642 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f44216fcb80 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f24a6a73b0 0x55f24a6a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f44216fcb80 4 client.18642 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f44216fcb80 1 -- 172.21.15.19:0/2055331295 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f24a548a70 con 0x55f24a6a73b0 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.964+0000 7f4410a82700 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f24a6a73b0 0x55f24a6a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f4410a82700 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f24a6a73b0 0x55f24a6a97d0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2cdf51f13fb8327 server_cookie=b7959c31202f02aa in_seq=0 out_seq=0 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f43ebfff700 10 client.18642 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f43ebfff700 1 -- 172.21.15.19:0/2055331295 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f24a548a70 con 0x55f24a6a73b0 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f43ebfff700 10 client.18642 handle_command_reply: tid=0 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 client.18642 shutdown 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 -- 172.21.15.19:0/2055331295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a627400 msgr2=0x55f24a62a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a627400 0x55f24a62a050 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f440400d010 tx=0x7f44040aaa20 comp rx=0 tx=0).stop 2024-05-09T01:37:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 -- 172.21.15.19:0/2055331295 shutdown_connections 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55f24a6a73b0 0x55f24a6a97d0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f24a627400 0x55f24a62a050 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f24a4a14a0 0x55f24a6295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f43ebfff700 0 client.18642 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 --2- 172.21.15.19:0/2055331295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f24a58b160 0x55f24a629b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 -- 172.21.15.19:0/2055331295 >> 172.21.15.19:0/2055331295 conn(0x55f24a5016a0 msgr2=0x55f24a62adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 -- 172.21.15.19:0/2055331295 shutdown_connections 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.965+0000 7f44216fcb80 1 -- 172.21.15.19:0/2055331295 wait complete. 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f44216fcb80 20 client.18642 trim_cache size 0 max 16384 2024-05-09T01:37:22.967 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:37:22.968 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:37:22.968 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:37:22.968 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:37:22.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 -- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f43fc0b2880 msgr2=0x7f43fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f43fc0b2880 0x7f43fc0b4d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f440c19b260 tx=0x7f43f8007f40 comp rx=0 tx=0).stop 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 -- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 msgr2=0x7f440c199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 0x7f440c199b30 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f44040095c0 tx=0x7f4404007a20 comp rx=0 tx=0).stop 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 -- 172.21.15.19:0/2112108639 shutdown_connections 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f43fc0b2880 0x7f43fc0b4d90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f440c108740 0x7f440c199b30 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440c1034c0 0x7f440c19f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 --2- 172.21.15.19:0/2112108639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f440c107df0 0x7f440c19fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 -- 172.21.15.19:0/2112108639 >> 172.21.15.19:0/2112108639 conn(0x7f440c0fedf0 msgr2=0x7f440c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 -- 172.21.15.19:0/2112108639 shutdown_connections 2024-05-09T01:37:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:22.966+0000 7f4412ce6700 1 -- 172.21.15.19:0/2112108639 wait complete. 2024-05-09T01:37:23.766 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:37:23.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:37:24.033 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:37:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:24 smithi059 ceph-mon[34351]: pgmap v734: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-09T01:37:24.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:24 smithi019 ceph-mon[30158]: pgmap v734: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-09T01:37:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:24 smithi196 ceph-mon[35041]: pgmap v734: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-09T01:37:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.903+0000 7fca5568b700 1 -- 172.21.15.19:0/1501765026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca501034c0 msgr2=0x7fca501038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.903+0000 7fca5568b700 1 --2- 172.21.15.19:0/1501765026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca501034c0 0x7fca501038d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fca400099c0 tx=0x7fca40009cd0 comp rx=0 tx=0).stop 2024-05-09T01:37:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.903+0000 7fca5568b700 1 -- 172.21.15.19:0/1501765026 shutdown_connections 2024-05-09T01:37:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.903+0000 7fca5568b700 1 --2- 172.21.15.19:0/1501765026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca50108740 0x7fca501100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.903+0000 7fca5568b700 1 --2- 172.21.15.19:0/1501765026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca501034c0 0x7fca501038d0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.903+0000 7fca5568b700 1 --2- 172.21.15.19:0/1501765026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50107df0 0x7fca50108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.903+0000 7fca5568b700 1 -- 172.21.15.19:0/1501765026 >> 172.21.15.19:0/1501765026 conn(0x7fca500fedc0 msgr2=0x7fca501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.903+0000 7fca5568b700 1 -- 172.21.15.19:0/1501765026 shutdown_connections 2024-05-09T01:37:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.903+0000 7fca5568b700 1 -- 172.21.15.19:0/1501765026 wait complete. 2024-05-09T01:37:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.904+0000 7fca5568b700 1 Processor -- start 2024-05-09T01:37:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.904+0000 7fca5568b700 1 -- start start 2024-05-09T01:37:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca5568b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501034c0 0x7fca5019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca5568b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50107df0 0x7fca5019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca5568b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca50108740 0x7fca5019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca5568b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca50113370 con 0x7fca50108740 2024-05-09T01:37:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca5568b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca501131f0 con 0x7fca501034c0 2024-05-09T01:37:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca5568b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca501134f0 con 0x7fca50107df0 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca50108740 0x7fca5019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca50108740 0x7fca5019f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39400/0 (socket says 172.21.15.19:39400) 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4f7fe700 1 -- 172.21.15.19:0/4290420255 learned_addr learned my addr 172.21.15.19:0/4290420255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4effd700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501034c0 0x7fca5019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4e7fc700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50107df0 0x7fca5019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4f7fe700 1 -- 172.21.15.19:0/4290420255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50107df0 msgr2=0x7fca5019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4f7fe700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50107df0 0x7fca5019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4f7fe700 1 -- 172.21.15.19:0/4290420255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501034c0 msgr2=0x7fca5019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4f7fe700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501034c0 0x7fca5019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.905+0000 7fca4f7fe700 1 -- 172.21.15.19:0/4290420255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5019fbd0 con 0x7fca50108740 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.906+0000 7fca4f7fe700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca50108740 0x7fca5019f5e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fca4800ef30 tx=0x7fca4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.906+0000 7fca37fff700 1 -- 172.21.15.19:0/4290420255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480092e0 con 0x7fca50108740 2024-05-09T01:37:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.906+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5019fe60 con 0x7fca50108740 2024-05-09T01:37:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.906+0000 7fca37fff700 1 -- 172.21.15.19:0/4290420255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca48004bb0 con 0x7fca50108740 2024-05-09T01:37:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.906+0000 7fca37fff700 1 -- 172.21.15.19:0/4290420255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48004020 con 0x7fca50108740 2024-05-09T01:37:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.906+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a0350 con 0x7fca50108740 2024-05-09T01:37:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.907+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca500512c0 con 0x7fca50108740 2024-05-09T01:37:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.910+0000 7fca37fff700 1 -- 172.21.15.19:0/4290420255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fca4801b030 con 0x7fca50108740 2024-05-09T01:37:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.910+0000 7fca37fff700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca380b2880 0x7fca380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.911+0000 7fca4effd700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca380b2880 0x7fca380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:25.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.911+0000 7fca37fff700 1 -- 172.21.15.19:0/4290420255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fca480e2650 con 0x7fca50108740 2024-05-09T01:37:25.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.911+0000 7fca4effd700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca380b2880 0x7fca380b4d90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fca40003760 tx=0x7fca4001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:25.914+0000 7fca37fff700 1 -- 172.21.15.19:0/4290420255 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca48022020 con 0x7fca50108740 2024-05-09T01:37:26.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.178+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca5019c110 con 0x7fca50108740 2024-05-09T01:37:26.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.179+0000 7fca37fff700 1 -- 172.21.15.19:0/4290420255 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fca48010040 con 0x7fca50108740 2024-05-09T01:37:26.176 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:37:26.177 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca380b2880 msgr2=0x7fca380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca380b2880 0x7fca380b4d90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fca40003760 tx=0x7fca4001a040 comp rx=0 tx=0).stop 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca50108740 msgr2=0x7fca5019f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca50108740 0x7fca5019f5e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fca4800ef30 tx=0x7fca4800c3a0 comp rx=0 tx=0).stop 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 shutdown_connections 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fca380b2880 0x7fca380b4d90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca50108740 0x7fca5019f5e0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501034c0 0x7fca5019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 --2- 172.21.15.19:0/4290420255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50107df0 0x7fca5019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 >> 172.21.15.19:0/4290420255 conn(0x7fca500fedc0 msgr2=0x7fca5006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:26.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 shutdown_connections 2024-05-09T01:37:26.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:26.181+0000 7fca5568b700 1 -- 172.21.15.19:0/4290420255 wait complete. 2024-05-09T01:37:26.179 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:37:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:26 smithi059 ceph-mon[34351]: pgmap v735: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:37:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:26 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/4290420255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:37:26.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:26 smithi019 ceph-mon[30158]: pgmap v735: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:37:26.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:26 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/4290420255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:37:26.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:26 smithi196 ceph-mon[35041]: pgmap v735: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-05-09T01:37:26.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:26 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/4290420255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:37:27.050 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:37:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:37:28.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:28 smithi019 ceph-mon[30158]: pgmap v736: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:37:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:28 smithi059 ceph-mon[34351]: pgmap v736: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:37:28.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:28 smithi196 ceph-mon[35041]: pgmap v736: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:37:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.955+0000 7fd093328700 1 -- 172.21.15.19:0/906119624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c108740 msgr2=0x7fd08c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.955+0000 7fd093328700 1 --2- 172.21.15.19:0/906119624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c108740 0x7fd08c1100a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00b0d0 tx=0x7fd07c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.955+0000 7fd093328700 1 -- 172.21.15.19:0/906119624 shutdown_connections 2024-05-09T01:37:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.955+0000 7fd093328700 1 --2- 172.21.15.19:0/906119624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c108740 0x7fd08c1100a0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.955+0000 7fd093328700 1 --2- 172.21.15.19:0/906119624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd08c1034c0 0x7fd08c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.955+0000 7fd093328700 1 --2- 172.21.15.19:0/906119624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd08c107df0 0x7fd08c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.955+0000 7fd093328700 1 -- 172.21.15.19:0/906119624 >> 172.21.15.19:0/906119624 conn(0x7fd08c0fedc0 msgr2=0x7fd08c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.956+0000 7fd093328700 1 -- 172.21.15.19:0/906119624 shutdown_connections 2024-05-09T01:37:28.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.956+0000 7fd093328700 1 -- 172.21.15.19:0/906119624 wait complete. 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.956+0000 7fd093328700 1 Processor -- start 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.956+0000 7fd093328700 1 -- start start 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd093328700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd08c1034c0 0x7fd08c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd093328700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c107df0 0x7fd08c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd093328700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd08c108740 0x7fd08c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd093328700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1133c0 con 0x7fd08c107df0 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd093328700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c113240 con 0x7fd08c108740 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd093328700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c113540 con 0x7fd08c1034c0 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0908c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c107df0 0x7fd08c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0908c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c107df0 0x7fd08c19c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39420/0 (socket says 172.21.15.19:39420) 2024-05-09T01:37:28.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0908c3700 1 -- 172.21.15.19:0/2814634392 learned_addr learned my addr 172.21.15.19:0/2814634392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd08c1034c0 0x7fd08c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0918c5700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd08c108740 0x7fd08c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0908c3700 1 -- 172.21.15.19:0/2814634392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd08c1034c0 msgr2=0x7fd08c19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0908c3700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd08c1034c0 0x7fd08c19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0908c3700 1 -- 172.21.15.19:0/2814634392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd08c108740 msgr2=0x7fd08c19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0908c3700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd08c108740 0x7fd08c19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0908c3700 1 -- 172.21.15.19:0/2814634392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c1a6f40 con 0x7fd08c107df0 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0918c5700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd08c108740 0x7fd08c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd08c1034c0 0x7fd08c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.957+0000 7fd0908c3700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c107df0 0x7fd08c19c1c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d980 tx=0x7fd08000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.958+0000 7fd08a7fc700 1 -- 172.21.15.19:0/2814634392 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080017070 con 0x7fd08c107df0 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.958+0000 7fd08a7fc700 1 -- 172.21.15.19:0/2814634392 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd080010460 con 0x7fd08c107df0 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.958+0000 7fd093328700 1 -- 172.21.15.19:0/2814634392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08c1a7230 con 0x7fd08c107df0 2024-05-09T01:37:28.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.958+0000 7fd08a7fc700 1 -- 172.21.15.19:0/2814634392 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08000de60 con 0x7fd08c107df0 2024-05-09T01:37:28.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.958+0000 7fd093328700 1 -- 172.21.15.19:0/2814634392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08c1a7770 con 0x7fd08c107df0 2024-05-09T01:37:28.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.960+0000 7fd08a7fc700 1 -- 172.21.15.19:0/2814634392 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd080010610 con 0x7fd08c107df0 2024-05-09T01:37:28.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.961+0000 7fd08a7fc700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd0740b2880 0x7fd0740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd0740b2880 0x7fd0740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0a1d3eb80 1 Processor -- start 2024-05-09T01:37:28.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd08a7fc700 1 -- 172.21.15.19:0/2814634392 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd0800e1d90 con 0x7fd08c107df0 2024-05-09T01:37:28.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0a1d3eb80 1 -- start start 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 0x558d80c67d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c6c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558d80b85ff0 con 0x558d80ae64a0 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558d80b247f0 con 0x558d80c6c400 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d80b81fb0 con 0x558d80bd0160 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd0740b2880 0x7fd0740b4d90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd084005fd0 tx=0x7fd084005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0908c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0908c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c682f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39430/0 (socket says 172.21.15.19:39430) 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0908c3700 1 -- 172.21.15.19:0/3642368679 learned_addr learned my addr 172.21.15.19:0/3642368679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0918c5700 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c6c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.962+0000 7fd0910c4700 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 0x558d80c67d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0908c3700 1 -- 172.21.15.19:0/3642368679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 msgr2=0x558d80c67d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0908c3700 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 0x558d80c67d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0908c3700 1 -- 172.21.15.19:0/3642368679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 msgr2=0x558d80c6c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0908c3700 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c6c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0908c3700 1 -- 172.21.15.19:0/3642368679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d80b1c370 con 0x558d80ae64a0 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0910c4700 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 0x558d80c67d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0908c3700 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c682f0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd0800a9320 tx=0x7fd0800a9a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e558499ef9356259 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0737fe700 1 -- 172.21.15.19:0/3642368679 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080020070 con 0x558d80ae64a0 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0737fe700 1 -- 172.21.15.19:0/3642368679 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0800ad740 con 0x558d80ae64a0 2024-05-09T01:37:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.963+0000 7fd0737fe700 1 -- 172.21.15.19:0/3642368679 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800a8850 con 0x558d80ae64a0 2024-05-09T01:37:28.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/3642368679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 msgr2=0x558d80c682f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c682f0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd0800a9320 tx=0x7fd0800a9a90 comp rx=0 tx=0).stop 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/3642368679 shutdown_connections 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c6c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 0x558d80c67d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/3642368679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c682f0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/3642368679 >> 172.21.15.19:0/3642368679 conn(0x558d80b466a0 msgr2=0x558d80c5e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/3642368679 shutdown_connections 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/3642368679 wait complete. 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 -- start start 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.964+0000 7fd0a1d3eb80 1 Processor -- start 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80ae64a0 0x558d80c71640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c71b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80c6c400 0x558d80c720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558d80a01ba0 con 0x558d80bd0160 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558d80b40150 con 0x558d80ae64a0 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d80c6d150 con 0x558d80c6c400 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0a1d3eb80 10 client.0 _ll_register_callbacks cb 0x558d809f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0908c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0908c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c71b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39442/0 (socket says 172.21.15.19:39442) 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0908c3700 1 -- 172.21.15.19:0/2585677645 learned_addr learned my addr 172.21.15.19:0/2585677645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80ae64a0 0x558d80c71640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.965+0000 7fd0918c5700 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80c6c400 0x558d80c720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0908c3700 1 -- 172.21.15.19:0/2585677645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80c6c400 msgr2=0x558d80c720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0908c3700 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80c6c400 0x558d80c720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0908c3700 1 -- 172.21.15.19:0/2585677645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80ae64a0 msgr2=0x558d80c71640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0908c3700 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80ae64a0 0x558d80c71640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0908c3700 1 -- 172.21.15.19:0/2585677645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d80a01490 con 0x558d80bd0160 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80ae64a0 0x558d80c71640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0908c3700 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c71b80 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd0800a99f0 tx=0x7fd0800b22a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c235c20ba71e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd070ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd070ff9700 1 -- 172.21.15.19:0/2585677645 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800c2070 con 0x558d80bd0160 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd070ff9700 1 -- 172.21.15.19:0/2585677645 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd080009b40 con 0x558d80bd0160 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2585677645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0800ad740 con 0x558d80bd0160 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd070ff9700 1 -- 172.21.15.19:0/2585677645 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd080020030 con 0x558d80bd0160 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0a1d3eb80 10 client.18660 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.966+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2585677645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d80c7eea0 con 0x558d80bd0160 2024-05-09T01:37:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.967+0000 7fd070ff9700 1 -- 172.21.15.19:0/2585677645 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0800ad740 con 0x558d80bd0160 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.967+0000 7fd070ff9700 1 -- 172.21.15.19:0/2585677645 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd0800b0ac0 con 0x558d80bd0160 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.967+0000 7fd0a1d3eb80 10 client.18660 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.967+0000 7fd0a1d3eb80 10 client.18660 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.967+0000 7fd0a1d3eb80 20 client.18660 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.967+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558d80c81370 0x558d80c83790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.967+0000 7fd0a1d3eb80 4 client.18660 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.967+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2585677645 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d80b939d0 con 0x558d80c81370 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.967+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558d80c81370 0x558d80c83790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.968+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558d80c81370 0x558d80c83790 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1900e907b182cf9d server_cookie=613fa777af1d4ee in_seq=0 out_seq=0 2024-05-09T01:37:28.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.968+0000 7fd070ff9700 10 client.18660 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:37:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.968+0000 7fd070ff9700 1 -- 172.21.15.19:0/2585677645 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d80b939d0 con 0x558d80c81370 2024-05-09T01:37:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.968+0000 7fd070ff9700 10 client.18660 handle_command_reply: tid=0 2024-05-09T01:37:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.968+0000 7fd0a1d3eb80 1 client.18660 shutdown 2024-05-09T01:37:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2585677645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 msgr2=0x558d80c71b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c71b80 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd0800a99f0 tx=0x7fd0800b22a0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2585677645 shutdown_connections 2024-05-09T01:37:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558d80c81370 0x558d80c83790 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80c6c400 0x558d80c720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80ae64a0 0x558d80c71640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2585677645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c71b80 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2585677645 >> 172.21.15.19:0/2585677645 conn(0x558d80b466a0 msgr2=0x558d80c5e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd070ff9700 0 client.18660 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:37:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2585677645 shutdown_connections 2024-05-09T01:37:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2585677645 wait complete. 2024-05-09T01:37:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.969+0000 7fd0a1d3eb80 20 client.18660 trim_cache size 0 max 16384 2024-05-09T01:37:28.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.984+0000 7fd0a1d3eb80 1 Processor -- start 2024-05-09T01:37:28.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.984+0000 7fd0a1d3eb80 1 -- start start 2024-05-09T01:37:28.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.984+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80ae64a0 0x558d80c7d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.984+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.984+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.984+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1acc30 con 0x558d80bd0160 2024-05-09T01:37:28.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.984+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1ae770 con 0x558d80c6c400 2024-05-09T01:37:28.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.984+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1ad300 con 0x558d80ae64a0 2024-05-09T01:37:28.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0908c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c863a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0908c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c863a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39458/0 (socket says 172.21.15.19:39458) 2024-05-09T01:37:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0908c3700 1 -- 172.21.15.19:0/1220737468 learned_addr learned my addr 172.21.15.19:0/1220737468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0910c4700 1 --2- 172.21.15.19:0/1220737468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80ae64a0 0x558d80c7d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0918c5700 1 --2- 172.21.15.19:0/1220737468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0908c3700 1 -- 172.21.15.19:0/1220737468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80ae64a0 msgr2=0x558d80c7d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0908c3700 1 --2- 172.21.15.19:0/1220737468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80ae64a0 0x558d80c7d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0908c3700 1 -- 172.21.15.19:0/1220737468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 msgr2=0x558d80c868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0908c3700 1 --2- 172.21.15.19:0/1220737468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c868e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0908c3700 1 -- 172.21.15.19:0/1220737468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d80c86e20 con 0x558d80bd0160 2024-05-09T01:37:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd0908c3700 1 --2- 172.21.15.19:0/1220737468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c863a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd0800b0eb0 tx=0x7fd080010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236121386e13e9c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd072ffd700 1 -- 172.21.15.19:0/1220737468 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800c2070 con 0x558d80bd0160 2024-05-09T01:37:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd072ffd700 1 -- 172.21.15.19:0/1220737468 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0800ad740 con 0x558d80bd0160 2024-05-09T01:37:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.985+0000 7fd072ffd700 1 -- 172.21.15.19:0/1220737468 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800a4d00 con 0x558d80bd0160 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.986+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/1220737468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 msgr2=0x558d80c863a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.986+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/1220737468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c863a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd0800b0eb0 tx=0x7fd080010c70 comp rx=0 tx=0).stop 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.986+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/1220737468 shutdown_connections 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.986+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/1220737468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.986+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/1220737468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80ae64a0 0x558d80c7d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.986+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/1220737468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80bd0160 0x558d80c863a0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.986+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/1220737468 >> 172.21.15.19:0/1220737468 conn(0x558d80b466a0 msgr2=0x558d80c5e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.986+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/1220737468 shutdown_connections 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.986+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/1220737468 wait complete. 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.987+0000 7fd0a1d3eb80 1 -- start start 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.987+0000 7fd0a1d3eb80 1 Processor -- start 2024-05-09T01:37:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.987+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c86600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.987+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 0x558d80c83960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.987+0000 7fd0a1d3eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.987+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1ae0a0 con 0x558d80ae64a0 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.987+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1ad9d0 con 0x558d80c6c400 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.987+0000 7fd0a1d3eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1aabd0 con 0x558d80bd0160 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.987+0000 7fd0a1d3eb80 10 client.0 _ll_register_callbacks cb 0x558d809f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0910c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c86600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0910c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c86600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39470/0 (socket says 172.21.15.19:39470) 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0910c4700 1 -- 172.21.15.19:0/2818525958 learned_addr learned my addr 172.21.15.19:0/2818525958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0918c5700 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0908c3700 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 0x558d80c83960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0910c4700 1 -- 172.21.15.19:0/2818525958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 msgr2=0x558d80c83960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 0x558d80c83960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0910c4700 1 -- 172.21.15.19:0/2818525958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 msgr2=0x558d80c83ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c83ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0910c4700 1 -- 172.21.15.19:0/2818525958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d80c843e0 con 0x558d80ae64a0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0910c4700 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c86600 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd084005ea0 tx=0x7fd0840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b92c3375278475 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd070ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd070ff9700 1 -- 172.21.15.19:0/2818525958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084028500 con 0x558d80ae64a0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd070ff9700 1 -- 172.21.15.19:0/2818525958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd0840286a0 con 0x558d80ae64a0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.988+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2818525958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0800ad740 con 0x558d80ae64a0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd070ff9700 1 -- 172.21.15.19:0/2818525958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0840288a0 con 0x558d80ae64a0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd0a1d3eb80 10 client.18666 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2818525958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d80ceb480 con 0x558d80ae64a0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd070ff9700 1 -- 172.21.15.19:0/2818525958 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0800ad740 con 0x558d80ae64a0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd070ff9700 1 -- 172.21.15.19:0/2818525958 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd08401e430 con 0x558d80ae64a0 2024-05-09T01:37:28.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd0a1d3eb80 10 client.18666 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd0a1d3eb80 10 client.18666 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd0a1d3eb80 20 client.18666 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558d80cebba0 0x558d80cedfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd0a1d3eb80 4 client.18666 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2818525958 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d80b8da70 con 0x558d80cebba0 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.989+0000 7fd0908c3700 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558d80cebba0 0x558d80cedfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0908c3700 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558d80cebba0 0x558d80cedfc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=915184a6b873ec3b server_cookie=a397a45262ee2ab2 in_seq=0 out_seq=0 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd070ff9700 10 client.18666 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd070ff9700 1 -- 172.21.15.19:0/2818525958 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d80b8da70 con 0x558d80cebba0 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd070ff9700 10 client.18666 handle_command_reply: tid=0 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 client.18666 shutdown 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2818525958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 msgr2=0x558d80c86600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c86600 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd084005ea0 tx=0x7fd0840047c0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2818525958 shutdown_connections 2024-05-09T01:37:28.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x558d80cebba0 0x558d80cedfc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d80c6c400 0x558d80c83ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d80ae64a0 0x558d80c86600 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 --2- 172.21.15.19:0/2818525958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d80bd0160 0x558d80c83960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2818525958 >> 172.21.15.19:0/2818525958 conn(0x558d80b466a0 msgr2=0x558d80c6b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:28.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd070ff9700 0 client.18666 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:37:28.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2818525958 shutdown_connections 2024-05-09T01:37:28.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 1 -- 172.21.15.19:0/2818525958 wait complete. 2024-05-09T01:37:28.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.990+0000 7fd0a1d3eb80 20 client.18666 trim_cache size 0 max 16384 2024-05-09T01:37:28.992 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 -- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd0740b2880 msgr2=0x7fd0740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd0740b2880 0x7fd0740b4d90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd084005fd0 tx=0x7fd084005e30 comp rx=0 tx=0).stop 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 -- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c107df0 msgr2=0x7fd08c19c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c107df0 0x7fd08c19c1c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d980 tx=0x7fd08000dd40 comp rx=0 tx=0).stop 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 -- 172.21.15.19:0/2814634392 shutdown_connections 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd08c108740 0x7fd08c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd08c1034c0 0x7fd08c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd0740b2880 0x7fd0740b4d90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 --2- 172.21.15.19:0/2814634392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd08c107df0 0x7fd08c19c1c0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 -- 172.21.15.19:0/2814634392 >> 172.21.15.19:0/2814634392 conn(0x7fd08c0fedc0 msgr2=0x7fd08c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 -- 172.21.15.19:0/2814634392 shutdown_connections 2024-05-09T01:37:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:28.991+0000 7fd093328700 1 -- 172.21.15.19:0/2814634392 wait complete. 2024-05-09T01:37:29.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:37:29.985 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:37:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:30 smithi059 ceph-mon[34351]: pgmap v737: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:37:30.887 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:30 smithi019 ceph-mon[30158]: pgmap v737: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:37:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:30 smithi196 ceph-mon[35041]: pgmap v737: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:37:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.805+0000 7ff646ac0700 1 -- 172.21.15.19:0/3369052620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64010e8f0 msgr2=0x7ff640110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.805+0000 7ff646ac0700 1 --2- 172.21.15.19:0/3369052620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64010e8f0 0x7ff640110dd0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b0a0 tx=0x7ff63800b3b0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.805+0000 7ff646ac0700 1 -- 172.21.15.19:0/3369052620 shutdown_connections 2024-05-09T01:37:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.805+0000 7ff646ac0700 1 --2- 172.21.15.19:0/3369052620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64010e8f0 0x7ff640110dd0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.805+0000 7ff646ac0700 1 --2- 172.21.15.19:0/3369052620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6401054c0 0x7ff6401058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.805+0000 7ff646ac0700 1 --2- 172.21.15.19:0/3369052620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff640105e10 0x7ff64010e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.805+0000 7ff646ac0700 1 -- 172.21.15.19:0/3369052620 >> 172.21.15.19:0/3369052620 conn(0x7ff6400fedf0 msgr2=0x7ff640101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.805+0000 7ff646ac0700 1 -- 172.21.15.19:0/3369052620 shutdown_connections 2024-05-09T01:37:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.805+0000 7ff646ac0700 1 -- 172.21.15.19:0/3369052620 wait complete. 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.806+0000 7ff646ac0700 1 Processor -- start 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.806+0000 7ff646ac0700 1 -- start start 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff646ac0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6401054c0 0x7ff64019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff646ac0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff640105e10 0x7ff64019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff646ac0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64010e8f0 0x7ff64019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff646ac0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff640114450 con 0x7ff6401054c0 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff646ac0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401142d0 con 0x7ff640105e10 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff646ac0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401145d0 con 0x7ff64010e8f0 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64485c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6401054c0 0x7ff64019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64485c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6401054c0 0x7ff64019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40242/0 (socket says 172.21.15.19:40242) 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64485c700 1 -- 172.21.15.19:0/1277515279 learned_addr learned my addr 172.21.15.19:0/1277515279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff63ffff700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff640105e10 0x7ff64019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64505d700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64010e8f0 0x7ff64019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64485c700 1 -- 172.21.15.19:0/1277515279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64010e8f0 msgr2=0x7ff64019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64485c700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64010e8f0 0x7ff64019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64485c700 1 -- 172.21.15.19:0/1277515279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff640105e10 msgr2=0x7ff64019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64485c700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff640105e10 0x7ff64019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64485c700 1 -- 172.21.15.19:0/1277515279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64019fc30 con 0x7ff6401054c0 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.807+0000 7ff64485c700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6401054c0 0x7ff64019ad60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff63400d980 tx=0x7ff63400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.808+0000 7ff63dffb700 1 -- 172.21.15.19:0/1277515279 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634017030 con 0x7ff6401054c0 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.808+0000 7ff63dffb700 1 -- 172.21.15.19:0/1277515279 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6340044d0 con 0x7ff6401054c0 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.808+0000 7ff646ac0700 1 -- 172.21.15.19:0/1277515279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64019ff20 con 0x7ff6401054c0 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.808+0000 7ff63dffb700 1 -- 172.21.15.19:0/1277515279 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634003970 con 0x7ff6401054c0 2024-05-09T01:37:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.808+0000 7ff646ac0700 1 -- 172.21.15.19:0/1277515279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6401a7840 con 0x7ff6401054c0 2024-05-09T01:37:31.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.811+0000 7ff63dffb700 1 -- 172.21.15.19:0/1277515279 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff63401a070 con 0x7ff6401054c0 2024-05-09T01:37:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.811+0000 7ff63dffb700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6300b2880 0x7ff6300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.811+0000 7ff63ffff700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6300b2880 0x7ff6300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.811+0000 7ff63dffb700 1 -- 172.21.15.19:0/1277515279 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff6340e1ae0 con 0x7ff6401054c0 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff6554d6b80 1 Processor -- start 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff6554d6b80 1 -- start start 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff63ffff700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6300b2880 0x7ff6300b4d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff640102230 tx=0x7ff62c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b2430e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 0x556b2430ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 0x556b24308ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556b2422aff0 con 0x556b24275160 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556b241c97f0 con 0x556b243086a0 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556b24226fb0 con 0x556b2418b4a0 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff64485c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b2430e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff64485c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b2430e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40248/0 (socket says 172.21.15.19:40248) 2024-05-09T01:37:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff64485c700 1 -- 172.21.15.19:0/851814339 learned_addr learned my addr 172.21.15.19:0/851814339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff63ffff700 1 --2- 172.21.15.19:0/851814339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 0x556b2430ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.812+0000 7ff64505d700 1 --2- 172.21.15.19:0/851814339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 0x556b24308ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.813+0000 7ff64485c700 1 -- 172.21.15.19:0/851814339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 msgr2=0x556b2430ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.813+0000 7ff64485c700 1 --2- 172.21.15.19:0/851814339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 0x556b2430ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.813+0000 7ff64485c700 1 -- 172.21.15.19:0/851814339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 msgr2=0x556b24308ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.813+0000 7ff64485c700 1 --2- 172.21.15.19:0/851814339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 0x556b24308ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.813+0000 7ff64485c700 1 -- 172.21.15.19:0/851814339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b241c1370 con 0x556b24275160 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.813+0000 7ff64485c700 1 --2- 172.21.15.19:0/851814339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b2430e4f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff6340a5160 tx=0x7ff63400fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50de542f2559bc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.813+0000 7ff62affd700 1 -- 172.21.15.19:0/851814339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6340a90e0 con 0x556b24275160 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.813+0000 7ff62affd700 1 -- 172.21.15.19:0/851814339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6340ad490 con 0x556b24275160 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.813+0000 7ff62affd700 1 -- 172.21.15.19:0/851814339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6340b1ab0 con 0x556b24275160 2024-05-09T01:37:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 -- 172.21.15.19:0/851814339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 msgr2=0x556b2430e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/851814339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b2430e4f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff6340a5160 tx=0x7ff63400fd80 comp rx=0 tx=0).stop 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 -- 172.21.15.19:0/851814339 shutdown_connections 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/851814339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 0x556b24308ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/851814339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b2430e4f0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/851814339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 0x556b2430ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 -- 172.21.15.19:0/851814339 >> 172.21.15.19:0/851814339 conn(0x556b241eb6a0 msgr2=0x556b24303400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 -- 172.21.15.19:0/851814339 shutdown_connections 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 -- 172.21.15.19:0/851814339 wait complete. 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.814+0000 7ff6554d6b80 1 -- start start 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff6554d6b80 1 Processor -- start 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2418b4a0 0x556b24312230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b24312770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b243086a0 0x556b24312cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556b240a6ba0 con 0x556b24275160 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556b241e5150 con 0x556b2418b4a0 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556b243151d0 con 0x556b243086a0 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff6554d6b80 10 client.0 _ll_register_callbacks cb 0x556b2409c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff63ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b24312770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff63ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b24312770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40264/0 (socket says 172.21.15.19:40264) 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff63ffff700 1 -- 172.21.15.19:0/3825112912 learned_addr learned my addr 172.21.15.19:0/3825112912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff64485c700 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2418b4a0 0x556b24312230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.815+0000 7ff64505d700 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b243086a0 0x556b24312cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff63ffff700 1 -- 172.21.15.19:0/3825112912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b243086a0 msgr2=0x556b24312cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff63ffff700 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b243086a0 0x556b24312cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff63ffff700 1 -- 172.21.15.19:0/3825112912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2418b4a0 msgr2=0x556b24312230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff63ffff700 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2418b4a0 0x556b24312230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff63ffff700 1 -- 172.21.15.19:0/3825112912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b240a6490 con 0x556b24275160 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff63ffff700 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b24312770 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7ff62c011fd0 tx=0x7ff62c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6574509a58dfcad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff61ffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff61ffff700 1 -- 172.21.15.19:0/3825112912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c02c060 con 0x556b24275160 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff61ffff700 1 -- 172.21.15.19:0/3825112912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff62c009040 con 0x556b24275160 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff6554d6b80 1 -- 172.21.15.19:0/3825112912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6340ad490 con 0x556b24275160 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff61ffff700 1 -- 172.21.15.19:0/3825112912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff62c01d6c0 con 0x556b24275160 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff6554d6b80 10 client.18678 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.816+0000 7ff6554d6b80 1 -- 172.21.15.19:0/3825112912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b2431fc70 con 0x556b24275160 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.817+0000 7ff61ffff700 1 -- 172.21.15.19:0/3825112912 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6340ad490 con 0x556b24275160 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.817+0000 7ff61ffff700 1 -- 172.21.15.19:0/3825112912 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff62c027990 con 0x556b24275160 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.817+0000 7ff6554d6b80 10 client.18678 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.817+0000 7ff6554d6b80 10 client.18678 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:37:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.817+0000 7ff6554d6b80 20 client.18678 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.817+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556b243222b0 0x556b243246d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.817+0000 7ff6554d6b80 4 client.18678 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.817+0000 7ff6554d6b80 1 -- 172.21.15.19:0/3825112912 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b242389d0 con 0x556b243222b0 2024-05-09T01:37:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.817+0000 7ff64485c700 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556b243222b0 0x556b243246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.818+0000 7ff64485c700 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556b243222b0 0x556b243246d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4e2a6893cbc74d8f server_cookie=23c7aa6d03edb282 in_seq=0 out_seq=0 2024-05-09T01:37:31.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.818+0000 7ff61ffff700 10 client.18678 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.818+0000 7ff61ffff700 1 -- 172.21.15.19:0/3825112912 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b242389d0 con 0x556b243222b0 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.818+0000 7ff61ffff700 10 client.18678 handle_command_reply: tid=0 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.818+0000 7ff6554d6b80 1 client.18678 shutdown 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 -- 172.21.15.19:0/3825112912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 msgr2=0x556b24312770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b24312770 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7ff62c011fd0 tx=0x7ff62c0047c0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 -- 172.21.15.19:0/3825112912 shutdown_connections 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556b243222b0 0x556b243246d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b243086a0 0x556b24312cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2418b4a0 0x556b24312230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/3825112912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b24275160 0x556b24312770 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff61ffff700 0 client.18678 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 -- 172.21.15.19:0/3825112912 >> 172.21.15.19:0/3825112912 conn(0x556b241eb6a0 msgr2=0x556b243031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 -- 172.21.15.19:0/3825112912 shutdown_connections 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 1 -- 172.21.15.19:0/3825112912 wait complete. 2024-05-09T01:37:31.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.819+0000 7ff6554d6b80 20 client.18678 trim_cache size 0 max 16384 2024-05-09T01:37:31.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.834+0000 7ff6554d6b80 1 Processor -- start 2024-05-09T01:37:31.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.834+0000 7ff6554d6b80 1 -- start start 2024-05-09T01:37:31.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.834+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2418b4a0 0x556b2431df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.834+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b24275160 0x556b2431e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.834+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 0x556b2430d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.834+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401aeda0 con 0x556b2418b4a0 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.834+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401b08e0 con 0x556b243086a0 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.834+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401af470 con 0x556b24275160 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64485c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2418b4a0 0x556b2431df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64485c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2418b4a0 0x556b2431df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40278/0 (socket says 172.21.15.19:40278) 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64485c700 1 -- 172.21.15.19:0/2227640235 learned_addr learned my addr 172.21.15.19:0/2227640235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64505d700 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 0x556b2430d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff63ffff700 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b24275160 0x556b2431e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64485c700 1 -- 172.21.15.19:0/2227640235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b24275160 msgr2=0x556b2431e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64485c700 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b24275160 0x556b2431e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64485c700 1 -- 172.21.15.19:0/2227640235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 msgr2=0x556b2430d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64485c700 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 0x556b2430d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64485c700 1 -- 172.21.15.19:0/2227640235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b24312000 con 0x556b2418b4a0 2024-05-09T01:37:31.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff63ffff700 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b24275160 0x556b2431e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:31.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff64485c700 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2418b4a0 0x556b2431df80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff6340a8de0 tx=0x7ff6340b2060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad2e7a72d820a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:31.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff62a7fc700 1 -- 172.21.15.19:0/2227640235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63400f690 con 0x556b2418b4a0 2024-05-09T01:37:31.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff62a7fc700 1 -- 172.21.15.19:0/2227640235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6340c2070 con 0x556b2418b4a0 2024-05-09T01:37:31.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.835+0000 7ff62a7fc700 1 -- 172.21.15.19:0/2227640235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6340b1d10 con 0x556b2418b4a0 2024-05-09T01:37:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.836+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2227640235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2418b4a0 msgr2=0x556b2431df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.836+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2418b4a0 0x556b2431df80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff6340a8de0 tx=0x7ff6340b2060 comp rx=0 tx=0).stop 2024-05-09T01:37:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.836+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2227640235 shutdown_connections 2024-05-09T01:37:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.836+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b243086a0 0x556b2430d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.836+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2418b4a0 0x556b2431df80 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.836+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2227640235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b24275160 0x556b2431e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.836+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2227640235 >> 172.21.15.19:0/2227640235 conn(0x556b241eb6a0 msgr2=0x556b24310b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.836+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2227640235 shutdown_connections 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.836+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2227640235 wait complete. 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.837+0000 7ff6554d6b80 1 -- start start 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.837+0000 7ff6554d6b80 1 Processor -- start 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.837+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 0x556b2431a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.837+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b24275160 0x556b2431a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.837+0000 7ff6554d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b243086a0 0x556b24328e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.837+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401b0210 con 0x556b243086a0 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.837+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401afb40 con 0x556b24275160 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.837+0000 7ff6554d6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401acd40 con 0x556b2418b4a0 2024-05-09T01:37:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.837+0000 7ff6554d6b80 10 client.0 _ll_register_callbacks cb 0x556b2409c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64505d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b243086a0 0x556b24328e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64505d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b243086a0 0x556b24328e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40290/0 (socket says 172.21.15.19:40290) 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64505d700 1 -- 172.21.15.19:0/2507648232 learned_addr learned my addr 172.21.15.19:0/2507648232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64485c700 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 0x556b2431a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff63ffff700 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b24275160 0x556b2431a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64505d700 1 -- 172.21.15.19:0/2507648232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 msgr2=0x556b2431a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64505d700 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 0x556b2431a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64505d700 1 -- 172.21.15.19:0/2507648232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b24275160 msgr2=0x556b2431a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64505d700 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b24275160 0x556b2431a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64505d700 1 -- 172.21.15.19:0/2507648232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b243293a0 con 0x556b243086a0 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff64505d700 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b243086a0 0x556b24328e60 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b0a0 tx=0x7ff638003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1930196864e82d32 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff61ffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff61ffff700 1 -- 172.21.15.19:0/2507648232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63802b060 con 0x556b243086a0 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2507648232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6340c2070 con 0x556b243086a0 2024-05-09T01:37:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff61ffff700 1 -- 172.21.15.19:0/2507648232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff6380075c0 con 0x556b243086a0 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff61ffff700 1 -- 172.21.15.19:0/2507648232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6380077c0 con 0x556b243086a0 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff6554d6b80 10 client.18684 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.838+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2507648232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b2438c230 con 0x556b243086a0 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.839+0000 7ff61ffff700 1 -- 172.21.15.19:0/2507648232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6340c2070 con 0x556b243086a0 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.839+0000 7ff61ffff700 1 -- 172.21.15.19:0/2507648232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff638025070 con 0x556b243086a0 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.839+0000 7ff6554d6b80 10 client.18684 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.839+0000 7ff6554d6b80 10 client.18684 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.839+0000 7ff6554d6b80 20 client.18684 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.839+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556b2438c920 0x556b2438ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.839+0000 7ff6554d6b80 4 client.18684 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.839+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2507648232 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b24232a70 con 0x556b2438c920 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.839+0000 7ff64485c700 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556b2438c920 0x556b2438ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff64485c700 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556b2438c920 0x556b2438ed40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e1d99da1a494eefc server_cookie=2a6c76d6442a8aac in_seq=0 out_seq=0 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff61ffff700 10 client.18684 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff61ffff700 1 -- 172.21.15.19:0/2507648232 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b24232a70 con 0x556b2438c920 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff61ffff700 10 client.18684 handle_command_reply: tid=0 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff6554d6b80 1 client.18684 shutdown 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2507648232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b243086a0 msgr2=0x556b24328e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b243086a0 0x556b24328e60 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b0a0 tx=0x7ff638003c30 comp rx=0 tx=0).stop 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2507648232 shutdown_connections 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556b2438c920 0x556b2438ed40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b243086a0 0x556b24328e60 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2418b4a0 0x556b2431a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff6554d6b80 1 --2- 172.21.15.19:0/2507648232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b24275160 0x556b2431a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff61ffff700 0 client.18684 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:37:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.840+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2507648232 >> 172.21.15.19:0/2507648232 conn(0x556b241eb6a0 msgr2=0x556b2430be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.841+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2507648232 shutdown_connections 2024-05-09T01:37:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.841+0000 7ff6554d6b80 1 -- 172.21.15.19:0/2507648232 wait complete. 2024-05-09T01:37:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.841+0000 7ff6554d6b80 20 client.18684 trim_cache size 0 max 16384 2024-05-09T01:37:31.842 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:37:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.841+0000 7ff646ac0700 1 -- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6300b2880 msgr2=0x7ff6300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.841+0000 7ff646ac0700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6300b2880 0x7ff6300b4d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff640102230 tx=0x7ff62c009260 comp rx=0 tx=0).stop 2024-05-09T01:37:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.841+0000 7ff646ac0700 1 -- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6401054c0 msgr2=0x7ff64019ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.841+0000 7ff646ac0700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6401054c0 0x7ff64019ad60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff63400d980 tx=0x7ff63400dc90 comp rx=0 tx=0).stop 2024-05-09T01:37:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.842+0000 7ff646ac0700 1 -- 172.21.15.19:0/1277515279 shutdown_connections 2024-05-09T01:37:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.842+0000 7ff646ac0700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64010e8f0 0x7ff64019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.842+0000 7ff646ac0700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7ff6300b2880 0x7ff6300b4d90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.842+0000 7ff646ac0700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6401054c0 0x7ff64019ad60 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.842+0000 7ff646ac0700 1 --2- 172.21.15.19:0/1277515279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff640105e10 0x7ff64019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:31.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.842+0000 7ff646ac0700 1 -- 172.21.15.19:0/1277515279 >> 172.21.15.19:0/1277515279 conn(0x7ff6400fedf0 msgr2=0x7ff6400ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:31.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.842+0000 7ff646ac0700 1 -- 172.21.15.19:0/1277515279 shutdown_connections 2024-05-09T01:37:31.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:31.842+0000 7ff646ac0700 1 -- 172.21.15.19:0/1277515279 wait complete. 2024-05-09T01:37:32.597 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:37:32.744 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:32 smithi196 ceph-mon[35041]: pgmap v738: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:37:32.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:32 smithi059 ceph-mon[34351]: pgmap v738: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:37:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:37:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:32 smithi019 ceph-mon[30158]: pgmap v738: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:37:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.997+0000 7fe661b5f640 1 -- 172.21.15.19:0/2069307306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c103dc0 msgr2=0x7fe65c1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.997+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2069307306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c103dc0 0x7fe65c1041d0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe644009900 tx=0x7fe64402a410 comp rx=0 tx=0).stop 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.997+0000 7fe661b5f640 1 -- 172.21.15.19:0/2069307306 shutdown_connections 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.997+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2069307306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c108ac0 0x7fe65c10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.997+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2069307306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c103dc0 0x7fe65c1041d0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.997+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2069307306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c102b80 0x7fe65c103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.997+0000 7fe661b5f640 1 -- 172.21.15.19:0/2069307306 >> 172.21.15.19:0/2069307306 conn(0x7fe65c0fdd90 msgr2=0x7fe65c100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.997+0000 7fe661b5f640 1 -- 172.21.15.19:0/2069307306 shutdown_connections 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.997+0000 7fe661b5f640 1 -- 172.21.15.19:0/2069307306 wait complete. 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.998+0000 7fe661b5f640 1 Processor -- start 2024-05-09T01:37:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.998+0000 7fe661b5f640 1 -- start start 2024-05-09T01:37:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.998+0000 7fe661b5f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c102b80 0x7fe65c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.998+0000 7fe661b5f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c103dc0 0x7fe65c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.998+0000 7fe661b5f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c108ac0 0x7fe65c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.998+0000 7fe661b5f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c10e680 con 0x7fe65c108ac0 2024-05-09T01:37:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.998+0000 7fe661b5f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c10e500 con 0x7fe65c102b80 2024-05-09T01:37:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.998+0000 7fe661b5f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c10e800 con 0x7fe65c103dc0 2024-05-09T01:37:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c108ac0 0x7fe65c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c108ac0 0x7fe65c1a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40318/0 (socket says 172.21.15.19:40318) 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65bfff640 1 -- 172.21.15.19:0/2489911877 learned_addr learned my addr 172.21.15.19:0/2489911877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe652ffd640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c103dc0 0x7fe65c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c102b80 0x7fe65c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65bfff640 1 -- 172.21.15.19:0/2489911877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c103dc0 msgr2=0x7fe65c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65bfff640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c103dc0 0x7fe65c19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65bfff640 1 -- 172.21.15.19:0/2489911877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c102b80 msgr2=0x7fe65c19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65bfff640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c102b80 0x7fe65c19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65bfff640 1 -- 172.21.15.19:0/2489911877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c1a1e60 con 0x7fe65c108ac0 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c102b80 0x7fe65c19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe65bfff640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c108ac0 0x7fe65c1a17f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe65400eaa0 tx=0x7fe65400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe6597fa640 1 -- 172.21.15.19:0/2489911877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65400cdb0 con 0x7fe65c108ac0 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe6597fa640 1 -- 172.21.15.19:0/2489911877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe654005020 con 0x7fe65c108ac0 2024-05-09T01:37:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe661b5f640 1 -- 172.21.15.19:0/2489911877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c1a2150 con 0x7fe65c108ac0 2024-05-09T01:37:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe6597fa640 1 -- 172.21.15.19:0/2489911877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540116d0 con 0x7fe65c108ac0 2024-05-09T01:37:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:32.999+0000 7fe661b5f640 1 -- 172.21.15.19:0/2489911877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c1a2650 con 0x7fe65c108ac0 2024-05-09T01:37:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe668a0b740 1 Processor -- start 2024-05-09T01:37:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe6597fa640 1 -- 172.21.15.19:0/2489911877 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe6540051d0 con 0x7fe65c108ac0 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe668a0b740 1 -- start start 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e012f0 0x5648361cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 0x5648361d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe668a0b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5648361327c0 con 0x564835e012f0 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe668a0b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5648360ce0e0 con 0x564835e094d0 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe668a0b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56483612e780 con 0x564835e2a1b0 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e012f0 0x5648361cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe65b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e012f0 0x5648361cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40326/0 (socket says 172.21.15.19:40326) 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.002+0000 7fe65b7fe640 1 -- 172.21.15.19:0/658412762 learned_addr learned my addr 172.21.15.19:0/658412762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe652ffd640 1 --2- 172.21.15.19:0/658412762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe652ffd640 1 --2- 172.21.15.19:0/658412762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 0x5648361d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe65b7fe640 1 -- 172.21.15.19:0/658412762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 msgr2=0x5648361ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/658412762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe65b7fe640 1 -- 172.21.15.19:0/658412762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 msgr2=0x5648361d5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/658412762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 0x5648361d5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe65b7fe640 1 -- 172.21.15.19:0/658412762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648360f77c0 con 0x564835e012f0 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/658412762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e012f0 0x5648361cea80 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009c00 tx=0x7fe64c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52123bf80df69878 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe6597fa640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6480b27e0 0x7fe6480b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe651ffb640 1 -- 172.21.15.19:0/658412762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c016070 con 0x564835e012f0 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe652ffd640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6480b27e0 0x7fe6480b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe651ffb640 1 -- 172.21.15.19:0/658412762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe64c0054d0 con 0x564835e012f0 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe651ffb640 1 -- 172.21.15.19:0/658412762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c011470 con 0x564835e012f0 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.003+0000 7fe6597fa640 1 -- 172.21.15.19:0/2489911877 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe6540e24c0 con 0x7fe65c108ac0 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe668a0b740 1 -- 172.21.15.19:0/658412762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e012f0 msgr2=0x5648361cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe652ffd640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6480b27e0 0x7fe6480b4cf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe644000c00 tx=0x7fe644031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe668a0b740 1 --2- 172.21.15.19:0/658412762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e012f0 0x5648361cea80 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009c00 tx=0x7fe64c00e6c0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe668a0b740 1 -- 172.21.15.19:0/658412762 shutdown_connections 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe668a0b740 1 --2- 172.21.15.19:0/658412762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 0x5648361d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe668a0b740 1 --2- 172.21.15.19:0/658412762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e012f0 0x5648361cea80 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe668a0b740 1 --2- 172.21.15.19:0/658412762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe668a0b740 1 -- 172.21.15.19:0/658412762 >> 172.21.15.19:0/658412762 conn(0x564835e1e300 msgr2=0x5648361c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe668a0b740 1 -- 172.21.15.19:0/658412762 shutdown_connections 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.004+0000 7fe668a0b740 1 -- 172.21.15.19:0/658412762 wait complete. 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe668a0b740 1 -- start start 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe668a0b740 1 Processor -- start 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 0x5648361d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361d8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe668a0b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5648361d63d0 con 0x564835e094d0 2024-05-09T01:37:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe668a0b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5648360b99c0 con 0x564835e012f0 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe668a0b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648361d6550 con 0x564835e2a1b0 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe668a0b740 10 client.0 _ll_register_callbacks cb 0x5648360e6020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe652ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe652ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361d8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40328/0 (socket says 172.21.15.19:40328) 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe652ffd640 1 -- 172.21.15.19:0/1363007444 learned_addr learned my addr 172.21.15.19:0/1363007444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 0x5648361d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.005+0000 7fe65bfff640 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe652ffd640 1 -- 172.21.15.19:0/1363007444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 msgr2=0x5648361df660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe652ffd640 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361df660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe652ffd640 1 -- 172.21.15.19:0/1363007444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 msgr2=0x5648361d8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe652ffd640 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 0x5648361d8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe652ffd640 1 -- 172.21.15.19:0/1363007444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648360e01f0 con 0x564835e094d0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe652ffd640 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361d8770 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fe644009900 tx=0x7fe644005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2764fdbf910b0390 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe63affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe63affd640 1 -- 172.21.15.19:0/1363007444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64404d070 con 0x564835e094d0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe63affd640 1 -- 172.21.15.19:0/1363007444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe644004460 con 0x564835e094d0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe668a0b740 1 -- 172.21.15.19:0/1363007444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe64c0054d0 con 0x564835e094d0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe63affd640 1 -- 172.21.15.19:0/1363007444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe644059050 con 0x564835e094d0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe668a0b740 10 client.18696 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe668a0b740 1 -- 172.21.15.19:0/1363007444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564835e95790 con 0x564835e094d0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.006+0000 7fe63affd640 1 -- 172.21.15.19:0/1363007444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe64c0054d0 con 0x564835e094d0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe63affd640 1 -- 172.21.15.19:0/1363007444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe64403a040 con 0x564835e094d0 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe668a0b740 10 client.18696 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe668a0b740 10 client.18696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe668a0b740 20 client.18696 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe668a0b740 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564835de8d80 0x5648361e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe668a0b740 4 client.18696 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe668a0b740 1 -- 172.21.15.19:0/1363007444 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648361403d0 con 0x564835de8d80 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564835de8d80 0x5648361e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564835de8d80 0x5648361e5b70 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3726218f7432ddd9 server_cookie=c896ff58c0537753 in_seq=0 out_seq=0 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.007+0000 7fe63affd640 10 client.18696 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.008+0000 7fe63affd640 1 -- 172.21.15.19:0/1363007444 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5648361403d0 con 0x564835de8d80 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.008+0000 7fe63affd640 10 client.18696 handle_command_reply: tid=0 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.008+0000 7fe668a0b740 1 client.18696 shutdown 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 -- 172.21.15.19:0/1363007444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 msgr2=0x5648361d8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361d8770 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fe644009900 tx=0x7fe644005a20 comp rx=0 tx=0).stop 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 -- 172.21.15.19:0/1363007444 shutdown_connections 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564835de8d80 0x5648361e5b70 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361df660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361d8770 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe63affd640 0 client.18696 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 --2- 172.21.15.19:0/1363007444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 0x5648361d8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 -- 172.21.15.19:0/1363007444 >> 172.21.15.19:0/1363007444 conn(0x564835e1e300 msgr2=0x5648361c8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 -- 172.21.15.19:0/1363007444 shutdown_connections 2024-05-09T01:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 1 -- 172.21.15.19:0/1363007444 wait complete. 2024-05-09T01:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.009+0000 7fe668a0b740 20 client.18696 trim_cache size 0 max 16384 2024-05-09T01:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe668a0b740 1 Processor -- start 2024-05-09T01:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe668a0b740 1 -- start start 2024-05-09T01:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 0x5648361df970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe668a0b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1ab430 con 0x564835e094d0 2024-05-09T01:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe668a0b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1acf70 con 0x564835e012f0 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe668a0b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1abb00 con 0x564835e2a1b0 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe652ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe652ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40334/0 (socket says 172.21.15.19:40334) 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe652ffd640 1 -- 172.21.15.19:0/691050546 learned_addr learned my addr 172.21.15.19:0/691050546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 0x5648361df970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe65bfff640 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe652ffd640 1 -- 172.21.15.19:0/691050546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 msgr2=0x5648361cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe652ffd640 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe652ffd640 1 -- 172.21.15.19:0/691050546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 msgr2=0x5648361df970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe652ffd640 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 0x5648361df970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe652ffd640 1 -- 172.21.15.19:0/691050546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564835f0c5f0 con 0x564835e094d0 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe65bfff640 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 0x5648361df970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.024+0000 7fe652ffd640 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361cd3a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fe64402a930 tx=0x7fe6440055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb5ae2658a37260 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe6517fa640 1 -- 172.21.15.19:0/691050546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644062070 con 0x564835e094d0 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe6517fa640 1 -- 172.21.15.19:0/691050546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6440035f0 con 0x564835e094d0 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe6517fa640 1 -- 172.21.15.19:0/691050546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64404d040 con 0x564835e094d0 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe668a0b740 1 -- 172.21.15.19:0/691050546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 msgr2=0x5648361cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe668a0b740 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361cd3a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fe64402a930 tx=0x7fe6440055c0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe668a0b740 1 -- 172.21.15.19:0/691050546 shutdown_connections 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe668a0b740 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e2a1b0 0x5648361cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe668a0b740 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e094d0 0x5648361cd3a0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe668a0b740 1 --2- 172.21.15.19:0/691050546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e012f0 0x5648361df970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe668a0b740 1 -- 172.21.15.19:0/691050546 >> 172.21.15.19:0/691050546 conn(0x564835e1e300 msgr2=0x5648361ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe668a0b740 1 -- 172.21.15.19:0/691050546 shutdown_connections 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.025+0000 7fe668a0b740 1 -- 172.21.15.19:0/691050546 wait complete. 2024-05-09T01:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe668a0b740 1 -- start start 2024-05-09T01:37:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe668a0b740 1 Processor -- start 2024-05-09T01:37:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e012f0 0x564836249020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 0x564836249560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe668a0b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e2a1b0 0x5648362504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe668a0b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1ac8a0 con 0x564835e2a1b0 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe668a0b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1ac1d0 con 0x564835e094d0 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe668a0b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1a9410 con 0x564835e012f0 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe668a0b740 10 client.0 _ll_register_callbacks cb 0x7fe644009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.026+0000 7fe65bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e2a1b0 0x5648362504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e2a1b0 0x5648362504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40336/0 (socket says 172.21.15.19:40336) 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65bfff640 1 -- 172.21.15.19:0/3667425999 learned_addr learned my addr 172.21.15.19:0/3667425999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e012f0 0x564836249020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe652ffd640 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 0x564836249560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65bfff640 1 -- 172.21.15.19:0/3667425999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e012f0 msgr2=0x564836249020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65bfff640 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e012f0 0x564836249020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65bfff640 1 -- 172.21.15.19:0/3667425999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 msgr2=0x564836249560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65bfff640 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 0x564836249560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65bfff640 1 -- 172.21.15.19:0/3667425999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648360f6ed0 con 0x564835e2a1b0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e012f0 0x564836249020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe65bfff640 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e2a1b0 0x5648362504c0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe6540a5af0 tx=0x7fe6540aa1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7d65f1c76c67f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe63affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe63affd640 1 -- 172.21.15.19:0/3667425999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540c3060 con 0x564835e2a1b0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe63affd640 1 -- 172.21.15.19:0/3667425999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fe6540054f0 con 0x564835e2a1b0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe668a0b740 1 -- 172.21.15.19:0/3667425999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6440035f0 con 0x564835e2a1b0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe63affd640 1 -- 172.21.15.19:0/3667425999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe65401f030 con 0x564835e2a1b0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe668a0b740 10 client.18702 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.027+0000 7fe668a0b740 1 -- 172.21.15.19:0/3667425999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564836251af0 con 0x564835e2a1b0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.028+0000 7fe63affd640 1 -- 172.21.15.19:0/3667425999 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6440035f0 con 0x564835e2a1b0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.028+0000 7fe63affd640 1 -- 172.21.15.19:0/3667425999 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe65400f040 con 0x564835e2a1b0 2024-05-09T01:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.028+0000 7fe668a0b740 10 client.18702 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.028+0000 7fe668a0b740 10 client.18702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.028+0000 7fe668a0b740 20 client.18702 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.028+0000 7fe668a0b740 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564835dbe1c0 0x5648362544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.028+0000 7fe668a0b740 4 client.18702 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.028+0000 7fe668a0b740 1 -- 172.21.15.19:0/3667425999 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56483613a240 con 0x564835dbe1c0 2024-05-09T01:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.028+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564835dbe1c0 0x5648362544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.029+0000 7fe63affd640 10 client.18702 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.029+0000 7fe65b7fe640 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564835dbe1c0 0x5648362544e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1d59141311409fc server_cookie=6efd25039f1f379a in_seq=0 out_seq=0 2024-05-09T01:37:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe63affd640 1 -- 172.21.15.19:0/3667425999 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56483613a240 con 0x564835dbe1c0 2024-05-09T01:37:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe63affd640 10 client.18702 handle_command_reply: tid=0 2024-05-09T01:37:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 client.18702 shutdown 2024-05-09T01:37:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 -- 172.21.15.19:0/3667425999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e2a1b0 msgr2=0x5648362504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e2a1b0 0x5648362504c0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe6540a5af0 tx=0x7fe6540aa1c0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 -- 172.21.15.19:0/3667425999 shutdown_connections 2024-05-09T01:37:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x564835dbe1c0 0x5648362544e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564835e2a1b0 0x5648362504c0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564835e094d0 0x564836249560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe63affd640 0 client.18702 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 --2- 172.21.15.19:0/3667425999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564835e012f0 0x564836249020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 -- 172.21.15.19:0/3667425999 >> 172.21.15.19:0/3667425999 conn(0x564835e1e300 msgr2=0x5648361ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 -- 172.21.15.19:0/3667425999 shutdown_connections 2024-05-09T01:37:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 1 -- 172.21.15.19:0/3667425999 wait complete. 2024-05-09T01:37:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.047+0000 7fe668a0b740 20 client.18702 trim_cache size 0 max 16384 2024-05-09T01:37:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-05-09T01:37:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-05-09T01:37:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-05-09T01:37:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 -- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6480b27e0 msgr2=0x7fe6480b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6480b27e0 0x7fe6480b4cf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe644000c00 tx=0x7fe644031400 comp rx=0 tx=0).stop 2024-05-09T01:37:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 -- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c108ac0 msgr2=0x7fe65c1a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c108ac0 0x7fe65c1a17f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe65400eaa0 tx=0x7fe65400ef60 comp rx=0 tx=0).stop 2024-05-09T01:37:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 -- 172.21.15.19:0/2489911877 shutdown_connections 2024-05-09T01:37:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fe6480b27e0 0x7fe6480b4cf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe65c108ac0 0x7fe65c1a17f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c103dc0 0x7fe65c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 --2- 172.21.15.19:0/2489911877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c102b80 0x7fe65c19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 -- 172.21.15.19:0/2489911877 >> 172.21.15.19:0/2489911877 conn(0x7fe65c0fdd90 msgr2=0x7fe65c109f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 -- 172.21.15.19:0/2489911877 shutdown_connections 2024-05-09T01:37:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-05-09T01:37:33.048+0000 7fe661b5f640 1 -- 172.21.15.19:0/2489911877 wait complete. 2024-05-09T01:37:34.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.571+0000 7f3d9afe6700 1 -- 172.21.15.19:0/807977472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94108770 msgr2=0x7f3d941100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:34.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.571+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/807977472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94108770 0x7f3d941100d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00b0d0 tx=0x7f3d8c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.571+0000 7f3d9afe6700 1 -- 172.21.15.19:0/807977472 shutdown_connections 2024-05-09T01:37:34.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.571+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/807977472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94108770 0x7f3d941100d0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.571+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/807977472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d94103540 0x7f3d94103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.571+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/807977472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d94107e20 0x7f3d94108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.571+0000 7f3d9afe6700 1 -- 172.21.15.19:0/807977472 >> 172.21.15.19:0/807977472 conn(0x7f3d94078e50 msgr2=0x7f3d94079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:34.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.571+0000 7f3d9afe6700 1 -- 172.21.15.19:0/807977472 shutdown_connections 2024-05-09T01:37:34.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.571+0000 7f3d9afe6700 1 -- 172.21.15.19:0/807977472 wait complete. 2024-05-09T01:37:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.572+0000 7f3d9afe6700 1 Processor -- start 2024-05-09T01:37:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.572+0000 7f3d9afe6700 1 -- start start 2024-05-09T01:37:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d9afe6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94103540 0x7f3d9419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d9afe6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d94107e20 0x7f3d9419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d9afe6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d94108770 0x7f3d9419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d9afe6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d941133a0 con 0x7f3d94103540 2024-05-09T01:37:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d9afe6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d94113220 con 0x7f3d94108770 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d9afe6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d94113520 con 0x7f3d94107e20 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d98d82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94103540 0x7f3d9419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d98d82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94103540 0x7f3d9419bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40352/0 (socket says 172.21.15.19:40352) 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d98d82700 1 -- 172.21.15.19:0/1696667714 learned_addr learned my addr 172.21.15.19:0/1696667714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d99583700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d94108770 0x7f3d9419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d93fff700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d94107e20 0x7f3d9419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d98d82700 1 -- 172.21.15.19:0/1696667714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d94107e20 msgr2=0x7f3d9419c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d98d82700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d94107e20 0x7f3d9419c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d98d82700 1 -- 172.21.15.19:0/1696667714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d94108770 msgr2=0x7f3d9419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d98d82700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d94108770 0x7f3d9419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d98d82700 1 -- 172.21.15.19:0/1696667714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d941a6ee0 con 0x7f3d94103540 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.573+0000 7f3d93fff700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d94107e20 0x7f3d9419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.574+0000 7f3d98d82700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94103540 0x7f3d9419bc20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400c9b0 tx=0x7f3d8400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.574+0000 7f3d91ffb700 1 -- 172.21.15.19:0/1696667714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84007b00 con 0x7f3d94103540 2024-05-09T01:37:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.574+0000 7f3d91ffb700 1 -- 172.21.15.19:0/1696667714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d84007ca0 con 0x7f3d94103540 2024-05-09T01:37:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.574+0000 7f3d9afe6700 1 -- 172.21.15.19:0/1696667714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d941a71d0 con 0x7f3d94103540 2024-05-09T01:37:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.574+0000 7f3d91ffb700 1 -- 172.21.15.19:0/1696667714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84018660 con 0x7f3d94103540 2024-05-09T01:37:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.574+0000 7f3d9afe6700 1 -- 172.21.15.19:0/1696667714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d941a7710 con 0x7f3d94103540 2024-05-09T01:37:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.576+0000 7f3d91ffb700 1 -- 172.21.15.19:0/1696667714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3d8400f450 con 0x7f3d94103540 2024-05-09T01:37:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.577+0000 7f3d91ffb700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d7c0b2880 0x7f3d7c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.577+0000 7f3d91ffb700 1 -- 172.21.15.19:0/1696667714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3d840e2270 con 0x7f3d94103540 2024-05-09T01:37:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.577+0000 7f3d93fff700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d7c0b2880 0x7f3d7c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3da99fcb80 1 Processor -- start 2024-05-09T01:37:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3da99fcb80 1 -- start start 2024-05-09T01:37:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3d93fff700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d7c0b2880 0x7f3d7c0b4d90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005e10 tx=0x7f3d88005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f29ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191da84a0 0x557191f2a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191f2e1e0 0x557191f2e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557191e480a0 con 0x557191e22d00 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557191ded9a0 con 0x557191f2e1e0 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557191e44060 con 0x557191da84a0 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3d98d82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f29ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3d98d82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f29ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40362/0 (socket says 172.21.15.19:40362) 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3d98d82700 1 -- 172.21.15.19:0/1689451341 learned_addr learned my addr 172.21.15.19:0/1689451341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3d98d82700 1 -- 172.21.15.19:0/1689451341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191da84a0 msgr2=0x557191f2a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3d98d82700 1 --2- 172.21.15.19:0/1689451341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191da84a0 0x557191f2a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3d98d82700 1 -- 172.21.15.19:0/1689451341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191f2e1e0 msgr2=0x557191f2e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.578+0000 7f3d98d82700 1 --2- 172.21.15.19:0/1689451341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191f2e1e0 0x557191f2e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.579+0000 7f3d98d82700 1 -- 172.21.15.19:0/1689451341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557191dde500 con 0x557191e22d00 2024-05-09T01:37:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.579+0000 7f3d98d82700 1 --2- 172.21.15.19:0/1689451341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f29ad0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3d840a97a0 tx=0x7f3d840a91e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e3e4399fad1df0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.579+0000 7f3d7affd700 1 -- 172.21.15.19:0/1689451341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d840c3070 con 0x557191e22d00 2024-05-09T01:37:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.579+0000 7f3d7affd700 1 -- 172.21.15.19:0/1689451341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d840a9f10 con 0x557191e22d00 2024-05-09T01:37:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.579+0000 7f3d7affd700 1 -- 172.21.15.19:0/1689451341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d840a9a70 con 0x557191e22d00 2024-05-09T01:37:34.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 -- 172.21.15.19:0/1689451341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 msgr2=0x557191f29ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:34.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/1689451341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f29ad0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3d840a97a0 tx=0x7f3d840a91e0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 -- 172.21.15.19:0/1689451341 shutdown_connections 2024-05-09T01:37:34.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/1689451341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191f2e1e0 0x557191f2e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/1689451341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f29ad0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/1689451341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191da84a0 0x557191f2a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 -- 172.21.15.19:0/1689451341 >> 172.21.15.19:0/1689451341 conn(0x557191dabf70 msgr2=0x557191f20220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 -- 172.21.15.19:0/1689451341 shutdown_connections 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 -- 172.21.15.19:0/1689451341 wait complete. 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 -- start start 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.580+0000 7f3da99fcb80 1 Processor -- start 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191da84a0 0x557191f33500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191f2e1e0 0x557191f33f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557191f2ef00 con 0x557191e22d00 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557191de2b20 con 0x557191f2e1e0 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557191f36470 con 0x557191da84a0 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3da99fcb80 10 client.0 _ll_register_callbacks cb 0x557191cb96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40378/0 (socket says 172.21.15.19:40378) 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d93fff700 1 -- 172.21.15.19:0/3253558241 learned_addr learned my addr 172.21.15.19:0/3253558241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d99583700 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191f2e1e0 0x557191f33f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d98d82700 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191da84a0 0x557191f33500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d93fff700 1 -- 172.21.15.19:0/3253558241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191da84a0 msgr2=0x557191f33500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d93fff700 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191da84a0 0x557191f33500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d93fff700 1 -- 172.21.15.19:0/3253558241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191f2e1e0 msgr2=0x557191f33f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d93fff700 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191f2e1e0 0x557191f33f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.581+0000 7f3d93fff700 1 -- 172.21.15.19:0/3253558241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557191cc3490 con 0x557191e22d00 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3d93fff700 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33a40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800bfd0 tx=0x7f3d8800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef83dc13b5fb61b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3d73fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3d73fff700 1 -- 172.21.15.19:0/3253558241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88003a90 con 0x557191e22d00 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3d73fff700 1 -- 172.21.15.19:0/3253558241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3d88003c30 con 0x557191e22d00 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3da99fcb80 1 -- 172.21.15.19:0/3253558241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d840a9f10 con 0x557191e22d00 2024-05-09T01:37:34.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3d73fff700 1 -- 172.21.15.19:0/3253558241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d88003e50 con 0x557191e22d00 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3da99fcb80 10 client.18714 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3da99fcb80 1 -- 172.21.15.19:0/3253558241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557191f40ea0 con 0x557191e22d00 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3d73fff700 1 -- 172.21.15.19:0/3253558241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d840a9f10 con 0x557191e22d00 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.582+0000 7f3d73fff700 1 -- 172.21.15.19:0/3253558241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3d88011600 con 0x557191e22d00 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.583+0000 7f3da99fcb80 10 client.18714 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.583+0000 7f3da99fcb80 10 client.18714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.583+0000 7f3da99fcb80 20 client.18714 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.583+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557191f43580 0x557191f459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.583+0000 7f3da99fcb80 4 client.18714 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.583+0000 7f3da99fcb80 1 -- 172.21.15.19:0/3253558241 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557191e55a80 con 0x557191f43580 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.583+0000 7f3d98d82700 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557191f43580 0x557191f459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.583+0000 7f3d98d82700 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557191f43580 0x557191f459a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9429341d69968c8c server_cookie=b29f479aa3b18692 in_seq=0 out_seq=0 2024-05-09T01:37:34.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.583+0000 7f3d73fff700 10 client.18714 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.584+0000 7f3d73fff700 1 -- 172.21.15.19:0/3253558241 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557191e55a80 con 0x557191f43580 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.584+0000 7f3d73fff700 10 client.18714 handle_command_reply: tid=0 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.584+0000 7f3da99fcb80 1 client.18714 shutdown 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.584+0000 7f3da99fcb80 1 -- 172.21.15.19:0/3253558241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 msgr2=0x557191f33a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.584+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33a40 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800bfd0 tx=0x7f3d8800fbb0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.584+0000 7f3da99fcb80 1 -- 172.21.15.19:0/3253558241 shutdown_connections 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.584+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557191f43580 0x557191f459a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.585+0000 7f3d73fff700 0 client.18714 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.585+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191f2e1e0 0x557191f33f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.585+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191da84a0 0x557191f33500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.585+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/3253558241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33a40 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.585+0000 7f3da99fcb80 1 -- 172.21.15.19:0/3253558241 >> 172.21.15.19:0/3253558241 conn(0x557191dabf70 msgr2=0x557191f243f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.585+0000 7f3da99fcb80 1 -- 172.21.15.19:0/3253558241 shutdown_connections 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.585+0000 7f3da99fcb80 1 -- 172.21.15.19:0/3253558241 wait complete. 2024-05-09T01:37:34.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.585+0000 7f3da99fcb80 20 client.18714 trim_cache size 0 max 16384 2024-05-09T01:37:34.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3da99fcb80 1 Processor -- start 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3da99fcb80 1 -- start start 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191da84a0 0x557191f1aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 0x557191f24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d941acb10 con 0x557191e22d00 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d941ae650 con 0x557191da84a0 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d941ad1e0 con 0x557191f2e1e0 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3d93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3d93fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40382/0 (socket says 172.21.15.19:40382) 2024-05-09T01:37:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3d93fff700 1 -- 172.21.15.19:0/2336024170 learned_addr learned my addr 172.21.15.19:0/2336024170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.600+0000 7f3d98d82700 1 --2- 172.21.15.19:0/2336024170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191da84a0 0x557191f1aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d93fff700 1 -- 172.21.15.19:0/2336024170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 msgr2=0x557191f24b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d99583700 1 --2- 172.21.15.19:0/2336024170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 0x557191f24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d93fff700 1 --2- 172.21.15.19:0/2336024170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 0x557191f24b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d93fff700 1 -- 172.21.15.19:0/2336024170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191da84a0 msgr2=0x557191f1aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d93fff700 1 --2- 172.21.15.19:0/2336024170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191da84a0 0x557191f1aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d93fff700 1 -- 172.21.15.19:0/2336024170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557191f1c9d0 con 0x557191e22d00 2024-05-09T01:37:34.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:34 smithi019 ceph-mon[30158]: pgmap v739: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d93fff700 1 --2- 172.21.15.19:0/2336024170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33dc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3d88000c00 tx=0x7f3d88003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ba9f95479d0040 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d7a7fc700 1 -- 172.21.15.19:0/2336024170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88003c30 con 0x557191e22d00 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d7a7fc700 1 -- 172.21.15.19:0/2336024170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d8803b070 con 0x557191e22d00 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.601+0000 7f3d7a7fc700 1 -- 172.21.15.19:0/2336024170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88036b50 con 0x557191e22d00 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 -- 172.21.15.19:0/2336024170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 msgr2=0x557191f33dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/2336024170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33dc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3d88000c00 tx=0x7f3d88003a20 comp rx=0 tx=0).stop 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 -- 172.21.15.19:0/2336024170 shutdown_connections 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/2336024170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 0x557191f24b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/2336024170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191da84a0 0x557191f1aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/2336024170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191e22d00 0x557191f33dc0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 -- 172.21.15.19:0/2336024170 >> 172.21.15.19:0/2336024170 conn(0x557191dabf70 msgr2=0x557191f243f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 -- 172.21.15.19:0/2336024170 shutdown_connections 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 -- 172.21.15.19:0/2336024170 wait complete. 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.602+0000 7f3da99fcb80 1 -- start start 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3da99fcb80 1 Processor -- start 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191da84a0 0x557191f36930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191e22d00 0x557191f36e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3da99fcb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 0x557191fabca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d941adf80 con 0x557191da84a0 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d941ad8b0 con 0x557191e22d00 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3da99fcb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d941aaab0 con 0x557191f2e1e0 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3da99fcb80 10 client.0 _ll_register_callbacks cb 0x557191cb96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3d98d82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191da84a0 0x557191f36930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3d98d82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191da84a0 0x557191f36930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40396/0 (socket says 172.21.15.19:40396) 2024-05-09T01:37:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3d98d82700 1 -- 172.21.15.19:0/4023053508 learned_addr learned my addr 172.21.15.19:0/4023053508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3d93fff700 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191e22d00 0x557191f36e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.603+0000 7f3d99583700 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 0x557191fabca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d98d82700 1 -- 172.21.15.19:0/4023053508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 msgr2=0x557191fabca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d98d82700 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 0x557191fabca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d98d82700 1 -- 172.21.15.19:0/4023053508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191e22d00 msgr2=0x557191f36e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d98d82700 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191e22d00 0x557191f36e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d98d82700 1 -- 172.21.15.19:0/4023053508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557191fac1e0 con 0x557191da84a0 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d98d82700 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191da84a0 0x557191f36930 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3d840a9510 tx=0x7f3d840a5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd0ae065b630709 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d73fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d73fff700 1 -- 172.21.15.19:0/4023053508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84087020 con 0x557191da84a0 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d73fff700 1 -- 172.21.15.19:0/4023053508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3d8400f740 con 0x557191da84a0 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3da99fcb80 1 -- 172.21.15.19:0/4023053508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d8803b070 con 0x557191da84a0 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d73fff700 1 -- 172.21.15.19:0/4023053508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d840c3040 con 0x557191da84a0 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3da99fcb80 10 client.18720 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3da99fcb80 1 -- 172.21.15.19:0/4023053508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557191fad410 con 0x557191da84a0 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.604+0000 7f3d73fff700 1 -- 172.21.15.19:0/4023053508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d8803b070 con 0x557191da84a0 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3d73fff700 1 -- 172.21.15.19:0/4023053508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3d8406b040 con 0x557191da84a0 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3da99fcb80 10 client.18720 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3da99fcb80 10 client.18720 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3da99fcb80 20 client.18720 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557191f43580 0x557191faf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3da99fcb80 4 client.18720 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3da99fcb80 1 -- 172.21.15.19:0/4023053508 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557191e4fb20 con 0x557191f43580 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3d93fff700 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557191f43580 0x557191faf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3d93fff700 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557191f43580 0x557191faf6b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f47e8daa6f16ffb6 server_cookie=966779faed571960 in_seq=0 out_seq=0 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.605+0000 7f3d73fff700 10 client.18720 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3d73fff700 1 -- 172.21.15.19:0/4023053508 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557191e4fb20 con 0x557191f43580 2024-05-09T01:37:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3d73fff700 10 client.18720 handle_command_reply: tid=0 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 client.18720 shutdown 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 -- 172.21.15.19:0/4023053508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191da84a0 msgr2=0x557191f36930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191da84a0 0x557191f36930 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3d840a9510 tx=0x7f3d840a5040 comp rx=0 tx=0).stop 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 -- 172.21.15.19:0/4023053508 shutdown_connections 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557191f2e1e0 0x557191fabca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557191da84a0 0x557191f36930 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x557191f43580 0x557191faf6b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3d73fff700 0 client.18720 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 --2- 172.21.15.19:0/4023053508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557191e22d00 0x557191f36e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 -- 172.21.15.19:0/4023053508 >> 172.21.15.19:0/4023053508 conn(0x557191dabf70 msgr2=0x557191f1a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 -- 172.21.15.19:0/4023053508 shutdown_connections 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 1 -- 172.21.15.19:0/4023053508 wait complete. 2024-05-09T01:37:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.606+0000 7f3da99fcb80 20 client.18720 trim_cache size 0 max 16384 2024-05-09T01:37:34.608 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:37:34.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 -- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d7c0b2880 msgr2=0x7f3d7c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:34.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d7c0b2880 0x7f3d7c0b4d90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005e10 tx=0x7f3d88005d60 comp rx=0 tx=0).stop 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 -- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94103540 msgr2=0x7f3d9419bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94103540 0x7f3d9419bc20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400c9b0 tx=0x7f3d8400ccc0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 -- 172.21.15.19:0/1696667714 shutdown_connections 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d94108770 0x7f3d9419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3d7c0b2880 0x7f3d7c0b4d90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d94103540 0x7f3d9419bc20 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 --2- 172.21.15.19:0/1696667714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d94107e20 0x7f3d9419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 -- 172.21.15.19:0/1696667714 >> 172.21.15.19:0/1696667714 conn(0x7f3d94078e50 msgr2=0x7f3d94106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 -- 172.21.15.19:0/1696667714 shutdown_connections 2024-05-09T01:37:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:34.607+0000 7f3d9afe6700 1 -- 172.21.15.19:0/1696667714 wait complete. 2024-05-09T01:37:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:34 smithi059 ceph-mon[34351]: pgmap v739: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-05-09T01:37:34.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:34 smithi196 ceph-mon[35041]: pgmap v739: 97 pgs: 97 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-05-09T01:37:35.233 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:37:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:37:36.592 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:36 smithi019 ceph-mon[30158]: pgmap v740: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:37:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:36 smithi059 ceph-mon[34351]: pgmap v740: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:37:36.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:36 smithi196 ceph-mon[35041]: pgmap v740: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-05-09T01:37:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 -- 172.21.15.19:0/615921951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc1087a0 msgr2=0x7f81bc110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 --2- 172.21.15.19:0/615921951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc1087a0 0x7f81bc110140 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f81b40099c0 tx=0x7f81b4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 -- 172.21.15.19:0/615921951 shutdown_connections 2024-05-09T01:37:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 --2- 172.21.15.19:0/615921951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc1087a0 0x7f81bc110140 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 --2- 172.21.15.19:0/615921951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc103c50 0x7f81bc104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 --2- 172.21.15.19:0/615921951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81bc107e50 0x7f81bc108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 -- 172.21.15.19:0/615921951 >> 172.21.15.19:0/615921951 conn(0x7f81bc0fe540 msgr2=0x7f81bc100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 -- 172.21.15.19:0/615921951 shutdown_connections 2024-05-09T01:37:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 -- 172.21.15.19:0/615921951 wait complete. 2024-05-09T01:37:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.139+0000 7f81c47a4700 1 Processor -- start 2024-05-09T01:37:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c47a4700 1 -- start start 2024-05-09T01:37:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c47a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc103c50 0x7f81bc19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c47a4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81bc107e50 0x7f81bc19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c47a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc1087a0 0x7f81bc19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c47a4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc113480 con 0x7f81bc103c50 2024-05-09T01:37:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c47a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc113300 con 0x7f81bc1087a0 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c47a4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc113600 con 0x7f81bc107e50 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2540700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc103c50 0x7f81bc19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2540700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc103c50 0x7f81bc19bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40416/0 (socket says 172.21.15.19:40416) 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2540700 1 -- 172.21.15.19:0/3026833706 learned_addr learned my addr 172.21.15.19:0/3026833706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81bc107e50 0x7f81bc19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2d41700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc1087a0 0x7f81bc19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2540700 1 -- 172.21.15.19:0/3026833706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81bc107e50 msgr2=0x7f81bc19c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2540700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81bc107e50 0x7f81bc19c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2540700 1 -- 172.21.15.19:0/3026833706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc1087a0 msgr2=0x7f81bc19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2540700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc1087a0 0x7f81bc19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2540700 1 -- 172.21.15.19:0/3026833706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc199af0 con 0x7f81bc103c50 2024-05-09T01:37:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81bc107e50 0x7f81bc19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.140+0000 7f81c2d41700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc1087a0 0x7f81bc19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.141+0000 7f81c2540700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc103c50 0x7f81bc19bd00 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0060b0 tx=0x7f81ac007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.141+0000 7f81bb7fe700 1 -- 172.21.15.19:0/3026833706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac01a070 con 0x7f81bc103c50 2024-05-09T01:37:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.141+0000 7f81c47a4700 1 -- 172.21.15.19:0/3026833706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81bc199de0 con 0x7f81bc103c50 2024-05-09T01:37:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.141+0000 7f81bb7fe700 1 -- 172.21.15.19:0/3026833706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac00de20 con 0x7f81bc103c50 2024-05-09T01:37:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.141+0000 7f81bb7fe700 1 -- 172.21.15.19:0/3026833706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac014720 con 0x7f81bc103c50 2024-05-09T01:37:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.142+0000 7f81c47a4700 1 -- 172.21.15.19:0/3026833706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81bc19a340 con 0x7f81bc103c50 2024-05-09T01:37:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.144+0000 7f81bb7fe700 1 -- 172.21.15.19:0/3026833706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f81ac01d070 con 0x7f81bc103c50 2024-05-09T01:37:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.144+0000 7f81bb7fe700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f81a40b2880 0x7f81a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.145+0000 7f81bb7fe700 1 -- 172.21.15.19:0/3026833706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f81ac0e1b50 con 0x7f81bc103c50 2024-05-09T01:37:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.145+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f81a40b2880 0x7f81a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.145+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f81a40b2880 0x7f81a40b4d90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f81bc106720 tx=0x7f81b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:37.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81d31bab80 1 Processor -- start 2024-05-09T01:37:37.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81d31bab80 1 -- start start 2024-05-09T01:37:37.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 0x5645d37404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d35bd4a0 0x5645d3740a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d373aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81d31bab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5645d365cff0 con 0x5645d373a6a0 2024-05-09T01:37:37.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81d31bab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5645d35fb7f0 con 0x5645d36a7160 2024-05-09T01:37:37.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81d31bab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5645d3658fb0 con 0x5645d35bd4a0 2024-05-09T01:37:37.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81c2d41700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d373aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81c2d41700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d373aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40424/0 (socket says 172.21.15.19:40424) 2024-05-09T01:37:37.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.146+0000 7f81c2d41700 1 -- 172.21.15.19:0/3759849594 learned_addr learned my addr 172.21.15.19:0/3759849594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/3759849594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d35bd4a0 0x5645d3740a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81c2540700 1 --2- 172.21.15.19:0/3759849594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 0x5645d37404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81c2d41700 1 -- 172.21.15.19:0/3759849594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d35bd4a0 msgr2=0x5645d3740a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81c2d41700 1 --2- 172.21.15.19:0/3759849594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d35bd4a0 0x5645d3740a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81c2d41700 1 -- 172.21.15.19:0/3759849594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 msgr2=0x5645d37404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81c2d41700 1 --2- 172.21.15.19:0/3759849594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 0x5645d37404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81c2d41700 1 -- 172.21.15.19:0/3759849594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d35f3370 con 0x5645d373a6a0 2024-05-09T01:37:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81c2d41700 1 --2- 172.21.15.19:0/3759849594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d373aad0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f81b40098e0 tx=0x7f81b4009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d49cf921c8f0c14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81b8ff9700 1 -- 172.21.15.19:0/3759849594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4003a60 con 0x5645d373a6a0 2024-05-09T01:37:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81b8ff9700 1 -- 172.21.15.19:0/3759849594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b4005580 con 0x5645d373a6a0 2024-05-09T01:37:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.147+0000 7f81b8ff9700 1 -- 172.21.15.19:0/3759849594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b400bd70 con 0x5645d373a6a0 2024-05-09T01:37:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.148+0000 7f81d31bab80 1 -- 172.21.15.19:0/3759849594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 msgr2=0x5645d373aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.148+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3759849594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d373aad0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f81b40098e0 tx=0x7f81b4009590 comp rx=0 tx=0).stop 2024-05-09T01:37:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.148+0000 7f81d31bab80 1 -- 172.21.15.19:0/3759849594 shutdown_connections 2024-05-09T01:37:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.148+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3759849594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d373aad0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.148+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3759849594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 0x5645d37404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.148+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3759849594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d35bd4a0 0x5645d3740a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.148+0000 7f81d31bab80 1 -- 172.21.15.19:0/3759849594 >> 172.21.15.19:0/3759849594 conn(0x5645d361d6a0 msgr2=0x5645d3735400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:37.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.148+0000 7f81d31bab80 1 -- 172.21.15.19:0/3759849594 shutdown_connections 2024-05-09T01:37:37.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.148+0000 7f81d31bab80 1 -- 172.21.15.19:0/3759849594 wait complete. 2024-05-09T01:37:37.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81d31bab80 1 -- start start 2024-05-09T01:37:37.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81d31bab80 1 Processor -- start 2024-05-09T01:37:37.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d37440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 0x5645d37445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d373a6a0 0x5645d3744b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81d31bab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5645d34d8ba0 con 0x5645d35bd4a0 2024-05-09T01:37:37.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81d31bab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5645d3617150 con 0x5645d373a6a0 2024-05-09T01:37:37.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81d31bab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5645d373b4e0 con 0x5645d36a7160 2024-05-09T01:37:37.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81d31bab80 10 client.0 _ll_register_callbacks cb 0x5645d34ce6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81c2540700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d37440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81c2540700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d37440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40428/0 (socket says 172.21.15.19:40428) 2024-05-09T01:37:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.149+0000 7f81c2540700 1 -- 172.21.15.19:0/3987476800 learned_addr learned my addr 172.21.15.19:0/3987476800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81c2d41700 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d373a6a0 0x5645d3744b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 0x5645d37445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81c2540700 1 -- 172.21.15.19:0/3987476800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 msgr2=0x5645d37445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81c2540700 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 0x5645d37445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81c2540700 1 -- 172.21.15.19:0/3987476800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d373a6a0 msgr2=0x5645d3744b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:37.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81c2540700 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d373a6a0 0x5645d3744b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81c2540700 1 -- 172.21.15.19:0/3987476800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d34d8490 con 0x5645d35bd4a0 2024-05-09T01:37:37.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81c2540700 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d37440b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0df630 tx=0x7f81ac015c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd92024c837f0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:37.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:37.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81a1ffb700 1 -- 172.21.15.19:0/3987476800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0a9850 con 0x5645d35bd4a0 2024-05-09T01:37:37.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81a1ffb700 1 -- 172.21.15.19:0/3987476800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f81ac0b1b20 con 0x5645d35bd4a0 2024-05-09T01:37:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81d31bab80 1 -- 172.21.15.19:0/3987476800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b4005580 con 0x5645d35bd4a0 2024-05-09T01:37:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81a1ffb700 1 -- 172.21.15.19:0/3987476800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81ac0a9150 con 0x5645d35bd4a0 2024-05-09T01:37:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81d31bab80 10 client.18732 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.150+0000 7f81d31bab80 1 -- 172.21.15.19:0/3987476800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645d3751a10 con 0x5645d35bd4a0 2024-05-09T01:37:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.151+0000 7f81a1ffb700 1 -- 172.21.15.19:0/3987476800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b4005580 con 0x5645d35bd4a0 2024-05-09T01:37:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.151+0000 7f81a1ffb700 1 -- 172.21.15.19:0/3987476800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f81ac0dfb80 con 0x5645d35bd4a0 2024-05-09T01:37:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.151+0000 7f81d31bab80 10 client.18732 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.151+0000 7f81d31bab80 10 client.18732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:37:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.151+0000 7f81d31bab80 20 client.18732 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.151+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5645d3754050 0x5645d3756470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.151+0000 7f81d31bab80 4 client.18732 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.151+0000 7f81d31bab80 1 -- 172.21.15.19:0/3987476800 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645d366a9d0 con 0x5645d3754050 2024-05-09T01:37:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.151+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5645d3754050 0x5645d3756470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.152+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5645d3754050 0x5645d3756470 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d3a9180ab64dafd server_cookie=23817fd907dba772 in_seq=0 out_seq=0 2024-05-09T01:37:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.152+0000 7f81a1ffb700 10 client.18732 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:37:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81a1ffb700 1 -- 172.21.15.19:0/3987476800 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645d366a9d0 con 0x5645d3754050 2024-05-09T01:37:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81a1ffb700 10 client.18732 handle_command_reply: tid=0 2024-05-09T01:37:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 client.18732 shutdown 2024-05-09T01:37:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 -- 172.21.15.19:0/3987476800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 msgr2=0x5645d37440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d37440b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0df630 tx=0x7f81ac015c70 comp rx=0 tx=0).stop 2024-05-09T01:37:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 -- 172.21.15.19:0/3987476800 shutdown_connections 2024-05-09T01:37:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5645d3754050 0x5645d3756470 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d373a6a0 0x5645d3744b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d37440b0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81a1ffb700 0 client.18732 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:37:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 --2- 172.21.15.19:0/3987476800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 0x5645d37445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 -- 172.21.15.19:0/3987476800 >> 172.21.15.19:0/3987476800 conn(0x5645d361d6a0 msgr2=0x5645d37351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 -- 172.21.15.19:0/3987476800 shutdown_connections 2024-05-09T01:37:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 1 -- 172.21.15.19:0/3987476800 wait complete. 2024-05-09T01:37:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.153+0000 7f81d31bab80 20 client.18732 trim_cache size 0 max 16384 2024-05-09T01:37:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.168+0000 7f81d31bab80 1 Processor -- start 2024-05-09T01:37:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.168+0000 7f81d31bab80 1 -- start start 2024-05-09T01:37:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d373e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 0x5645d373f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d373a6a0 0x5645d3739690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81d31bab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1b0db0 con 0x5645d35bd4a0 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81d31bab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1b28f0 con 0x5645d36a7160 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81d31bab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1b1480 con 0x5645d373a6a0 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2540700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d373e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2540700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d373e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40430/0 (socket says 172.21.15.19:40430) 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2540700 1 -- 172.21.15.19:0/4171281048 learned_addr learned my addr 172.21.15.19:0/4171281048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 0x5645d373f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2d41700 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d373a6a0 0x5645d3739690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2540700 1 -- 172.21.15.19:0/4171281048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d373a6a0 msgr2=0x5645d3739690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:37.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2540700 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d373a6a0 0x5645d3739690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2540700 1 -- 172.21.15.19:0/4171281048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 msgr2=0x5645d373f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2540700 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 0x5645d373f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2540700 1 -- 172.21.15.19:0/4171281048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d37358f0 con 0x5645d35bd4a0 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2d41700 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d373a6a0 0x5645d3739690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81c2540700 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d373e760 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0a8e50 tx=0x7f81ac00d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9751ba6e1008948e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81a3fff700 1 -- 172.21.15.19:0/4171281048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0a9850 con 0x5645d35bd4a0 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81a3fff700 1 -- 172.21.15.19:0/4171281048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac0c3070 con 0x5645d35bd4a0 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.169+0000 7f81a3fff700 1 -- 172.21.15.19:0/4171281048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac005e60 con 0x5645d35bd4a0 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.170+0000 7f81d31bab80 1 -- 172.21.15.19:0/4171281048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 msgr2=0x5645d373e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.170+0000 7f81d31bab80 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d373e760 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0a8e50 tx=0x7f81ac00d8b0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.170+0000 7f81d31bab80 1 -- 172.21.15.19:0/4171281048 shutdown_connections 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.170+0000 7f81d31bab80 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d373a6a0 0x5645d3739690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.170+0000 7f81d31bab80 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d35bd4a0 0x5645d373e760 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.170+0000 7f81d31bab80 1 --2- 172.21.15.19:0/4171281048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d36a7160 0x5645d373f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.170+0000 7f81d31bab80 1 -- 172.21.15.19:0/4171281048 >> 172.21.15.19:0/4171281048 conn(0x5645d361d6a0 msgr2=0x5645d373f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.170+0000 7f81d31bab80 1 -- 172.21.15.19:0/4171281048 shutdown_connections 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.170+0000 7f81d31bab80 1 -- 172.21.15.19:0/4171281048 wait complete. 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81d31bab80 1 -- start start 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81d31bab80 1 Processor -- start 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d35bd4a0 0x5645d374c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 0x5645d375b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81d31bab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d375b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81d31bab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1b2220 con 0x5645d373a6a0 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81d31bab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1b1b50 con 0x5645d35bd4a0 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81d31bab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1aed50 con 0x5645d36a7160 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81d31bab80 10 client.0 _ll_register_callbacks cb 0x5645d34ce6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81c2d41700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d375b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81c2d41700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d375b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40444/0 (socket says 172.21.15.19:40444) 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81c2d41700 1 -- 172.21.15.19:0/2090059716 learned_addr learned my addr 172.21.15.19:0/2090059716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.171+0000 7f81c2540700 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d35bd4a0 0x5645d374c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 0x5645d375b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81c2d41700 1 -- 172.21.15.19:0/2090059716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 msgr2=0x5645d375b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81c2d41700 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 0x5645d375b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81c2d41700 1 -- 172.21.15.19:0/2090059716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d35bd4a0 msgr2=0x5645d374c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81c2d41700 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d35bd4a0 0x5645d374c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81c2d41700 1 -- 172.21.15.19:0/2090059716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d375bad0 con 0x5645d373a6a0 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81c1d3f700 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 0x5645d375b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81c2d41700 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d375b560 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x5645d37410c0 tx=0x7f81b4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f33fa771e4cdbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81a1ffb700 1 -- 172.21.15.19:0/2090059716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4003a70 con 0x5645d373a6a0 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81a1ffb700 1 -- 172.21.15.19:0/2090059716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f81b4003c10 con 0x5645d373a6a0 2024-05-09T01:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81d31bab80 1 -- 172.21.15.19:0/2090059716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81ac0c3070 con 0x5645d373a6a0 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81a1ffb700 1 -- 172.21.15.19:0/2090059716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81b403e050 con 0x5645d373a6a0 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81d31bab80 10 client.18738 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81d31bab80 1 -- 172.21.15.19:0/2090059716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645d374bf60 con 0x5645d373a6a0 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.172+0000 7f81a1ffb700 1 -- 172.21.15.19:0/2090059716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac0c3070 con 0x5645d373a6a0 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81a1ffb700 1 -- 172.21.15.19:0/2090059716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f81b400f770 con 0x5645d373a6a0 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81d31bab80 10 client.18738 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81d31bab80 10 client.18738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81d31bab80 20 client.18738 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81d31bab80 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5645d37be7e0 0x5645d37c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81d31bab80 4 client.18738 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81d31bab80 1 -- 172.21.15.19:0/2090059716 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645d3664a70 con 0x5645d37be7e0 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81c2540700 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5645d37be7e0 0x5645d37c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81a1ffb700 10 client.18738 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.173+0000 7f81c2540700 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5645d37be7e0 0x5645d37c0c00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d25887da1b35491 server_cookie=1a55eac413f88552 in_seq=0 out_seq=0 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81a1ffb700 1 -- 172.21.15.19:0/2090059716 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645d3664a70 con 0x5645d37be7e0 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81a1ffb700 10 client.18738 handle_command_reply: tid=0 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 client.18738 shutdown 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 -- 172.21.15.19:0/2090059716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 msgr2=0x5645d375b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d375b560 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x5645d37410c0 tx=0x7f81b4005a00 comp rx=0 tx=0).stop 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 -- 172.21.15.19:0/2090059716 shutdown_connections 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x5645d37be7e0 0x5645d37c0c00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81a1ffb700 0 client.18738 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645d373a6a0 0x5645d375b560 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645d35bd4a0 0x5645d374c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 --2- 172.21.15.19:0/2090059716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645d36a7160 0x5645d375b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 -- 172.21.15.19:0/2090059716 >> 172.21.15.19:0/2090059716 conn(0x5645d361d6a0 msgr2=0x5645d373de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:37.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 -- 172.21.15.19:0/2090059716 shutdown_connections 2024-05-09T01:37:37.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 1 -- 172.21.15.19:0/2090059716 wait complete. 2024-05-09T01:37:37.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81d31bab80 20 client.18738 trim_cache size 0 max 16384 2024-05-09T01:37:37.180 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:37:37.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81c47a4700 1 -- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f81a40b2880 msgr2=0x7f81a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:37.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81c47a4700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f81a40b2880 0x7f81a40b4d90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f81bc106720 tx=0x7f81b0008040 comp rx=0 tx=0).stop 2024-05-09T01:37:37.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81c47a4700 1 -- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc103c50 msgr2=0x7f81bc19bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:37.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.174+0000 7f81c47a4700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc103c50 0x7f81bc19bd00 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0060b0 tx=0x7f81ac007f90 comp rx=0 tx=0).stop 2024-05-09T01:37:37.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.175+0000 7f81c47a4700 1 -- 172.21.15.19:0/3026833706 shutdown_connections 2024-05-09T01:37:37.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.175+0000 7f81c47a4700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc1087a0 0x7f81bc19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.175+0000 7f81c47a4700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f81a40b2880 0x7f81a40b4d90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.175+0000 7f81c47a4700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81bc103c50 0x7f81bc19bd00 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.175+0000 7f81c47a4700 1 --2- 172.21.15.19:0/3026833706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81bc107e50 0x7f81bc19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.175+0000 7f81c47a4700 1 -- 172.21.15.19:0/3026833706 >> 172.21.15.19:0/3026833706 conn(0x7f81bc0fe540 msgr2=0x7f81bc1075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.175+0000 7f81c47a4700 1 -- 172.21.15.19:0/3026833706 shutdown_connections 2024-05-09T01:37:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:37.175+0000 7f81c47a4700 1 -- 172.21.15.19:0/3026833706 wait complete. 2024-05-09T01:37:37.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:37:38.133 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:37:38.528 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:38 smithi019 ceph-mon[30158]: pgmap v741: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:37:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:38 smithi059 ceph-mon[34351]: pgmap v741: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:37:38.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:38 smithi196 ceph-mon[35041]: pgmap v741: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-05-09T01:37:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.910+0000 7f395cce9700 1 -- 172.21.15.19:0/1033828904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958109880 msgr2=0x7f39581111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.910+0000 7f395cce9700 1 --2- 172.21.15.19:0/1033828904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958109880 0x7f39581111e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f395000b0d0 tx=0x7f395000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.910+0000 7f395cce9700 1 -- 172.21.15.19:0/1033828904 shutdown_connections 2024-05-09T01:37:39.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.910+0000 7f395cce9700 1 --2- 172.21.15.19:0/1033828904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958109880 0x7f39581111e0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.910+0000 7f395cce9700 1 --2- 172.21.15.19:0/1033828904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3958104650 0x7f3958104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.910+0000 7f395cce9700 1 --2- 172.21.15.19:0/1033828904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3958108f30 0x7f3958109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.910+0000 7f395cce9700 1 -- 172.21.15.19:0/1033828904 >> 172.21.15.19:0/1033828904 conn(0x7f3958079f00 msgr2=0x7f395807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:39.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.910+0000 7f395cce9700 1 -- 172.21.15.19:0/1033828904 shutdown_connections 2024-05-09T01:37:39.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.910+0000 7f395cce9700 1 -- 172.21.15.19:0/1033828904 wait complete. 2024-05-09T01:37:39.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395cce9700 1 Processor -- start 2024-05-09T01:37:39.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395cce9700 1 -- start start 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395cce9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958104650 0x7f395819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395cce9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3958108f30 0x7f395819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395cce9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3958109880 0x7f395819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395cce9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3958114520 con 0x7f3958104650 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395cce9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39581143a0 con 0x7f3958109880 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395cce9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39581146a0 con 0x7f3958108f30 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958104650 0x7f395819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958104650 0x7f395819ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59790/0 (socket says 172.21.15.19:59790) 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f395659c700 1 -- 172.21.15.19:0/4175411039 learned_addr learned my addr 172.21.15.19:0/4175411039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.911+0000 7f3956d9d700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3958109880 0x7f395819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f394dd9b700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3958108f30 0x7f395819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f395659c700 1 -- 172.21.15.19:0/4175411039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3958108f30 msgr2=0x7f395819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f395659c700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3958108f30 0x7f395819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f395659c700 1 -- 172.21.15.19:0/4175411039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3958109880 msgr2=0x7f395819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f395659c700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3958109880 0x7f395819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f395659c700 1 -- 172.21.15.19:0/4175411039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395819fd60 con 0x7f3958104650 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f395659c700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958104650 0x7f395819ae90 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f394800c920 tx=0x7f394800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f394ffff700 1 -- 172.21.15.19:0/4175411039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948007b00 con 0x7f3958104650 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f395cce9700 1 -- 172.21.15.19:0/4175411039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39581a0050 con 0x7f3958104650 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f394ffff700 1 -- 172.21.15.19:0/4175411039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3948004ce0 con 0x7f3958104650 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f394ffff700 1 -- 172.21.15.19:0/4175411039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948005500 con 0x7f3958104650 2024-05-09T01:37:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.912+0000 7f395cce9700 1 -- 172.21.15.19:0/4175411039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39581a7970 con 0x7f3958104650 2024-05-09T01:37:39.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.915+0000 7f394ffff700 1 -- 172.21.15.19:0/4175411039 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f394800f450 con 0x7f3958104650 2024-05-09T01:37:39.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.916+0000 7f394ffff700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f39440b2880 0x7f39440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.916+0000 7f394dd9b700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f39440b2880 0x7f39440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.916+0000 7f394ffff700 1 -- 172.21.15.19:0/4175411039 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f39480e1e30 con 0x7f3958104650 2024-05-09T01:37:39.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.916+0000 7f396b6ffb80 1 Processor -- start 2024-05-09T01:37:39.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.916+0000 7f394dd9b700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f39440b2880 0x7f39440b4d90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3940006fd0 tx=0x7f3940008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:39.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.916+0000 7f396b6ffb80 1 -- start start 2024-05-09T01:37:39.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 0x55e3d7510ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d7511030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d75151e0 0x55e3d75155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e3d742f0a0 con 0x55e3d738f4a0 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e3d73d49a0 con 0x55e3d75151e0 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3d742b060 con 0x55e3d7409d00 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f394dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d7511030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f394dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d7511030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59794/0 (socket says 172.21.15.19:59794) 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f394dd9b700 1 -- 172.21.15.19:0/643808945 learned_addr learned my addr 172.21.15.19:0/643808945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f395659c700 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 0x55e3d7510ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f3956d9d700 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d75151e0 0x55e3d75155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f394dd9b700 1 -- 172.21.15.19:0/643808945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 msgr2=0x55e3d7510ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f394dd9b700 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 0x55e3d7510ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f394dd9b700 1 -- 172.21.15.19:0/643808945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d75151e0 msgr2=0x55e3d75155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:39.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f394dd9b700 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d75151e0 0x55e3d75155f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f394dd9b700 1 -- 172.21.15.19:0/643808945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d73c5500 con 0x55e3d738f4a0 2024-05-09T01:37:39.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f3956d9d700 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d75151e0 0x55e3d75155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:39.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.917+0000 7f394dd9b700 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d7511030 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f394000d6a0 tx=0x7f3940010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e441f071b922f5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:39.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.918+0000 7f394cd99700 1 -- 172.21.15.19:0/643808945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940024070 con 0x55e3d738f4a0 2024-05-09T01:37:39.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.918+0000 7f394cd99700 1 -- 172.21.15.19:0/643808945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394000de40 con 0x55e3d738f4a0 2024-05-09T01:37:39.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.918+0000 7f394cd99700 1 -- 172.21.15.19:0/643808945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394001fae0 con 0x55e3d738f4a0 2024-05-09T01:37:39.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 -- 172.21.15.19:0/643808945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 msgr2=0x55e3d7511030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:39.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d7511030 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f394000d6a0 tx=0x7f3940010000 comp rx=0 tx=0).stop 2024-05-09T01:37:39.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 -- 172.21.15.19:0/643808945 shutdown_connections 2024-05-09T01:37:39.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d75151e0 0x55e3d75155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 0x55e3d7510ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/643808945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d7511030 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 -- 172.21.15.19:0/643808945 >> 172.21.15.19:0/643808945 conn(0x55e3d7392f70 msgr2=0x55e3d7507220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:39.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 -- 172.21.15.19:0/643808945 shutdown_connections 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 -- 172.21.15.19:0/643808945 wait complete. 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 -- start start 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.919+0000 7f396b6ffb80 1 Processor -- start 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d751a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 0x55e3d751a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 0x55e3d751aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e3d7515f00 con 0x55e3d738f4a0 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e3d73c9b20 con 0x55e3d7409d00 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3d751d470 con 0x55e3d75151e0 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f395659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d751a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f396b6ffb80 10 client.0 _ll_register_callbacks cb 0x55e3d72a06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f395659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d751a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59798/0 (socket says 172.21.15.19:59798) 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f395659c700 1 -- 172.21.15.19:0/3260570652 learned_addr learned my addr 172.21.15.19:0/3260570652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f394dd9b700 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 0x55e3d751a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f3956d9d700 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 0x55e3d751aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f395659c700 1 -- 172.21.15.19:0/3260570652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 msgr2=0x55e3d751aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f395659c700 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 0x55e3d751aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f395659c700 1 -- 172.21.15.19:0/3260570652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 msgr2=0x55e3d751a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f395659c700 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 0x55e3d751a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f395659c700 1 -- 172.21.15.19:0/3260570652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d72aa490 con 0x55e3d738f4a0 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f395659c700 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d751a460 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f39480df8b0 tx=0x7f39480a9ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38523fb3de60431a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.920+0000 7f393dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f393dffb700 1 -- 172.21.15.19:0/3260570652 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480c3060 con 0x55e3d738f4a0 2024-05-09T01:37:39.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f393dffb700 1 -- 172.21.15.19:0/3260570652 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f39480a9840 con 0x55e3d738f4a0 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f396b6ffb80 1 -- 172.21.15.19:0/3260570652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f394000de40 con 0x55e3d738f4a0 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f393dffb700 1 -- 172.21.15.19:0/3260570652 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f394801d040 con 0x55e3d738f4a0 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f396b6ffb80 10 client.18750 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f396b6ffb80 1 -- 172.21.15.19:0/3260570652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e3d7527e50 con 0x55e3d738f4a0 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f393dffb700 1 -- 172.21.15.19:0/3260570652 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394000de40 con 0x55e3d738f4a0 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f393dffb700 1 -- 172.21.15.19:0/3260570652 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f39480b0b00 con 0x55e3d738f4a0 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f396b6ffb80 10 client.18750 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f396b6ffb80 10 client.18750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f396b6ffb80 20 client.18750 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e3d752a3e0 0x55e3d752c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f396b6ffb80 4 client.18750 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.921+0000 7f396b6ffb80 1 -- 172.21.15.19:0/3260570652 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3d743ca80 con 0x55e3d752a3e0 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.922+0000 7f394dd9b700 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e3d752a3e0 0x55e3d752c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.922+0000 7f394dd9b700 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e3d752a3e0 0x55e3d752c800 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87593f01cb3f17c server_cookie=7b4baebade4bf9c5 in_seq=0 out_seq=0 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.922+0000 7f393dffb700 10 client.18750 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f393dffb700 1 -- 172.21.15.19:0/3260570652 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3d743ca80 con 0x55e3d752a3e0 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f393dffb700 10 client.18750 handle_command_reply: tid=0 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 client.18750 shutdown 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 -- 172.21.15.19:0/3260570652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 msgr2=0x55e3d751a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d751a460 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f39480df8b0 tx=0x7f39480a9ad0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 -- 172.21.15.19:0/3260570652 shutdown_connections 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e3d752a3e0 0x55e3d752c800 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 0x55e3d751aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d751a460 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f393dffb700 0 client.18750 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/3260570652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 0x55e3d751a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 -- 172.21.15.19:0/3260570652 >> 172.21.15.19:0/3260570652 conn(0x55e3d7392f70 msgr2=0x55e3d7500ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:39.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.923+0000 7f396b6ffb80 1 -- 172.21.15.19:0/3260570652 shutdown_connections 2024-05-09T01:37:39.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.924+0000 7f396b6ffb80 1 -- 172.21.15.19:0/3260570652 wait complete. 2024-05-09T01:37:39.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.924+0000 7f396b6ffb80 20 client.18750 trim_cache size 0 max 16384 2024-05-09T01:37:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f396b6ffb80 1 Processor -- start 2024-05-09T01:37:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f396b6ffb80 1 -- start start 2024-05-09T01:37:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d750be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 0x55e3d751ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 0x55e3d7514720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39581acae0 con 0x55e3d738f4a0 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39581ae620 con 0x55e3d7409d00 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39581ad1b0 con 0x55e3d75151e0 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f395659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d750be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f395659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d750be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59806/0 (socket says 172.21.15.19:59806) 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f395659c700 1 -- 172.21.15.19:0/1669268234 learned_addr learned my addr 172.21.15.19:0/1669268234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f394dd9b700 1 --2- 172.21.15.19:0/1669268234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 0x55e3d751ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f3956d9d700 1 --2- 172.21.15.19:0/1669268234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 0x55e3d7514720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f395659c700 1 -- 172.21.15.19:0/1669268234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 msgr2=0x55e3d7514720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f395659c700 1 --2- 172.21.15.19:0/1669268234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 0x55e3d7514720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f395659c700 1 -- 172.21.15.19:0/1669268234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 msgr2=0x55e3d751ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f395659c700 1 --2- 172.21.15.19:0/1669268234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 0x55e3d751ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.939+0000 7f395659c700 1 -- 172.21.15.19:0/1669268234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d751b260 con 0x55e3d738f4a0 2024-05-09T01:37:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.940+0000 7f395659c700 1 --2- 172.21.15.19:0/1669268234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d750be20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f394800c8f0 tx=0x7f39480a9490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2114e975c38341b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:39.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.940+0000 7f393ffff700 1 -- 172.21.15.19:0/1669268234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480a9840 con 0x55e3d738f4a0 2024-05-09T01:37:39.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.940+0000 7f393ffff700 1 -- 172.21.15.19:0/1669268234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39480a3af0 con 0x55e3d738f4a0 2024-05-09T01:37:39.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.940+0000 7f393ffff700 1 -- 172.21.15.19:0/1669268234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480a4fa0 con 0x55e3d738f4a0 2024-05-09T01:37:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.940+0000 7f396b6ffb80 1 -- 172.21.15.19:0/1669268234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 msgr2=0x55e3d750be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.940+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/1669268234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d750be20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f394800c8f0 tx=0x7f39480a9490 comp rx=0 tx=0).stop 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 -- 172.21.15.19:0/1669268234 shutdown_connections 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/1669268234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d75151e0 0x55e3d7514720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/1669268234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d738f4a0 0x55e3d750be20 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/1669268234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d7409d00 0x55e3d751ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 -- 172.21.15.19:0/1669268234 >> 172.21.15.19:0/1669268234 conn(0x55e3d7392f70 msgr2=0x55e3d7514010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 -- 172.21.15.19:0/1669268234 shutdown_connections 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 -- 172.21.15.19:0/1669268234 wait complete. 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 -- start start 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 Processor -- start 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d738f4a0 0x55e3d7522c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 0x55e3d751dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d75151e0 0x55e3d751e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39581adf50 con 0x55e3d75151e0 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39581ad880 con 0x55e3d738f4a0 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39581aaa80 con 0x55e3d7409d00 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.941+0000 7f396b6ffb80 10 client.0 _ll_register_callbacks cb 0x55e3d72a06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f3956d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d75151e0 0x55e3d751e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f395659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d738f4a0 0x55e3d7522c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f3956d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d75151e0 0x55e3d751e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59812/0 (socket says 172.21.15.19:59812) 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f3956d9d700 1 -- 172.21.15.19:0/716129510 learned_addr learned my addr 172.21.15.19:0/716129510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f394dd9b700 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 0x55e3d751dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f3956d9d700 1 -- 172.21.15.19:0/716129510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 msgr2=0x55e3d751dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f3956d9d700 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 0x55e3d751dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f3956d9d700 1 -- 172.21.15.19:0/716129510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d738f4a0 msgr2=0x55e3d7522c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f3956d9d700 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d738f4a0 0x55e3d7522c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f3956d9d700 1 -- 172.21.15.19:0/716129510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d751edf0 con 0x55e3d75151e0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f394dd9b700 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 0x55e3d751dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f3956d9d700 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d75151e0 0x55e3d751e670 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f395000d010 tx=0x7f3950003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fd0e09d1569800 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f393dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f393dffb700 1 -- 172.21.15.19:0/716129510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500079d0 con 0x55e3d75151e0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f393dffb700 1 -- 172.21.15.19:0/716129510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3950007b70 con 0x55e3d75151e0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f396b6ffb80 1 -- 172.21.15.19:0/716129510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39480a3af0 con 0x55e3d75151e0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f393dffb700 1 -- 172.21.15.19:0/716129510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3950007d70 con 0x55e3d75151e0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f396b6ffb80 10 client.18756 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.942+0000 7f396b6ffb80 1 -- 172.21.15.19:0/716129510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e3d7594bc0 con 0x55e3d75151e0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.943+0000 7f393dffb700 1 -- 172.21.15.19:0/716129510 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39480a3af0 con 0x55e3d75151e0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.943+0000 7f393dffb700 1 -- 172.21.15.19:0/716129510 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3950024070 con 0x55e3d75151e0 2024-05-09T01:37:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.943+0000 7f396b6ffb80 10 client.18756 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.943+0000 7f396b6ffb80 10 client.18756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.943+0000 7f396b6ffb80 20 client.18756 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.943+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e3d75952d0 0x55e3d75976f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.943+0000 7f396b6ffb80 4 client.18756 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.943+0000 7f396b6ffb80 1 -- 172.21.15.19:0/716129510 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3d7436b20 con 0x55e3d75952d0 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.943+0000 7f395659c700 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e3d75952d0 0x55e3d75976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f395659c700 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e3d75952d0 0x55e3d75976f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b6a8ce8d2ea50e7 server_cookie=f32ea05ba78cf887 in_seq=0 out_seq=0 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f393dffb700 10 client.18756 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f393dffb700 1 -- 172.21.15.19:0/716129510 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3d7436b20 con 0x55e3d75952d0 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f393dffb700 10 client.18756 handle_command_reply: tid=0 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 client.18756 shutdown 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 -- 172.21.15.19:0/716129510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d75151e0 msgr2=0x55e3d751e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d75151e0 0x55e3d751e670 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f395000d010 tx=0x7f3950003ca0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 -- 172.21.15.19:0/716129510 shutdown_connections 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x55e3d75952d0 0x55e3d75976f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3d75151e0 0x55e3d751e670 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e3d738f4a0 0x55e3d7522c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f393dffb700 0 client.18756 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 --2- 172.21.15.19:0/716129510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3d7409d00 0x55e3d751dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 -- 172.21.15.19:0/716129510 >> 172.21.15.19:0/716129510 conn(0x55e3d7392f70 msgr2=0x55e3d750bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 -- 172.21.15.19:0/716129510 shutdown_connections 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 1 -- 172.21.15.19:0/716129510 wait complete. 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.944+0000 7f396b6ffb80 20 client.18756 trim_cache size 0 max 16384 2024-05-09T01:37:39.946 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:37:39.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 -- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f39440b2880 msgr2=0x7f39440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:39.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f39440b2880 0x7f39440b4d90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3940006fd0 tx=0x7f3940008040 comp rx=0 tx=0).stop 2024-05-09T01:37:39.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 -- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958104650 msgr2=0x7f395819ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:39.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958104650 0x7f395819ae90 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f394800c920 tx=0x7f394800cc30 comp rx=0 tx=0).stop 2024-05-09T01:37:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 -- 172.21.15.19:0/4175411039 shutdown_connections 2024-05-09T01:37:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3958109880 0x7f395819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f39440b2880 0x7f39440b4d90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3958104650 0x7f395819ae90 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 --2- 172.21.15.19:0/4175411039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3958108f30 0x7f395819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 -- 172.21.15.19:0/4175411039 >> 172.21.15.19:0/4175411039 conn(0x7f3958079f00 msgr2=0x7f3958102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 -- 172.21.15.19:0/4175411039 shutdown_connections 2024-05-09T01:37:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:39.945+0000 7f395cce9700 1 -- 172.21.15.19:0/4175411039 wait complete. 2024-05-09T01:37:40.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:40 smithi019 ceph-mon[30158]: pgmap v742: 97 pgs: 97 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:37:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:40 smithi059 ceph-mon[34351]: pgmap v742: 97 pgs: 97 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:37:40.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:40 smithi196 ceph-mon[35041]: pgmap v742: 97 pgs: 97 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-05-09T01:37:41.579 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:37:41.579 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:37:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:37:42.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:42 smithi019 ceph-mon[30158]: pgmap v743: 97 pgs: 97 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:37:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:42 smithi059 ceph-mon[34351]: pgmap v743: 97 pgs: 97 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:37:42.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:42 smithi196 ceph-mon[35041]: pgmap v743: 97 pgs: 97 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-05-09T01:37:43.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.602+0000 7f1d44b28700 1 -- 172.21.15.19:0/2448301663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40107e20 msgr2=0x7f1d40108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:43.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.602+0000 7f1d44b28700 1 --2- 172.21.15.19:0/2448301663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40107e20 0x7f1d40108230 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f1d280099c0 tx=0x7f1d28009cd0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.602+0000 7f1d44b28700 1 -- 172.21.15.19:0/2448301663 shutdown_connections 2024-05-09T01:37:43.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.602+0000 7f1d44b28700 1 --2- 172.21.15.19:0/2448301663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d40108770 0x7f1d401100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.602+0000 7f1d44b28700 1 --2- 172.21.15.19:0/2448301663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40103c20 0x7f1d40104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.602+0000 7f1d44b28700 1 --2- 172.21.15.19:0/2448301663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40107e20 0x7f1d40108230 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.602+0000 7f1d44b28700 1 -- 172.21.15.19:0/2448301663 >> 172.21.15.19:0/2448301663 conn(0x7f1d400fe6f0 msgr2=0x7f1d40100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:43.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.603+0000 7f1d44b28700 1 -- 172.21.15.19:0/2448301663 shutdown_connections 2024-05-09T01:37:43.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.603+0000 7f1d44b28700 1 -- 172.21.15.19:0/2448301663 wait complete. 2024-05-09T01:37:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.603+0000 7f1d44b28700 1 Processor -- start 2024-05-09T01:37:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d44b28700 1 -- start start 2024-05-09T01:37:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d44b28700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40103c20 0x7f1d4019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d44b28700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d40107e20 0x7f1d4019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d44b28700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40108770 0x7f1d4019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d44b28700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40113390 con 0x7f1d40103c20 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d44b28700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40113210 con 0x7f1d40107e20 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d44b28700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40113510 con 0x7f1d40108770 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40103c20 0x7f1d4019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40103c20 0x7f1d4019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59826/0 (socket says 172.21.15.19:59826) 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3e59c700 1 -- 172.21.15.19:0/3510995529 learned_addr learned my addr 172.21.15.19:0/3510995529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d40107e20 0x7f1d4019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3ed9d700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40108770 0x7f1d4019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3e59c700 1 -- 172.21.15.19:0/3510995529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40108770 msgr2=0x7f1d4019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40108770 0x7f1d4019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3e59c700 1 -- 172.21.15.19:0/3510995529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d40107e20 msgr2=0x7f1d4019b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d40107e20 0x7f1d4019b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3e59c700 1 -- 172.21.15.19:0/3510995529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4019fdc0 con 0x7f1d40103c20 2024-05-09T01:37:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.604+0000 7f1d3ed9d700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40108770 0x7f1d4019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.605+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40103c20 0x7f1d4019aef0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000d980 tx=0x7f1d3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.605+0000 7f1d37fff700 1 -- 172.21.15.19:0/3510995529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30017070 con 0x7f1d40103c20 2024-05-09T01:37:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.605+0000 7f1d44b28700 1 -- 172.21.15.19:0/3510995529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d401a00b0 con 0x7f1d40103c20 2024-05-09T01:37:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.605+0000 7f1d37fff700 1 -- 172.21.15.19:0/3510995529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d30009de0 con 0x7f1d40103c20 2024-05-09T01:37:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.605+0000 7f1d37fff700 1 -- 172.21.15.19:0/3510995529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000f3c0 con 0x7f1d40103c20 2024-05-09T01:37:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.605+0000 7f1d44b28700 1 -- 172.21.15.19:0/3510995529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d401a79d0 con 0x7f1d40103c20 2024-05-09T01:37:43.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.607+0000 7f1d37fff700 1 -- 172.21.15.19:0/3510995529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1d3000f560 con 0x7f1d40103c20 2024-05-09T01:37:43.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.608+0000 7f1d37fff700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d2c0b2880 0x7f1d2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.608+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d2c0b2880 0x7f1d2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d37fff700 1 -- 172.21.15.19:0/3510995529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f1d300e2170 con 0x7f1d40103c20 2024-05-09T01:37:43.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d5353eb80 1 Processor -- start 2024-05-09T01:37:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d5353eb80 1 -- start start 2024-05-09T01:37:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d2c0b2880 0x7f1d2c0b4d90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d28000c00 tx=0x7f1d2801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800da11e0 0x560800e38dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e39370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e3d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560800d57070 con 0x560800cb7520 2024-05-09T01:37:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560800cf5870 con 0x560800da11e0 2024-05-09T01:37:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560800d53030 con 0x560800e3d480 2024-05-09T01:37:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d35d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e39370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d35d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e39370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59834/0 (socket says 172.21.15.19:59834) 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d3e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800da11e0 0x560800e38dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.609+0000 7f1d35d9b700 1 -- 172.21.15.19:0/4100056547 learned_addr learned my addr 172.21.15.19:0/4100056547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d3ed9d700 1 --2- 172.21.15.19:0/4100056547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e3d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d35d9b700 1 -- 172.21.15.19:0/4100056547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 msgr2=0x560800e3d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/4100056547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e3d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d35d9b700 1 -- 172.21.15.19:0/4100056547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800da11e0 msgr2=0x560800e38dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/4100056547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800da11e0 0x560800e38dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d35d9b700 1 -- 172.21.15.19:0/4100056547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560800ced3f0 con 0x560800cb7520 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/4100056547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e39370 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1d280099c0 tx=0x7f1d28009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd46475a4cd752b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d34d99700 1 -- 172.21.15.19:0/4100056547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d280039c0 con 0x560800cb7520 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d34d99700 1 -- 172.21.15.19:0/4100056547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d2800bc40 con 0x560800cb7520 2024-05-09T01:37:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.610+0000 7f1d34d99700 1 -- 172.21.15.19:0/4100056547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28026850 con 0x560800cb7520 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.611+0000 7f1d5353eb80 1 -- 172.21.15.19:0/4100056547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 msgr2=0x560800e39370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.611+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/4100056547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e39370 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1d280099c0 tx=0x7f1d28009cd0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.611+0000 7f1d5353eb80 1 -- 172.21.15.19:0/4100056547 shutdown_connections 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.611+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/4100056547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e3d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.611+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/4100056547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800da11e0 0x560800e38dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.611+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/4100056547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e39370 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.611+0000 7f1d5353eb80 1 -- 172.21.15.19:0/4100056547 >> 172.21.15.19:0/4100056547 conn(0x560800d17720 msgr2=0x560800e2f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.611+0000 7f1d5353eb80 1 -- 172.21.15.19:0/4100056547 shutdown_connections 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.611+0000 7f1d5353eb80 1 -- 172.21.15.19:0/4100056547 wait complete. 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d5353eb80 1 -- start start 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d5353eb80 1 Processor -- start 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 0x560800e426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800da11e0 0x560800e42c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e43160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560800bd2c20 con 0x560800da11e0 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560800d111d0 con 0x560800cb7520 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560800e3e1d0 con 0x560800e3d480 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d5353eb80 10 client.0 _ll_register_callbacks cb 0x560800bc8740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d35d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800da11e0 0x560800e42c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d3e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 0x560800e426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d35d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800da11e0 0x560800e42c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59848/0 (socket says 172.21.15.19:59848) 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.612+0000 7f1d35d9b700 1 -- 172.21.15.19:0/1889164054 learned_addr learned my addr 172.21.15.19:0/1889164054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d3ed9d700 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e43160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d35d9b700 1 -- 172.21.15.19:0/1889164054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 msgr2=0x560800e43160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e43160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d35d9b700 1 -- 172.21.15.19:0/1889164054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 msgr2=0x560800e426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 0x560800e426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d35d9b700 1 -- 172.21.15.19:0/1889164054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560800bd2510 con 0x560800da11e0 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 0x560800e426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800da11e0 0x560800e42c20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009990 tx=0x7f1d28005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2de735aa9580009 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1889164054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28038020 con 0x560800da11e0 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1889164054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f1d280039c0 con 0x560800da11e0 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1889164054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2800bc40 con 0x560800da11e0 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1889164054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1d28043050 con 0x560800da11e0 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d5353eb80 10 client.18768 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.613+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1889164054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560800e4ff40 con 0x560800da11e0 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.614+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1889164054 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d2800bc40 con 0x560800da11e0 2024-05-09T01:37:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.614+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1889164054 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1d28030850 con 0x560800da11e0 2024-05-09T01:37:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.614+0000 7f1d5353eb80 10 client.18768 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.614+0000 7f1d5353eb80 10 client.18768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.614+0000 7f1d5353eb80 20 client.18768 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.614+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560800e52410 0x560800e54830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.614+0000 7f1d5353eb80 4 client.18768 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.614+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1889164054 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560800d64a50 con 0x560800e52410 2024-05-09T01:37:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.614+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560800e52410 0x560800e54830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.615+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560800e52410 0x560800e54830 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9051d3cd97419bb server_cookie=335e0b8df919b909 in_seq=0 out_seq=0 2024-05-09T01:37:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.615+0000 7f1d25ffb700 10 client.18768 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:43.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1889164054 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560800d64a50 con 0x560800e52410 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d25ffb700 10 client.18768 handle_command_reply: tid=0 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 client.18768 shutdown 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1889164054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800da11e0 msgr2=0x560800e42c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800da11e0 0x560800e42c20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009990 tx=0x7f1d28005a00 comp rx=0 tx=0).stop 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1889164054 shutdown_connections 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560800e52410 0x560800e54830 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e43160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 0x560800e426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1889164054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800da11e0 0x560800e42c20 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1889164054 >> 172.21.15.19:0/1889164054 conn(0x560800d17720 msgr2=0x560800e2f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d25ffb700 0 client.18768 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1889164054 shutdown_connections 2024-05-09T01:37:43.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1889164054 wait complete. 2024-05-09T01:37:43.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.616+0000 7f1d5353eb80 20 client.18768 trim_cache size 0 max 16384 2024-05-09T01:37:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.637+0000 7f1d5353eb80 1 Processor -- start 2024-05-09T01:37:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.637+0000 7f1d5353eb80 1 -- start start 2024-05-09T01:37:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.637+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 0x560800e34140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.637+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e4e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.637+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800e415d0 0x560800e3b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.637+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401acaa0 con 0x560800e415d0 2024-05-09T01:37:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.637+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401ae5e0 con 0x560800cb7520 2024-05-09T01:37:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.637+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401ad170 con 0x560800e3d480 2024-05-09T01:37:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.637+0000 7f1d3ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800e415d0 0x560800e3b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d3ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800e415d0 0x560800e3b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59856/0 (socket says 172.21.15.19:59856) 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d3ed9d700 1 -- 172.21.15.19:0/2460784179 learned_addr learned my addr 172.21.15.19:0/2460784179 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/2460784179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e4e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/2460784179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 0x560800e34140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d3ed9d700 1 -- 172.21.15.19:0/2460784179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 msgr2=0x560800e4e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d3ed9d700 1 --2- 172.21.15.19:0/2460784179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e4e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d3ed9d700 1 -- 172.21.15.19:0/2460784179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 msgr2=0x560800e34140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d3ed9d700 1 --2- 172.21.15.19:0/2460784179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 0x560800e34140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d3ed9d700 1 -- 172.21.15.19:0/2460784179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560800e3cda0 con 0x560800e415d0 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d3ed9d700 1 --2- 172.21.15.19:0/2460784179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800e415d0 0x560800e3b380 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007670 tx=0x7f1d3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7416727d0b4c743 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d27fff700 1 -- 172.21.15.19:0/2460784179 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3801b070 con 0x560800e415d0 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d27fff700 1 -- 172.21.15.19:0/2460784179 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d38017d00 con 0x560800e415d0 2024-05-09T01:37:43.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.638+0000 7f1d27fff700 1 -- 172.21.15.19:0/2460784179 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38016d50 con 0x560800e415d0 2024-05-09T01:37:43.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 -- 172.21.15.19:0/2460784179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800e415d0 msgr2=0x560800e3b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:43.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/2460784179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800e415d0 0x560800e3b380 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007670 tx=0x7f1d3800b040 comp rx=0 tx=0).stop 2024-05-09T01:37:43.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 -- 172.21.15.19:0/2460784179 shutdown_connections 2024-05-09T01:37:43.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/2460784179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800e415d0 0x560800e3b380 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/2460784179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800cb7520 0x560800e34140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/2460784179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e3d480 0x560800e4e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 -- 172.21.15.19:0/2460784179 >> 172.21.15.19:0/2460784179 conn(0x560800d17720 msgr2=0x560800e40ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 -- 172.21.15.19:0/2460784179 shutdown_connections 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 -- 172.21.15.19:0/2460784179 wait complete. 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 -- start start 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.639+0000 7f1d5353eb80 1 Processor -- start 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e57230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800e3d480 0x560800e57770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d5353eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e415d0 0x560800e556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401adf10 con 0x560800cb7520 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401ad840 con 0x560800e3d480 2024-05-09T01:37:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d5353eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401aaa60 con 0x560800e415d0 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d5353eb80 10 client.0 _ll_register_callbacks cb 0x560800bc8740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e57230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e57230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59870/0 (socket says 172.21.15.19:59870) 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3e59c700 1 -- 172.21.15.19:0/1820783634 learned_addr learned my addr 172.21.15.19:0/1820783634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800e3d480 0x560800e57770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3ed9d700 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e415d0 0x560800e556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3e59c700 1 -- 172.21.15.19:0/1820783634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e415d0 msgr2=0x560800e556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e415d0 0x560800e556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3e59c700 1 -- 172.21.15.19:0/1820783634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800e3d480 msgr2=0x560800e57770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800e3d480 0x560800e57770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3e59c700 1 -- 172.21.15.19:0/1820783634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560800e55d80 con 0x560800cb7520 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d3e59c700 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e57230 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f1d300dfbf0 tx=0x7f1d300cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf9cd1c3e2eb664 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.640+0000 7f1d25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1820783634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30020040 con 0x560800cb7520 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1820783634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f1d300106a0 con 0x560800cb7520 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1820783634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d38017d00 con 0x560800cb7520 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1820783634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1d300adac0 con 0x560800cb7520 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d5353eb80 10 client.18774 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1820783634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560800ebd2b0 con 0x560800cb7520 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1820783634 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d38017d00 con 0x560800cb7520 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1820783634 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1d300b20e0 con 0x560800cb7520 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d5353eb80 10 client.18774 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d5353eb80 10 client.18774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:43.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d5353eb80 20 client.18774 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560800ebd9c0 0x560800ebfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d5353eb80 4 client.18774 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.641+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1820783634 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560800d5eaf0 con 0x560800ebd9c0 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.642+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560800ebd9c0 0x560800ebfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.642+0000 7f1d35d9b700 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560800ebd9c0 0x560800ebfde0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8505c36666aa0179 server_cookie=4851eb46d314251 in_seq=0 out_seq=0 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.642+0000 7f1d25ffb700 10 client.18774 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d25ffb700 1 -- 172.21.15.19:0/1820783634 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560800d5eaf0 con 0x560800ebd9c0 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d25ffb700 10 client.18774 handle_command_reply: tid=0 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 client.18774 shutdown 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1820783634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 msgr2=0x560800e57230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e57230 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f1d300dfbf0 tx=0x7f1d300cc040 comp rx=0 tx=0).stop 2024-05-09T01:37:43.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1820783634 shutdown_connections 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x560800ebd9c0 0x560800ebfde0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560800e415d0 0x560800e556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560800cb7520 0x560800e57230 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 --2- 172.21.15.19:0/1820783634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560800e3d480 0x560800e57770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1820783634 >> 172.21.15.19:0/1820783634 conn(0x560800d17720 msgr2=0x560800e40ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d25ffb700 0 client.18774 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.643+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1820783634 shutdown_connections 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d5353eb80 1 -- 172.21.15.19:0/1820783634 wait complete. 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d5353eb80 20 client.18774 trim_cache size 0 max 16384 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:37:43.645 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T01:37:43.646 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "779223ba-f240-4016-9b97-a9b67fd9d44b", 2024-05-09T01:37:43.646 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T01:37:43.646 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:37:43.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 -- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d2c0b2880 msgr2=0x7f1d2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:43.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d2c0b2880 0x7f1d2c0b4d90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d28000c00 tx=0x7f1d2801a040 comp rx=0 tx=0).stop 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 -- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40103c20 msgr2=0x7f1d4019aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40103c20 0x7f1d4019aef0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000d980 tx=0x7f1d3000dc90 comp rx=0 tx=0).stop 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 -- 172.21.15.19:0/3510995529 shutdown_connections 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40108770 0x7f1d4019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1d2c0b2880 0x7f1d2c0b4d90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d40103c20 0x7f1d4019aef0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 --2- 172.21.15.19:0/3510995529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d40107e20 0x7f1d4019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 -- 172.21.15.19:0/3510995529 >> 172.21.15.19:0/3510995529 conn(0x7f1d400fe6f0 msgr2=0x7f1d40102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 -- 172.21.15.19:0/3510995529 shutdown_connections 2024-05-09T01:37:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:43.644+0000 7f1d44b28700 1 -- 172.21.15.19:0/3510995529 wait complete. 2024-05-09T01:37:44.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:37:44.609 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:37:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:44 smithi019 ceph-mon[30158]: pgmap v744: 97 pgs: 97 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:37:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:44 smithi019 ceph-mon[30158]: scrub queued for path: / 2024-05-09T01:37:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:44 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:37:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:44 smithi019 ceph-mon[30158]: scrub summary: active paths [/] 2024-05-09T01:37:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:44 smithi019 ceph-mon[30158]: scrub summary: idle+waiting paths [/] 2024-05-09T01:37:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:44 smithi059 ceph-mon[34351]: pgmap v744: 97 pgs: 97 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:37:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:44 smithi059 ceph-mon[34351]: scrub queued for path: / 2024-05-09T01:37:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:44 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:37:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:44 smithi059 ceph-mon[34351]: scrub summary: active paths [/] 2024-05-09T01:37:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:44 smithi059 ceph-mon[34351]: scrub summary: idle+waiting paths [/] 2024-05-09T01:37:44.920 INFO:tasks.workunit.client.0.smithi019.stdout:561/600: open fd = 3 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:562/600: open fd = 3 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:563/600: open fd = 3 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.921 INFO:tasks.workunit.client.0.smithi019.stdout:564/600: open fd = 3 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:565/600: open fd = 3 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:566/600: open fd = 3 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.922 INFO:tasks.workunit.client.0.smithi019.stdout:567/600: open fd = 3 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:568/600: open fd = 3 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:569/600: open fd = 3 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:570/600: open fd = 3 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:571/600: open fd = 3 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:572/600: open fd = 3 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:573/600: open fd = 3 2024-05-09T01:37:44.924 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:574/600: open fd = 3 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:575/600: open fd = 3 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:576/600: open fd = 3 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:577/600: open fd = 3 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:578/600: open fd = 3 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.926 INFO:tasks.workunit.client.0.smithi019.stdout:579/600: open fd = 3 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:580/600: open fd = 3 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:581/600: open fd = 3 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:582/600: open fd = 3 2024-05-09T01:37:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:583/600: open fd = 3 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:584/600: open fd = 3 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:585/600: open fd = 3 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:586/600: open fd = 3 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:587/600: open fd = 3 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:588/600: open fd = 3 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:589/600: open fd = 3 2024-05-09T01:37:44.929 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:590/600: open fd = 3 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:591/600: open fd = 3 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:592/600: open fd = 3 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.930 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:593/600: open fd = 3 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:594/600: open fd = 3 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:595/600: open fd = 3 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.931 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:596/600: open fd = 3 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:597/600: open fd = 3 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:598/600: open fd = 3 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.932 INFO:tasks.workunit.client.0.smithi019.stdout:599/600: open fd = 3 2024-05-09T01:37:44.933 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.933 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.933 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.933 INFO:tasks.workunit.client.0.smithi019.stdout:600/600: open fd = 3 2024-05-09T01:37:44.933 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.933 INFO:tasks.workunit.client.0.smithi019.stdout:write ret = 32 2024-05-09T01:37:44.933 INFO:tasks.workunit.client.0.smithi019.stdout:pread ret = 64 2024-05-09T01:37:44.933 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo OK 2024-05-09T01:37:44.934 INFO:tasks.workunit.client.0.smithi019.stdout:OK 2024-05-09T01:37:44.935 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T01:37:44.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T01:37:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:44 smithi196 ceph-mon[35041]: pgmap v744: 97 pgs: 97 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T01:37:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:44 smithi196 ceph-mon[35041]: scrub queued for path: / 2024-05-09T01:37:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:44 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:37:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:44 smithi196 ceph-mon[35041]: scrub summary: active paths [/] 2024-05-09T01:37:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:44 smithi196 ceph-mon[35041]: scrub summary: idle+waiting paths [/] 2024-05-09T01:37:44.973 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-05-09T01:37:44.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T01:37:45.322 DEBUG:teuthology.parallel:result is None 2024-05-09T01:37:45.323 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T01:37:45.356 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T01:37:45.356 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-09T01:37:45.367 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T01:37:45.404 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-09T01:37:45.414 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-09T01:37:46.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.414+0000 7f84c651c700 1 -- 172.21.15.19:0/3439126563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c0105480 msgr2=0x7f84c0105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:46.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.414+0000 7f84c651c700 1 --2- 172.21.15.19:0/3439126563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c0105480 0x7f84c0105890 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009990 tx=0x7f84b0009ca0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.414+0000 7f84c651c700 1 -- 172.21.15.19:0/3439126563 shutdown_connections 2024-05-09T01:37:46.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.414+0000 7f84c651c700 1 --2- 172.21.15.19:0/3439126563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84c010e8f0 0x7f84c0110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.414+0000 7f84c651c700 1 --2- 172.21.15.19:0/3439126563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c0105480 0x7f84c0105890 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.414+0000 7f84c651c700 1 --2- 172.21.15.19:0/3439126563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c0105dd0 0x7f84c010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.414+0000 7f84c651c700 1 -- 172.21.15.19:0/3439126563 >> 172.21.15.19:0/3439126563 conn(0x7f84c00fef30 msgr2=0x7f84c0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:46.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.415+0000 7f84c651c700 1 -- 172.21.15.19:0/3439126563 shutdown_connections 2024-05-09T01:37:46.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.415+0000 7f84c651c700 1 -- 172.21.15.19:0/3439126563 wait complete. 2024-05-09T01:37:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.415+0000 7f84c651c700 1 Processor -- start 2024-05-09T01:37:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.415+0000 7f84c651c700 1 -- start start 2024-05-09T01:37:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c651c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84c0105480 0x7f84c0103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c651c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c0105dd0 0x7f84c01036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c651c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c010e8f0 0x7f84c0103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c651c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0114440 con 0x7f84c010e8f0 2024-05-09T01:37:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c651c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01142c0 con 0x7f84c0105480 2024-05-09T01:37:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c651c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01145c0 con 0x7f84c0105dd0 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c4ab9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c010e8f0 0x7f84c0103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c4ab9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c010e8f0 0x7f84c0103bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59890/0 (socket says 172.21.15.19:59890) 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c4ab9700 1 -- 172.21.15.19:0/416400508 learned_addr learned my addr 172.21.15.19:0/416400508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84b7fff700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c0105dd0 0x7f84c01036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84bffff700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84c0105480 0x7f84c0103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c4ab9700 1 -- 172.21.15.19:0/416400508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c0105dd0 msgr2=0x7f84c01036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c0105dd0 0x7f84c01036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c4ab9700 1 -- 172.21.15.19:0/416400508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84c0105480 msgr2=0x7f84c0103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84c0105480 0x7f84c0103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c4ab9700 1 -- 172.21.15.19:0/416400508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c01a91c0 con 0x7f84c010e8f0 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.416+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c010e8f0 0x7f84c0103bf0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f84b800ef30 tx=0x7f84b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.417+0000 7f84bdffb700 1 -- 172.21.15.19:0/416400508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80092e0 con 0x7f84c010e8f0 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.417+0000 7f84bdffb700 1 -- 172.21.15.19:0/416400508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b8004bb0 con 0x7f84c010e8f0 2024-05-09T01:37:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.417+0000 7f84c651c700 1 -- 172.21.15.19:0/416400508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c01a9450 con 0x7f84c010e8f0 2024-05-09T01:37:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.417+0000 7f84bdffb700 1 -- 172.21.15.19:0/416400508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004020 con 0x7f84c010e8f0 2024-05-09T01:37:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.417+0000 7f84c651c700 1 -- 172.21.15.19:0/416400508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a9940 con 0x7f84c010e8f0 2024-05-09T01:37:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.420+0000 7f84bdffb700 1 -- 172.21.15.19:0/416400508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f84b801b030 con 0x7f84c010e8f0 2024-05-09T01:37:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84bdffb700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f84ac0b2880 0x7f84ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84d4f32b80 1 Processor -- start 2024-05-09T01:37:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84d4f32b80 1 -- start start 2024-05-09T01:37:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84bffff700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f84ac0b2880 0x7f84ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84bdffb700 1 -- 172.21.15.19:0/416400508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f84b80e1cb0 con 0x7f84c010e8f0 2024-05-09T01:37:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583a8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d582cb0a0 con 0x561d582a5d00 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d582709a0 con 0x561d583a8400 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d582c7060 con 0x561d5822b4a0 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.421+0000 7f84bffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84bffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59892/0 (socket says 172.21.15.19:59892) 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84bffff700 1 -- 172.21.15.19:0/2632159302 learned_addr learned my addr 172.21.15.19:0/2632159302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b7fff700 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84bffff700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f84ac0b2880 0x7f84ac0b4d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f84b000b2e0 tx=0x7f84b0005050 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b7fff700 1 -- 172.21.15.19:0/2632159302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 msgr2=0x561d583a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:46.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b7fff700 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583a8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b7fff700 1 -- 172.21.15.19:0/2632159302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 msgr2=0x561d583ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:46.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b7fff700 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b7fff700 1 -- 172.21.15.19:0/2632159302 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d58261500 con 0x561d5822b4a0 2024-05-09T01:37:46.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84bffff700 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:37:46.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b7fff700 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583ae780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009c80 tx=0x7f84a8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a184ccbfa52be0b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:46.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b67fc700 1 -- 172.21.15.19:0/2632159302 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004310 con 0x561d5822b4a0 2024-05-09T01:37:46.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b67fc700 1 -- 172.21.15.19:0/2632159302 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84a8011de0 con 0x561d5822b4a0 2024-05-09T01:37:46.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.422+0000 7f84b67fc700 1 -- 172.21.15.19:0/2632159302 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8005280 con 0x561d5822b4a0 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.423+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2632159302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 msgr2=0x561d583ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.423+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583ae780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009c80 tx=0x7f84a8009f90 comp rx=0 tx=0).stop 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.423+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2632159302 shutdown_connections 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.423+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.423+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.423+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2632159302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583ae780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.423+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2632159302 >> 172.21.15.19:0/2632159302 conn(0x561d5822ef70 msgr2=0x561d583a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.423+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2632159302 shutdown_connections 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.423+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2632159302 wait complete. 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.424+0000 7f84d4f32b80 1 -- start start 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.424+0000 7f84d4f32b80 1 Processor -- start 2024-05-09T01:37:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d583a9160 con 0x561d582a5d00 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d58265b20 con 0x561d583a8400 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d583b5020 con 0x561d5822b4a0 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84d4f32b80 10 client.0 _ll_register_callbacks cb 0x561d5813c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84b7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84b7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59894/0 (socket says 172.21.15.19:59894) 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84b7fff700 1 -- 172.21.15.19:0/2554534452 learned_addr learned my addr 172.21.15.19:0/2554534452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84bffff700 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84b7fff700 1 -- 172.21.15.19:0/2554534452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 msgr2=0x561d583b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:46.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84b7fff700 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84b7fff700 1 -- 172.21.15.19:0/2554534452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 msgr2=0x561d583b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84b7fff700 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84b7fff700 1 -- 172.21.15.19:0/2554534452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d58146490 con 0x561d582a5d00 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f84b7fff700 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583b24f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f84a8004ce0 tx=0x7f84a8011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81126cc3bd34abc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f849f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.425+0000 7f849f7fe700 1 -- 172.21.15.19:0/2554534452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004320 con 0x561d582a5d00 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f849f7fe700 1 -- 172.21.15.19:0/2554534452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f84a80044c0 con 0x561d582a5d00 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2554534452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a8011de0 con 0x561d582a5d00 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f849f7fe700 1 -- 172.21.15.19:0/2554534452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84a8003680 con 0x561d582a5d00 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f84d4f32b80 10 client.18786 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2554534452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d583bf990 con 0x561d582a5d00 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f849f7fe700 1 -- 172.21.15.19:0/2554534452 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84a8011de0 con 0x561d582a5d00 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f849f7fe700 1 -- 172.21.15.19:0/2554534452 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f84a8018070 con 0x561d582a5d00 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f84d4f32b80 10 client.18786 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f84d4f32b80 10 client.18786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f84d4f32b80 20 client.18786 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d583c1eb0 0x561d583c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f84d4f32b80 4 client.18786 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.426+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2554534452 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d582d8a80 con 0x561d583c1eb0 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.427+0000 7f84bffff700 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d583c1eb0 0x561d583c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.427+0000 7f849f7fe700 10 client.18786 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.427+0000 7f84bffff700 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d583c1eb0 0x561d583c42d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eeabbf6ce0b84d4 server_cookie=abea206f0048ea9e in_seq=0 out_seq=0 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f849f7fe700 1 -- 172.21.15.19:0/2554534452 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d582d8a80 con 0x561d583c1eb0 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f849f7fe700 10 client.18786 handle_command_reply: tid=0 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 client.18786 shutdown 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2554534452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 msgr2=0x561d583b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583b24f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f84a8004ce0 tx=0x7f84a8011a20 comp rx=0 tx=0).stop 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2554534452 shutdown_connections 2024-05-09T01:37:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d583c1eb0 0x561d583c42d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/2554534452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583b24f0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f849f7fe700 0 client.18786 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2554534452 >> 172.21.15.19:0/2554534452 conn(0x561d5822ef70 msgr2=0x561d5839cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2554534452 shutdown_connections 2024-05-09T01:37:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 1 -- 172.21.15.19:0/2554534452 wait complete. 2024-05-09T01:37:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.428+0000 7f84d4f32b80 20 client.18786 trim_cache size 0 max 16384 2024-05-09T01:37:46.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84d4f32b80 1 Processor -- start 2024-05-09T01:37:46.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84d4f32b80 1 -- start start 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d5822b4a0 0x561d583bde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d582a5d00 0x561d583b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d583a8400 0x561d583c9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b0e80 con 0x561d583a8400 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b29c0 con 0x561d5822b4a0 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b1550 con 0x561d582a5d00 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84c4ab9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d583a8400 0x561d583c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84c4ab9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d583a8400 0x561d583c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59908/0 (socket says 172.21.15.19:59908) 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84c4ab9700 1 -- 172.21.15.19:0/848163011 learned_addr learned my addr 172.21.15.19:0/848163011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.449+0000 7f84bffff700 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d5822b4a0 0x561d583bde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84b7fff700 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d582a5d00 0x561d583b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84c4ab9700 1 -- 172.21.15.19:0/848163011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d582a5d00 msgr2=0x561d583b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d582a5d00 0x561d583b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84c4ab9700 1 -- 172.21.15.19:0/848163011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d5822b4a0 msgr2=0x561d583bde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d5822b4a0 0x561d583bde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84c4ab9700 1 -- 172.21.15.19:0/848163011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5839f9d0 con 0x561d583a8400 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84b7fff700 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d582a5d00 0x561d583b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d583a8400 0x561d583c9930 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f84b80a8da0 tx=0x7f84b8009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fda23b5f55528fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84b5ffb700 1 -- 172.21.15.19:0/848163011 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8072070 con 0x561d583a8400 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84b5ffb700 1 -- 172.21.15.19:0/848163011 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b80a92e0 con 0x561d583a8400 2024-05-09T01:37:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.450+0000 7f84b5ffb700 1 -- 172.21.15.19:0/848163011 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80b1e80 con 0x561d583a8400 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 -- 172.21.15.19:0/848163011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d583a8400 msgr2=0x561d583c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d583a8400 0x561d583c9930 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f84b80a8da0 tx=0x7f84b8009480 comp rx=0 tx=0).stop 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 -- 172.21.15.19:0/848163011 shutdown_connections 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d583a8400 0x561d583c9930 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d5822b4a0 0x561d583bde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/848163011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d582a5d00 0x561d583b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 -- 172.21.15.19:0/848163011 >> 172.21.15.19:0/848163011 conn(0x561d5822ef70 msgr2=0x561d583a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 -- 172.21.15.19:0/848163011 shutdown_connections 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 -- 172.21.15.19:0/848163011 wait complete. 2024-05-09T01:37:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.451+0000 7f84d4f32b80 1 -- start start 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84d4f32b80 1 Processor -- start 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84d4f32b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b22f0 con 0x561d582a5d00 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b1c20 con 0x561d583a8400 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84d4f32b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01aee20 con 0x561d5822b4a0 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84d4f32b80 10 client.0 _ll_register_callbacks cb 0x561d5813c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84b7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84b7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59914/0 (socket says 172.21.15.19:59914) 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84b7fff700 1 -- 172.21.15.19:0/967982888 learned_addr learned my addr 172.21.15.19:0/967982888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:37:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84bffff700 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.452+0000 7f84b7fff700 1 -- 172.21.15.19:0/967982888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 msgr2=0x561d583c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f84b7fff700 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f84b7fff700 1 -- 172.21.15.19:0/967982888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 msgr2=0x561d583c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f84b7fff700 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f84b7fff700 1 -- 172.21.15.19:0/967982888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5842b620 con 0x561d582a5d00 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f84c4ab9700 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f84b7fff700 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583c9900 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f84a8005410 tx=0x7f84a8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aad0d4563d115aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f849f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f849f7fe700 1 -- 172.21.15.19:0/967982888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a803b060 con 0x561d582a5d00 2024-05-09T01:37:46.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f849f7fe700 1 -- 172.21.15.19:0/967982888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f84a802b400 con 0x561d582a5d00 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f84d4f32b80 1 -- 172.21.15.19:0/967982888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b80a92e0 con 0x561d582a5d00 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f849f7fe700 1 -- 172.21.15.19:0/967982888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84a8011d50 con 0x561d582a5d00 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f84d4f32b80 10 client.18792 fetch_fsmap learned FSMap version 28 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f84d4f32b80 1 -- 172.21.15.19:0/967982888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d5842c800 con 0x561d582a5d00 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.453+0000 7f849f7fe700 1 -- 172.21.15.19:0/967982888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b80a92e0 con 0x561d582a5d00 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f849f7fe700 1 -- 172.21.15.19:0/967982888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f84a8018070 con 0x561d582a5d00 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f84d4f32b80 10 client.18792 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f84d4f32b80 10 client.18792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f84d4f32b80 20 client.18792 populate_metadata read hostname 'smithi019' 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d5842cee0 0x561d5842f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f84d4f32b80 4 client.18792 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f84d4f32b80 1 -- 172.21.15.19:0/967982888 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d582d2b20 con 0x561d5842cee0 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f84bffff700 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d5842cee0 0x561d5842f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f84bffff700 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d5842cee0 0x561d5842f300 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4814bcd572cd83f server_cookie=1b5e0eb20ab5b32d in_seq=0 out_seq=0 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.454+0000 7f849f7fe700 10 client.18792 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f849f7fe700 1 -- 172.21.15.19:0/967982888 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561d582d2b20 con 0x561d5842cee0 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f849f7fe700 10 client.18792 handle_command_reply: tid=0 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 client.18792 shutdown 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 -- 172.21.15.19:0/967982888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 msgr2=0x561d583c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583c9900 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f84a8005410 tx=0x7f84a8005dc0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 -- 172.21.15.19:0/967982888 shutdown_connections 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x561d5842cee0 0x561d5842f300 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d583a8400 0x561d583c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d5822b4a0 0x561d583c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 --2- 172.21.15.19:0/967982888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d582a5d00 0x561d583c9900 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 -- 172.21.15.19:0/967982888 >> 172.21.15.19:0/967982888 conn(0x561d5822ef70 msgr2=0x561d5839cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f849f7fe700 0 client.18792 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 -- 172.21.15.19:0/967982888 shutdown_connections 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 1 -- 172.21.15.19:0/967982888 wait complete. 2024-05-09T01:37:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.455+0000 7f84d4f32b80 20 client.18792 trim_cache size 0 max 16384 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "779223ba-f240-4016-9b97-a9b67fd9d44b": { 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "779223ba-f240-4016-9b97-a9b67fd9d44b", 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:37:46.458 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:37:46.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 -- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f84ac0b2880 msgr2=0x7f84ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:46.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f84ac0b2880 0x7f84ac0b4d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f84b000b2e0 tx=0x7f84b0005050 comp rx=0 tx=0).stop 2024-05-09T01:37:46.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 -- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c010e8f0 msgr2=0x7f84c0103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:37:46.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c010e8f0 0x7f84c0103bf0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f84b800ef30 tx=0x7f84b800c3a0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 -- 172.21.15.19:0/416400508 shutdown_connections 2024-05-09T01:37:46.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f84ac0b2880 0x7f84ac0b4d90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c010e8f0 0x7f84c0103bf0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84c0105480 0x7f84c0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 --2- 172.21.15.19:0/416400508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c0105dd0 0x7f84c01036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:37:46.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 -- 172.21.15.19:0/416400508 >> 172.21.15.19:0/416400508 conn(0x7f84c00fef30 msgr2=0x7f84c00ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:37:46.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 -- 172.21.15.19:0/416400508 shutdown_connections 2024-05-09T01:37:46.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:37:46.456+0000 7f84c651c700 1 -- 172.21.15.19:0/416400508 wait complete. 2024-05-09T01:37:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:46 smithi059 ceph-mon[34351]: pgmap v745: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:37:46.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:46 smithi019 ceph-mon[30158]: pgmap v745: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:37:46.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:46 smithi196 ceph-mon[35041]: pgmap v745: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-05-09T01:37:47.187 INFO:tasks.cephfs.filesystem:scrub status for tag:779223ba-f240-4016-9b97-a9b67fd9d44b - {'path': '/', 'tag': '779223ba-f240-4016-9b97-a9b67fd9d44b', 'options': 'recursive,force'} 2024-05-09T01:37:47.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:47 smithi059 ceph-mon[34351]: scrub summary: idle 2024-05-09T01:37:47.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:47 smithi019 ceph-mon[30158]: scrub summary: idle 2024-05-09T01:37:47.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:47 smithi196 ceph-mon[35041]: scrub summary: idle 2024-05-09T01:37:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:48 smithi059 ceph-mon[34351]: pgmap v746: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:37:48.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:48 smithi019 ceph-mon[30158]: pgmap v746: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:37:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:48 smithi196 ceph-mon[35041]: pgmap v746: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-05-09T01:37:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:50 smithi059 ceph-mon[34351]: pgmap v747: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:37:50.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:50 smithi019 ceph-mon[30158]: pgmap v747: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:37:50.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:50 smithi196 ceph-mon[35041]: pgmap v747: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-05-09T01:37:52.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:52 smithi059 ceph-mon[34351]: pgmap v748: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-05-09T01:37:52.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:52 smithi019 ceph-mon[30158]: pgmap v748: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-05-09T01:37:52.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:52 smithi196 ceph-mon[35041]: pgmap v748: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-05-09T01:37:54.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:54 smithi059 ceph-mon[34351]: pgmap v749: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-05-09T01:37:54.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:54 smithi019 ceph-mon[30158]: pgmap v749: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-05-09T01:37:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:54 smithi196 ceph-mon[35041]: pgmap v749: 97 pgs: 97 active+clean; 4.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-05-09T01:37:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:56 smithi059 ceph-mon[34351]: pgmap v750: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-05-09T01:37:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:56 smithi019 ceph-mon[30158]: pgmap v750: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-05-09T01:37:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:56 smithi196 ceph-mon[35041]: pgmap v750: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-05-09T01:37:58.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:37:58 smithi059 ceph-mon[34351]: pgmap v751: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T01:37:58.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:37:58 smithi019 ceph-mon[30158]: pgmap v751: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T01:37:58.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:37:58 smithi196 ceph-mon[35041]: pgmap v751: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T01:38:00.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:00 smithi059 ceph-mon[34351]: pgmap v752: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T01:38:00.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:00 smithi019 ceph-mon[30158]: pgmap v752: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T01:38:00.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:00 smithi196 ceph-mon[35041]: pgmap v752: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T01:38:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:02 smithi059 ceph-mon[34351]: pgmap v753: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T01:38:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:02 smithi019 ceph-mon[30158]: pgmap v753: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T01:38:02.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:02 smithi196 ceph-mon[35041]: pgmap v753: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T01:38:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:04 smithi059 ceph-mon[34351]: pgmap v754: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-09T01:38:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:38:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:38:04.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:38:04.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:38:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:04 smithi019 ceph-mon[30158]: pgmap v754: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-09T01:38:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:38:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:38:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:38:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:38:04.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:04 smithi196 ceph-mon[35041]: pgmap v754: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-09T01:38:04.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:38:04.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:38:04.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:38:04.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:38:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:06 smithi059 ceph-mon[34351]: pgmap v755: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-05-09T01:38:06.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:06 smithi019 ceph-mon[30158]: pgmap v755: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-05-09T01:38:06.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:06 smithi196 ceph-mon[35041]: pgmap v755: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-05-09T01:38:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:08 smithi059 ceph-mon[34351]: pgmap v756: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T01:38:08.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:08 smithi019 ceph-mon[30158]: pgmap v756: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T01:38:08.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:08 smithi196 ceph-mon[35041]: pgmap v756: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T01:38:10.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:10 smithi059 ceph-mon[34351]: pgmap v757: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-09T01:38:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:10 smithi019 ceph-mon[30158]: pgmap v757: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-09T01:38:10.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:10 smithi196 ceph-mon[35041]: pgmap v757: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-09T01:38:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:12 smithi059 ceph-mon[34351]: pgmap v758: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-09T01:38:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:12 smithi019 ceph-mon[30158]: pgmap v758: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-09T01:38:12.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:12 smithi196 ceph-mon[35041]: pgmap v758: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-09T01:38:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:14 smithi059 ceph-mon[34351]: pgmap v759: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-09T01:38:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:14 smithi019 ceph-mon[30158]: pgmap v759: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-09T01:38:14.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:14 smithi196 ceph-mon[35041]: pgmap v759: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-09T01:38:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:16 smithi059 ceph-mon[34351]: pgmap v760: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-09T01:38:16.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:16 smithi019 ceph-mon[30158]: pgmap v760: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-09T01:38:16.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:16 smithi196 ceph-mon[35041]: pgmap v760: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-09T01:38:17.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:38:17.457 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:18.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:18 smithi019 ceph-mon[30158]: pgmap v761: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:38:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:18 smithi059 ceph-mon[34351]: pgmap v761: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:38:18.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:18 smithi196 ceph-mon[35041]: pgmap v761: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:38:19.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.242+0000 7f2a11eb7700 1 -- 172.21.15.19:0/4128556318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 msgr2=0x7f2a0c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:19.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.242+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/4128556318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 0x7f2a0c1038d0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0099c0 tx=0x7f29fc009cd0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.242+0000 7f2a11eb7700 1 -- 172.21.15.19:0/4128556318 shutdown_connections 2024-05-09T01:38:19.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.242+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/4128556318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a0c108740 0x7f2a0c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.242+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/4128556318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 0x7f2a0c1038d0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.242+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/4128556318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a0c107df0 0x7f2a0c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.242+0000 7f2a11eb7700 1 -- 172.21.15.19:0/4128556318 >> 172.21.15.19:0/4128556318 conn(0x7f2a0c0fedc0 msgr2=0x7f2a0c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:19.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.242+0000 7f2a11eb7700 1 -- 172.21.15.19:0/4128556318 shutdown_connections 2024-05-09T01:38:19.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.242+0000 7f2a11eb7700 1 -- 172.21.15.19:0/4128556318 wait complete. 2024-05-09T01:38:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.243+0000 7f2a11eb7700 1 Processor -- start 2024-05-09T01:38:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.243+0000 7f2a11eb7700 1 -- start start 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.243+0000 7f2a11eb7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 0x7f2a0c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.243+0000 7f2a11eb7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a0c107df0 0x7f2a0c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.243+0000 7f2a11eb7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a0c108740 0x7f2a0c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.243+0000 7f2a11eb7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1133c0 con 0x7f2a0c1034c0 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.243+0000 7f2a11eb7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c113240 con 0x7f2a0c107df0 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.243+0000 7f2a11eb7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c113540 con 0x7f2a0c108740 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 0x7f2a0c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 0x7f2a0c19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49752/0 (socket says 172.21.15.19:49752) 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0b7fe700 1 -- 172.21.15.19:0/1135267728 learned_addr learned my addr 172.21.15.19:0/1135267728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0affd700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a0c107df0 0x7f2a0c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0bfff700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a0c108740 0x7f2a0c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0b7fe700 1 -- 172.21.15.19:0/1135267728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a0c108740 msgr2=0x7f2a0c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a0c108740 0x7f2a0c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0b7fe700 1 -- 172.21.15.19:0/1135267728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a0c107df0 msgr2=0x7f2a0c19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:19.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a0c107df0 0x7f2a0c19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0b7fe700 1 -- 172.21.15.19:0/1135267728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c19fd10 con 0x7f2a0c1034c0 2024-05-09T01:38:19.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 0x7f2a0c19ae40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005910 tx=0x7f29fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:19.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a08ff9700 1 -- 172.21.15.19:0/1135267728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc026070 con 0x7f2a0c1034c0 2024-05-09T01:38:19.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a08ff9700 1 -- 172.21.15.19:0/1135267728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29fc00f460 con 0x7f2a0c1034c0 2024-05-09T01:38:19.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a11eb7700 1 -- 172.21.15.19:0/1135267728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c19ffa0 con 0x7f2a0c1034c0 2024-05-09T01:38:19.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.244+0000 7f2a08ff9700 1 -- 172.21.15.19:0/1135267728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc009e60 con 0x7f2a0c1034c0 2024-05-09T01:38:19.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.245+0000 7f2a11eb7700 1 -- 172.21.15.19:0/1135267728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c1a0480 con 0x7f2a0c1034c0 2024-05-09T01:38:19.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.248+0000 7f2a08ff9700 1 -- 172.21.15.19:0/1135267728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f29fc029070 con 0x7f2a0c1034c0 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.249+0000 7f2a08ff9700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f29f40b2880 0x7f29f40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.249+0000 7f2a208cdb80 1 Processor -- start 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.249+0000 7f2a208cdb80 1 -- start start 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.249+0000 7f2a0affd700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f29f40b2880 0x7f29f40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a08ff9700 1 -- 172.21.15.19:0/1135267728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f29fc0e2b30 con 0x7f2a0c1034c0 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23788ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c23789030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 0x558c2378d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c236a70a0 con 0x558c23681d00 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558c2364c9a0 con 0x558c236074a0 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c236a3060 con 0x558c2378d1e0 2024-05-09T01:38:19.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23788ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23788ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49758/0 (socket says 172.21.15.19:49758) 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0b7fe700 1 -- 172.21.15.19:0/3322806408 learned_addr learned my addr 172.21.15.19:0/3322806408 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0bfff700 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 0x558c2378d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0affd700 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c23789030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0affd700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f29f40b2880 0x7f29f40b4d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c109910 tx=0x7f2a00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0bfff700 1 -- 172.21.15.19:0/3322806408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 msgr2=0x558c23789030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0bfff700 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c23789030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0bfff700 1 -- 172.21.15.19:0/3322806408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 msgr2=0x558c23788ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0bfff700 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23788ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0bfff700 1 -- 172.21.15.19:0/3322806408 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c2363d500 con 0x558c2378d1e0 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0affd700 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c23789030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:38:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.250+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23788ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:38:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.251+0000 7f2a0bfff700 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 0x558c2378d5f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009c90 tx=0x7f2a04007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578d20df069efddd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.251+0000 7f29f1ffb700 1 -- 172.21.15.19:0/3322806408 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04007cd0 con 0x558c2378d1e0 2024-05-09T01:38:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.251+0000 7f29f1ffb700 1 -- 172.21.15.19:0/3322806408 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a04004bb0 con 0x558c2378d1e0 2024-05-09T01:38:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.251+0000 7f29f1ffb700 1 -- 172.21.15.19:0/3322806408 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04014410 con 0x558c2378d1e0 2024-05-09T01:38:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 -- 172.21.15.19:0/3322806408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 msgr2=0x558c2378d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:19.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 0x558c2378d5f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009c90 tx=0x7f2a04007500 comp rx=0 tx=0).stop 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 -- 172.21.15.19:0/3322806408 shutdown_connections 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 0x558c2378d5f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23788ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/3322806408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c23789030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 -- 172.21.15.19:0/3322806408 >> 172.21.15.19:0/3322806408 conn(0x558c2360af70 msgr2=0x558c2377f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 -- 172.21.15.19:0/3322806408 shutdown_connections 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 -- 172.21.15.19:0/3322806408 wait complete. 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.252+0000 7f2a208cdb80 1 -- start start 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a208cdb80 1 Processor -- start 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 0x558c23792350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c23681d00 0x558c23792890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c2378d1e0 0x558c23792dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c2378df00 con 0x558c2378d1e0 2024-05-09T01:38:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558c23641b20 con 0x558c23681d00 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c23795470 con 0x558c236074a0 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a0bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c2378d1e0 0x558c23792dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a0bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c2378d1e0 0x558c23792dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49764/0 (socket says 172.21.15.19:49764) 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a0bfff700 1 -- 172.21.15.19:0/2522179182 learned_addr learned my addr 172.21.15.19:0/2522179182 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 0x558c23792350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a208cdb80 10 client.0 _ll_register_callbacks cb 0x558c235186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.253+0000 7f2a0affd700 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c23681d00 0x558c23792890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f2a0b7fe700 1 -- 172.21.15.19:0/2522179182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c23681d00 msgr2=0x558c23792890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c23681d00 0x558c23792890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f2a0b7fe700 1 -- 172.21.15.19:0/2522179182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c2378d1e0 msgr2=0x558c23792dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c2378d1e0 0x558c23792dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f2a0b7fe700 1 -- 172.21.15.19:0/2522179182 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c23522490 con 0x558c236074a0 2024-05-09T01:38:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f2a0bfff700 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c2378d1e0 0x558c23792dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 0x558c23792350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009990 tx=0x7f29fc0a6100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90024b01cd0e13d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f29eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f29eaffd700 1 -- 172.21.15.19:0/2522179182 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0aa160 con 0x558c236074a0 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f2a208cdb80 1 -- 172.21.15.19:0/2522179182 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a04004bb0 con 0x558c236074a0 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f29eaffd700 1 -- 172.21.15.19:0/2522179182 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f29fc0aa5d0 con 0x558c236074a0 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.254+0000 7f29eaffd700 1 -- 172.21.15.19:0/2522179182 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29fc0ae510 con 0x558c236074a0 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f2a208cdb80 10 client.27880 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f2a208cdb80 1 -- 172.21.15.19:0/2522179182 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c2379fce0 con 0x558c236074a0 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f29eaffd700 1 -- 172.21.15.19:0/2522179182 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a04004bb0 con 0x558c236074a0 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f29eaffd700 1 -- 172.21.15.19:0/2522179182 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f29fc0039d0 con 0x558c236074a0 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f2a208cdb80 10 client.27880 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f2a208cdb80 10 client.27880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f2a208cdb80 20 client.27880 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558c237a22f0 0x558c237a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f2a208cdb80 4 client.27880 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:38:19.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.255+0000 7f2a208cdb80 1 -- 172.21.15.19:0/2522179182 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c236b4a80 con 0x558c237a22f0 2024-05-09T01:38:19.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.256+0000 7f2a0affd700 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558c237a22f0 0x558c237a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.256+0000 7f2a0affd700 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558c237a22f0 0x558c237a4750 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f63d17e873512d server_cookie=2ae1b3c0e43a6813 in_seq=0 out_seq=0 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.256+0000 7f29eaffd700 10 client.27880 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.257+0000 7f29eaffd700 1 -- 172.21.15.19:0/2522179182 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c236b4a80 con 0x558c237a22f0 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.257+0000 7f29eaffd700 10 client.27880 handle_command_reply: tid=0 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.257+0000 7f2a208cdb80 1 client.27880 shutdown 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.257+0000 7f2a208cdb80 1 -- 172.21.15.19:0/2522179182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 msgr2=0x558c23792350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.257+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 0x558c23792350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009990 tx=0x7f29fc0a6100 comp rx=0 tx=0).stop 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f2a208cdb80 1 -- 172.21.15.19:0/2522179182 shutdown_connections 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558c237a22f0 0x558c237a4750 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c2378d1e0 0x558c23792dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f29eaffd700 0 client.27880 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:38:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 0x558c23792350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/2522179182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c23681d00 0x558c23792890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f2a208cdb80 1 -- 172.21.15.19:0/2522179182 >> 172.21.15.19:0/2522179182 conn(0x558c2360af70 msgr2=0x558c23778ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f2a208cdb80 1 -- 172.21.15.19:0/2522179182 shutdown_connections 2024-05-09T01:38:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f2a208cdb80 1 -- 172.21.15.19:0/2522179182 wait complete. 2024-05-09T01:38:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.258+0000 7f2a208cdb80 20 client.27880 trim_cache size 0 max 16384 2024-05-09T01:38:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.278+0000 7f2a208cdb80 1 Processor -- start 2024-05-09T01:38:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.278+0000 7f2a208cdb80 1 -- start start 2024-05-09T01:38:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 0x558c2379e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23783b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c2378d1e0 0x558c237a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1aca80 con 0x558c23681d00 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1ae5c0 con 0x558c2378d1e0 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1ad150 con 0x558c236074a0 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23783b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23783b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49768/0 (socket says 172.21.15.19:49768) 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0affd700 1 -- 172.21.15.19:0/294327263 learned_addr learned my addr 172.21.15.19:0/294327263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 0x558c2379e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0affd700 1 -- 172.21.15.19:0/294327263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 msgr2=0x558c2379e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0affd700 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 0x558c2379e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0affd700 1 -- 172.21.15.19:0/294327263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c2378d1e0 msgr2=0x558c237a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0bfff700 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c2378d1e0 0x558c237a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0affd700 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c2378d1e0 0x558c237a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0affd700 1 -- 172.21.15.19:0/294327263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c2378dde0 con 0x558c23681d00 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0bfff700 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c2378d1e0 0x558c237a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.279+0000 7f2a0affd700 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23783b00 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x558c23780a00 tx=0x7f2a00014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cbc92482d9284d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:19.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.280+0000 7f29f17fa700 1 -- 172.21.15.19:0/294327263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00004560 con 0x558c23681d00 2024-05-09T01:38:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.280+0000 7f29f17fa700 1 -- 172.21.15.19:0/294327263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a00004830 con 0x558c23681d00 2024-05-09T01:38:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.280+0000 7f29f17fa700 1 -- 172.21.15.19:0/294327263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a000055e0 con 0x558c23681d00 2024-05-09T01:38:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 -- 172.21.15.19:0/294327263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 msgr2=0x558c23783b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23783b00 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x558c23780a00 tx=0x7f2a00014000 comp rx=0 tx=0).stop 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 -- 172.21.15.19:0/294327263 shutdown_connections 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c2378d1e0 0x558c237a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c236074a0 0x558c2379e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/294327263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c23783b00 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 -- 172.21.15.19:0/294327263 >> 172.21.15.19:0/294327263 conn(0x558c2360af70 msgr2=0x558c237833f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 -- 172.21.15.19:0/294327263 shutdown_connections 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 -- 172.21.15.19:0/294327263 wait complete. 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.281+0000 7f2a208cdb80 1 -- start start 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a208cdb80 1 Processor -- start 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c2379ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c2378a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a208cdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 0x558c2378ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1adef0 con 0x558c23681d00 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1ad820 con 0x558c236074a0 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a208cdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1aaa20 con 0x558c2378d1e0 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a208cdb80 10 client.0 _ll_register_callbacks cb 0x558c235186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c2378a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c2378a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49782/0 (socket says 172.21.15.19:49782) 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a0b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c2379ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a0affd700 1 -- 172.21.15.19:0/119867073 learned_addr learned my addr 172.21.15.19:0/119867073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.282+0000 7f2a0bfff700 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 0x558c2378ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a0affd700 1 -- 172.21.15.19:0/119867073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 msgr2=0x558c2378ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a0affd700 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 0x558c2378ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a0affd700 1 -- 172.21.15.19:0/119867073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 msgr2=0x558c2379ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a0affd700 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c2379ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a0affd700 1 -- 172.21.15.19:0/119867073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c2378b510 con 0x558c23681d00 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c2379ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a0affd700 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c2378a8a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2a000047d0 tx=0x7f2a0001cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b266827c06bf14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f29eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f29eaffd700 1 -- 172.21.15.19:0/119867073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0001cdc0 con 0x558c23681d00 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f29eaffd700 1 -- 172.21.15.19:0/119867073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2a000049e0 con 0x558c23681d00 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a208cdb80 1 -- 172.21.15.19:0/119867073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a00004830 con 0x558c23681d00 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f29eaffd700 1 -- 172.21.15.19:0/119867073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a00004be0 con 0x558c23681d00 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a208cdb80 10 client.18810 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.283+0000 7f2a208cdb80 1 -- 172.21.15.19:0/119867073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c2380c940 con 0x558c23681d00 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.284+0000 7f29eaffd700 1 -- 172.21.15.19:0/119867073 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a00004830 con 0x558c23681d00 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.284+0000 7f29eaffd700 1 -- 172.21.15.19:0/119867073 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2a00003a40 con 0x558c23681d00 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.284+0000 7f2a208cdb80 10 client.18810 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.284+0000 7f2a208cdb80 10 client.18810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.284+0000 7f2a208cdb80 20 client.18810 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:19.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.284+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558c2380cfe0 0x558c2380f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.284+0000 7f2a208cdb80 4 client.18810 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:38:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.284+0000 7f2a208cdb80 1 -- 172.21.15.19:0/119867073 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c236aeb20 con 0x558c2380cfe0 2024-05-09T01:38:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.284+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558c2380cfe0 0x558c2380f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f29eaffd700 10 client.18810 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:38:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a0b7fe700 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558c2380cfe0 0x558c2380f400 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3186a48a3adb69 server_cookie=de2c86d401e0e9aa in_seq=0 out_seq=0 2024-05-09T01:38:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f29eaffd700 1 -- 172.21.15.19:0/119867073 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558c236aeb20 con 0x558c2380cfe0 2024-05-09T01:38:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f29eaffd700 10 client.18810 handle_command_reply: tid=0 2024-05-09T01:38:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 client.18810 shutdown 2024-05-09T01:38:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 -- 172.21.15.19:0/119867073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 msgr2=0x558c2378a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c2378a8a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2a000047d0 tx=0x7f2a0001cb10 comp rx=0 tx=0).stop 2024-05-09T01:38:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 -- 172.21.15.19:0/119867073 shutdown_connections 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x558c2380cfe0 0x558c2380f400 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c2378d1e0 0x558c2378ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c236074a0 0x558c2379ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 --2- 172.21.15.19:0/119867073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23681d00 0x558c2378a8a0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 -- 172.21.15.19:0/119867073 >> 172.21.15.19:0/119867073 conn(0x558c2360af70 msgr2=0x558c23779780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f29eaffd700 0 client.18810 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 -- 172.21.15.19:0/119867073 shutdown_connections 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 1 -- 172.21.15.19:0/119867073 wait complete. 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.285+0000 7f2a208cdb80 20 client.18810 trim_cache size 0 max 16384 2024-05-09T01:38:19.286 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:38:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T01:38:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T01:38:19.287 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:38:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 -- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f29f40b2880 msgr2=0x7f29f40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f29f40b2880 0x7f29f40b4d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c109910 tx=0x7f2a00009260 comp rx=0 tx=0).stop 2024-05-09T01:38:19.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 -- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 msgr2=0x7f2a0c19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 0x7f2a0c19ae40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005910 tx=0x7f29fc0045e0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 -- 172.21.15.19:0/1135267728 shutdown_connections 2024-05-09T01:38:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a0c108740 0x7f2a0c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f29f40b2880 0x7f29f40b4d90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a0c1034c0 0x7f2a0c19ae40 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 --2- 172.21.15.19:0/1135267728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a0c107df0 0x7f2a0c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 -- 172.21.15.19:0/1135267728 >> 172.21.15.19:0/1135267728 conn(0x7f2a0c0fedc0 msgr2=0x7f2a0c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 -- 172.21.15.19:0/1135267728 shutdown_connections 2024-05-09T01:38:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:19.286+0000 7f2a11eb7700 1 -- 172.21.15.19:0/1135267728 wait complete. 2024-05-09T01:38:20.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:19 smithi019 ceph-mon[30158]: pgmap v762: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T01:38:20.194 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:19 smithi196 ceph-mon[35041]: pgmap v762: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T01:38:20.274 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:38:20.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:38:20.346 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:19 smithi059 ceph-mon[34351]: pgmap v762: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T01:38:20.531 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:22.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.266+0000 7fa7a9999700 1 -- 172.21.15.19:0/4206020808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a4107df0 msgr2=0x7fa7a4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:22.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.266+0000 7fa7a9999700 1 --2- 172.21.15.19:0/4206020808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a4107df0 0x7fa7a4108200 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0099c0 tx=0x7fa78c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.266+0000 7fa7a9999700 1 -- 172.21.15.19:0/4206020808 shutdown_connections 2024-05-09T01:38:22.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.266+0000 7fa7a9999700 1 --2- 172.21.15.19:0/4206020808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7a4108740 0x7fa7a41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.266+0000 7fa7a9999700 1 --2- 172.21.15.19:0/4206020808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a41034c0 0x7fa7a41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.266+0000 7fa7a9999700 1 --2- 172.21.15.19:0/4206020808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a4107df0 0x7fa7a4108200 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.266+0000 7fa7a9999700 1 -- 172.21.15.19:0/4206020808 >> 172.21.15.19:0/4206020808 conn(0x7fa7a40fedc0 msgr2=0x7fa7a41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.266+0000 7fa7a9999700 1 -- 172.21.15.19:0/4206020808 shutdown_connections 2024-05-09T01:38:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.266+0000 7fa7a9999700 1 -- 172.21.15.19:0/4206020808 wait complete. 2024-05-09T01:38:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.267+0000 7fa7a9999700 1 Processor -- start 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.267+0000 7fa7a9999700 1 -- start start 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.267+0000 7fa7a9999700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a41034c0 0x7fa7a419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.267+0000 7fa7a9999700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4107df0 0x7fa7a419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.267+0000 7fa7a9999700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7a4108740 0x7fa7a419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.267+0000 7fa7a9999700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41144c0 con 0x7fa7a41034c0 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.267+0000 7fa7a9999700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4114340 con 0x7fa7a4107df0 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.267+0000 7fa7a9999700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4114640 con 0x7fa7a4108740 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a41034c0 0x7fa7a419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a41034c0 0x7fa7a419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35640/0 (socket says 172.21.15.19:35640) 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a2ffd700 1 -- 172.21.15.19:0/1499306154 learned_addr learned my addr 172.21.15.19:0/1499306154 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a37fe700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7a4108740 0x7fa7a419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a27fc700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4107df0 0x7fa7a419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a2ffd700 1 -- 172.21.15.19:0/1499306154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7a4108740 msgr2=0x7fa7a419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a2ffd700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7a4108740 0x7fa7a419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a2ffd700 1 -- 172.21.15.19:0/1499306154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4107df0 msgr2=0x7fa7a419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a2ffd700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4107df0 0x7fa7a419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a2ffd700 1 -- 172.21.15.19:0/1499306154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a419fce0 con 0x7fa7a41034c0 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a2ffd700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a41034c0 0x7fa7a419ae10 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fa79400ef10 tx=0x7fa79400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a8997700 1 -- 172.21.15.19:0/1499306154 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79400c9b0 con 0x7fa7a41034c0 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a419ff70 con 0x7fa7a41034c0 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a8997700 1 -- 172.21.15.19:0/1499306154 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa794019070 con 0x7fa7a41034c0 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.268+0000 7fa7a8997700 1 -- 172.21.15.19:0/1499306154 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794014480 con 0x7fa7a41034c0 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.269+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a41a78f0 con 0x7fa7a41034c0 2024-05-09T01:38:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.269+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a40512c0 con 0x7fa7a41034c0 2024-05-09T01:38:22.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.271+0000 7fa7a8997700 1 -- 172.21.15.19:0/1499306154 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa794012050 con 0x7fa7a41034c0 2024-05-09T01:38:22.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.272+0000 7fa7a8997700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa7900b2880 0x7fa7900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:22.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.272+0000 7fa7a27fc700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa7900b2880 0x7fa7900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:22.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.272+0000 7fa7a8997700 1 -- 172.21.15.19:0/1499306154 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa7940e27d0 con 0x7fa7a41034c0 2024-05-09T01:38:22.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.273+0000 7fa7a27fc700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa7900b2880 0x7fa7900b4d90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa78c000c00 tx=0x7fa78c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.276+0000 7fa7a8997700 1 -- 172.21.15.19:0/1499306154 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa794010040 con 0x7fa7a41034c0 2024-05-09T01:38:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.539+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7a402e5f0 con 0x7fa7a41034c0 2024-05-09T01:38:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.540+0000 7fa7a8997700 1 -- 172.21.15.19:0/1499306154 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fa7940aa4d0 con 0x7fa7a41034c0 2024-05-09T01:38:22.537 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:38:22.537 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:38:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa7900b2880 msgr2=0x7fa7900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa7900b2880 0x7fa7900b4d90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa78c000c00 tx=0x7fa78c01a040 comp rx=0 tx=0).stop 2024-05-09T01:38:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a41034c0 msgr2=0x7fa7a419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a41034c0 0x7fa7a419ae10 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fa79400ef10 tx=0x7fa79400c3a0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 shutdown_connections 2024-05-09T01:38:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7a4108740 0x7fa7a419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fa7900b2880 0x7fa7900b4d90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7a41034c0 0x7fa7a419ae10 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 --2- 172.21.15.19:0/1499306154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4107df0 0x7fa7a419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.542+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 >> 172.21.15.19:0/1499306154 conn(0x7fa7a40fedc0 msgr2=0x7fa7a4101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.543+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 shutdown_connections 2024-05-09T01:38:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:22.543+0000 7fa7a9999700 1 -- 172.21.15.19:0/1499306154 wait complete. 2024-05-09T01:38:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:38:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:22 smithi059 ceph-mon[34351]: pgmap v763: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T01:38:22.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:22 smithi019 ceph-mon[30158]: pgmap v763: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T01:38:22.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:22 smithi196 ceph-mon[35041]: pgmap v763: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T01:38:23.159 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:38:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:23.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:23 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1499306154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:38:23.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:23 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1499306154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:38:23.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:23 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1499306154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:38:24.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:24 smithi019 ceph-mon[30158]: pgmap v764: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:38:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:24 smithi059 ceph-mon[34351]: pgmap v764: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:38:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:24 smithi196 ceph-mon[35041]: pgmap v764: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T01:38:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.195+0000 7fdc4b95c700 1 -- 172.21.15.19:0/2710272261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 msgr2=0x7fdc44103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.195+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/2710272261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 0x7fdc44103df0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009990 tx=0x7fdc38009ca0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.196+0000 7fdc4b95c700 1 -- 172.21.15.19:0/2710272261 shutdown_connections 2024-05-09T01:38:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.196+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/2710272261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc44104330 0x7fdc44110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.196+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/2710272261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc441092d0 0x7fdc441096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.196+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/2710272261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 0x7fdc44103df0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.196+0000 7fdc4b95c700 1 -- 172.21.15.19:0/2710272261 >> 172.21.15.19:0/2710272261 conn(0x7fdc440fe4e0 msgr2=0x7fdc44100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.196+0000 7fdc4b95c700 1 -- 172.21.15.19:0/2710272261 shutdown_connections 2024-05-09T01:38:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.196+0000 7fdc4b95c700 1 -- 172.21.15.19:0/2710272261 wait complete. 2024-05-09T01:38:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc4b95c700 1 Processor -- start 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc4b95c700 1 -- start start 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc4b95c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 0x7fdc4419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc4b95c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc44104330 0x7fdc4419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc4b95c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc441092d0 0x7fdc4419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc4b95c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc441133c0 con 0x7fdc441039e0 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc4b95c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44113240 con 0x7fdc44104330 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc4b95c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44113540 con 0x7fdc441092d0 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc496f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 0x7fdc4419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc496f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 0x7fdc4419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35648/0 (socket says 172.21.15.19:35648) 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc496f8700 1 -- 172.21.15.19:0/1021202040 learned_addr learned my addr 172.21.15.19:0/1021202040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc44104330 0x7fdc4419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.197+0000 7fdc49ef9700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc441092d0 0x7fdc4419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc496f8700 1 -- 172.21.15.19:0/1021202040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc441092d0 msgr2=0x7fdc4419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc496f8700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc441092d0 0x7fdc4419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc496f8700 1 -- 172.21.15.19:0/1021202040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc44104330 msgr2=0x7fdc4419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc496f8700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc44104330 0x7fdc4419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc496f8700 1 -- 172.21.15.19:0/1021202040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4419fcf0 con 0x7fdc441039e0 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc49ef9700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc441092d0 0x7fdc4419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc44104330 0x7fdc4419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc496f8700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 0x7fdc4419ae20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000d950 tx=0x7fdc4000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc367fc700 1 -- 172.21.15.19:0/1021202040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40016070 con 0x7fdc441039e0 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc4b95c700 1 -- 172.21.15.19:0/1021202040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4419ffe0 con 0x7fdc441039e0 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc367fc700 1 -- 172.21.15.19:0/1021202040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc40010460 con 0x7fdc441039e0 2024-05-09T01:38:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc367fc700 1 -- 172.21.15.19:0/1021202040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000f620 con 0x7fdc441039e0 2024-05-09T01:38:25.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.198+0000 7fdc4b95c700 1 -- 172.21.15.19:0/1021202040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc441a7900 con 0x7fdc441039e0 2024-05-09T01:38:25.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.201+0000 7fdc367fc700 1 -- 172.21.15.19:0/1021202040 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdc40019070 con 0x7fdc441039e0 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.202+0000 7fdc367fc700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc300b2880 0x7fdc300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.202+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc300b2880 0x7fdc300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.202+0000 7fdc367fc700 1 -- 172.21.15.19:0/1021202040 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdc400e2200 con 0x7fdc441039e0 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.202+0000 7fdc5a372b80 1 Processor -- start 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.202+0000 7fdc5a372b80 1 -- start start 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.202+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc300b2880 0x7fdc300b4d90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006010 tx=0x7fdc3801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x5598339fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x5598339fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 0x5598339f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559833914ff0 con 0x55983395f160 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5598338b37f0 con 0x5598339f6ab0 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559833910fb0 con 0x5598338754a0 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc496f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x5598339fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc496f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x5598339fc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35660/0 (socket says 172.21.15.19:35660) 2024-05-09T01:38:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc496f8700 1 -- 172.21.15.19:0/2609105797 learned_addr learned my addr 172.21.15.19:0/2609105797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/2609105797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x5598339fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc49ef9700 1 --2- 172.21.15.19:0/2609105797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 0x5598339f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc496f8700 1 -- 172.21.15.19:0/2609105797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 msgr2=0x5598339fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc496f8700 1 --2- 172.21.15.19:0/2609105797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x5598339fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc496f8700 1 -- 172.21.15.19:0/2609105797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 msgr2=0x5598339f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc496f8700 1 --2- 172.21.15.19:0/2609105797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 0x5598339f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc496f8700 1 -- 172.21.15.19:0/2609105797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598338ab370 con 0x55983395f160 2024-05-09T01:38:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc496f8700 1 --2- 172.21.15.19:0/2609105797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x5598339fc8b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fdc400a6a70 tx=0x7fdc400a9390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877b61f3cccc4166 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:25.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.203+0000 7fdc2f7fe700 1 -- 172.21.15.19:0/2609105797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40020070 con 0x55983395f160 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.204+0000 7fdc2f7fe700 1 -- 172.21.15.19:0/2609105797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc400a9800 con 0x55983395f160 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.204+0000 7fdc2f7fe700 1 -- 172.21.15.19:0/2609105797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc400a9a60 con 0x55983395f160 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.204+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2609105797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 msgr2=0x5598339fc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.204+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2609105797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x5598339fc8b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fdc400a6a70 tx=0x7fdc400a9390 comp rx=0 tx=0).stop 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.204+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2609105797 shutdown_connections 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.204+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2609105797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 0x5598339f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.204+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2609105797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x5598339fc8b0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.204+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2609105797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x5598339fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.204+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2609105797 >> 172.21.15.19:0/2609105797 conn(0x5598338d56a0 msgr2=0x5598339ed400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.205+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2609105797 shutdown_connections 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.205+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2609105797 wait complete. 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.205+0000 7fdc5a372b80 1 -- start start 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.205+0000 7fdc5a372b80 1 Processor -- start 2024-05-09T01:38:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.205+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598338754a0 0x559833a00680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.205+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a00bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.205+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598339f6ab0 0x559833a01100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559833790ba0 con 0x55983395f160 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5598338cf150 con 0x5598338754a0 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598339f7830 con 0x5598339f6ab0 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc5a372b80 10 client.0 _ll_register_callbacks cb 0x5598337866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc48ef7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a00bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc48ef7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a00bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35670/0 (socket says 172.21.15.19:35670) 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc48ef7700 1 -- 172.21.15.19:0/2397231688 learned_addr learned my addr 172.21.15.19:0/2397231688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc496f8700 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598338754a0 0x559833a00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc49ef9700 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598339f6ab0 0x559833a01100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc48ef7700 1 -- 172.21.15.19:0/2397231688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598339f6ab0 msgr2=0x559833a01100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598339f6ab0 0x559833a01100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc48ef7700 1 -- 172.21.15.19:0/2397231688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598338754a0 msgr2=0x559833a00680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598338754a0 0x559833a00680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc48ef7700 1 -- 172.21.15.19:0/2397231688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559833790490 con 0x55983395f160 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a00bc0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009990 tx=0x7fdc38009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74a210f0a5b983f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.206+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2397231688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38034060 con 0x55983395f160 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2397231688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdc38003c70 con 0x55983395f160 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2397231688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc400a9800 con 0x55983395f160 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2397231688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc38005070 con 0x55983395f160 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc5a372b80 10 client.18828 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2397231688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559833a0df30 con 0x55983395f160 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2397231688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc400a9800 con 0x55983395f160 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2397231688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdc38022070 con 0x55983395f160 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc5a372b80 10 client.18828 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc5a372b80 10 client.18828 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc5a372b80 20 client.18828 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x559833a10570 0x559833a12990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc5a372b80 4 client.18828 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:38:25.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.207+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2397231688 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598339229d0 con 0x559833a10570 2024-05-09T01:38:25.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.208+0000 7fdc496f8700 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x559833a10570 0x559833a12990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.208+0000 7fdc2cff9700 10 client.18828 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:38:25.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.208+0000 7fdc496f8700 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x559833a10570 0x559833a12990 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=97451dbc1834330d server_cookie=c30a8fd748abf1b3 in_seq=0 out_seq=0 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2397231688 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598339229d0 con 0x559833a10570 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc2cff9700 10 client.18828 handle_command_reply: tid=0 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 client.18828 shutdown 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2397231688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 msgr2=0x559833a00bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a00bc0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009990 tx=0x7fdc38009ca0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2397231688 shutdown_connections 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x559833a10570 0x559833a12990 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598339f6ab0 0x559833a01100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598338754a0 0x559833a00680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2397231688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a00bc0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2397231688 >> 172.21.15.19:0/2397231688 conn(0x5598338d56a0 msgr2=0x5598339ed1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc2cff9700 0 client.18828 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:38:25.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2397231688 shutdown_connections 2024-05-09T01:38:25.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2397231688 wait complete. 2024-05-09T01:38:25.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.209+0000 7fdc5a372b80 20 client.18828 trim_cache size 0 max 16384 2024-05-09T01:38:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.224+0000 7fdc5a372b80 1 Processor -- start 2024-05-09T01:38:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.224+0000 7fdc5a372b80 1 -- start start 2024-05-09T01:38:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x559833a0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55983395f160 0x5598339ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598339f6ab0 0x5598339fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc441aca20 con 0x5598339f6ab0 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc441ae560 con 0x55983395f160 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc441ad0f0 con 0x5598338754a0 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc49ef9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598339f6ab0 0x5598339fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc49ef9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598339f6ab0 0x5598339fe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35682/0 (socket says 172.21.15.19:35682) 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc49ef9700 1 -- 172.21.15.19:0/1499257367 learned_addr learned my addr 172.21.15.19:0/1499257367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc496f8700 1 --2- 172.21.15.19:0/1499257367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x559833a0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/1499257367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55983395f160 0x5598339ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc49ef9700 1 -- 172.21.15.19:0/1499257367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 msgr2=0x559833a0c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc49ef9700 1 --2- 172.21.15.19:0/1499257367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x559833a0c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc49ef9700 1 -- 172.21.15.19:0/1499257367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55983395f160 msgr2=0x5598339ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc49ef9700 1 --2- 172.21.15.19:0/1499257367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55983395f160 0x5598339ff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc49ef9700 1 -- 172.21.15.19:0/1499257367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598339ee360 con 0x5598339f6ab0 2024-05-09T01:38:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.225+0000 7fdc49ef9700 1 --2- 172.21.15.19:0/1499257367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598339f6ab0 0x5598339fe270 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00c6e0 tx=0x7fdc3c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39d1d99d19321f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc2effd700 1 -- 172.21.15.19:0/1499257367 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c01c020 con 0x5598339f6ab0 2024-05-09T01:38:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc2effd700 1 -- 172.21.15.19:0/1499257367 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc3c018900 con 0x5598339f6ab0 2024-05-09T01:38:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc2effd700 1 -- 172.21.15.19:0/1499257367 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00a790 con 0x5598339f6ab0 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc5a372b80 1 -- 172.21.15.19:0/1499257367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598339f6ab0 msgr2=0x5598339fe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/1499257367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598339f6ab0 0x5598339fe270 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00c6e0 tx=0x7fdc3c009850 comp rx=0 tx=0).stop 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc5a372b80 1 -- 172.21.15.19:0/1499257367 shutdown_connections 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/1499257367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598339f6ab0 0x5598339fe270 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/1499257367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x559833a0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/1499257367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55983395f160 0x5598339ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.226+0000 7fdc5a372b80 1 -- 172.21.15.19:0/1499257367 >> 172.21.15.19:0/1499257367 conn(0x5598338d56a0 msgr2=0x5598339fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.227+0000 7fdc5a372b80 1 -- 172.21.15.19:0/1499257367 shutdown_connections 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.227+0000 7fdc5a372b80 1 -- 172.21.15.19:0/1499257367 wait complete. 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.227+0000 7fdc5a372b80 1 -- start start 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.227+0000 7fdc5a372b80 1 Processor -- start 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.227+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x559833a08420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a08960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc5a372b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 0x559833a170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc441ade90 con 0x55983395f160 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc441ad7c0 con 0x5598339f6ab0 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc5a372b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc441aa9e0 con 0x5598338754a0 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc5a372b80 10 client.0 _ll_register_callbacks cb 0x5598337866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc48ef7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a08960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc48ef7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a08960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35688/0 (socket says 172.21.15.19:35688) 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc48ef7700 1 -- 172.21.15.19:0/2816811875 learned_addr learned my addr 172.21.15.19:0/2816811875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc496f8700 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x559833a08420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc49ef9700 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 0x559833a170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc48ef7700 1 -- 172.21.15.19:0/2816811875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 msgr2=0x559833a08420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x559833a08420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc48ef7700 1 -- 172.21.15.19:0/2816811875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 msgr2=0x559833a170f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 0x559833a170f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc48ef7700 1 -- 172.21.15.19:0/2816811875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559833a17730 con 0x55983395f160 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc49ef9700 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 0x559833a170f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc48ef7700 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a08960 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fdc380098e0 tx=0x7fdc3802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342fc2f686933795 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2816811875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38034040 con 0x55983395f160 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2816811875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdc38005210 con 0x55983395f160 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.228+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2816811875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc3c018900 con 0x55983395f160 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2816811875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc38049070 con 0x55983395f160 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc5a372b80 10 client.18834 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2816811875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559833a7a550 con 0x55983395f160 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2816811875 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc3c018900 con 0x55983395f160 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2816811875 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdc38022040 con 0x55983395f160 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc5a372b80 10 client.18834 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc5a372b80 10 client.18834 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc5a372b80 20 client.18834 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x559833a7aba0 0x559833a7cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc5a372b80 4 client.18834 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2816811875 --> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55983391ca70 con 0x559833a7aba0 2024-05-09T01:38:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.229+0000 7fdc496f8700 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x559833a7aba0 0x559833a7cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc2cff9700 10 client.18834 ms_handle_connect on v2:172.21.15.196:6836/3887501643 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc496f8700 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x559833a7aba0 0x559833a7cfc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34d307e4420aef09 server_cookie=963f8dd7f3f6ee19 in_seq=0 out_seq=0 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc2cff9700 1 -- 172.21.15.19:0/2816811875 <== mds.4 v2:172.21.15.196:6836/3887501643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55983391ca70 con 0x559833a7aba0 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc2cff9700 10 client.18834 handle_command_reply: tid=0 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc5a372b80 1 client.18834 shutdown 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2816811875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 msgr2=0x559833a08960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a08960 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fdc380098e0 tx=0x7fdc3802f710 comp rx=0 tx=0).stop 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2816811875 shutdown_connections 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.196:6836/3887501643,v1:172.21.15.196:6837/3887501643] conn(0x559833a7aba0 0x559833a7cfc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598339f6ab0 0x559833a170f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598338754a0 0x559833a08420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc5a372b80 1 --2- 172.21.15.19:0/2816811875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55983395f160 0x559833a08960 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc2cff9700 0 client.18834 ms_handle_reset on v2:172.21.15.196:6836/3887501643 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.230+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2816811875 >> 172.21.15.19:0/2816811875 conn(0x5598338d56a0 msgr2=0x5598339fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2816811875 shutdown_connections 2024-05-09T01:38:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc5a372b80 1 -- 172.21.15.19:0/2816811875 wait complete. 2024-05-09T01:38:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc5a372b80 20 client.18834 trim_cache size 0 max 16384 2024-05-09T01:38:25.232 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:38:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 -- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc300b2880 msgr2=0x7fdc300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc300b2880 0x7fdc300b4d90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006010 tx=0x7fdc3801a040 comp rx=0 tx=0).stop 2024-05-09T01:38:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 -- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 msgr2=0x7fdc4419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 0x7fdc4419ae20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000d950 tx=0x7fdc4000dd10 comp rx=0 tx=0).stop 2024-05-09T01:38:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 -- 172.21.15.19:0/1021202040 shutdown_connections 2024-05-09T01:38:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc441092d0 0x7fdc4419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdc300b2880 0x7fdc300b4d90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441039e0 0x7fdc4419ae20 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 --2- 172.21.15.19:0/1021202040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc44104330 0x7fdc4419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 -- 172.21.15.19:0/1021202040 >> 172.21.15.19:0/1021202040 conn(0x7fdc440fe4e0 msgr2=0x7fdc44101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 -- 172.21.15.19:0/1021202040 shutdown_connections 2024-05-09T01:38:25.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:25.231+0000 7fdc4b95c700 1 -- 172.21.15.19:0/1021202040 wait complete. 2024-05-09T01:38:26.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:38:26.275 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:26.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:26 smithi019 ceph-mon[30158]: pgmap v765: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:38:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:26 smithi059 ceph-mon[34351]: pgmap v765: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:38:26.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:26 smithi196 ceph-mon[35041]: pgmap v765: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T01:38:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.177+0000 7f527b48e700 1 -- 172.21.15.19:0/3138578115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5274107df0 msgr2=0x7f5274108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.177+0000 7f527b48e700 1 --2- 172.21.15.19:0/3138578115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5274107df0 0x7f5274108200 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f52640099c0 tx=0x7f5264009cd0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.177+0000 7f527b48e700 1 -- 172.21.15.19:0/3138578115 shutdown_connections 2024-05-09T01:38:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.177+0000 7f527b48e700 1 --2- 172.21.15.19:0/3138578115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5274108740 0x7f52741100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.177+0000 7f527b48e700 1 --2- 172.21.15.19:0/3138578115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52741034c0 0x7f52741038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.177+0000 7f527b48e700 1 --2- 172.21.15.19:0/3138578115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5274107df0 0x7f5274108200 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.177+0000 7f527b48e700 1 -- 172.21.15.19:0/3138578115 >> 172.21.15.19:0/3138578115 conn(0x7f52740fedf0 msgr2=0x7f5274101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.177+0000 7f527b48e700 1 -- 172.21.15.19:0/3138578115 shutdown_connections 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.177+0000 7f527b48e700 1 -- 172.21.15.19:0/3138578115 wait complete. 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.178+0000 7f527b48e700 1 Processor -- start 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.178+0000 7f527b48e700 1 -- start start 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.178+0000 7f527b48e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52741034c0 0x7f52740ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.178+0000 7f527b48e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5274107df0 0x7f52740ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.178+0000 7f527b48e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5274108740 0x7f52740ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.178+0000 7f527b48e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5274113350 con 0x7f52741034c0 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527b48e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52741131d0 con 0x7f5274107df0 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527b48e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52741134d0 con 0x7f5274108740 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527922a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52741034c0 0x7f52740ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527922a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52741034c0 0x7f52740ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35700/0 (socket says 172.21.15.19:35700) 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527922a700 1 -- 172.21.15.19:0/3713364080 learned_addr learned my addr 172.21.15.19:0/3713364080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f5278a29700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5274107df0 0x7f52740ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527922a700 1 -- 172.21.15.19:0/3713364080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5274108740 msgr2=0x7f52740ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527922a700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5274108740 0x7f52740ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527922a700 1 -- 172.21.15.19:0/3713364080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5274107df0 msgr2=0x7f52740ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527922a700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5274107df0 0x7f52740ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527922a700 1 -- 172.21.15.19:0/3713364080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52741a9120 con 0x7f52741034c0 2024-05-09T01:38:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527922a700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52741034c0 0x7f52740ff2d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f527000c9b0 tx=0x7f527000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f526a7fc700 1 -- 172.21.15.19:0/3713364080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270007b00 con 0x7f52741034c0 2024-05-09T01:38:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527b48e700 1 -- 172.21.15.19:0/3713364080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52741a9410 con 0x7f52741034c0 2024-05-09T01:38:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f526a7fc700 1 -- 172.21.15.19:0/3713364080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5270007ca0 con 0x7f52741034c0 2024-05-09T01:38:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f526a7fc700 1 -- 172.21.15.19:0/3713364080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270018660 con 0x7f52741034c0 2024-05-09T01:38:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.179+0000 7f527b48e700 1 -- 172.21.15.19:0/3713364080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52741a9920 con 0x7f52741034c0 2024-05-09T01:38:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.182+0000 7f526a7fc700 1 -- 172.21.15.19:0/3713364080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f527000f450 con 0x7f52741034c0 2024-05-09T01:38:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.183+0000 7f526a7fc700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52600b2880 0x7f52600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.183+0000 7f5278a29700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52600b2880 0x7f52600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.183+0000 7f526a7fc700 1 -- 172.21.15.19:0/3713364080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f52700e2210 con 0x7f52741034c0 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.183+0000 7f5289ea4b80 1 Processor -- start 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.183+0000 7f5289ea4b80 1 -- start start 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.183+0000 7f5278a29700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52600b2880 0x7f52600b4d90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5264006010 tx=0x7f5264000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 0x556c09fe24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 0x556c09fe2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09fdc6a0 0x556c09fdcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c09efeff0 con 0x556c09fdc6a0 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556c09e9d7f0 con 0x556c09f49160 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556c09efafb0 con 0x556c09e5f4a0 2024-05-09T01:38:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5279a2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09fdc6a0 0x556c09fdcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5279a2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09fdc6a0 0x556c09fdcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35716/0 (socket says 172.21.15.19:35716) 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5279a2b700 1 -- 172.21.15.19:0/3130550205 learned_addr learned my addr 172.21.15.19:0/3130550205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f527922a700 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 0x556c09fe24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5278a29700 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 0x556c09fe2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5279a2b700 1 -- 172.21.15.19:0/3130550205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 msgr2=0x556c09fe2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5279a2b700 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 0x556c09fe2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5279a2b700 1 -- 172.21.15.19:0/3130550205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 msgr2=0x556c09fe24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5279a2b700 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 0x556c09fe24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5279a2b700 1 -- 172.21.15.19:0/3130550205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c09e95370 con 0x556c09fdc6a0 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f527922a700 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 0x556c09fe24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.184+0000 7f5279a2b700 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09fdc6a0 0x556c09fdcad0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f526c009be0 tx=0x7f526c009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f490f912e16b8c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.185+0000 7f525f7fe700 1 -- 172.21.15.19:0/3130550205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c004bb0 con 0x556c09fdc6a0 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.185+0000 7f525f7fe700 1 -- 172.21.15.19:0/3130550205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c004d50 con 0x556c09fdc6a0 2024-05-09T01:38:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.185+0000 7f525f7fe700 1 -- 172.21.15.19:0/3130550205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c013630 con 0x556c09fdc6a0 2024-05-09T01:38:28.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.185+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3130550205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09fdc6a0 msgr2=0x556c09fdcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:28.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.185+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09fdc6a0 0x556c09fdcad0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f526c009be0 tx=0x7f526c009ef0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.186+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3130550205 shutdown_connections 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.186+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09fdc6a0 0x556c09fdcad0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.186+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 0x556c09fe24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.186+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3130550205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 0x556c09fe2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.186+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3130550205 >> 172.21.15.19:0/3130550205 conn(0x556c09ebf6a0 msgr2=0x556c09fd7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.186+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3130550205 shutdown_connections 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.186+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3130550205 wait complete. 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.186+0000 7f5289ea4b80 1 -- start start 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.186+0000 7f5289ea4b80 1 Processor -- start 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09e5f4a0 0x556c09fe61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 0x556c09fe66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 0x556c09fe6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c09d7aba0 con 0x556c09e5f4a0 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556c09eb9150 con 0x556c09f49160 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556c09fe91d0 con 0x556c09fdc6a0 2024-05-09T01:38:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f5289ea4b80 10 client.0 _ll_register_callbacks cb 0x556c09d706c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f527922a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09e5f4a0 0x556c09fe61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f527922a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09e5f4a0 0x556c09fe61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35718/0 (socket says 172.21.15.19:35718) 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f527922a700 1 -- 172.21.15.19:0/3368498530 learned_addr learned my addr 172.21.15.19:0/3368498530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f5278a29700 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 0x556c09fe66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f5279a2b700 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 0x556c09fe6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f527922a700 1 -- 172.21.15.19:0/3368498530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 msgr2=0x556c09fe6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f527922a700 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 0x556c09fe6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f527922a700 1 -- 172.21.15.19:0/3368498530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 msgr2=0x556c09fe66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f527922a700 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 0x556c09fe66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f527922a700 1 -- 172.21.15.19:0/3368498530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c09d7a490 con 0x556c09e5f4a0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f527922a700 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09e5f4a0 0x556c09fe61b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f5270000c00 tx=0x7f52700a91d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ed3fff8057f943 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.187+0000 7f525cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f525cff9700 1 -- 172.21.15.19:0/3368498530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52700a9320 con 0x556c09e5f4a0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f525cff9700 1 -- 172.21.15.19:0/3368498530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f52700a5e00 con 0x556c09e5f4a0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3368498530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f526c004d50 con 0x556c09e5f4a0 2024-05-09T01:38:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f525cff9700 1 -- 172.21.15.19:0/3368498530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52700a9f10 con 0x556c09e5f4a0 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f5289ea4b80 10 client.18846 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3368498530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c09ff3b90 con 0x556c09e5f4a0 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f525cff9700 1 -- 172.21.15.19:0/3368498530 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c004d50 con 0x556c09e5f4a0 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f525cff9700 1 -- 172.21.15.19:0/3368498530 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f52700a8a70 con 0x556c09e5f4a0 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f5289ea4b80 10 client.18846 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f5289ea4b80 10 client.18846 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f5289ea4b80 20 client.18846 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556c09ff61d0 0x556c09ff8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f5289ea4b80 4 client.18846 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.188+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3368498530 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c09f0c9d0 con 0x556c09ff61d0 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.189+0000 7f5278a29700 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556c09ff61d0 0x556c09ff8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.189+0000 7f5278a29700 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556c09ff61d0 0x556c09ff8650 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f294715d14ef26d8 server_cookie=f203b660db6a256c in_seq=0 out_seq=0 2024-05-09T01:38:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.189+0000 7f525cff9700 10 client.18846 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:38:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f525cff9700 1 -- 172.21.15.19:0/3368498530 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c09f0c9d0 con 0x556c09ff61d0 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f525cff9700 10 client.18846 handle_command_reply: tid=0 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 client.18846 shutdown 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3368498530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09e5f4a0 msgr2=0x556c09fe61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09e5f4a0 0x556c09fe61b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f5270000c00 tx=0x7f52700a91d0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3368498530 shutdown_connections 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556c09ff61d0 0x556c09ff8650 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 0x556c09fe6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09e5f4a0 0x556c09fe61b0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3368498530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09f49160 0x556c09fe66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3368498530 >> 172.21.15.19:0/3368498530 conn(0x556c09ebf6a0 msgr2=0x556c09fd71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f525cff9700 0 client.18846 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3368498530 shutdown_connections 2024-05-09T01:38:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3368498530 wait complete. 2024-05-09T01:38:28.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.190+0000 7f5289ea4b80 20 client.18846 trim_cache size 0 max 16384 2024-05-09T01:38:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.205+0000 7f5289ea4b80 1 Processor -- start 2024-05-09T01:38:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.205+0000 7f5289ea4b80 1 -- start start 2024-05-09T01:38:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 0x556c09fe2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09fe09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09fdc6a0 0x556c09fdbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52741b0f30 con 0x556c09f49160 2024-05-09T01:38:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52741b2a70 con 0x556c09fdc6a0 2024-05-09T01:38:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52741b1600 con 0x556c09e5f4a0 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5278a29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09fe09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5278a29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09fe09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35726/0 (socket says 172.21.15.19:35726) 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5278a29700 1 -- 172.21.15.19:0/3881211264 learned_addr learned my addr 172.21.15.19:0/3881211264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5279a2b700 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09fdc6a0 0x556c09fdbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f527922a700 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 0x556c09fe2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5278a29700 1 -- 172.21.15.19:0/3881211264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 msgr2=0x556c09fe2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5278a29700 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 0x556c09fe2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5278a29700 1 -- 172.21.15.19:0/3881211264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09fdc6a0 msgr2=0x556c09fdbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5278a29700 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09fdc6a0 0x556c09fdbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5278a29700 1 -- 172.21.15.19:0/3881211264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c09fd0f60 con 0x556c09f49160 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5279a2b700 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09fdc6a0 0x556c09fdbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.206+0000 7f5278a29700 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09fe09b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5264009640 tx=0x7f5264021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff2ec506b28e690 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f525effd700 1 -- 172.21.15.19:0/3881211264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264039070 con 0x556c09f49160 2024-05-09T01:38:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f525effd700 1 -- 172.21.15.19:0/3881211264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526400b790 con 0x556c09f49160 2024-05-09T01:38:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f525effd700 1 -- 172.21.15.19:0/3881211264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526400f750 con 0x556c09f49160 2024-05-09T01:38:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3881211264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 msgr2=0x556c09fe09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09fe09b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5264009640 tx=0x7f5264021000 comp rx=0 tx=0).stop 2024-05-09T01:38:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3881211264 shutdown_connections 2024-05-09T01:38:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09fdc6a0 0x556c09fdbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09e5f4a0 0x556c09fe2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/3881211264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09fe09b0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.207+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3881211264 >> 172.21.15.19:0/3881211264 conn(0x556c09ebf6a0 msgr2=0x556c09fdb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.208+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3881211264 shutdown_connections 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.208+0000 7f5289ea4b80 1 -- 172.21.15.19:0/3881211264 wait complete. 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.208+0000 7f5289ea4b80 1 -- start start 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.208+0000 7f5289ea4b80 1 Processor -- start 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09e5f4a0 0x556c09fee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09ffd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5289ea4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 0x556c09ffd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52741b23a0 con 0x556c09f49160 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52741b1cd0 con 0x556c09e5f4a0 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5289ea4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52741aeed0 con 0x556c09fdc6a0 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5289ea4b80 10 client.0 _ll_register_callbacks cb 0x556c09d706c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5278a29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09ffd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5278a29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09ffd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35742/0 (socket says 172.21.15.19:35742) 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5278a29700 1 -- 172.21.15.19:0/1724868405 learned_addr learned my addr 172.21.15.19:0/1724868405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f527922a700 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09e5f4a0 0x556c09fee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5279a2b700 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 0x556c09ffd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5278a29700 1 -- 172.21.15.19:0/1724868405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 msgr2=0x556c09ffd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5278a29700 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 0x556c09ffd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5278a29700 1 -- 172.21.15.19:0/1724868405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09e5f4a0 msgr2=0x556c09fee5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5278a29700 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09e5f4a0 0x556c09fee5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5278a29700 1 -- 172.21.15.19:0/1724868405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c09ffdc80 con 0x556c09f49160 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f527922a700 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09e5f4a0 0x556c09fee5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5278a29700 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09ffd200 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5264005a40 tx=0x7f5264003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a492b84ba6ba9e53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f525cff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f525cff9700 1 -- 172.21.15.19:0/1724868405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264039070 con 0x556c09f49160 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f5289ea4b80 1 -- 172.21.15.19:0/1724868405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f526400b790 con 0x556c09f49160 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.209+0000 7f525cff9700 1 -- 172.21.15.19:0/1724868405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f5264034430 con 0x556c09f49160 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f525cff9700 1 -- 172.21.15.19:0/1724868405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5264034630 con 0x556c09f49160 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f5289ea4b80 10 client.18852 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:28.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f5289ea4b80 1 -- 172.21.15.19:0/1724868405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c09fedfb0 con 0x556c09f49160 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f525cff9700 1 -- 172.21.15.19:0/1724868405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526400b790 con 0x556c09f49160 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f525cff9700 1 -- 172.21.15.19:0/1724868405 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5264022070 con 0x556c09f49160 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f5289ea4b80 10 client.18852 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f5289ea4b80 10 client.18852 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f5289ea4b80 20 client.18852 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556c0a060c20 0x556c0a063040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f5289ea4b80 4 client.18852 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.210+0000 7f5289ea4b80 1 -- 172.21.15.19:0/1724868405 --> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c09f06a70 con 0x556c0a060c20 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f527922a700 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556c0a060c20 0x556c0a063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f527922a700 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556c0a060c20 0x556c0a063040 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=512931e08a6fa63e server_cookie=edd60bc7c7ea4443 in_seq=0 out_seq=0 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f525cff9700 10 client.18852 ms_handle_connect on v2:172.21.15.59:6834/3872629659 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f525cff9700 1 -- 172.21.15.19:0/1724868405 <== mds.3 v2:172.21.15.59:6834/3872629659 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c09f06a70 con 0x556c0a060c20 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f525cff9700 10 client.18852 handle_command_reply: tid=0 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f5289ea4b80 1 client.18852 shutdown 2024-05-09T01:38:28.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f5289ea4b80 1 -- 172.21.15.19:0/1724868405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 msgr2=0x556c09ffd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09ffd200 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5264005a40 tx=0x7f5264003bf0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f5289ea4b80 1 -- 172.21.15.19:0/1724868405 shutdown_connections 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.59:6834/3872629659,v1:172.21.15.59:6835/3872629659] conn(0x556c0a060c20 0x556c0a063040 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c09fdc6a0 0x556c09ffd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c09e5f4a0 0x556c09fee5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f525cff9700 0 client.18852 ms_handle_reset on v2:172.21.15.59:6834/3872629659 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f5289ea4b80 1 --2- 172.21.15.19:0/1724868405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c09f49160 0x556c09ffd200 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.211+0000 7f5289ea4b80 1 -- 172.21.15.19:0/1724868405 >> 172.21.15.19:0/1724868405 conn(0x556c09ebf6a0 msgr2=0x556c09fe4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f5289ea4b80 1 -- 172.21.15.19:0/1724868405 shutdown_connections 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f5289ea4b80 1 -- 172.21.15.19:0/1724868405 wait complete. 2024-05-09T01:38:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f5289ea4b80 20 client.18852 trim_cache size 0 max 16384 2024-05-09T01:38:28.213 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:38:28.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 -- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52600b2880 msgr2=0x7f52600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:28.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52600b2880 0x7f52600b4d90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5264006010 tx=0x7f5264000bc0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 -- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52741034c0 msgr2=0x7f52740ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:28.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52741034c0 0x7f52740ff2d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f527000c9b0 tx=0x7f527000ccc0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 -- 172.21.15.19:0/3713364080 shutdown_connections 2024-05-09T01:38:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5274108740 0x7f52740ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f52600b2880 0x7f52600b4d90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52741034c0 0x7f52740ff2d0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 --2- 172.21.15.19:0/3713364080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5274107df0 0x7f52740ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.212+0000 7f527b48e700 1 -- 172.21.15.19:0/3713364080 >> 172.21.15.19:0/3713364080 conn(0x7f52740fedf0 msgr2=0x7f52741096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.213+0000 7f527b48e700 1 -- 172.21.15.19:0/3713364080 shutdown_connections 2024-05-09T01:38:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:28.213+0000 7f527b48e700 1 -- 172.21.15.19:0/3713364080 wait complete. 2024-05-09T01:38:28.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:28 smithi019 ceph-mon[30158]: pgmap v766: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:28 smithi059 ceph-mon[34351]: pgmap v766: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:28.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:38:28.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:28 smithi196 ceph-mon[35041]: pgmap v766: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:29.110 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:30.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:30 smithi019 ceph-mon[30158]: pgmap v767: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:38:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.799+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/2430316874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc109890 msgr2=0x7fb2dc1111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.799+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/2430316874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc109890 0x7fb2dc1111f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400b0d0 tx=0x7fb2d400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.799+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/2430316874 shutdown_connections 2024-05-09T01:38:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.799+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/2430316874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc109890 0x7fb2dc1111f0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.799+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/2430316874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2dc102b20 0x7fb2dc102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.799+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/2430316874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2dc108f40 0x7fb2dc109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.799+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/2430316874 >> 172.21.15.19:0/2430316874 conn(0x7fb2dc0fe4e0 msgr2=0x7fb2dc100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.799+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/2430316874 shutdown_connections 2024-05-09T01:38:30.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.799+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/2430316874 wait complete. 2024-05-09T01:38:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2e0eaa700 1 Processor -- start 2024-05-09T01:38:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2e0eaa700 1 -- start start 2024-05-09T01:38:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2e0eaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2dc102b20 0x7fb2dc19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2e0eaa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc108f40 0x7fb2dc19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2e0eaa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2dc109890 0x7fb2dc19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2e0eaa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1144d0 con 0x7fb2dc108f40 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2e0eaa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc114350 con 0x7fb2dc102b20 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2e0eaa700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc114650 con 0x7fb2dc109890 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2d9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc108f40 0x7fb2dc19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2d9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc108f40 0x7fb2dc19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47544/0 (socket says 172.21.15.19:47544) 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.800+0000 7fb2d9d9b700 1 -- 172.21.15.19:0/77999751 learned_addr learned my addr 172.21.15.19:0/77999751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2da59c700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2dc102b20 0x7fb2dc19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2dad9d700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2dc109890 0x7fb2dc19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2d9d9b700 1 -- 172.21.15.19:0/77999751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2dc109890 msgr2=0x7fb2dc19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2dc109890 0x7fb2dc19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2d9d9b700 1 -- 172.21.15.19:0/77999751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2dc102b20 msgr2=0x7fb2dc19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2dc102b20 0x7fb2dc19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2d9d9b700 1 -- 172.21.15.19:0/77999751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc19fc70 con 0x7fb2dc108f40 2024-05-09T01:38:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc108f40 0x7fb2dc19b340 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00d950 tx=0x7fb2cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2d37fe700 1 -- 172.21.15.19:0/77999751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc017070 con 0x7fb2dc108f40 2024-05-09T01:38:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2d37fe700 1 -- 172.21.15.19:0/77999751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2cc005020 con 0x7fb2dc108f40 2024-05-09T01:38:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/77999751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc19ff60 con 0x7fb2dc108f40 2024-05-09T01:38:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2d37fe700 1 -- 172.21.15.19:0/77999751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00fa10 con 0x7fb2dc108f40 2024-05-09T01:38:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.801+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/77999751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc1a7880 con 0x7fb2dc108f40 2024-05-09T01:38:30.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.804+0000 7fb2d37fe700 1 -- 172.21.15.19:0/77999751 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb2cc00fbd0 con 0x7fb2dc108f40 2024-05-09T01:38:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.805+0000 7fb2d37fe700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb2c80b2900 0x7fb2c80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.805+0000 7fb2ef8c0b80 1 Processor -- start 2024-05-09T01:38:30.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.805+0000 7fb2da59c700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb2c80b2900 0x7fb2c80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.805+0000 7fb2d37fe700 1 -- 172.21.15.19:0/77999751 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb2cc0e1a50 con 0x7fb2dc108f40 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2ef8c0b80 1 -- start start 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec3438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec343df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec33dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562eec25bff0 con 0x562eec33dab0 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562eec1fa7f0 con 0x562eec2a6160 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562eec257fb0 con 0x562eec1bc4a0 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2da59c700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb2c80b2900 0x7fb2c80b4e10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc06bc50 tx=0x7fb2c4007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2dad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec33dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2dad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec33dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47558/0 (socket says 172.21.15.19:47558) 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2dad9d700 1 -- 172.21.15.19:0/2793183122 learned_addr learned my addr 172.21.15.19:0/2793183122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2da59c700 1 --2- 172.21.15.19:0/2793183122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec3438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2da59c700 1 -- 172.21.15.19:0/2793183122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 msgr2=0x562eec343df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T01:38:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.806+0000 7fb2da59c700 1 --2- 172.21.15.19:0/2793183122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec343df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.807+0000 7fb2da59c700 1 -- 172.21.15.19:0/2793183122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 msgr2=0x562eec33dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.807+0000 7fb2da59c700 1 --2- 172.21.15.19:0/2793183122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec33dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.807+0000 7fb2da59c700 1 -- 172.21.15.19:0/2793183122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eec1f2370 con 0x562eec2a6160 2024-05-09T01:38:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.807+0000 7fb2dad9d700 1 --2- 172.21.15.19:0/2793183122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec33dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:38:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.807+0000 7fb2da59c700 1 --2- 172.21.15.19:0/2793183122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec3438b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4010760 tx=0x7fb2c4013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad64b1dca621bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.807+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/2793183122 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4010d90 con 0x562eec2a6160 2024-05-09T01:38:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.807+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/2793183122 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c4024070 con 0x562eec2a6160 2024-05-09T01:38:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.807+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/2793183122 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40118c0 con 0x562eec2a6160 2024-05-09T01:38:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.808+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/2793183122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 msgr2=0x562eec3438b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.808+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/2793183122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec3438b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4010760 tx=0x7fb2c4013000 comp rx=0 tx=0).stop 2024-05-09T01:38:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.808+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/2793183122 shutdown_connections 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.808+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/2793183122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec33dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.808+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/2793183122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec3438b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.808+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/2793183122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec343df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.808+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/2793183122 >> 172.21.15.19:0/2793183122 conn(0x562eec21c6a0 msgr2=0x562eec334400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.808+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/2793183122 shutdown_connections 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.808+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/2793183122 wait complete. 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2ef8c0b80 1 -- start start 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2ef8c0b80 1 Processor -- start 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec1bc4a0 0x562eec3475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec347b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec33dab0 0x562eec348050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562eec0d7ba0 con 0x562eec1bc4a0 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562eec216150 con 0x562eec2a6160 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562eec33e8f0 con 0x562eec33dab0 2024-05-09T01:38:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2ef8c0b80 10 client.0 _ll_register_callbacks cb 0x562eec0cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec1bc4a0 0x562eec3475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec1bc4a0 0x562eec3475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47560/0 (socket says 172.21.15.19:47560) 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.809+0000 7fb2da59c700 1 -- 172.21.15.19:0/676257366 learned_addr learned my addr 172.21.15.19:0/676257366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec347b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2dad9d700 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec33dab0 0x562eec348050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2da59c700 1 -- 172.21.15.19:0/676257366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec33dab0 msgr2=0x562eec348050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2da59c700 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec33dab0 0x562eec348050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2da59c700 1 -- 172.21.15.19:0/676257366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 msgr2=0x562eec347b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2da59c700 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec347b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2da59c700 1 -- 172.21.15.19:0/676257366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eec0d7490 con 0x562eec1bc4a0 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec347b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2dad9d700 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec33dab0 0x562eec348050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2da59c700 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec1bc4a0 0x562eec3475d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4005e30 tx=0x7fb2c40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4075b06ddb4e50de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:30.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/676257366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40118f0 con 0x562eec1bc4a0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/676257366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c4024070 con 0x562eec1bc4a0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.810+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/676257366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb2c4003680 con 0x562eec1bc4a0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/676257366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2c4012e10 con 0x562eec1bc4a0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2ef8c0b80 10 client.18864 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/676257366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562eec354f40 con 0x562eec1bc4a0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/676257366 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c4024070 con 0x562eec1bc4a0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/676257366 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb2c4004750 con 0x562eec1bc4a0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2ef8c0b80 10 client.18864 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2ef8c0b80 10 client.18864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2ef8c0b80 20 client.18864 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x562eec3574d0 0x562eec3598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2ef8c0b80 4 client.18864 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.811+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/676257366 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eec2699d0 con 0x562eec3574d0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.812+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x562eec3574d0 0x562eec3598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.812+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x562eec3574d0 0x562eec3598f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e99c4711ff8e82f7 server_cookie=ed05fb5013585a63 in_seq=0 out_seq=0 2024-05-09T01:38:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.812+0000 7fb2c1ffb700 10 client.18864 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/676257366 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562eec2699d0 con 0x562eec3574d0 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2c1ffb700 10 client.18864 handle_command_reply: tid=0 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 client.18864 shutdown 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/676257366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec1bc4a0 msgr2=0x562eec3475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec1bc4a0 0x562eec3475d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4005e30 tx=0x7fb2c40044c0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/676257366 shutdown_connections 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec33dab0 0x562eec348050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec1bc4a0 0x562eec3475d0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x562eec3574d0 0x562eec3598f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/676257366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec347b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2c1ffb700 0 client.18864 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:38:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/676257366 >> 172.21.15.19:0/676257366 conn(0x562eec21c6a0 msgr2=0x562eec3341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/676257366 shutdown_connections 2024-05-09T01:38:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/676257366 wait complete. 2024-05-09T01:38:30.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.813+0000 7fb2ef8c0b80 20 client.18864 trim_cache size 0 max 16384 2024-05-09T01:38:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2ef8c0b80 1 Processor -- start 2024-05-09T01:38:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2ef8c0b80 1 -- start start 2024-05-09T01:38:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec345230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec2a6160 0x562eec346890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec33dab0 0x562eec3390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1acae0 con 0x562eec2a6160 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1ae620 con 0x562eec33dab0 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1ad1b0 con 0x562eec1bc4a0 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2d9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec2a6160 0x562eec346890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2d9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec2a6160 0x562eec346890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47570/0 (socket says 172.21.15.19:47570) 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2d9d9b700 1 -- 172.21.15.19:0/439609003 learned_addr learned my addr 172.21.15.19:0/439609003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2dad9d700 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec33dab0 0x562eec3390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2da59c700 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec345230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2d9d9b700 1 -- 172.21.15.19:0/439609003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 msgr2=0x562eec345230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec345230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2d9d9b700 1 -- 172.21.15.19:0/439609003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec33dab0 msgr2=0x562eec3390c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec33dab0 0x562eec3390c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2d9d9b700 1 -- 172.21.15.19:0/439609003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eec32df60 con 0x562eec2a6160 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.829+0000 7fb2da59c700 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec345230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.830+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec2a6160 0x562eec346890 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0df8a0 tx=0x7fb2cc0df400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6553f817426fbdd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.830+0000 7fb2c3fff700 1 -- 172.21.15.19:0/439609003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0c2070 con 0x562eec2a6160 2024-05-09T01:38:30.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.830+0000 7fb2c3fff700 1 -- 172.21.15.19:0/439609003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2cc0ad400 con 0x562eec2a6160 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.830+0000 7fb2c3fff700 1 -- 172.21.15.19:0/439609003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc020050 con 0x562eec2a6160 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.830+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/439609003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec2a6160 msgr2=0x562eec346890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.830+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec2a6160 0x562eec346890 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0df8a0 tx=0x7fb2cc0df400 comp rx=0 tx=0).stop 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/439609003 shutdown_connections 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec33dab0 0x562eec3390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec345230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/439609003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec2a6160 0x562eec346890 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/439609003 >> 172.21.15.19:0/439609003 conn(0x562eec21c6a0 msgr2=0x562eec341200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/439609003 shutdown_connections 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/439609003 wait complete. 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 -- start start 2024-05-09T01:38:30.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 Processor -- start 2024-05-09T01:38:30.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec35c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec3594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec359a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1adf50 con 0x562eec33dab0 2024-05-09T01:38:30.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1ad880 con 0x562eec2a6160 2024-05-09T01:38:30.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.831+0000 7fb2ef8c0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1aaa80 con 0x562eec1bc4a0 2024-05-09T01:38:30.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2ef8c0b80 10 client.0 _ll_register_callbacks cb 0x562eec0cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:30.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2dad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec359a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2dad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec359a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47576/0 (socket says 172.21.15.19:47576) 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2dad9d700 1 -- 172.21.15.19:0/1708536081 learned_addr learned my addr 172.21.15.19:0/1708536081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2da59c700 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec35c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2d9d9b700 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec3594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2dad9d700 1 -- 172.21.15.19:0/1708536081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 msgr2=0x562eec35c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2dad9d700 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec35c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2dad9d700 1 -- 172.21.15.19:0/1708536081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 msgr2=0x562eec3594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2dad9d700 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec3594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2dad9d700 1 -- 172.21.15.19:0/1708536081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eec359f60 con 0x562eec33dab0 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2dad9d700 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec359a20 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4003c30 tx=0x7fb2d4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80f5c3490a24eaa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/1708536081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d402c060 con 0x562eec33dab0 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/1708536081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb2d40052b0 con 0x562eec33dab0 2024-05-09T01:38:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.832+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/1708536081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2cc0ad400 con 0x562eec33dab0 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/1708536081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2d40054b0 con 0x562eec33dab0 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2ef8c0b80 10 client.18870 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/1708536081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562eec3c0540 con 0x562eec33dab0 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/1708536081 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2cc0ad400 con 0x562eec33dab0 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/1708536081 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb2d4026070 con 0x562eec33dab0 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2ef8c0b80 10 client.18870 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2ef8c0b80 10 client.18870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2ef8c0b80 20 client.18870 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x562eec3c0c50 0x562eec3c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2ef8c0b80 4 client.18870 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/1708536081 --> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562eec263a70 con 0x562eec3c0c50 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.833+0000 7fb2da59c700 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x562eec3c0c50 0x562eec3c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2c1ffb700 10 client.18870 ms_handle_connect on v2:172.21.15.196:6838/165915317 2024-05-09T01:38:30.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2da59c700 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x562eec3c0c50 0x562eec3c3070 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6eafc609f151e36 server_cookie=5c269897633f076a in_seq=0 out_seq=0 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2c1ffb700 1 -- 172.21.15.19:0/1708536081 <== mds.2 v2:172.21.15.196:6838/165915317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562eec263a70 con 0x562eec3c0c50 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2c1ffb700 10 client.18870 handle_command_reply: tid=0 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 client.18870 shutdown 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/1708536081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 msgr2=0x562eec359a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec359a20 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4003c30 tx=0x7fb2d4009580 comp rx=0 tx=0).stop 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/1708536081 shutdown_connections 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eec33dab0 0x562eec359a20 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562eec1bc4a0 0x562eec35c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.196:6838/165915317,v1:172.21.15.196:6839/165915317] conn(0x562eec3c0c50 0x562eec3c3070 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 --2- 172.21.15.19:0/1708536081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562eec2a6160 0x562eec3594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/1708536081 >> 172.21.15.19:0/1708536081 conn(0x562eec21c6a0 msgr2=0x562eec342da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2c1ffb700 0 client.18870 ms_handle_reset on v2:172.21.15.196:6838/165915317 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/1708536081 shutdown_connections 2024-05-09T01:38:30.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.834+0000 7fb2ef8c0b80 1 -- 172.21.15.19:0/1708536081 wait complete. 2024-05-09T01:38:30.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2ef8c0b80 20 client.18870 trim_cache size 0 max 16384 2024-05-09T01:38:30.836 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb2c80b2900 msgr2=0x7fb2c80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb2c80b2900 0x7fb2c80b4e10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc06bc50 tx=0x7fb2c4007400 comp rx=0 tx=0).stop 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc108f40 msgr2=0x7fb2dc19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc108f40 0x7fb2dc19b340 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00d950 tx=0x7fb2cc00dc60 comp rx=0 tx=0).stop 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/77999751 shutdown_connections 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2dc109890 0x7fb2dc19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2dc102b20 0x7fb2dc19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fb2c80b2900 0x7fb2c80b4e10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 --2- 172.21.15.19:0/77999751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2dc108f40 0x7fb2dc19b340 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/77999751 >> 172.21.15.19:0/77999751 conn(0x7fb2dc0fe4e0 msgr2=0x7fb2dc1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/77999751 shutdown_connections 2024-05-09T01:38:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:30.835+0000 7fb2e0eaa700 1 -- 172.21.15.19:0/77999751 wait complete. 2024-05-09T01:38:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:30 smithi059 ceph-mon[34351]: pgmap v767: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:38:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:30 smithi196 ceph-mon[35041]: pgmap v767: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:38:31.530 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:38:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:32.746 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:32 smithi196 ceph-mon[35041]: pgmap v768: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-05-09T01:38:32.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:32 smithi059 ceph-mon[34351]: pgmap v768: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-05-09T01:38:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:32 smithi019 ceph-mon[30158]: pgmap v768: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-05-09T01:38:33.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.542+0000 7f434c795700 1 -- 172.21.15.19:0/194105851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441034c0 msgr2=0x7f43441038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:33.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.542+0000 7f434c795700 1 --2- 172.21.15.19:0/194105851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441034c0 0x7f43441038d0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f433400b0d0 tx=0x7f433400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.542+0000 7f434c795700 1 -- 172.21.15.19:0/194105851 shutdown_connections 2024-05-09T01:38:33.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.542+0000 7f434c795700 1 --2- 172.21.15.19:0/194105851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4344108740 0x7f43441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.542+0000 7f434c795700 1 --2- 172.21.15.19:0/194105851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43441034c0 0x7f43441038d0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.542+0000 7f434c795700 1 --2- 172.21.15.19:0/194105851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4344107df0 0x7f4344108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.542+0000 7f434c795700 1 -- 172.21.15.19:0/194105851 >> 172.21.15.19:0/194105851 conn(0x7f43440fedf0 msgr2=0x7f4344101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:33.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.542+0000 7f434c795700 1 -- 172.21.15.19:0/194105851 shutdown_connections 2024-05-09T01:38:33.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.542+0000 7f434c795700 1 -- 172.21.15.19:0/194105851 wait complete. 2024-05-09T01:38:33.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.543+0000 7f434c795700 1 Processor -- start 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.543+0000 7f434c795700 1 -- start start 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.543+0000 7f434c795700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43441034c0 0x7f434419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.543+0000 7f434c795700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4344107df0 0x7f434419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.543+0000 7f434c795700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4344108740 0x7f434419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f434c795700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4344113370 con 0x7f4344107df0 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f434c795700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43441131f0 con 0x7f43441034c0 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f434c795700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43441134f0 con 0x7f4344108740 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f4349d30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4344107df0 0x7f434419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f4349d30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4344107df0 0x7f434419b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47584/0 (socket says 172.21.15.19:47584) 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f4349d30700 1 -- 172.21.15.19:0/1135466717 learned_addr learned my addr 172.21.15.19:0/1135466717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f434a531700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43441034c0 0x7f434419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f434ad32700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4344108740 0x7f434419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f4349d30700 1 -- 172.21.15.19:0/1135466717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4344108740 msgr2=0x7f434419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f4349d30700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4344108740 0x7f434419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f4349d30700 1 -- 172.21.15.19:0/1135466717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43441034c0 msgr2=0x7f434419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f4349d30700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43441034c0 0x7f434419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f4349d30700 1 -- 172.21.15.19:0/1135466717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434419fd20 con 0x7f4344107df0 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f4349d30700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4344107df0 0x7f434419b3f0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f433800e970 tx=0x7f433800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f43437fe700 1 -- 172.21.15.19:0/1135466717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338004a30 con 0x7f4344107df0 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f434c795700 1 -- 172.21.15.19:0/1135466717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43441a0010 con 0x7f4344107df0 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.544+0000 7f43437fe700 1 -- 172.21.15.19:0/1135466717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433800cc90 con 0x7f4344107df0 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.545+0000 7f43437fe700 1 -- 172.21.15.19:0/1135466717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338005610 con 0x7f4344107df0 2024-05-09T01:38:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.545+0000 7f434c795700 1 -- 172.21.15.19:0/1135466717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441a7930 con 0x7f4344107df0 2024-05-09T01:38:33.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.547+0000 7f43437fe700 1 -- 172.21.15.19:0/1135466717 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f433800c7e0 con 0x7f4344107df0 2024-05-09T01:38:33.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.548+0000 7f43437fe700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f432c0b2900 0x7f432c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.548+0000 7f434a531700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f432c0b2900 0x7f432c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.548+0000 7f43437fe700 1 -- 172.21.15.19:0/1135466717 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f43380e2550 con 0x7f4344107df0 2024-05-09T01:38:33.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f434a531700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f432c0b2900 0x7f432c0b4e10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4334009fd0 tx=0x7f433400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:33.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f435b1abb80 1 Processor -- start 2024-05-09T01:38:33.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f435b1abb80 1 -- start start 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4eb8160 0x559db4f514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4dce4a0 0x559db4f51a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 0x559db4f4bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f435b1abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559db4e6dff0 con 0x559db4dce4a0 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f435b1abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559db4e0c7f0 con 0x559db4f4b6a0 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f435b1abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559db4e69fb0 con 0x559db4eb8160 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.549+0000 7f4349d30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4dce4a0 0x559db4f51a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4349d30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4dce4a0 0x559db4f51a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47600/0 (socket says 172.21.15.19:47600) 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4349d30700 1 -- 172.21.15.19:0/926400302 learned_addr learned my addr 172.21.15.19:0/926400302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f434ad32700 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 0x559db4f4bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f434a531700 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4eb8160 0x559db4f514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4349d30700 1 -- 172.21.15.19:0/926400302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4eb8160 msgr2=0x559db4f514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4349d30700 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4eb8160 0x559db4f514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4349d30700 1 -- 172.21.15.19:0/926400302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 msgr2=0x559db4f4bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4349d30700 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 0x559db4f4bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4349d30700 1 -- 172.21.15.19:0/926400302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db4e04370 con 0x559db4dce4a0 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f434a531700 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4eb8160 0x559db4f514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f434ad32700 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 0x559db4f4bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4349d30700 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4dce4a0 0x559db4f51a30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f43380dff10 tx=0x7f433800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e11bb97242f79f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4340ff9700 1 -- 172.21.15.19:0/926400302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43380a9260 con 0x559db4dce4a0 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4340ff9700 1 -- 172.21.15.19:0/926400302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43380b22a0 con 0x559db4dce4a0 2024-05-09T01:38:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.550+0000 7f4340ff9700 1 -- 172.21.15.19:0/926400302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43380a5280 con 0x559db4dce4a0 2024-05-09T01:38:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.551+0000 7f435b1abb80 1 -- 172.21.15.19:0/926400302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4dce4a0 msgr2=0x559db4f51a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.551+0000 7f435b1abb80 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4dce4a0 0x559db4f51a30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f43380dff10 tx=0x7f433800efd0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.551+0000 7f435b1abb80 1 -- 172.21.15.19:0/926400302 shutdown_connections 2024-05-09T01:38:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.551+0000 7f435b1abb80 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 0x559db4f4bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.551+0000 7f435b1abb80 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4eb8160 0x559db4f514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.551+0000 7f435b1abb80 1 --2- 172.21.15.19:0/926400302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4dce4a0 0x559db4f51a30 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.551+0000 7f435b1abb80 1 -- 172.21.15.19:0/926400302 >> 172.21.15.19:0/926400302 conn(0x559db4e2e6a0 msgr2=0x559db4f46400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.551+0000 7f435b1abb80 1 -- 172.21.15.19:0/926400302 shutdown_connections 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.551+0000 7f435b1abb80 1 -- 172.21.15.19:0/926400302 wait complete. 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f435b1abb80 1 -- start start 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f435b1abb80 1 Processor -- start 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4dce4a0 0x559db4f55230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4f4b6a0 0x559db4f55cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f435b1abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559db4ce9ba0 con 0x559db4eb8160 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f435b1abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559db4e28150 con 0x559db4dce4a0 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f435b1abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559db4f4c4e0 con 0x559db4f4b6a0 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f435b1abb80 10 client.0 _ll_register_callbacks cb 0x559db4cdf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f4349d30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f4349d30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47604/0 (socket says 172.21.15.19:47604) 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.552+0000 7f4349d30700 1 -- 172.21.15.19:0/2401386796 learned_addr learned my addr 172.21.15.19:0/2401386796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f434a531700 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4dce4a0 0x559db4f55230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f434ad32700 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4f4b6a0 0x559db4f55cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4349d30700 1 -- 172.21.15.19:0/2401386796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4f4b6a0 msgr2=0x559db4f55cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4349d30700 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4f4b6a0 0x559db4f55cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4349d30700 1 -- 172.21.15.19:0/2401386796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4dce4a0 msgr2=0x559db4f55230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4349d30700 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4dce4a0 0x559db4f55230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4349d30700 1 -- 172.21.15.19:0/2401386796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db4ce9490 con 0x559db4eb8160 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f434ad32700 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4f4b6a0 0x559db4f55cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4349d30700 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55770 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f43380a6d60 tx=0x7f4338004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a56cf768947de6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4329ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4329ffb700 1 -- 172.21.15.19:0/2401386796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433800cad0 con 0x559db4eb8160 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4329ffb700 1 -- 172.21.15.19:0/2401386796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f43380a9d50 con 0x559db4eb8160 2024-05-09T01:38:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f435b1abb80 1 -- 172.21.15.19:0/2401386796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43380b22a0 con 0x559db4eb8160 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f4329ffb700 1 -- 172.21.15.19:0/2401386796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43380a9af0 con 0x559db4eb8160 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f435b1abb80 10 client.18882 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.553+0000 7f435b1abb80 1 -- 172.21.15.19:0/2401386796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559db4f62b50 con 0x559db4eb8160 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f4329ffb700 1 -- 172.21.15.19:0/2401386796 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43380b22a0 con 0x559db4eb8160 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f4329ffb700 1 -- 172.21.15.19:0/2401386796 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f43380b24c0 con 0x559db4eb8160 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f435b1abb80 10 client.18882 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f435b1abb80 10 client.18882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f435b1abb80 20 client.18882 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x559db4f65020 0x559db4f67440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f435b1abb80 4 client.18882 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f435b1abb80 1 -- 172.21.15.19:0/2401386796 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559db4e7b9d0 con 0x559db4f65020 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f434a531700 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x559db4f65020 0x559db4f67440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f4329ffb700 10 client.18882 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:38:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.554+0000 7f434a531700 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x559db4f65020 0x559db4f67440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e66f21117a15ecf server_cookie=62edc106c281cd19 in_seq=0 out_seq=0 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f4329ffb700 1 -- 172.21.15.19:0/2401386796 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559db4e7b9d0 con 0x559db4f65020 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f4329ffb700 10 client.18882 handle_command_reply: tid=0 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f435b1abb80 1 client.18882 shutdown 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f435b1abb80 1 -- 172.21.15.19:0/2401386796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 msgr2=0x559db4f55770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55770 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f43380a6d60 tx=0x7f4338004e90 comp rx=0 tx=0).stop 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f435b1abb80 1 -- 172.21.15.19:0/2401386796 shutdown_connections 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x559db4f65020 0x559db4f67440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4f4b6a0 0x559db4f55cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4dce4a0 0x559db4f55230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2401386796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55770 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f435b1abb80 1 -- 172.21.15.19:0/2401386796 >> 172.21.15.19:0/2401386796 conn(0x559db4e2e6a0 msgr2=0x559db4f461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.555+0000 7f4329ffb700 0 client.18882 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.556+0000 7f435b1abb80 1 -- 172.21.15.19:0/2401386796 shutdown_connections 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.556+0000 7f435b1abb80 1 -- 172.21.15.19:0/2401386796 wait complete. 2024-05-09T01:38:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.556+0000 7f435b1abb80 20 client.18882 trim_cache size 0 max 16384 2024-05-09T01:38:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f435b1abb80 1 Processor -- start 2024-05-09T01:38:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f435b1abb80 1 -- start start 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 0x559db4f54190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 0x559db4f612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f435b1abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43441aede0 con 0x559db4eb8160 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f435b1abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43441b0920 con 0x559db4f4b6a0 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f435b1abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43441af4b0 con 0x559db4dce4a0 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f4349d30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f4349d30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47608/0 (socket says 172.21.15.19:47608) 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.571+0000 7f4349d30700 1 -- 172.21.15.19:0/2470519535 learned_addr learned my addr 172.21.15.19:0/2470519535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f434a531700 1 --2- 172.21.15.19:0/2470519535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 0x559db4f54190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f4349d30700 1 -- 172.21.15.19:0/2470519535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 msgr2=0x559db4f54190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f4349d30700 1 --2- 172.21.15.19:0/2470519535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 0x559db4f54190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f4349d30700 1 -- 172.21.15.19:0/2470519535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 msgr2=0x559db4f612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f4349d30700 1 --2- 172.21.15.19:0/2470519535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 0x559db4f612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f4349d30700 1 -- 172.21.15.19:0/2470519535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db4f468f0 con 0x559db4eb8160 2024-05-09T01:38:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f434a531700 1 --2- 172.21.15.19:0/2470519535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 0x559db4f54190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:38:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f4349d30700 1 --2- 172.21.15.19:0/2470519535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55af0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f43380a9ff0 tx=0x7f433800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43912e9252cd781b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f432bfff700 1 -- 172.21.15.19:0/2470519535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43380a98c0 con 0x559db4eb8160 2024-05-09T01:38:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f432bfff700 1 -- 172.21.15.19:0/2470519535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43380b22a0 con 0x559db4eb8160 2024-05-09T01:38:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.572+0000 7f432bfff700 1 -- 172.21.15.19:0/2470519535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43380a9260 con 0x559db4eb8160 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 -- 172.21.15.19:0/2470519535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 msgr2=0x559db4f55af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2470519535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55af0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f43380a9ff0 tx=0x7f433800efd0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 -- 172.21.15.19:0/2470519535 shutdown_connections 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2470519535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4f4b6a0 0x559db4f612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2470519535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 0x559db4f54190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 --2- 172.21.15.19:0/2470519535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4eb8160 0x559db4f55af0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 -- 172.21.15.19:0/2470519535 >> 172.21.15.19:0/2470519535 conn(0x559db4e2e6a0 msgr2=0x559db4f4ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 -- 172.21.15.19:0/2470519535 shutdown_connections 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 -- 172.21.15.19:0/2470519535 wait complete. 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.573+0000 7f435b1abb80 1 -- start start 2024-05-09T01:38:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f435b1abb80 1 Processor -- start 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 0x559db4fc6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4eb8160 0x559db4fc6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f435b1abb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4f4b6a0 0x559db4f67610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f435b1abb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43441b0250 con 0x559db4f4b6a0 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f435b1abb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43441afb80 con 0x559db4eb8160 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f435b1abb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43441acd80 con 0x559db4dce4a0 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f435b1abb80 10 client.0 _ll_register_callbacks cb 0x559db4f4bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f434ad32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4f4b6a0 0x559db4f67610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f434ad32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4f4b6a0 0x559db4f67610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47618/0 (socket says 172.21.15.19:47618) 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f434ad32700 1 -- 172.21.15.19:0/3369858047 learned_addr learned my addr 172.21.15.19:0/3369858047 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f4349d30700 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4eb8160 0x559db4fc6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f434a531700 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 0x559db4fc6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f434ad32700 1 -- 172.21.15.19:0/3369858047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 msgr2=0x559db4fc6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f434ad32700 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 0x559db4fc6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f434ad32700 1 -- 172.21.15.19:0/3369858047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4eb8160 msgr2=0x559db4fc6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f434ad32700 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4eb8160 0x559db4fc6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.574+0000 7f434ad32700 1 -- 172.21.15.19:0/3369858047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db4f67d40 con 0x559db4f4b6a0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f434ad32700 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4f4b6a0 0x559db4f67610 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f433c010a00 tx=0x7f433c010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1903c0fa26d65339 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f4329ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f4329ffb700 1 -- 172.21.15.19:0/3369858047 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c00fca0 con 0x559db4f4b6a0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f4329ffb700 1 -- 172.21.15.19:0/3369858047 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f433c00fe40 con 0x559db4f4b6a0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f435b1abb80 1 -- 172.21.15.19:0/3369858047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43380b22a0 con 0x559db4f4b6a0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f4329ffb700 1 -- 172.21.15.19:0/3369858047 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f433c01b3f0 con 0x559db4f4b6a0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f435b1abb80 10 client.18888 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f435b1abb80 1 -- 172.21.15.19:0/3369858047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559db4fcef30 con 0x559db4f4b6a0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f4329ffb700 1 -- 172.21.15.19:0/3369858047 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43380b22a0 con 0x559db4f4b6a0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f4329ffb700 1 -- 172.21.15.19:0/3369858047 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f433c025710 con 0x559db4f4b6a0 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f435b1abb80 10 client.18888 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f435b1abb80 10 client.18888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.575+0000 7f435b1abb80 20 client.18888 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.576+0000 7f435b1abb80 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x559db4fcf6c0 0x559db4fd1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.576+0000 7f435b1abb80 4 client.18888 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.576+0000 7f435b1abb80 1 -- 172.21.15.19:0/3369858047 --> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559db4e75a70 con 0x559db4fcf6c0 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.576+0000 7f434a531700 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x559db4fcf6c0 0x559db4fd1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.576+0000 7f4329ffb700 10 client.18888 ms_handle_connect on v2:172.21.15.59:6836/2120139910 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.576+0000 7f434a531700 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x559db4fcf6c0 0x559db4fd1ae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2dd514a549295aa7 server_cookie=7d2730f3bb62cb27 in_seq=0 out_seq=0 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.576+0000 7f4329ffb700 1 -- 172.21.15.19:0/3369858047 <== mds.1 v2:172.21.15.59:6836/2120139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559db4e75a70 con 0x559db4fcf6c0 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.576+0000 7f4329ffb700 10 client.18888 handle_command_reply: tid=0 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.576+0000 7f435b1abb80 1 client.18888 shutdown 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 -- 172.21.15.19:0/3369858047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4f4b6a0 msgr2=0x559db4f67610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4f4b6a0 0x559db4f67610 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f433c010a00 tx=0x7f433c010d10 comp rx=0 tx=0).stop 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 -- 172.21.15.19:0/3369858047 shutdown_connections 2024-05-09T01:38:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.59:6836/2120139910,v1:172.21.15.59:6837/2120139910] conn(0x559db4fcf6c0 0x559db4fd1ae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db4f4b6a0 0x559db4f67610 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559db4dce4a0 0x559db4fc6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 --2- 172.21.15.19:0/3369858047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db4eb8160 0x559db4fc6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 -- 172.21.15.19:0/3369858047 >> 172.21.15.19:0/3369858047 conn(0x559db4e2e6a0 msgr2=0x559db4f4ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f4329ffb700 0 client.18888 ms_handle_reset on v2:172.21.15.59:6836/2120139910 2024-05-09T01:38:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 -- 172.21.15.19:0/3369858047 shutdown_connections 2024-05-09T01:38:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 1 -- 172.21.15.19:0/3369858047 wait complete. 2024-05-09T01:38:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f435b1abb80 20 client.18888 trim_cache size 0 max 16384 2024-05-09T01:38:33.578 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:38:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f434c795700 1 -- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f432c0b2900 msgr2=0x7f432c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f434c795700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f432c0b2900 0x7f432c0b4e10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4334009fd0 tx=0x7f433400b040 comp rx=0 tx=0).stop 2024-05-09T01:38:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f434c795700 1 -- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4344107df0 msgr2=0x7f434419b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.577+0000 7f434c795700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4344107df0 0x7f434419b3f0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f433800e970 tx=0x7f433800ec80 comp rx=0 tx=0).stop 2024-05-09T01:38:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.578+0000 7f434c795700 1 -- 172.21.15.19:0/1135466717 shutdown_connections 2024-05-09T01:38:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.578+0000 7f434c795700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4344108740 0x7f434419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.578+0000 7f434c795700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43441034c0 0x7f434419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.578+0000 7f434c795700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f432c0b2900 0x7f432c0b4e10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.578+0000 7f434c795700 1 --2- 172.21.15.19:0/1135466717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4344107df0 0x7f434419b3f0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.578+0000 7f434c795700 1 -- 172.21.15.19:0/1135466717 >> 172.21.15.19:0/1135466717 conn(0x7f43440fedf0 msgr2=0x7f4344075df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.578+0000 7f434c795700 1 -- 172.21.15.19:0/1135466717 shutdown_connections 2024-05-09T01:38:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:33.578+0000 7f434c795700 1 -- 172.21.15.19:0/1135466717 wait complete. 2024-05-09T01:38:34.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:38:34.637 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:34.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:34 smithi019 ceph-mon[30158]: pgmap v769: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:38:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:34 smithi059 ceph-mon[34351]: pgmap v769: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:38:34.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:34 smithi196 ceph-mon[35041]: pgmap v769: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.473+0000 7fd3f028b700 1 -- 172.21.15.19:0/2737979220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e81054f0 msgr2=0x7fd3e8105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.473+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2737979220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e81054f0 0x7fd3e8105900 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009990 tx=0x7fd3e4009ca0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.473+0000 7fd3f028b700 1 -- 172.21.15.19:0/2737979220 shutdown_connections 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.473+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2737979220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e810e920 0x7fd3e8110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.473+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2737979220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e81054f0 0x7fd3e8105900 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.473+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2737979220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e8105e40 0x7fd3e810e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.473+0000 7fd3f028b700 1 -- 172.21.15.19:0/2737979220 >> 172.21.15.19:0/2737979220 conn(0x7fd3e8078e00 msgr2=0x7fd3e8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.473+0000 7fd3f028b700 1 -- 172.21.15.19:0/2737979220 shutdown_connections 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.473+0000 7fd3f028b700 1 -- 172.21.15.19:0/2737979220 wait complete. 2024-05-09T01:38:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.474+0000 7fd3f028b700 1 Processor -- start 2024-05-09T01:38:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.474+0000 7fd3f028b700 1 -- start start 2024-05-09T01:38:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3f028b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e81054f0 0x7fd3e819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3f028b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e8105e40 0x7fd3e819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3f028b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e810e920 0x7fd3e819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3f028b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8114470 con 0x7fd3e810e920 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3f028b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81142f0 con 0x7fd3e81054f0 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3f028b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81145f0 con 0x7fd3e8105e40 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee828700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e810e920 0x7fd3e819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee027700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e81054f0 0x7fd3e819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee828700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e810e920 0x7fd3e819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47634/0 (socket says 172.21.15.19:47634) 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee027700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e81054f0 0x7fd3e819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35534/0 (socket says 172.21.15.19:35534) 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee828700 1 -- 172.21.15.19:0/2729207534 learned_addr learned my addr 172.21.15.19:0/2729207534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ed826700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e8105e40 0x7fd3e819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee828700 1 -- 172.21.15.19:0/2729207534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e8105e40 msgr2=0x7fd3e819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee828700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e8105e40 0x7fd3e819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee828700 1 -- 172.21.15.19:0/2729207534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e81054f0 msgr2=0x7fd3e819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee828700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e81054f0 0x7fd3e819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.475+0000 7fd3ee828700 1 -- 172.21.15.19:0/2729207534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e819fc00 con 0x7fd3e810e920 2024-05-09T01:38:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.476+0000 7fd3ee828700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e810e920 0x7fd3e819f610 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0007ee0 tx=0x7fd3e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.476+0000 7fd3df7fe700 1 -- 172.21.15.19:0/2729207534 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0004bb0 con 0x7fd3e810e920 2024-05-09T01:38:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.476+0000 7fd3f028b700 1 -- 172.21.15.19:0/2729207534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e819fef0 con 0x7fd3e810e920 2024-05-09T01:38:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.476+0000 7fd3df7fe700 1 -- 172.21.15.19:0/2729207534 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e0004d50 con 0x7fd3e810e920 2024-05-09T01:38:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.476+0000 7fd3f028b700 1 -- 172.21.15.19:0/2729207534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e81a0430 con 0x7fd3e810e920 2024-05-09T01:38:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.476+0000 7fd3df7fe700 1 -- 172.21.15.19:0/2729207534 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0005920 con 0x7fd3e810e920 2024-05-09T01:38:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.479+0000 7fd3df7fe700 1 -- 172.21.15.19:0/2729207534 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd3e001a020 con 0x7fd3e810e920 2024-05-09T01:38:36.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.480+0000 7fd3feca1b80 1 Processor -- start 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.480+0000 7fd3feca1b80 1 -- start start 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.480+0000 7fd3df7fe700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3d40b2900 0x7fd3d40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.480+0000 7fd3ed826700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3d40b2900 0x7fd3d40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.480+0000 7fd3df7fe700 1 -- 172.21.15.19:0/2729207534 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd3e00e17b0 con 0x7fd3e810e920 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.480+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.480+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 0x555a153b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 0x555a153acad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a152ceff0 con 0x555a15319160 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555a1526d7f0 con 0x555a1522f4a0 2024-05-09T01:38:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555a152cafb0 con 0x555a153ac6a0 2024-05-09T01:38:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee027700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee027700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47638/0 (socket says 172.21.15.19:47638) 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee027700 1 -- 172.21.15.19:0/1020301562 learned_addr learned my addr 172.21.15.19:0/1020301562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee828700 1 --2- 172.21.15.19:0/1020301562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 0x555a153acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ed826700 1 --2- 172.21.15.19:0/1020301562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 0x555a153b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee027700 1 -- 172.21.15.19:0/1020301562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 msgr2=0x555a153acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee027700 1 --2- 172.21.15.19:0/1020301562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 0x555a153acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee027700 1 -- 172.21.15.19:0/1020301562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 msgr2=0x555a153b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee027700 1 --2- 172.21.15.19:0/1020301562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 0x555a153b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee027700 1 -- 172.21.15.19:0/1020301562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a15265370 con 0x555a15319160 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ed826700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3d40b2900 0x7fd3d40b4e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8079800 tx=0x7fd3d8013900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.481+0000 7fd3ee027700 1 --2- 172.21.15.19:0/1020301562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153b24f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009990 tx=0x7fd3e4009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae8c09b41da5e6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.484+0000 7fd3dcd47700 1 -- 172.21.15.19:0/1020301562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4003b20 con 0x555a15319160 2024-05-09T01:38:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.484+0000 7fd3dcd47700 1 -- 172.21.15.19:0/1020301562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e4003cc0 con 0x555a15319160 2024-05-09T01:38:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.484+0000 7fd3dcd47700 1 -- 172.21.15.19:0/1020301562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400bd30 con 0x555a15319160 2024-05-09T01:38:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1020301562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 msgr2=0x555a153b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1020301562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153b24f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009990 tx=0x7fd3e4009640 comp rx=0 tx=0).stop 2024-05-09T01:38:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1020301562 shutdown_connections 2024-05-09T01:38:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1020301562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 0x555a153acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1020301562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153b24f0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1020301562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 0x555a153b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1020301562 >> 172.21.15.19:0/1020301562 conn(0x555a1528f6a0 msgr2=0x555a153a7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1020301562 shutdown_connections 2024-05-09T01:38:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1020301562 wait complete. 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 -- start start 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.485+0000 7fd3feca1b80 1 Processor -- start 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1522f4a0 0x555a153b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a15319160 0x555a153b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 0x555a153b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555a1514aba0 con 0x555a1522f4a0 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555a15289150 con 0x555a153ac6a0 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555a153ad4e0 con 0x555a15319160 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3feca1b80 10 client.0 _ll_register_callbacks cb 0x555a151406c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee027700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1522f4a0 0x555a153b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee027700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1522f4a0 0x555a153b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47652/0 (socket says 172.21.15.19:47652) 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee027700 1 -- 172.21.15.19:0/1544375659 learned_addr learned my addr 172.21.15.19:0/1544375659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee828700 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 0x555a153b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ed826700 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a15319160 0x555a153b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee027700 1 -- 172.21.15.19:0/1544375659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a15319160 msgr2=0x555a153b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee027700 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a15319160 0x555a153b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee027700 1 -- 172.21.15.19:0/1544375659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 msgr2=0x555a153b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee027700 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 0x555a153b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee027700 1 -- 172.21.15.19:0/1544375659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1514a490 con 0x555a1522f4a0 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.486+0000 7fd3ee828700 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 0x555a153b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3ee027700 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1522f4a0 0x555a153b61a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4000c00 tx=0x7fd3e4003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a94346785aa8685 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/1544375659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e402f020 con 0x555a1522f4a0 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/1544375659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd3e4005420 con 0x555a1522f4a0 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1544375659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e4003cc0 con 0x555a1522f4a0 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/1544375659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd3e400b910 con 0x555a1522f4a0 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3feca1b80 10 client.18900 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1544375659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a153c3b00 con 0x555a1522f4a0 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/1544375659 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e4003cc0 con 0x555a1522f4a0 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/1544375659 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd3e400fb20 con 0x555a1522f4a0 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3feca1b80 10 client.18900 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3feca1b80 10 client.18900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3feca1b80 20 client.18900 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555a153c5fd0 0x555a153c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3feca1b80 4 client.18900 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.487+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1544375659 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a152dc9d0 con 0x555a153c5fd0 2024-05-09T01:38:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.488+0000 7fd3ed826700 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555a153c5fd0 0x555a153c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.488+0000 7fd3ed826700 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555a153c5fd0 0x555a153c83f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ae252c8b5eda0e server_cookie=ab574ebd348a3693 in_seq=0 out_seq=0 2024-05-09T01:38:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.488+0000 7fd3d1ffb700 10 client.18900 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/1544375659 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a152dc9d0 con 0x555a153c5fd0 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3d1ffb700 10 client.18900 handle_command_reply: tid=0 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3feca1b80 1 client.18900 shutdown 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1544375659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1522f4a0 msgr2=0x555a153b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1522f4a0 0x555a153b61a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4000c00 tx=0x7fd3e4003b50 comp rx=0 tx=0).stop 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1544375659 shutdown_connections 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555a153c5fd0 0x555a153c83f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 0x555a153b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3d1ffb700 0 client.18900 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a1522f4a0 0x555a153b61a0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/1544375659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a15319160 0x555a153b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.489+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1544375659 >> 172.21.15.19:0/1544375659 conn(0x555a1528f6a0 msgr2=0x555a153a71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.490+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1544375659 shutdown_connections 2024-05-09T01:38:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.490+0000 7fd3feca1b80 1 -- 172.21.15.19:0/1544375659 wait complete. 2024-05-09T01:38:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.490+0000 7fd3feca1b80 20 client.18900 trim_cache size 0 max 16384 2024-05-09T01:38:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3feca1b80 1 Processor -- start 2024-05-09T01:38:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3feca1b80 1 -- start start 2024-05-09T01:38:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522f4a0 0x555a153b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 0x555a153b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81aec00 con 0x555a15319160 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81b0740 con 0x555a153ac6a0 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81af2d0 con 0x555a1522f4a0 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ed826700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ed826700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47668/0 (socket says 172.21.15.19:47668) 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ed826700 1 -- 172.21.15.19:0/4075174756 learned_addr learned my addr 172.21.15.19:0/4075174756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ee027700 1 --2- 172.21.15.19:0/4075174756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522f4a0 0x555a153b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ee828700 1 --2- 172.21.15.19:0/4075174756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 0x555a153b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ed826700 1 -- 172.21.15.19:0/4075174756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522f4a0 msgr2=0x555a153b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ed826700 1 --2- 172.21.15.19:0/4075174756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522f4a0 0x555a153b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ed826700 1 -- 172.21.15.19:0/4075174756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 msgr2=0x555a153b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ed826700 1 --2- 172.21.15.19:0/4075174756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 0x555a153b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.505+0000 7fd3ed826700 1 -- 172.21.15.19:0/4075174756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a153a8d00 con 0x555a15319160 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.506+0000 7fd3ed826700 1 --2- 172.21.15.19:0/4075174756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c22f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8004000 tx=0x7fd3d801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553bff365716b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.506+0000 7fd3d3fff700 1 -- 172.21.15.19:0/4075174756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8003ce0 con 0x555a15319160 2024-05-09T01:38:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.506+0000 7fd3d3fff700 1 -- 172.21.15.19:0/4075174756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d802d070 con 0x555a15319160 2024-05-09T01:38:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.506+0000 7fd3d3fff700 1 -- 172.21.15.19:0/4075174756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80207b0 con 0x555a15319160 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.506+0000 7fd3feca1b80 1 -- 172.21.15.19:0/4075174756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 msgr2=0x555a153c22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/4075174756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c22f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8004000 tx=0x7fd3d801b000 comp rx=0 tx=0).stop 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 -- 172.21.15.19:0/4075174756 shutdown_connections 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/4075174756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a153ac6a0 0x555a153b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/4075174756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522f4a0 0x555a153b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/4075174756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c22f0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 -- 172.21.15.19:0/4075174756 >> 172.21.15.19:0/4075174756 conn(0x555a1528f6a0 msgr2=0x555a153afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 -- 172.21.15.19:0/4075174756 shutdown_connections 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 -- 172.21.15.19:0/4075174756 wait complete. 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 -- start start 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 Processor -- start 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 0x555a153c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 0x555a153c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81b0070 con 0x555a15319160 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81af9a0 con 0x555a1522f4a0 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.507+0000 7fd3feca1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81acba0 con 0x555a153ac6a0 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3feca1b80 10 client.0 _ll_register_callbacks cb 0x555a151406c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ed826700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ed826700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47678/0 (socket says 172.21.15.19:47678) 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ed826700 1 -- 172.21.15.19:0/2666380027 learned_addr learned my addr 172.21.15.19:0/2666380027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ee027700 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 0x555a153c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ee828700 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 0x555a153c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ed826700 1 -- 172.21.15.19:0/2666380027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 msgr2=0x555a153c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ed826700 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 0x555a153c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ed826700 1 -- 172.21.15.19:0/2666380027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 msgr2=0x555a153c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ed826700 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 0x555a153c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ed826700 1 -- 172.21.15.19:0/2666380027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1542e640 con 0x555a15319160 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ee828700 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 0x555a153c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3ed826700 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c8930 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005fa0 tx=0x7fd3d8003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2310b4d7af5f340f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/2666380027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8005450 con 0x555a15319160 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/2666380027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fd3d80055f0 con 0x555a15319160 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.508+0000 7fd3feca1b80 1 -- 172.21.15.19:0/2666380027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d802d070 con 0x555a15319160 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/2666380027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd3d8028380 con 0x555a15319160 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3feca1b80 10 client.18906 fetch_fsmap learned FSMap version 28 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3feca1b80 1 -- 172.21.15.19:0/2666380027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a1542f8a0 con 0x555a15319160 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/2666380027 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d802d070 con 0x555a15319160 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/2666380027 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd3d8010430 con 0x555a15319160 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3feca1b80 10 client.18906 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3feca1b80 10 client.18906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T01:38:36.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3feca1b80 20 client.18906 populate_metadata read hostname 'smithi019' 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555a153c5fd0 0x555a15431f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3feca1b80 4 client.18906 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.509+0000 7fd3feca1b80 1 -- 172.21.15.19:0/2666380027 --> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a152d6a70 con 0x555a153c5fd0 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.510+0000 7fd3ee027700 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555a153c5fd0 0x555a15431f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.510+0000 7fd3d1ffb700 10 client.18906 ms_handle_connect on v2:172.21.15.59:6838/1954332745 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.510+0000 7fd3ee027700 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555a153c5fd0 0x555a15431f00 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d38af192bfcb76f9 server_cookie=5359c6da88b6a6c1 in_seq=0 out_seq=0 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3d1ffb700 1 -- 172.21.15.19:0/2666380027 <== mds.0 v2:172.21.15.59:6838/1954332745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a152d6a70 con 0x555a153c5fd0 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3d1ffb700 10 client.18906 handle_command_reply: tid=0 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 client.18906 shutdown 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 -- 172.21.15.19:0/2666380027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 msgr2=0x555a153c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c8930 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005fa0 tx=0x7fd3d8003ae0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 -- 172.21.15.19:0/2666380027 shutdown_connections 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.59:6838/1954332745,v1:172.21.15.59:6839/1954332745] conn(0x555a153c5fd0 0x555a15431f00 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3d1ffb700 0 client.18906 ms_handle_reset on v2:172.21.15.59:6838/1954332745 2024-05-09T01:38:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a153ac6a0 0x555a153c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555a1522f4a0 0x555a153c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 --2- 172.21.15.19:0/2666380027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555a15319160 0x555a153c8930 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 -- 172.21.15.19:0/2666380027 >> 172.21.15.19:0/2666380027 conn(0x555a1528f6a0 msgr2=0x555a153b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 -- 172.21.15.19:0/2666380027 shutdown_connections 2024-05-09T01:38:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 1 -- 172.21.15.19:0/2666380027 wait complete. 2024-05-09T01:38:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.511+0000 7fd3feca1b80 20 client.18906 trim_cache size 0 max 16384 2024-05-09T01:38:36.512 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T01:38:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 -- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3d40b2900 msgr2=0x7fd3d40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3d40b2900 0x7fd3d40b4e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8079800 tx=0x7fd3d8013900 comp rx=0 tx=0).stop 2024-05-09T01:38:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 -- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e810e920 msgr2=0x7fd3e819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e810e920 0x7fd3e819f610 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0007ee0 tx=0x7fd3e000d3b0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 -- 172.21.15.19:0/2729207534 shutdown_connections 2024-05-09T01:38:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fd3d40b2900 0x7fd3d40b4e10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e810e920 0x7fd3e819f610 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e81054f0 0x7fd3e819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 --2- 172.21.15.19:0/2729207534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e8105e40 0x7fd3e819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 -- 172.21.15.19:0/2729207534 >> 172.21.15.19:0/2729207534 conn(0x7fd3e8078e00 msgr2=0x7fd3e8100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 -- 172.21.15.19:0/2729207534 shutdown_connections 2024-05-09T01:38:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:36.512+0000 7fd3f028b700 1 -- 172.21.15.19:0/2729207534 wait complete. 2024-05-09T01:38:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:36 smithi059 ceph-mon[34351]: pgmap v770: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:38:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:36 smithi019 ceph-mon[30158]: pgmap v770: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:38:36.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:36 smithi196 ceph-mon[35041]: pgmap v770: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:38:37.297 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-09T01:38:37.298 INFO:tasks.fwd_scrub:done joining 2024-05-09T01:38:37.298 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T01:38:37.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T01:38:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.433+0000 7f2f37eb1640 1 -- 172.21.15.19:0/295371554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30109b70 msgr2=0x7f2f3010bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.433+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/295371554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30109b70 0x7f2f3010bf90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009ca0 tx=0x7f2f2802a8c0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.433+0000 7f2f37eb1640 1 -- 172.21.15.19:0/295371554 shutdown_connections 2024-05-09T01:38:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.433+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/295371554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30109b70 0x7f2f3010bf90 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.433+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/295371554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f30104e70 0x7f2f30105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.433+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/295371554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f30103c30 0x7f2f30104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.433+0000 7f2f37eb1640 1 -- 172.21.15.19:0/295371554 >> 172.21.15.19:0/295371554 conn(0x7f2f300fee40 msgr2=0x7f2f301012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.434+0000 7f2f37eb1640 1 -- 172.21.15.19:0/295371554 shutdown_connections 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.434+0000 7f2f37eb1640 1 -- 172.21.15.19:0/295371554 wait complete. 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.434+0000 7f2f37eb1640 1 Processor -- start 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.434+0000 7f2f37eb1640 1 -- start start 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f37eb1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f30103c30 0x7f2f3019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f37eb1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30104e70 0x7f2f3019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f37eb1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f30109b70 0x7f2f301a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f37eb1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3010f740 con 0x7f2f30104e70 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f37eb1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3010f5c0 con 0x7f2f30109b70 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f37eb1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3010f8c0 con 0x7f2f30103c30 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3544c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30104e70 0x7f2f3019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3544c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30104e70 0x7f2f3019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47700/0 (socket says 172.21.15.19:47700) 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3544c640 1 -- 172.21.15.19:0/916910570 learned_addr learned my addr 172.21.15.19:0/916910570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f35c4d640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f30103c30 0x7f2f3019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3644e640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f30109b70 0x7f2f301a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3544c640 1 -- 172.21.15.19:0/916910570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f30103c30 msgr2=0x7f2f3019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3544c640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f30103c30 0x7f2f3019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3544c640 1 -- 172.21.15.19:0/916910570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f30109b70 msgr2=0x7f2f301a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3544c640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f30109b70 0x7f2f301a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3544c640 1 -- 172.21.15.19:0/916910570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f301a1df0 con 0x7f2f30104e70 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f35c4d640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f30103c30 0x7f2f3019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f3544c640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30104e70 0x7f2f3019a7f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000d950 tx=0x7f2f2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f26ffd640 1 -- 172.21.15.19:0/916910570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20017070 con 0x7f2f30104e70 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f26ffd640 1 -- 172.21.15.19:0/916910570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f20009c30 con 0x7f2f30104e70 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f301a20e0 con 0x7f2f30104e70 2024-05-09T01:38:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f26ffd640 1 -- 172.21.15.19:0/916910570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000f620 con 0x7f2f30104e70 2024-05-09T01:38:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.435+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f301a2630 con 0x7f2f30104e70 2024-05-09T01:38:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.436+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f30108980 con 0x7f2f30104e70 2024-05-09T01:38:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.438+0000 7f2f26ffd640 1 -- 172.21.15.19:0/916910570 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2f20010460 con 0x7f2f30104e70 2024-05-09T01:38:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.439+0000 7f2f26ffd640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f1c0b27f0 0x7f2f1c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.439+0000 7f2f35c4d640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f1c0b27f0 0x7f2f1c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.439+0000 7f2f26ffd640 1 -- 172.21.15.19:0/916910570 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2f200e1f50 con 0x7f2f30104e70 2024-05-09T01:38:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.439+0000 7f2f35c4d640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f1c0b27f0 0x7f2f1c0b4d00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2f30104830 tx=0x7f2f2c00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.443+0000 7f2f26ffd640 1 -- 172.21.15.19:0/916910570 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f200a9bf0 con 0x7f2f30104e70 2024-05-09T01:38:37.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.640+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f300638d0 con 0x7f2f30104e70 2024-05-09T01:38:37.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.642+0000 7f2f26ffd640 1 -- 172.21.15.19:0/916910570 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22716 (secure 0 0 0) 0x7f2f20009de0 con 0x7f2f30104e70 2024-05-09T01:38:37.639 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:38:37.639 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:23:22.298486+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f1c0b27f0 msgr2=0x7f2f1c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f1c0b27f0 0x7f2f1c0b4d00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2f30104830 tx=0x7f2f2c00a560 comp rx=0 tx=0).stop 2024-05-09T01:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30104e70 msgr2=0x7f2f3019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30104e70 0x7f2f3019a7f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000d950 tx=0x7f2f2000de10 comp rx=0 tx=0).stop 2024-05-09T01:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 shutdown_connections 2024-05-09T01:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f2f1c0b27f0 0x7f2f1c0b4d00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f30109b70 0x7f2f301a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f30104e70 0x7f2f3019a7f0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 --2- 172.21.15.19:0/916910570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f30103c30 0x7f2f3019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.644+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 >> 172.21.15.19:0/916910570 conn(0x7f2f300fee40 msgr2=0x7f2f30105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.645+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 shutdown_connections 2024-05-09T01:38:37.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.645+0000 7f2f37eb1640 1 -- 172.21.15.19:0/916910570 wait complete. 2024-05-09T01:38:37.653 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T01:17:49.120859+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T01:22:34.932415+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T01:22:38.555946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T01:38:37.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T01:38:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 -- 172.21.15.19:0/317044770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8106d40 msgr2=0x7f98d8109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 --2- 172.21.15.19:0/317044770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8106d40 0x7f98d8109220 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009900 tx=0x7f98cc02a410 comp rx=0 tx=0).stop 2024-05-09T01:38:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 -- 172.21.15.19:0/317044770 shutdown_connections 2024-05-09T01:38:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 --2- 172.21.15.19:0/317044770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98d81098b0 0x7f98d810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 --2- 172.21.15.19:0/317044770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8106d40 0x7f98d8109220 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 --2- 172.21.15.19:0/317044770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d8101f60 0x7f98d8102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 -- 172.21.15.19:0/317044770 >> 172.21.15.19:0/317044770 conn(0x7f98d80fd470 msgr2=0x7f98d80ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 -- 172.21.15.19:0/317044770 shutdown_connections 2024-05-09T01:38:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 -- 172.21.15.19:0/317044770 wait complete. 2024-05-09T01:38:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.775+0000 7f98dd984640 1 Processor -- start 2024-05-09T01:38:37.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98dd984640 1 -- start start 2024-05-09T01:38:37.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98dd984640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8101f60 0x7f98d819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:37.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98dd984640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98d8106d40 0x7f98d819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98dd984640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d81098b0 0x7f98d81a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98dd984640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98d810f740 con 0x7f98d8101f60 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98dd984640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98d810f5c0 con 0x7f98d8106d40 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98dd984640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98d810f8c0 con 0x7f98d81098b0 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8101f60 0x7f98d819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8101f60 0x7f98d819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47708/0 (socket says 172.21.15.19:47708) 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d6ffd640 1 -- 172.21.15.19:0/3882808483 learned_addr learned my addr 172.21.15.19:0/3882808483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d67fc640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98d8106d40 0x7f98d819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d77fe640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d81098b0 0x7f98d81a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d6ffd640 1 -- 172.21.15.19:0/3882808483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d81098b0 msgr2=0x7f98d81a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d6ffd640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d81098b0 0x7f98d81a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d6ffd640 1 -- 172.21.15.19:0/3882808483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98d8106d40 msgr2=0x7f98d819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d6ffd640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98d8106d40 0x7f98d819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d6ffd640 1 -- 172.21.15.19:0/3882808483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d81a1eb0 con 0x7f98d8101f60 2024-05-09T01:38:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d77fe640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d81098b0 0x7f98d81a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:38:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.776+0000 7f98d6ffd640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8101f60 0x7f98d819a370 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e9f0 tx=0x7f98c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.777+0000 7f98dc982640 1 -- 172.21.15.19:0/3882808483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c800ccc0 con 0x7f98d8101f60 2024-05-09T01:38:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.777+0000 7f98dc982640 1 -- 172.21.15.19:0/3882808483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c80054d0 con 0x7f98d8101f60 2024-05-09T01:38:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.777+0000 7f98dc982640 1 -- 172.21.15.19:0/3882808483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8010430 con 0x7f98d8101f60 2024-05-09T01:38:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.777+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d81a21a0 con 0x7f98d8101f60 2024-05-09T01:38:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.777+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d81a26a0 con 0x7f98d8101f60 2024-05-09T01:38:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.778+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d80fef90 con 0x7f98d8101f60 2024-05-09T01:38:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.780+0000 7f98dc982640 1 -- 172.21.15.19:0/3882808483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f98c80046c0 con 0x7f98d8101f60 2024-05-09T01:38:37.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.781+0000 7f98dc982640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c00b27f0 0x7f98c00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:37.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.781+0000 7f98d67fc640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c00b27f0 0x7f98c00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:37.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.781+0000 7f98dc982640 1 -- 172.21.15.19:0/3882808483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f98c80e2930 con 0x7f98d8101f60 2024-05-09T01:38:37.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.782+0000 7f98d67fc640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c00b27f0 0x7f98c00b4d00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f98cc000c00 tx=0x7f98cc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:37.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.784+0000 7f98dc982640 1 -- 172.21.15.19:0/3882808483 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98c8054df0 con 0x7f98d8101f60 2024-05-09T01:38:37.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.978+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98d802dff0 con 0x7f98d8101f60 2024-05-09T01:38:37.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.979+0000 7f98dc982640 1 -- 172.21.15.19:0/3882808483 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f98c80aa5d0 con 0x7f98d8101f60 2024-05-09T01:38:37.976 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-09T01:38:37.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c00b27f0 msgr2=0x7f98c00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c00b27f0 0x7f98c00b4d00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f98cc000c00 tx=0x7f98cc031400 comp rx=0 tx=0).stop 2024-05-09T01:38:37.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8101f60 msgr2=0x7f98d819a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:37.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8101f60 0x7f98d819a370 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e9f0 tx=0x7f98c800eeb0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 shutdown_connections 2024-05-09T01:38:37.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f98c00b27f0 0x7f98c00b4d00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d81098b0 0x7f98d81a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98d8106d40 0x7f98d819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 --2- 172.21.15.19:0/3882808483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8101f60 0x7f98d819a370 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:37.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 >> 172.21.15.19:0/3882808483 conn(0x7f98d80fd470 msgr2=0x7f98d810a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:37.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 shutdown_connections 2024-05-09T01:38:37.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:37.981+0000 7f98dd984640 1 -- 172.21.15.19:0/3882808483 wait complete. 2024-05-09T01:38:37.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T01:38:38.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 -- 172.21.15.19:0/914961019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc104590 msgr2=0x7f9acc10acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/914961019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc104590 0x7f9acc10acd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009ca0 tx=0x7f9ac402a8c0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 -- 172.21.15.19:0/914961019 shutdown_connections 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/914961019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc104590 0x7f9acc10acd0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/914961019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9acc103a90 0x7f9acc103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/914961019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc1030b0 0x7f9acc1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 -- 172.21.15.19:0/914961019 >> 172.21.15.19:0/914961019 conn(0x7f9acc0fd470 msgr2=0x7f9acc0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 -- 172.21.15.19:0/914961019 shutdown_connections 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 -- 172.21.15.19:0/914961019 wait complete. 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.112+0000 7f9ad15d5640 1 Processor -- start 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9ad15d5640 1 -- start start 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9ad15d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc1030b0 0x7f9acc1023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9ad15d5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc103a90 0x7f9acc19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9ad15d5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9acc104590 0x7f9acc1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9ad15d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc10e630 con 0x7f9acc1030b0 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9ad15d5640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc10e4b0 con 0x7f9acc104590 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9ad15d5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc10e7b0 con 0x7f9acc103a90 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc1030b0 0x7f9acc1023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc1030b0 0x7f9acc1023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47722/0 (socket says 172.21.15.19:47722) 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acaffd640 1 -- 172.21.15.19:0/51022033 learned_addr learned my addr 172.21.15.19:0/51022033 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acb7fe640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9acc104590 0x7f9acc1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9aca7fc640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc103a90 0x7f9acc19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acaffd640 1 -- 172.21.15.19:0/51022033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc103a90 msgr2=0x7f9acc19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acaffd640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc103a90 0x7f9acc19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acaffd640 1 -- 172.21.15.19:0/51022033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9acc104590 msgr2=0x7f9acc1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acaffd640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9acc104590 0x7f9acc1a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acaffd640 1 -- 172.21.15.19:0/51022033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc1a1df0 con 0x7f9acc1030b0 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9aca7fc640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc103a90 0x7f9acc19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acb7fe640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9acc104590 0x7f9acc1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.113+0000 7f9acaffd640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc1030b0 0x7f9acc1023b0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d740 tx=0x7f9abc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.114+0000 7f9ab3fff640 1 -- 172.21.15.19:0/51022033 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc017070 con 0x7f9acc1030b0 2024-05-09T01:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.114+0000 7f9ab3fff640 1 -- 172.21.15.19:0/51022033 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9abc00de50 con 0x7f9acc1030b0 2024-05-09T01:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.114+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc1a20e0 con 0x7f9acc1030b0 2024-05-09T01:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.114+0000 7f9ab3fff640 1 -- 172.21.15.19:0/51022033 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc003c20 con 0x7f9acc1030b0 2024-05-09T01:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.114+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc1a2680 con 0x7f9acc1030b0 2024-05-09T01:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.115+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc109a60 con 0x7f9acc1030b0 2024-05-09T01:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.116+0000 7f9ab3fff640 1 -- 172.21.15.19:0/51022033 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9abc003dc0 con 0x7f9acc1030b0 2024-05-09T01:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.117+0000 7f9ab3fff640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ab40b2870 0x7f9ab40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.117+0000 7f9aca7fc640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ab40b2870 0x7f9ab40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.117+0000 7f9ab3fff640 1 -- 172.21.15.19:0/51022033 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f9abc0e2110 con 0x7f9acc1030b0 2024-05-09T01:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.118+0000 7f9aca7fc640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ab40b2870 0x7f9ab40b4d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9acc101d50 tx=0x7f9ac0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:38.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.121+0000 7f9ab3fff640 1 -- 172.21.15.19:0/51022033 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9abc0a9e30 con 0x7f9acc1030b0 2024-05-09T01:38:38.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.313+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9acc02dea0 con 0x7f9acc1030b0 2024-05-09T01:38:38.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.314+0000 7f9ab3fff640 1 -- 172.21.15.19:0/51022033 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f9abc0a9730 con 0x7f9acc1030b0 2024-05-09T01:38:38.311 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-09T01:38:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ab40b2870 msgr2=0x7f9ab40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ab40b2870 0x7f9ab40b4d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9acc101d50 tx=0x7f9ac0005f90 comp rx=0 tx=0).stop 2024-05-09T01:38:38.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc1030b0 msgr2=0x7f9acc1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc1030b0 0x7f9acc1023b0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d740 tx=0x7f9abc00dc00 comp rx=0 tx=0).stop 2024-05-09T01:38:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 shutdown_connections 2024-05-09T01:38:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f9ab40b2870 0x7f9ab40b4d80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9acc104590 0x7f9acc1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc103a90 0x7f9acc19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 --2- 172.21.15.19:0/51022033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc1030b0 0x7f9acc1023b0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 >> 172.21.15.19:0/51022033 conn(0x7f9acc0fd470 msgr2=0x7f9acc106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 shutdown_connections 2024-05-09T01:38:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.316+0000 7f9ad15d5640 1 -- 172.21.15.19:0/51022033 wait complete. 2024-05-09T01:38:38.324 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T01:38:38.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 -- 172.21.15.19:0/3818891177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 msgr2=0x7fced4106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 --2- 172.21.15.19:0/3818891177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 0x7fced4106b40 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009900 tx=0x7fcec402a410 comp rx=0 tx=0).stop 2024-05-09T01:38:38.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 -- 172.21.15.19:0/3818891177 shutdown_connections 2024-05-09T01:38:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 --2- 172.21.15.19:0/3818891177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced41099e0 0x7fced410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 --2- 172.21.15.19:0/3818891177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fced4107080 0x7fced41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 --2- 172.21.15.19:0/3818891177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 0x7fced4106b40 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 -- 172.21.15.19:0/3818891177 >> 172.21.15.19:0/3818891177 conn(0x7fced40fdd20 msgr2=0x7fced41001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 -- 172.21.15.19:0/3818891177 shutdown_connections 2024-05-09T01:38:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 -- 172.21.15.19:0/3818891177 wait complete. 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 Processor -- start 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.447+0000 7fced9d94640 1 -- start start 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced9d94640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 0x7fced419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced9d94640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced4107080 0x7fced419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced9d94640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fced41099e0 0x7fced41a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced9d94640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fced410f730 con 0x7fced4104720 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced9d94640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fced410f5b0 con 0x7fced4107080 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced9d94640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fced410f8b0 con 0x7fced41099e0 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 0x7fced419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 0x7fced419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47738/0 (socket says 172.21.15.19:47738) 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced37fe640 1 -- 172.21.15.19:0/43647565 learned_addr learned my addr 172.21.15.19:0/43647565 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fcecaffd640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced4107080 0x7fced419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced3fff640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fced41099e0 0x7fced41a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced37fe640 1 -- 172.21.15.19:0/43647565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fced41099e0 msgr2=0x7fced41a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced37fe640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fced41099e0 0x7fced41a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced37fe640 1 -- 172.21.15.19:0/43647565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced4107080 msgr2=0x7fced419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced37fe640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced4107080 0x7fced419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced37fe640 1 -- 172.21.15.19:0/43647565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced41a1e50 con 0x7fced4104720 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fcecaffd640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced4107080 0x7fced419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.448+0000 7fced37fe640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 0x7fced419a3c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fcec4032fd0 tx=0x7fcec40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.449+0000 7fced17fa640 1 -- 172.21.15.19:0/43647565 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec403f030 con 0x7fced4104720 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.449+0000 7fced17fa640 1 -- 172.21.15.19:0/43647565 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcec40044d0 con 0x7fced4104720 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.449+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced41a20e0 con 0x7fced4104720 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.449+0000 7fced17fa640 1 -- 172.21.15.19:0/43647565 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4035070 con 0x7fced4104720 2024-05-09T01:38:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.449+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced41a25d0 con 0x7fced4104720 2024-05-09T01:38:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.450+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced40ff840 con 0x7fced4104720 2024-05-09T01:38:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.451+0000 7fced17fa640 1 -- 172.21.15.19:0/43647565 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcec4042030 con 0x7fced4104720 2024-05-09T01:38:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.452+0000 7fced17fa640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcec00b27f0 0x7fcec00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.453+0000 7fcecaffd640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcec00b27f0 0x7fcec00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.453+0000 7fced17fa640 1 -- 172.21.15.19:0/43647565 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcec4102b20 con 0x7fced4104720 2024-05-09T01:38:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.453+0000 7fcecaffd640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcec00b27f0 0x7fcec00b4d00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005fd0 tx=0x7fcebc005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.457+0000 7fced17fa640 1 -- 172.21.15.19:0/43647565 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcec4102e20 con 0x7fced4104720 2024-05-09T01:38:38.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:38 smithi019 ceph-mon[30158]: pgmap v771: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-05-09T01:38:38.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:38 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/916910570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:38:38.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:38 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/3882808483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:38:38.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:38 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/51022033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T01:38:38.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.651+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fced402ddc0 con 0x7fced4104720 2024-05-09T01:38:38.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.652+0000 7fced17fa640 1 -- 172.21.15.19:0/43647565 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7fcec40ca820 con 0x7fced4104720 2024-05-09T01:38:38.648 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-05-09T01:38:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcec00b27f0 msgr2=0x7fcec00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcec00b27f0 0x7fcec00b4d00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005fd0 tx=0x7fcebc005ea0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 msgr2=0x7fced419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 0x7fced419a3c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fcec4032fd0 tx=0x7fcec40042e0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 shutdown_connections 2024-05-09T01:38:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fcec00b27f0 0x7fcec00b4d00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fced41099e0 0x7fced41a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced4107080 0x7fced419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 --2- 172.21.15.19:0/43647565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced4104720 0x7fced419a3c0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 >> 172.21.15.19:0/43647565 conn(0x7fced40fdd20 msgr2=0x7fced4106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:38.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 shutdown_connections 2024-05-09T01:38:38.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.654+0000 7fced9d94640 1 -- 172.21.15.19:0/43647565 wait complete. 2024-05-09T01:38:38.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T01:38:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.781+0000 7fdb20901640 1 -- 172.21.15.19:0/3141167778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c103c40 msgr2=0x7fdb1c104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.781+0000 7fdb20901640 1 --2- 172.21.15.19:0/3141167778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c103c40 0x7fdb1c104940 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009900 tx=0x7fdb0c02a410 comp rx=0 tx=0).stop 2024-05-09T01:38:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.781+0000 7fdb20901640 1 -- 172.21.15.19:0/3141167778 shutdown_connections 2024-05-09T01:38:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.781+0000 7fdb20901640 1 --2- 172.21.15.19:0/3141167778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c109b80 0x7fdb1c10bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.781+0000 7fdb20901640 1 --2- 172.21.15.19:0/3141167778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb1c104e80 0x7fdb1c105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.781+0000 7fdb20901640 1 --2- 172.21.15.19:0/3141167778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c103c40 0x7fdb1c104940 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.781+0000 7fdb20901640 1 -- 172.21.15.19:0/3141167778 >> 172.21.15.19:0/3141167778 conn(0x7fdb1c0fee70 msgr2=0x7fdb1c1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.781+0000 7fdb20901640 1 -- 172.21.15.19:0/3141167778 shutdown_connections 2024-05-09T01:38:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.781+0000 7fdb20901640 1 -- 172.21.15.19:0/3141167778 wait complete. 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.782+0000 7fdb20901640 1 Processor -- start 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.782+0000 7fdb20901640 1 -- start start 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.782+0000 7fdb20901640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb1c103c40 0x7fdb1c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.782+0000 7fdb20901640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c104e80 0x7fdb1c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.782+0000 7fdb20901640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c109b80 0x7fdb1c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.782+0000 7fdb20901640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c10f720 con 0x7fdb1c104e80 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.782+0000 7fdb20901640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c10f5a0 con 0x7fdb1c109b80 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.782+0000 7fdb20901640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c10f8a0 con 0x7fdb1c103c40 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb19d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c104e80 0x7fdb1c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb19d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c104e80 0x7fdb1c19a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47756/0 (socket says 172.21.15.19:47756) 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb19d9b640 1 -- 172.21.15.19:0/4073954173 learned_addr learned my addr 172.21.15.19:0/4073954173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb1ad9d640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c109b80 0x7fdb1c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb1a59c640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb1c103c40 0x7fdb1c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb19d9b640 1 -- 172.21.15.19:0/4073954173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb1c103c40 msgr2=0x7fdb1c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb19d9b640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb1c103c40 0x7fdb1c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb19d9b640 1 -- 172.21.15.19:0/4073954173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c109b80 msgr2=0x7fdb1c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb19d9b640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c109b80 0x7fdb1c1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb19d9b640 1 -- 172.21.15.19:0/4073954173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c1a1ed0 con 0x7fdb1c104e80 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb19d9b640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c104e80 0x7fdb1c19a920 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000d950 tx=0x7fdb1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb0b7fe640 1 -- 172.21.15.19:0/4073954173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10017030 con 0x7fdb1c104e80 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c1a21c0 con 0x7fdb1c104e80 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb0b7fe640 1 -- 172.21.15.19:0/4073954173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb10004020 con 0x7fdb1c104e80 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb0b7fe640 1 -- 172.21.15.19:0/4073954173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb100047a0 con 0x7fdb1c104e80 2024-05-09T01:38:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.783+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c1a2710 con 0x7fdb1c104e80 2024-05-09T01:38:38.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.784+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1c108990 con 0x7fdb1c104e80 2024-05-09T01:38:38.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.786+0000 7fdb0b7fe640 1 -- 172.21.15.19:0/4073954173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdb10009a30 con 0x7fdb1c104e80 2024-05-09T01:38:38.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.787+0000 7fdb0b7fe640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb040b27f0 0x7fdb040b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:38:38.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.787+0000 7fdb1a59c640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb040b27f0 0x7fdb040b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:38:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.787+0000 7fdb0b7fe640 1 -- 172.21.15.19:0/4073954173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fdb100e2320 con 0x7fdb1c104e80 2024-05-09T01:38:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.788+0000 7fdb1a59c640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb040b27f0 0x7fdb040b4d00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c005ab0 tx=0x7fdb0c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:38:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:38.790+0000 7fdb0b7fe640 1 -- 172.21.15.19:0/4073954173 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb100a9fc0 con 0x7fdb1c104e80 2024-05-09T01:38:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:38 smithi059 ceph-mon[34351]: pgmap v771: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-05-09T01:38:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:38 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/916910570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:38:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:38 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/3882808483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:38:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:38 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/51022033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T01:38:38.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:38 smithi196 ceph-mon[35041]: pgmap v771: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-05-09T01:38:38.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:38 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/916910570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:38:38.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:38 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/3882808483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:38:38.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:38 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/51022033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T01:38:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.033+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fdb1c0638d0 con 0x7fdb1c104e80 2024-05-09T01:38:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.043+0000 7fdb0b7fe640 1 -- 172.21.15.19:0/4073954173 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+815643 (secure 0 0 0) 0x7fdb100a98c0 con 0x7fdb1c104e80 2024-05-09T01:38:39.050 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 20, 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 14150, 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2331192605 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-09T01:38:39.051 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2331192605 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/2331192605", 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-05-09T01:15:21.027314+0000", 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24109, 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-05-09T01:38:39.052 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T01:38:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.054 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-09T01:38:39.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-09T01:38:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-09T01:38:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-09T01:38:39.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-09T01:38:39.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-09T01:38:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.072 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-09T01:38:39.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-09T01:38:39.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-09T01:38:39.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.076 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.077 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.078 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.082 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.097 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.099 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-09T01:38:39.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-09T01:38:39.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-09T01:38:39.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-09T01:38:39.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-09T01:38:39.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T01:38:39.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T01:38:39.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T01:38:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T01:38:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T01:38:39.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.124 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.125 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.126 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T01:38:39.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.129 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-09T01:38:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-09T01:38:39.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T01:38:39.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.145 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.149 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.157 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-09T01:38:39.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T01:38:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T01:38:39.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T01:38:39.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-09T01:38:39.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T01:38:39.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.166 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T01:38:39.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T01:38:39.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.189 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.190 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.192 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.193 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.194 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T01:38:39.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T01:38:39.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T01:38:39.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T01:38:39.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T01:38:39.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T01:38:39.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.213 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.214 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T01:38:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.226 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.227 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-09T01:38:39.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-09T01:38:39.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-09T01:38:39.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-09T01:38:39.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-09T01:38:39.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.240 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.244 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T01:38:39.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-09T01:38:39.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.257 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-09T01:38:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-09T01:38:39.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T01:38:39.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.265 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-09T01:38:39.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-09T01:38:39.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.268 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.277 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.279 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-09T01:38:39.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-09T01:38:39.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.283 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.288 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T01:38:39.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T01:38:39.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-09T01:38:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T01:38:39.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-09T01:38:39.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T01:38:39.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.304 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.307 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-09T01:38:39.313 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.314 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.321 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.325 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-09T01:38:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.329 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.337 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.341 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.344 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T01:38:39.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-09T01:38:39.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T01:38:39.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.352 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.353 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.360 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.365 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.366 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T01:38:39.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T01:38:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-09T01:38:39.371 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.380 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.384 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.385 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.389 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.390 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.391 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.395 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.396 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.397 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.399 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.400 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.401 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-09T01:38:39.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.405 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.406 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.408 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.417 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.418 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.419 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-09T01:38:39.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-09T01:38:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.423 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.425 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.426 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.429 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.432 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.434 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.444 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T01:38:39.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.453 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.454 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.455 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-09T01:38:39.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-09T01:38:39.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T01:38:39.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-09T01:38:39.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-09T01:38:39.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.476 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.482 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.483 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.486 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.489 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T01:38:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-09T01:38:39.499 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24113, 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.511 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.513 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.516 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-09T01:38:39.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-09T01:38:39.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-09T01:38:39.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-09T01:38:39.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.521 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.522 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.524 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.526 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T01:38:39.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-09T01:38:39.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T01:38:39.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.538 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-09T01:38:39.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-09T01:38:39.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.548 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-09T01:38:39.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.555 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-09T01:38:39.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-09T01:38:39.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T01:38:39.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.572 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-09T01:38:39.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T01:38:39.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T01:38:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.581 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.586 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T01:38:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-09T01:38:39.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.592 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-09T01:38:39.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-09T01:38:39.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-09T01:38:39.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.602 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-09T01:38:39.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-09T01:38:39.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.606 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-09T01:38:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.611 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.612 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.614 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.617 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.619 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T01:38:39.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.631 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.632 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T01:38:39.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.642 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.648 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T01:38:39.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T01:38:39.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T01:38:39.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T01:38:39.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T01:38:39.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.654 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.655 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T01:38:39.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T01:38:39.657 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T01:38:39.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T01:38:39.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T01:38:39.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-09T01:38:39.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-09T01:38:39.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.666 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.667 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-09T01:38:39.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.673 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.674 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T01:38:39.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-09T01:38:39.676 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T01:38:39.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.682 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.688 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-09T01:38:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-09T01:38:39.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.692 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.694 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.697 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.698 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.703 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.704 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-09T01:38:39.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.707 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.711 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.712 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-09T01:38:39.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.717 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-09T01:38:39.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.723 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.726 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.735 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-09T01:38:39.740 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.745 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.747 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-09T01:38:39.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-09T01:38:39.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.752 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.755 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.756 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.759 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.762 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.766 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.772 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-09T01:38:39.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.778 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.782 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.788 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.791 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.794 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-09T01:38:39.798 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.801 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.804 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.808 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T01:38:39.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.814 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.818 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.821 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.828 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.829 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.831 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.832 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-09T01:38:39.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T01:38:39.835 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.837 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.840 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.841 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.845 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.846 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.847 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.850 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.851 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.852 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-09T01:38:39.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.856 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.858 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.862 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.864 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.866 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.868 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-09T01:38:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.870 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-09T01:38:39.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.875 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.878 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.880 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.881 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.886 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.889 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.892 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.894 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.896 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.897 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-09T01:38:39.901 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.904 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.909 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T01:38:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.912 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-09T01:38:39.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-09T01:38:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.931 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-09T01:38:39.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-09T01:38:39.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.948 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.950 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.952 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.954 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.956 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:38:39.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "restful" 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.963 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.970 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.980 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-09T01:38:39.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.982 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.983 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.985 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:39.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-09T01:38:39.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.988 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.990 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.992 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.993 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:39.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T01:38:39.999 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T01:38:40.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-09T01:38:40.001 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.002 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T01:38:40.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T01:38:40.006 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T01:38:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:40.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.017 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-09T01:38:40.024 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-09T01:38:40.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.029 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T01:38:40.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.032 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.035 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-09T01:38:40.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-09T01:38:40.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T01:38:40.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T01:38:40.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.049 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.050 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-09T01:38:40.058 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.062 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-09T01:38:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.066 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.069 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.073 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T01:38:40.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T01:38:40.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.082 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.096 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T01:38:40.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-09T01:38:40.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T01:38:40.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.115 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-09T01:38:40.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T01:38:40.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T01:38:40.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T01:38:40.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-09T01:38:40.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.128 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T01:38:40.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.135 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T01:38:40.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.142 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-09T01:38:40.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.146 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.148 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.153 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-09T01:38:40.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-09T01:38:40.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.164 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T01:38:40.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-09T01:38:40.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.172 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-09T01:38:40.181 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.184 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.189 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.191 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T01:38:40.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.197 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.198 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-09T01:38:40.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.203 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.205 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.211 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.225 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.233 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.236 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.237 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.240 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.241 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.246 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.251 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.257 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T01:38:40.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T01:38:40.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.264 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T01:38:40.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-09T01:38:40.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T01:38:40.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T01:38:40.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-09T01:38:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.278 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.281 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.284 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-09T01:38:40.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.286 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.289 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.297 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.298 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.304 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T01:38:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.314 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-09T01:38:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.332 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T01:38:40.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-09T01:38:40.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.339 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.344 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-09T01:38:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T01:38:40.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T01:38:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T01:38:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T01:38:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T01:38:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T01:38:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T01:38:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T01:38:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:38:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 3, 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1679793166 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T01:38:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1315263502 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2117409099 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:38:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2153532114 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:38:40.422 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:38:40.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:39 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/43647565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T01:38:40.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:39 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/4073954173' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T01:38:40.424 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:39 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/43647565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T01:38:40.424 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:39 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/4073954173' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb040b27f0 msgr2=0x7fdb040b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb040b27f0 0x7fdb040b4d00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c005ab0 tx=0x7fdb0c031400 comp rx=0 tx=0).stop 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c104e80 msgr2=0x7fdb1c19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c104e80 0x7fdb1c19a920 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000d950 tx=0x7fdb1000de10 comp rx=0 tx=0).stop 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 shutdown_connections 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fdb040b27f0 0x7fdb040b4d00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c109b80 0x7fdb1c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb1c104e80 0x7fdb1c19a920 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 --2- 172.21.15.19:0/4073954173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb1c103c40 0x7fdb1c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:38:40.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 >> 172.21.15.19:0/4073954173 conn(0x7fdb1c0fee70 msgr2=0x7fdb1c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:38:40.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 shutdown_connections 2024-05-09T01:38:40.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:38:39.049+0000 7fdb20901640 1 -- 172.21.15.19:0/4073954173 wait complete. 2024-05-09T01:38:40.431 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T01:38:40.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T01:38:40.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:39 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/43647565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T01:38:40.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:39 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/4073954173' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T01:38:40.698 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:40.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:40 smithi019 ceph-mon[30158]: pgmap v772: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:38:40.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:40 smithi059 ceph-mon[34351]: pgmap v772: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:38:40.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:40 smithi196 ceph-mon[35041]: pgmap v772: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-05-09T01:38:42.466 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:38:42.466 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:38:42.466 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 968, 2024-05-09T01:38:42.466 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 970, 2024-05-09T01:38:42.466 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 324290, 2024-05-09T01:38:42.466 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 307825, 2024-05-09T01:38:42.466 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-05-09T01:38:42.466 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-09T01:38:42.466 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.117245494, 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.038820763, 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.030892610, 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.035691142, 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 968, 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.050709863, 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000052386 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.467 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 277, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 270, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 857246, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 163086, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.045687436, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.018138361, 2024-05-09T01:38:42.468 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.024207461, 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.005279246, 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 269, 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.014465703, 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000053775 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 855856, 2024-05-09T01:38:42.469 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 162064 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1787, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1982, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 456571, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 995126, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.149670511, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.048231273, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.037218021, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.051950012, 2024-05-09T01:38:42.470 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1981, 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.077749165, 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000039247 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 428768 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-09T01:38:42.471 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4626, 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.138919649, 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000030030 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-05-09T01:38:42.473 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-05-09T01:38:42.474 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:38:42.475 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:38:42.476 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-05-09T01:38:42.477 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:38:42.478 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 4, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-05-09T01:38:42.479 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4195424, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4195283, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 4 2024-05-09T01:38:42.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 3, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 3, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 1, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 0, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 0, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 119080, 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215440 2024-05-09T01:38:42.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.482 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.483 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.486 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-05-09T01:38:42.486 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.486 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.486 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.486 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-05-09T01:38:42.486 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.487 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-05-09T01:38:42.488 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-05-09T01:38:42.489 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.490 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.492 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 0, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 1, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 1, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 1092, 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-05-09T01:38:42.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:38:42.494 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:38:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 197463, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 54, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-09T01:38:42.496 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 7720, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 162, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 8512, 2024-05-09T01:38:42.497 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 2752, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 2752, 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-05-09T01:38:42.498 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2752, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.429979531, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000882986 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2752, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2753 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 2752, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 2752, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 1802, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-05-09T01:38:42.499 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 948, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-05-09T01:38:42.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-05-09T01:38:42.501 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 83, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 2, 2024-05-09T01:38:42.502 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 6, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 2, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.503 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:42.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3028, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1348486, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3028, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3028, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1348486, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.505 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2752, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 1125, 2024-05-09T01:38:42.506 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2752, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1125, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T01:38:42.507 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2752, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2752, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2752, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2752, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:42.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:42.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55e8a9bc08f0": { 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:42.510 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:42.511 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:42.511 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:42.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:42.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:42.511 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:38:42.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:42 smithi019 ceph-mon[30158]: pgmap v773: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:42.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:42 smithi059 ceph-mon[34351]: pgmap v773: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:42.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:42 smithi196 ceph-mon[35041]: pgmap v773: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:43.265 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-09T01:38:43.265 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-09T01:38:43.265 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T01:38:43.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T01:38:43.525 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:44.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:44 smithi019 ceph-mon[30158]: pgmap v774: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:38:44.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:44 smithi059 ceph-mon[34351]: pgmap v774: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:38:44.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:44 smithi196 ceph-mon[35041]: pgmap v774: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:38:45.492 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:38:45.492 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 273, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 268, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 828208, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 165184, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.045099482, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.017452114, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.023763871, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.005310900, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 267, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.012845051, 2024-05-09T01:38:45.493 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000048108 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 828208, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 165184 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 6, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 207, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3312, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 194064, 2024-05-09T01:38:45.494 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.017959847, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.013187681, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002535198, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 206, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.011395798, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000055319 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:45.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 194064 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.000484247, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000409204, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T01:38:45.496 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-09T01:38:45.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:38:45.498 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:38:45.499 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 147960, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 28, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-09T01:38:45.500 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-09T01:38:45.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 275, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 789635, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 275, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 275, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 789635, 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:45.502 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:45.503 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:45.503 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:45.503 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:45.503 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:45.503 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:38:46.297 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T01:38:46.297 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T01:38:46.298 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T01:38:46.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T01:38:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:46.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:46 smithi019 ceph-mon[30158]: pgmap v775: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:38:46.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:46 smithi059 ceph-mon[34351]: pgmap v775: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:38:46.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:46 smithi196 ceph-mon[35041]: pgmap v775: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:38:48.393 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:38:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:38:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 950, 2024-05-09T01:38:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 951, 2024-05-09T01:38:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 319235, 2024-05-09T01:38:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 301783, 2024-05-09T01:38:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-05-09T01:38:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-09T01:38:48.393 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.109244300, 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.046809424, 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.023503331, 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.027133000, 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 949, 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.049817767, 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000052495 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1825, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2023, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 467756, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 1009553, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.129909873, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.042371372, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.031899708, 2024-05-09T01:38:48.395 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.045231380, 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2022, 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.061562170, 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000030446 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.396 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 430736 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 280, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 273, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 858020, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 164949, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.044487134, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.017413204, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.023708933, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.005189307, 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:48.397 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 272, 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.015736872, 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000057856 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 856672, 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 163984 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-05-09T01:38:48.398 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4658, 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.115755563, 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000024850 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.399 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-05-09T01:38:48.400 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-05-09T01:38:48.401 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:38:48.402 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:38:48.403 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:38:48.404 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-05-09T01:38:48.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 6, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-05-09T01:38:48.406 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4195956, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4195815, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 6 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 3, 2024-05-09T01:38:48.407 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 3, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 1, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 0, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 0, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 118468, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215440 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-05-09T01:38:48.408 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-05-09T01:38:48.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.410 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.411 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.412 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-05-09T01:38:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-05-09T01:38:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.414 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.417 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.418 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:38:48.419 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 0, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 2, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 2, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-05-09T01:38:48.420 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 1272, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:38:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:38:48.422 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:38:48.423 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 197480, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 55, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 7720, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 162, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:38:48.424 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 8512, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 2772, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-05-09T01:38:48.425 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 2772, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2772, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.365980143, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000853528 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2772, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2773 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 2772, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 2772, 2024-05-09T01:38:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 1814, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 956, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-05-09T01:38:48.427 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-05-09T01:38:48.428 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-05-09T01:38:48.429 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 83, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 2, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 6, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 2, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-05-09T01:38:48.430 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-05-09T01:38:48.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3051, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1353164, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3051, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3051, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1353164, 2024-05-09T01:38:48.432 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:48.433 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2772, 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 1657, 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2772, 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1657, 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:48.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2772, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2772, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2772, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2772, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:48.436 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55a96e27e8f0": { 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T01:38:48.437 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-09T01:38:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-09T01:38:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:48.438 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:48.438 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:38:48.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:48 smithi019 ceph-mon[30158]: pgmap v776: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:48.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:48 smithi059 ceph-mon[34351]: pgmap v776: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:48.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:48 smithi196 ceph-mon[35041]: pgmap v776: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:49.357 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-09T01:38:49.357 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-09T01:38:49.357 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T01:38:49.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T01:38:49.614 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:38:50.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:50 smithi059 ceph-mon[34351]: pgmap v777: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:38:50.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:50 smithi019 ceph-mon[30158]: pgmap v777: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:38:50.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:50 smithi196 ceph-mon[35041]: pgmap v777: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:38:51.476 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:38:51.476 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:38:51.476 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-05-09T01:38:51.476 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-05-09T01:38:51.476 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001150961, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000118463, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:51.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2, 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 207, 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 320, 2024-05-09T01:38:51.478 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 193968, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.017415162, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.012544421, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002561403, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 205, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.010902214, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000053181 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:51.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 193968 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 281, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 274, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 832352, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 167712, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.047196342, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.019640355, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.025025549, 2024-05-09T01:38:51.480 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.005293068, 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 274, 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.014458317, 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000052767 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 832352, 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 167712 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-09T01:38:51.481 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:38:51.482 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-09T01:38:51.483 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 147898, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 27, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-09T01:38:51.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-09T01:38:51.485 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 279, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 790219, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 279, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 279, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 790219, 2024-05-09T01:38:51.486 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T01:38:51.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T01:38:51.487 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T01:38:51.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T01:38:51.487 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:51.487 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T01:38:51.487 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:38:52.242 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T01:38:52.242 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T01:38:52.242 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T01:38:52.243 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T01:38:52.503 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:38:52.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:52 smithi059 ceph-mon[34351]: pgmap v778: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:52.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:52 smithi019 ceph-mon[30158]: pgmap v778: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:52.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:52 smithi196 ceph-mon[35041]: pgmap v778: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:54.511 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T01:38:54.511 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:38:54.511 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 6506, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 7938, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1970412, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 11014307, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 39, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.768195493, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.274210353, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.316265659, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.157740943, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 7937, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.245200656, 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000030893 2024-05-09T01:38:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 5876, 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.001597927, 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000271 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 896048, 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 1479424 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1478, 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 1329, 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1158486, 2024-05-09T01:38:54.513 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 609564, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 8, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 7, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.168835989, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.068812148, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.086342205, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.023428538, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1328, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.199898276, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000150525 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:54.514 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1103, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000346973, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000314 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 892624, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 190672 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2151, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2295, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 597521, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 3609607, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 224, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 7, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.310081927, 2024-05-09T01:38:54.515 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.096813054, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.204827412, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.021638850, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2293, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.159733460, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000069661 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3060, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000914735, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000298 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 39936, 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 443456 2024-05-09T01:38:54.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 6271, 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2.070904601, 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000330235 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T01:38:54.517 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 24, 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000827752, 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000034489 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 647, 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 647, 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 647, 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 3.470499926, 2024-05-09T01:38:54.518 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.005363987 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 345, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 24, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 5, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 19, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-05-09T01:38:54.519 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 16, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 2, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 2, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 2, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 680, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 674, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-05-09T01:38:54.520 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 262, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 4, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 234, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 262, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 169, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 19, 2024-05-09T01:38:54.521 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 150, 2024-05-09T01:38:54.522 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 93, 2024-05-09T01:38:54.522 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:38:54.522 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 611, 2024-05-09T01:38:54.522 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 612, 2024-05-09T01:38:54.522 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 599, 2024-05-09T01:38:54.522 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:38:54.522 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:38:54.522 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 667, 2024-05-09T01:38:54.523 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 650, 2024-05-09T01:38:54.523 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 2, 2024-05-09T01:38:54.523 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 622 2024-05-09T01:38:54.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.523 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-09T01:38:54.523 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-09T01:38:54.523 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 0, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 6, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 3, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 3, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-09T01:38:54.524 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 19, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-09T01:38:54.525 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 2536, 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 2472, 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 2472, 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 64, 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 54, 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 55, 2024-05-09T01:38:54.526 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 54, 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 54, 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 1, 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 1, 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 1, 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 11864483, 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 12046883, 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 0, 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-09T01:38:54.527 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2523, 2024-05-09T01:38:54.528 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 6.202161410, 2024-05-09T01:38:54.528 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002458248 2024-05-09T01:38:54.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.528 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 0 2024-05-09T01:38:54.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.528 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-09T01:38:54.528 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 23, 2024-05-09T01:38:54.528 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 45, 2024-05-09T01:38:54.528 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 22, 2024-05-09T01:38:54.529 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 18, 2024-05-09T01:38:54.529 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 20, 2024-05-09T01:38:54.529 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 2, 2024-05-09T01:38:54.529 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 24, 2024-05-09T01:38:54.529 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 29, 2024-05-09T01:38:54.529 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 5, 2024-05-09T01:38:54.529 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 2, 2024-05-09T01:38:54.529 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 17, 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 15, 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 126048, 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 231824 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 1862, 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 1, 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 647, 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 21, 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 2, 2024-05-09T01:38:54.530 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 5, 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.003751622, 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000750324 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 17, 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.004108575, 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000241680 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.532 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1, 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000159268, 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000159268 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-09T01:38:54.533 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.534 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.535 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.535 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-09T01:38:54.535 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 8, 2024-05-09T01:38:54.535 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.013472293, 2024-05-09T01:38:54.535 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001684036 2024-05-09T01:38:54.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.535 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-09T01:38:54.535 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1, 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000327855, 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000327855 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.536 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.004114595, 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002057297 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.002316517, 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001158258 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.537 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.009088802, 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.004544401 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.538 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 600, 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 3.407628821, 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.005679381 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 8, 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.022989987, 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002873748 2024-05-09T01:38:54.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.002541591, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002541591 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.541 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 1, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 3, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 2, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 1, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 89, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 89, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 775, 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-09T01:38:54.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:38:54.543 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:38:54.544 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 315335, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 129, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-09T01:38:54.545 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 7720, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 162, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 94082, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 594, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-09T01:38:54.546 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 4606, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 8211163, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 32, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 4606, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 4606, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 29.298617343, 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.006360967 2024-05-09T01:38:54.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 4606, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 5340 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 4606, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 302, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 4304, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 325, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 25, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 21, 2024-05-09T01:38:54.548 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 2525, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 171, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 517, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 281, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 50, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:38:54.549 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 599, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-09T01:38:54.550 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 83, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 2, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 12, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 12, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-09T01:38:54.551 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 814, 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 0, 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 12 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 1, 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-09T01:38:54.552 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-09T01:38:54.553 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 2, 2024-05-09T01:38:54.553 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-09T01:38:54.553 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 2, 2024-05-09T01:38:54.553 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 3, 2024-05-09T01:38:54.553 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 3, 2024-05-09T01:38:54.553 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-09T01:38:54.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.553 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:38:54.553 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 10131, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 2691412, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 10131, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 10131, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2691412, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.554 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 4606, 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 8211466, 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 4606, 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 8211466, 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-09T01:38:54.556 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 4606, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 4606, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 4606, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 4606, 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:54.557 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 3, 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 303, 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:54.558 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 3, 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2, 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 303, 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x55765913a680": { 2024-05-09T01:38:54.559 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 2536, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 7852579, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 2536, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2523, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 7852579, 2024-05-09T01:38:54.560 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:54.561 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:54.561 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:54.561 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:54.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:54.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:54.561 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T01:38:54.643 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:54 smithi059 ceph-mon[34351]: pgmap v779: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:38:54.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:54 smithi019 ceph-mon[30158]: pgmap v779: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:38:54.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:54 smithi196 ceph-mon[35041]: pgmap v779: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:38:55.330 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-09T01:38:55.330 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-09T01:38:55.330 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T01:38:55.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T01:38:55.582 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:38:56.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:56 smithi059 ceph-mon[34351]: pgmap v780: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:38:56.893 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:56 smithi019 ceph-mon[30158]: pgmap v780: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:38:56.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:56 smithi196 ceph-mon[35041]: pgmap v780: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:38:57.317 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T01:38:57.317 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:38:57.317 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 514, 2024-05-09T01:38:57.317 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 644, 2024-05-09T01:38:57.317 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 112055, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 151323, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 10, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.092273750, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.038855961, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.041808689, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.005792258, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 646, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.054264719, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000084001 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.318 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1109, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000320948, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000289 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 19, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 222, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 6849, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 529378, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 9, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-05-09T01:38:57.319 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.014807967, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.008894911, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.003433167, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000324725, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 221, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.022717373, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000102793 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.320 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 523248 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 369, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 363, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 882694, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 660118, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 49, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 5, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.071658266, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.020144107, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.046301247, 2024-05-09T01:38:57.321 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.007569579, 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 358, 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.023868723, 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000066672 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 47, 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000006218, 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000132 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 875312, 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 173984 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.322 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 48, 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.001384119, 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000028835 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T01:38:57.323 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000007997, 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000003998 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 0, 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 0, 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.324 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 11, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 10, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 10, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 10, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 0, 2024-05-09T01:38:57.325 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 1, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-05-09T01:38:57.326 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 5, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 298, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:38:57.327 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-09T01:38:57.328 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 1, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:38:57.329 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 4, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 4, 2024-05-09T01:38:57.330 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 2, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 2, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 4195424, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 0, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.006629070, 2024-05-09T01:38:57.331 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002209690 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 0 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 13, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 13, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 0, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 11, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 12, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 1, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 10, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 10, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 0, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 0, 2024-05-09T01:38:57.332 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 0, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 120892, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 223632 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 0, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 0, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 4, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.334 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-09T01:38:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.338 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.338 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.338 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-09T01:38:57.338 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.338 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.338 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.338 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.340 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.341 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-09T01:38:57.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.343 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 0, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 1, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 1, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 0, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 0, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 0, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 74, 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-09T01:38:57.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:38:57.346 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:38:57.347 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 221879, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 66, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-09T01:38:57.348 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 7720, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 162, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 50872, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 198, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-09T01:38:57.349 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 38, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 11332, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 33, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 38, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 38, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.210997344, 2024-05-09T01:38:57.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.005552561 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 38, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 60 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 38, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 0, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 38, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 0, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 10, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 0, 2024-05-09T01:38:57.351 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 3, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 6, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 6, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 20, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-09T01:38:57.352 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-09T01:38:57.353 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 83, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 2, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 8, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 13, 2024-05-09T01:38:57.354 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 5, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 14, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 0, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 1 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 1, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.355 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 898, 2024-05-09T01:38:57.356 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 901329, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 898, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 898, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 901329, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-09T01:38:57.357 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 38, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 11332, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 38, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 11332, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.358 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 38, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 38, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 38, 2024-05-09T01:38:57.359 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 38, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:57.360 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x55699c50e680": { 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T01:38:57.361 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 4, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1120, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 4, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 3, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1120, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:57.362 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:38:57.363 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T01:38:58.147 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T01:38:58.147 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T01:38:58.147 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T01:38:58.148 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T01:38:58.395 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:38:58.765 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:38:58 smithi059 ceph-mon[34351]: pgmap v781: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:58.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:38:58 smithi019 ceph-mon[30158]: pgmap v781: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:38:58.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:38:58 smithi196 ceph-mon[35041]: pgmap v781: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 36, 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 42, 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 8117, 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 60538, 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 13, 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.021509293, 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.003573204, 2024-05-09T01:39:00.071 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.012184267, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000524053, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 36, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.017910264, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000497507 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 29, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000006463, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000222 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2528, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 2544 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:39:00.072 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 509, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 848, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 109032, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 667019, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 9, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.104638240, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.050439962, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.043983802, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.007146633, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 850, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.067170787, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000079024 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.073 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1126, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000374081, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000332 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 435840 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 372, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 356, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 893990, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 635208, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 44, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-05-09T01:39:00.074 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.073892472, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.020374514, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.064742254, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.007877522, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 356, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.050586923, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000142098 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 38, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000004900, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000128 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 886272, 2024-05-09T01:39:00.075 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 173056 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 34, 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.002174286, 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000063949 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-09T01:39:00.076 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000009905, 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000004952 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 2, 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 1, 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1, 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000106014, 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000106014 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.077 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 1, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 11, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 10, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 0, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 10, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 10, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 1, 2024-05-09T01:39:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 1, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 1, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-09T01:39:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 1, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 5, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 298, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:39:00.080 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 1, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-09T01:39:00.081 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-09T01:39:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 3, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 3, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 2, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 2, 2024-05-09T01:39:00.083 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 4195200, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 0, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.003699958, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001849979 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 0 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 13, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 13, 2024-05-09T01:39:00.084 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 0, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 11, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 12, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 1, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 10, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 10, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 0, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 1, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 1, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 121136, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 223632 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 2, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-05-09T01:39:00.085 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 2, 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 11, 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-09T01:39:00.086 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.087 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1, 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000106014, 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000106014 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-09T01:39:00.088 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-09T01:39:00.089 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.090 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.090 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.090 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-09T01:39:00.090 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.090 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.090 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.091 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-09T01:39:00.092 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-09T01:39:00.093 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-09T01:39:00.094 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-09T01:39:00.095 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.096 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 1, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 1, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 0, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 1, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 1, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 1, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 769, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-09T01:39:00.097 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:39:00.098 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-09T01:39:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 238247, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 74, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 7720, 2024-05-09T01:39:00.100 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 162, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 51240, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 200, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-09T01:39:00.101 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 37, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 11108, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 33, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 37, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 37, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.247996862, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.006702617 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 37, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 59 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 37, 2024-05-09T01:39:00.102 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 37, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 10, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 2, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 6, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 6, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-09T01:39:00.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 20, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-09T01:39:00.104 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 83, 2024-05-09T01:39:00.105 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 2, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 9, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 12, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 3, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 14, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 0, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 1 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 1, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-09T01:39:00.106 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:00.107 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 913, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 909048, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 913, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 913, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 909048, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.108 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 37, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 11108, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 37, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 11108, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.109 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 37, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 37, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 37, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 37, 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:00.110 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-09T01:39:00.111 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x555c0acee410": { 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 3, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 896, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 3, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:39:00.112 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2, 2024-05-09T01:39:00.113 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 896, 2024-05-09T01:39:00.113 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:00.113 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:00.113 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:00.113 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:00.113 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:00.113 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:00.113 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T01:39:00.791 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:00 smithi059 ceph-mon[34351]: pgmap v782: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:00.820 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-09T01:39:00.821 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-09T01:39:00.821 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T01:39:00.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T01:39:00.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:00 smithi019 ceph-mon[30158]: pgmap v782: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:00.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:00 smithi196 ceph-mon[35041]: pgmap v782: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:01.071 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:39:02.812 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:02 smithi059 ceph-mon[34351]: pgmap v783: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:02.812 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T01:39:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:39:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2703, 2024-05-09T01:39:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2705, 2024-05-09T01:39:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 768535, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 859561, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 2, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.231437803, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.074550155, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.057739521, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.079759942, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2703, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.093625901, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000034637 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 4, 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 212, 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 876, 2024-05-09T01:39:02.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 525642, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.013389416, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.008799046, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.002267804, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000052454, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 210, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.012624121, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000060114 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.815 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 524944 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 295, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 286, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 863692, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 171774, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 2, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.054842699, 2024-05-09T01:39:02.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.019755851, 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.029876789, 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.007470001, 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 286, 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.017058291, 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000059644 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 863424, 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 171456 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 4630, 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.127309138, 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000027496 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:02.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 0, 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 0, 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.819 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 0, 2024-05-09T01:39:02.820 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 1, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-05-09T01:39:02.821 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:39:02.822 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-09T01:39:02.823 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-09T01:39:02.824 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-05-09T01:39:02.825 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 3, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 0, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 2, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 4195200, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 4194892, 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-09T01:39:02.826 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 3 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 3, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 3, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 0, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 1, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 1, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 0, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 0, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 0, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 0, 2024-05-09T01:39:02.827 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 0, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 0, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 118516, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 215440 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 0, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 0, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 0, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.828 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.829 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.830 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.832 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-09T01:39:02.833 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.833 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.833 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.833 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-09T01:39:02.834 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.835 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.836 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-09T01:39:02.837 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.838 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-09T01:39:02.839 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 1, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 1, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 0, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 1, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 0, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 0, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 2203, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-09T01:39:02.841 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:39:02.842 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-09T01:39:02.843 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 181287, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 45, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:39:02.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 7720, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 162, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 8512, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 13, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-09T01:39:02.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 2706, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 21, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 3, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 2706, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2706, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 1.527979028, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000564663 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-09T01:39:02.846 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2706, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2707 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 2706, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 2706, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 0, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 1736, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 0, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 968, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 0, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 0, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-09T01:39:02.847 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 1, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:39:02.848 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-09T01:39:02.849 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 83, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 2, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 4, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 6, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 2, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 0, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 2, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 0 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.850 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 0, 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-05-09T01:39:02.851 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 2998, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1345314, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 2998, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2998, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1345314, 2024-05-09T01:39:02.852 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 2706, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 901, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2706, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 901, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:02.854 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 2706, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 2706, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2706, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2706, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:02.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:02.856 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x55c7bf8c24e0": { 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T01:39:02.857 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-09T01:39:02.858 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-09T01:39:02.858 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T01:39:02.858 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T01:39:02.858 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T01:39:02.858 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T01:39:02.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:02.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T01:39:02.858 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T01:39:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:02 smithi019 ceph-mon[30158]: pgmap v783: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:02.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:02 smithi196 ceph-mon[35041]: pgmap v783: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:03.570 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-09T01:39:03.570 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-09T01:39:03.571 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T01:39:03.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T01:39:03.821 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:39:04.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:04 smithi196 ceph-mon[35041]: pgmap v784: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:39:04.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:04 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:39:04.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:04 smithi019 ceph-mon[30158]: pgmap v784: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:39:04.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:04 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:39:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:04 smithi059 ceph-mon[34351]: pgmap v784: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:39:04.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:04 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:39:05.494 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 955, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 791, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 225004, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 175925, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 9, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.096024998, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.035403634, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.055005269, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.006824160, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 794, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.038815516, 2024-05-09T01:39:05.495 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000048886 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1677, 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000489749, 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000292 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 63, 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 272, 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 11707, 2024-05-09T01:39:05.496 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1015855, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 53, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 7, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.033202018, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.010843914, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.018571255, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000320525, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 266, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.022464794, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000084454 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 62, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000011134, 2024-05-09T01:39:05.497 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000179 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 528480 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 336, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 328, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 872922, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 205405, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.056830102, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.019155961, 2024-05-09T01:39:05.498 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.038976018, 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.008332197, 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 327, 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.043039872, 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000131620 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000717, 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000358 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 867968, 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 173248 2024-05-09T01:39:05.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 45, 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002131604, 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000047368 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T01:39:05.500 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000012484, 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000006242 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 4, 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 3, 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3, 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002762209, 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000920736 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-05-09T01:39:05.501 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 1, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 12, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 12, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 11, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-05-09T01:39:05.502 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 2, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 8, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 6, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 2, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-05-09T01:39:05.503 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 5, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 298, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:39:05.504 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-05-09T01:39:05.505 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 3, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:39:05.506 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 6, 2024-05-09T01:39:05.507 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 6, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195956, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-05-09T01:39:05.508 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.017868469, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003573693 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 15, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 15, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 13, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 14, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 1, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 12, 2024-05-09T01:39:05.509 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 12, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 0, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 3, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 3, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 121684, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223632 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 9, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 4, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 8, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 1, 2024-05-09T01:39:05.510 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001550498, 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001550498 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-05-09T01:39:05.511 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-05-09T01:39:05.512 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001211711, 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000605855 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.515 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-05-09T01:39:05.515 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.515 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.515 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.515 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-05-09T01:39:05.515 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.516 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-05-09T01:39:05.517 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.520 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:39:05.522 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 0, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 0, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 2, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 2, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 82, 2024-05-09T01:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:39:05.524 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:39:05.525 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 238919, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 79, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 88, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 1, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-05-09T01:39:05.526 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7720, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 162, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 59360, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 237, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-05-09T01:39:05.527 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 44, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 12269, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 44, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 44, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.266994390, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.006068054 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.528 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 44, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 68 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 44, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 44, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 5, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 8, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-05-09T01:39:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-05-09T01:39:05.530 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-05-09T01:39:05.531 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 83, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 2, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 9, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 14, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 5, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 17, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 2 2024-05-09T01:39:05.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-05-09T01:39:05.533 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 1350, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 968413, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 1350, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1350, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 968413, 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:05.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 44, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 12269, 2024-05-09T01:39:05.535 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 44, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 12269, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:05.536 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 44, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 44, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 44, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 44, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:05.537 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55f62f30e410": { 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:05.538 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 6, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1652, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 6, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1652, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:05.540 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:05.540 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T01:39:05.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:05 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:39:05.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:05 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:39:05.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:05 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:39:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:05 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:39:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:05 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:39:05.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:05 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:39:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:05 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:39:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:05 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:39:05.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:05 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' 2024-05-09T01:39:06.279 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-09T01:39:06.279 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-09T01:39:06.279 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T01:39:06.279 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T01:39:06.523 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:39:06.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:06 smithi019 ceph-mon[30158]: pgmap v785: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:06.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:06 smithi059 ceph-mon[34351]: pgmap v785: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:06.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:06 smithi196 ceph-mon[35041]: pgmap v785: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:08.144 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T01:39:08.144 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:39:08.144 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1926, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1926, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4767609, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 611654, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.216647880, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.065511102, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.063577294, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.067228035, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1925, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.084623216, 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000043960 2024-05-09T01:39:08.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 651, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 644, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 949772, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 285070, 2024-05-09T01:39:08.146 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 5, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.092058822, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.032895968, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.037534818, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.020645569, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 643, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.033031918, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000051371 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 860048, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 169888 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1255, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1461, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 3972894, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 834965, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 4, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.135709036, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.053524468, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.031781294, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.036896110, 2024-05-09T01:39:08.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1459, 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.061804515, 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000042360 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 436192 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-05-09T01:39:08.149 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5986, 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.177372800, 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000029631 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T01:39:08.150 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.151 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 17, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 4, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 13, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 10, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-05-09T01:39:08.152 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 2, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-05-09T01:39:08.153 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-05-09T01:39:08.154 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-05-09T01:39:08.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-05-09T01:39:08.156 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-05-09T01:39:08.157 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 2536, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 54, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 11864483, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 12046883, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 12044386, 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-05-09T01:39:08.158 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 2536 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 14, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 17, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 3, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 9, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 11, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 2, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 17, 2024-05-09T01:39:08.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 19, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 2, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 119872, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215440 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-05-09T01:39:08.160 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-05-09T01:39:08.161 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.163 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-05-09T01:39:08.165 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-05-09T01:39:08.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.172 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 3, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 3612, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-05-09T01:39:08.173 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-05-09T01:39:08.174 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-05-09T01:39:08.175 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 189440, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 52, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:39:08.176 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7720, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 162, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 54818, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 257, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-05-09T01:39:08.177 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 3538, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 26, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 4, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 3538, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3538, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.762961304, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000498293 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3538, 2024-05-09T01:39:08.178 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3539 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 3538, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 3538, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 1929, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1606, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-05-09T01:39:08.179 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 2, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-05-09T01:39:08.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-05-09T01:39:08.181 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 83, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 2, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 5, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 6, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 1, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-05-09T01:39:08.182 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:39:08.183 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3828, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 9327887, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3828, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:08.184 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3828, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 9327887, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:08.185 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3538, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 7852589, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3538, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 7852589, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.186 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3538, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3538, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3538, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3538, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.187 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-05-09T01:39:08.188 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55bab3cda8f0": { 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:08.190 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T01:39:08.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:08 smithi059 ceph-mon[34351]: pgmap v786: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:08.885 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:08 smithi196 ceph-mon[35041]: pgmap v786: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:08.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:08 smithi019 ceph-mon[30158]: pgmap v786: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:08.913 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-09T01:39:08.913 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-09T01:39:08.913 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T01:39:08.913 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T01:39:09.152 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:39:10.840 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:10 smithi196 ceph-mon[35041]: pgmap v787: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:10.840 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 384, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 370, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 893624, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 660432, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 49, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.073244197, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.021247712, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.042567121, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.007707304, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 370, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.031263146, 2024-05-09T01:39:10.841 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000084494 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 54, 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000008508, 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000157 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 883904, 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 175008 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 18, 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 223, 2024-05-09T01:39:10.842 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4399, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 563599, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 11, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.015747606, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.008915246, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.004027589, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000428344, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 221, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.021230182, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000096064 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000231, 2024-05-09T01:39:10.843 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000115 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 527648 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 511, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 652, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 107662, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 135080, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 7, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.078294731, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.029714496, 2024-05-09T01:39:10.844 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.039313018, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.007258703, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 650, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.053207292, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000081857 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1124, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000378201, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000336 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-05-09T01:39:10.845 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 46, 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002268245, 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000049309 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T01:39:10.846 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000009763, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000004881 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-05-09T01:39:10.847 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 10, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 10, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 10, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-05-09T01:39:10.848 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:39:10.849 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 5, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 298, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:39:10.850 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 1, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-05-09T01:39:10.851 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:39:10.852 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 6, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 6, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-05-09T01:39:10.853 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195956, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.012157294, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002431458 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.854 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 13, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 13, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 11, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 12, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 1, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 10, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 10, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 0, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-05-09T01:39:10.855 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 121196, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223632 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 8, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.856 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-05-09T01:39:10.857 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-05-09T01:39:10.860 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.861 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:10 smithi059 ceph-mon[34351]: pgmap v787: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:10.861 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.861 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.861 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-05-09T01:39:10.861 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.862 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.866 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:39:10.867 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 0, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 0, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 83, 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-05-09T01:39:10.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:39:10.869 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:39:10.870 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 229943, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 70, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-05-09T01:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7720, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 162, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 50872, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 198, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-05-09T01:39:10.872 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 42, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 12044, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 28, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 42, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 42, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.230995116, 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005499883 2024-05-09T01:39:10.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 42, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 64 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 42, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 42, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-05-09T01:39:10.874 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 5, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 8, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-05-09T01:39:10.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-05-09T01:39:10.876 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 83, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 2, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 8, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 12, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 4, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-05-09T01:39:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 14, 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 1 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-05-09T01:39:10.878 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 909, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 904336, 2024-05-09T01:39:10.879 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 909, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 909, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 904336, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T01:39:10.880 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 42, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 12044, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 42, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 12044, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.881 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 42, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 42, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 42, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 42, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.882 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-05-09T01:39:10.883 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x561dd985c680": { 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 6, 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1652, 2024-05-09T01:39:10.884 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 6, 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5, 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1652, 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:10.885 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T01:39:10.893 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:10 smithi019 ceph-mon[30158]: pgmap v787: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:11.603 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-09T01:39:11.603 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-09T01:39:11.603 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T01:39:11.603 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T01:39:11.840 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:39:12.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:12 smithi196 ceph-mon[35041]: pgmap v788: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:12.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:12 smithi059 ceph-mon[34351]: pgmap v788: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:12 smithi019 ceph-mon[30158]: pgmap v788: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:13.519 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T01:39:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:39:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 986, 2024-05-09T01:39:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 986, 2024-05-09T01:39:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 262487, 2024-05-09T01:39:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 312788, 2024-05-09T01:39:13.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 7, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.106113295, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.033949877, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.029044968, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.031906530, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 985, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.044419428, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000045095 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 894, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1105, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 219860, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 724283, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 5, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.111402294, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.035093968, 2024-05-09T01:39:13.521 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.026170076, 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.039823394, 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1103, 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.046027958, 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000041729 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 440304 2024-05-09T01:39:13.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1267, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1260, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1188512, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 482660, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 3, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.179952121, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.056986837, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.062845521, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.050162492, 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:39:13.523 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1259, 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.076214656, 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000060535 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 861248, 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 172928 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-05-09T01:39:13.524 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4717, 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.161066946, 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000034146 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.525 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-05-09T01:39:13.526 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-05-09T01:39:13.527 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-05-09T01:39:13.528 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-05-09T01:39:13.529 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-05-09T01:39:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-05-09T01:39:13.531 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-05-09T01:39:13.532 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 6, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195956, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 4195815, 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-05-09T01:39:13.533 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 6 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 3, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 3, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 1, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 1, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 0, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 0, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 0, 2024-05-09T01:39:13.534 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 121428, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215440 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.535 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-05-09T01:39:13.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-05-09T01:39:13.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-05-09T01:39:13.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-05-09T01:39:13.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.540 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-05-09T01:39:13.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.544 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 0, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1493, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-05-09T01:39:13.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:39:13.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:39:13.549 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 189463, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 52, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:39:13.550 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7720, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 162, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 8512, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 13, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-05-09T01:39:13.551 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 2849, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 2849, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2849, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.637964057, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000574925 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-05-09T01:39:13.552 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2849, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2850 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 2849, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 2849, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 1865, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 982, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-05-09T01:39:13.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-05-09T01:39:13.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-05-09T01:39:13.555 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 83, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 2, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 5, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 6, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 1, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-05-09T01:39:13.556 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:39:13.557 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3143, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1369907, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3143, 2024-05-09T01:39:13.558 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3143, 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1369907, 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T01:39:13.559 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2849, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 1657, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2849, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1657, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:13.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2849, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2849, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2849, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2849, 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:13.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55dc77372410": { 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T01:39:13.563 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T01:39:13.564 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T01:39:14.294 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-09T01:39:14.294 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-09T01:39:14.294 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T01:39:14.305 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T01:39:14.305 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:39:14.305 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T01:39:14.328 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:39:14.328 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T01:39:14.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T01:39:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:39:14.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:14 smithi019 ceph-mon[30158]: pgmap v789: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:39:14.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:14 smithi059 ceph-mon[34351]: pgmap v789: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:39:14.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:14 smithi196 ceph-mon[35041]: pgmap v789: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T01:39:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.480+0000 7f85c5bcc700 1 -- 172.21.15.19:0/889407087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0108740 msgr2=0x7f85c01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.480+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/889407087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0108740 0x7f85c01100a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f85b800b0d0 tx=0x7f85b800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.480+0000 7f85c5bcc700 1 -- 172.21.15.19:0/889407087 shutdown_connections 2024-05-09T01:39:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.480+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/889407087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0108740 0x7f85c01100a0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.480+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/889407087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85c01034c0 0x7f85c01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.480+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/889407087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85c0107df0 0x7f85c0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.480+0000 7f85c5bcc700 1 -- 172.21.15.19:0/889407087 >> 172.21.15.19:0/889407087 conn(0x7f85c00fedf0 msgr2=0x7f85c0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.480+0000 7f85c5bcc700 1 -- 172.21.15.19:0/889407087 shutdown_connections 2024-05-09T01:39:16.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.480+0000 7f85c5bcc700 1 -- 172.21.15.19:0/889407087 wait complete. 2024-05-09T01:39:16.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.481+0000 7f85c5bcc700 1 Processor -- start 2024-05-09T01:39:16.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.481+0000 7f85c5bcc700 1 -- start start 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85c5bcc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85c01034c0 0x7f85c019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85c5bcc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0107df0 0x7f85c019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85c5bcc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85c0108740 0x7f85c0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85c5bcc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0113370 con 0x7f85c0107df0 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85c5bcc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f85c01131f0 con 0x7f85c0108740 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85c5bcc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85c01134f0 con 0x7f85c01034c0 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85beffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0107df0 0x7f85c019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85beffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0107df0 0x7f85c019fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58828/0 (socket says 172.21.15.19:58828) 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85beffd700 1 -- 172.21.15.19:0/1891696256 learned_addr learned my addr 172.21.15.19:0/1891696256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85bf7fe700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85c01034c0 0x7f85c019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85bffff700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85c0108740 0x7f85c0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85beffd700 1 -- 172.21.15.19:0/1891696256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85c01034c0 msgr2=0x7f85c019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85beffd700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85c01034c0 0x7f85c019f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85beffd700 1 -- 172.21.15.19:0/1891696256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85c0108740 msgr2=0x7f85c0199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85beffd700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85c0108740 0x7f85c0199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85beffd700 1 -- 172.21.15.19:0/1891696256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c019a1d0 con 0x7f85c0107df0 2024-05-09T01:39:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85bf7fe700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85c01034c0 0x7f85c019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:39:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.482+0000 7f85beffd700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0107df0 0x7f85c019fe10 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f85b400c8f0 tx=0x7f85b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.483+0000 7f85bcff9700 1 -- 172.21.15.19:0/1891696256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4007b00 con 0x7f85c0107df0 2024-05-09T01:39:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.483+0000 7f85bcff9700 1 -- 172.21.15.19:0/1891696256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b4007ca0 con 0x7f85c0107df0 2024-05-09T01:39:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.483+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c019a4c0 con 0x7f85c0107df0 2024-05-09T01:39:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.483+0000 7f85bcff9700 1 -- 172.21.15.19:0/1891696256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4018660 con 0x7f85c0107df0 2024-05-09T01:39:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.483+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c01a6760 con 0x7f85c0107df0 2024-05-09T01:39:16.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.484+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c00512c0 con 0x7f85c0107df0 2024-05-09T01:39:16.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.485+0000 7f85bcff9700 1 -- 172.21.15.19:0/1891696256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f85b400f450 con 0x7f85c0107df0 2024-05-09T01:39:16.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.486+0000 7f85bcff9700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85a80b2900 0x7f85a80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:16.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.486+0000 7f85bf7fe700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85a80b2900 0x7f85a80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:16.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.486+0000 7f85bcff9700 1 -- 172.21.15.19:0/1891696256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f85b40e21d0 con 0x7f85c0107df0 2024-05-09T01:39:16.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.487+0000 7f85bf7fe700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85a80b2900 0x7f85a80b4e10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f85b0005950 tx=0x7f85b0005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:16.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.490+0000 7f85bcff9700 1 -- 172.21.15.19:0/1891696256 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85b4014030 con 0x7f85c0107df0 2024-05-09T01:39:16.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:16 smithi019 ceph-mon[30158]: pgmap v790: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.721+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f85c019afe0 con 0x7f85c0107df0 2024-05-09T01:39:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.722+0000 7f85bcff9700 1 -- 172.21.15.19:0/1891696256 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+586 (secure 0 0 0) 0x7f85b40aee20 con 0x7f85c0107df0 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3911883227 2024-05-10T01:15:21.027129+0000 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/272437452 2024-05-10T01:15:21.027129+0000 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/582848527 2024-05-10T01:14:53.392608+0000 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/455701507 2024-05-10T01:14:53.392608+0000 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/112627556 2024-05-10T01:15:21.027129+0000 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/292723758 2024-05-10T01:15:21.027129+0000 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/112627556 2024-05-10T01:15:21.027129+0000 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3332485717 2024-05-10T01:14:53.392608+0000 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3332485717 2024-05-10T01:14:53.392608+0000 2024-05-09T01:39:16.719 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3768545435 2024-05-10T01:14:53.392608+0000 2024-05-09T01:39:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85a80b2900 msgr2=0x7f85a80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85a80b2900 0x7f85a80b4e10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f85b0005950 tx=0x7f85b0005f50 comp rx=0 tx=0).stop 2024-05-09T01:39:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0107df0 msgr2=0x7f85c019fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0107df0 0x7f85c019fe10 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f85b400c8f0 tx=0x7f85b400ccb0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 shutdown_connections 2024-05-09T01:39:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85c0108740 0x7f85c0199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85c01034c0 0x7f85c019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f85a80b2900 0x7f85a80b4e10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 --2- 172.21.15.19:0/1891696256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c0107df0 0x7f85c019fe10 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 >> 172.21.15.19:0/1891696256 conn(0x7f85c00fedf0 msgr2=0x7f85c0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.724+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 shutdown_connections 2024-05-09T01:39:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:16.725+0000 7f85c5bcc700 1 -- 172.21.15.19:0/1891696256 wait complete. 2024-05-09T01:39:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-05-09T01:39:16.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:16 smithi059 ceph-mon[34351]: pgmap v790: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:16.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:16 smithi196 ceph-mon[35041]: pgmap v790: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T01:39:17.380 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T01:39:17.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:39:17.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T01:39:17.433 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T01:39:17.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:39:17.433 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T01:39:17.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:39:17.471 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-09T01:39:17.528 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T01:39:17.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:39:17.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T01:39:17.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:39:17.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-05-09T01:39:17.607 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:17 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1891696256' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T01:39:17.657 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T01:39:17.668 INFO:tasks.cephadm:Teardown begin 2024-05-09T01:39:17.669 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T01:39:17.669 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:39:17.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:17 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1891696256' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T01:39:17.923 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:39:17.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:17 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1891696256' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T01:39:18.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:18 smithi019 ceph-mon[30158]: pgmap v791: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:18.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:18 smithi059 ceph-mon[34351]: pgmap v791: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:18.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:18 smithi196 ceph-mon[35041]: pgmap v791: 97 pgs: 97 active+clean; 4.1 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-05-09T01:39:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.592+0000 7f16d465d700 1 -- 172.21.15.19:0/3222079604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc107df0 msgr2=0x7f16cc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.592+0000 7f16d465d700 1 --2- 172.21.15.19:0/3222079604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc107df0 0x7f16cc108200 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0099c0 tx=0x7f16bc009cd0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.593+0000 7f16d465d700 1 -- 172.21.15.19:0/3222079604 shutdown_connections 2024-05-09T01:39:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.593+0000 7f16d465d700 1 --2- 172.21.15.19:0/3222079604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc108740 0x7f16cc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.593+0000 7f16d465d700 1 --2- 172.21.15.19:0/3222079604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16cc1034c0 0x7f16cc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.593+0000 7f16d465d700 1 --2- 172.21.15.19:0/3222079604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc107df0 0x7f16cc108200 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.593+0000 7f16d465d700 1 -- 172.21.15.19:0/3222079604 >> 172.21.15.19:0/3222079604 conn(0x7f16cc0fedf0 msgr2=0x7f16cc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.593+0000 7f16d465d700 1 -- 172.21.15.19:0/3222079604 shutdown_connections 2024-05-09T01:39:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.593+0000 7f16d465d700 1 -- 172.21.15.19:0/3222079604 wait complete. 2024-05-09T01:39:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.593+0000 7f16d465d700 1 Processor -- start 2024-05-09T01:39:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.593+0000 7f16d465d700 1 -- start start 2024-05-09T01:39:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d465d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc1034c0 0x7f16cc0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d465d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc107df0 0x7f16cc0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d465d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16cc108740 0x7f16cc0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d465d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc113350 con 0x7f16cc1034c0 2024-05-09T01:39:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d465d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1131d0 con 0x7f16cc108740 2024-05-09T01:39:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d465d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1134d0 con 0x7f16cc107df0 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d23f9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc1034c0 0x7f16cc0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d23f9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc1034c0 0x7f16cc0ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58856/0 (socket says 172.21.15.19:58856) 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d23f9700 1 -- 172.21.15.19:0/2258399259 learned_addr learned my addr 172.21.15.19:0/2258399259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d2bfa700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16cc108740 0x7f16cc0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d1bf8700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc107df0 0x7f16cc0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d23f9700 1 -- 172.21.15.19:0/2258399259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc107df0 msgr2=0x7f16cc0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d23f9700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc107df0 0x7f16cc0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d23f9700 1 -- 172.21.15.19:0/2258399259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16cc108740 msgr2=0x7f16cc0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d23f9700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16cc108740 0x7f16cc0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d23f9700 1 -- 172.21.15.19:0/2258399259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc1a91b0 con 0x7f16cc1034c0 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.594+0000 7f16d23f9700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc1034c0 0x7f16cc0ff2d0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f16c800ef10 tx=0x7f16c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.595+0000 7f16c37fe700 1 -- 172.21.15.19:0/2258399259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c800c9c0 con 0x7f16cc1034c0 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.595+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc1a9440 con 0x7f16cc1034c0 2024-05-09T01:39:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.595+0000 7f16c37fe700 1 -- 172.21.15.19:0/2258399259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c8019070 con 0x7f16cc1034c0 2024-05-09T01:39:19.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.595+0000 7f16c37fe700 1 -- 172.21.15.19:0/2258399259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8014480 con 0x7f16cc1034c0 2024-05-09T01:39:19.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.595+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc1a9950 con 0x7f16cc1034c0 2024-05-09T01:39:19.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.596+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16cc0512c0 con 0x7f16cc1034c0 2024-05-09T01:39:19.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.597+0000 7f16c37fe700 1 -- 172.21.15.19:0/2258399259 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f16c8012050 con 0x7f16cc1034c0 2024-05-09T01:39:19.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.598+0000 7f16c37fe700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16b80b2880 0x7f16b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:19.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.598+0000 7f16c37fe700 1 -- 172.21.15.19:0/2258399259 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f16c80e2820 con 0x7f16cc1034c0 2024-05-09T01:39:19.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.598+0000 7f16d1bf8700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16b80b2880 0x7f16b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:19.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.599+0000 7f16d1bf8700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16b80b2880 0x7f16b80b4d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f16bc000c00 tx=0x7f16bc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.602+0000 7f16c37fe700 1 -- 172.21.15.19:0/2258399259 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16c8010040 con 0x7f16cc1034c0 2024-05-09T01:39:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.863+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16cc02e5f0 con 0x7f16cc1034c0 2024-05-09T01:39:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.864+0000 7f16c37fe700 1 -- 172.21.15.19:0/2258399259 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f16c80aa520 con 0x7f16cc1034c0 2024-05-09T01:39:19.861 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:39:19.861 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:39:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.866+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16b80b2880 msgr2=0x7f16b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.866+0000 7f16d465d700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16b80b2880 0x7f16b80b4d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f16bc000c00 tx=0x7f16bc01a040 comp rx=0 tx=0).stop 2024-05-09T01:39:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.866+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc1034c0 msgr2=0x7f16cc0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.866+0000 7f16d465d700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc1034c0 0x7f16cc0ff2d0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f16c800ef10 tx=0x7f16c800c3a0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.867+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 shutdown_connections 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.867+0000 7f16d465d700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16cc108740 0x7f16cc0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.867+0000 7f16d465d700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f16b80b2880 0x7f16b80b4d90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.867+0000 7f16d465d700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16cc1034c0 0x7f16cc0ff2d0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.867+0000 7f16d465d700 1 --2- 172.21.15.19:0/2258399259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16cc107df0 0x7f16cc0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.867+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 >> 172.21.15.19:0/2258399259 conn(0x7f16cc0fedf0 msgr2=0x7f16cc075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.867+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 shutdown_connections 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:19.867+0000 7f16d465d700 1 -- 172.21.15.19:0/2258399259 wait complete. 2024-05-09T01:39:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:39:20.512 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:39:20.766 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:39:20.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:20 smithi059 ceph-mon[34351]: pgmap v792: 97 pgs: 97 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:39:20.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:20 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2258399259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:39:20.889 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:20 smithi019 ceph-mon[30158]: pgmap v792: 97 pgs: 97 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:39:20.889 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:20 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2258399259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:39:20.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:20 smithi196 ceph-mon[35041]: pgmap v792: 97 pgs: 97 active+clean; 4.1 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T01:39:20.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:20 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2258399259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:39:22.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.449+0000 7fde837d3700 1 -- 172.21.15.19:0/1721939004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c109850 msgr2=0x7fde7c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.449+0000 7fde837d3700 1 --2- 172.21.15.19:0/1721939004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c109850 0x7fde7c1111b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fde7400b0d0 tx=0x7fde7400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.449+0000 7fde837d3700 1 -- 172.21.15.19:0/1721939004 shutdown_connections 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.449+0000 7fde837d3700 1 --2- 172.21.15.19:0/1721939004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c109850 0x7fde7c1111b0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.449+0000 7fde837d3700 1 --2- 172.21.15.19:0/1721939004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7c1045d0 0x7fde7c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.449+0000 7fde837d3700 1 --2- 172.21.15.19:0/1721939004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde7c108f00 0x7fde7c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.449+0000 7fde837d3700 1 -- 172.21.15.19:0/1721939004 >> 172.21.15.19:0/1721939004 conn(0x7fde7c0ffea0 msgr2=0x7fde7c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.449+0000 7fde837d3700 1 -- 172.21.15.19:0/1721939004 shutdown_connections 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.449+0000 7fde837d3700 1 -- 172.21.15.19:0/1721939004 wait complete. 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde837d3700 1 Processor -- start 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde837d3700 1 -- start start 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde837d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c1045d0 0x7fde7c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde837d3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde7c108f00 0x7fde7c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde837d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7c109850 0x7fde7c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde837d3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c114480 con 0x7fde7c1045d0 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde837d3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c114300 con 0x7fde7c109850 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde837d3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c114600 con 0x7fde7c108f00 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde8156f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c1045d0 0x7fde7c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde8156f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c1045d0 0x7fde7c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39992/0 (socket says 172.21.15.19:39992) 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde8156f700 1 -- 172.21.15.19:0/1191940391 learned_addr learned my addr 172.21.15.19:0/1191940391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde80d6e700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde7c108f00 0x7fde7c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.450+0000 7fde81d70700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7c109850 0x7fde7c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde8156f700 1 -- 172.21.15.19:0/1191940391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde7c108f00 msgr2=0x7fde7c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde8156f700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde7c108f00 0x7fde7c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde8156f700 1 -- 172.21.15.19:0/1191940391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7c109850 msgr2=0x7fde7c19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde8156f700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7c109850 0x7fde7c19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde8156f700 1 -- 172.21.15.19:0/1191940391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c19fc30 con 0x7fde7c1045d0 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde8156f700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c1045d0 0x7fde7c19adb0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fde7800d950 tx=0x7fde7800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde727fc700 1 -- 172.21.15.19:0/1191940391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78016070 con 0x7fde7c1045d0 2024-05-09T01:39:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c19ff20 con 0x7fde7c1045d0 2024-05-09T01:39:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde727fc700 1 -- 172.21.15.19:0/1191940391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7800fd40 con 0x7fde7c1045d0 2024-05-09T01:39:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde727fc700 1 -- 172.21.15.19:0/1191940391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7800e3b0 con 0x7fde7c1045d0 2024-05-09T01:39:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.451+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c1a7840 con 0x7fde7c1045d0 2024-05-09T01:39:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.452+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c0512c0 con 0x7fde7c1045d0 2024-05-09T01:39:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.455+0000 7fde727fc700 1 -- 172.21.15.19:0/1191940391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fde78019070 con 0x7fde7c1045d0 2024-05-09T01:39:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.455+0000 7fde727fc700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fde680b2880 0x7fde680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.455+0000 7fde80d6e700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fde680b2880 0x7fde680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.456+0000 7fde727fc700 1 -- 172.21.15.19:0/1191940391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fde780e1c80 con 0x7fde7c1045d0 2024-05-09T01:39:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.456+0000 7fde80d6e700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fde680b2880 0x7fde680b4d90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fde7c103440 tx=0x7fde6c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.459+0000 7fde727fc700 1 -- 172.21.15.19:0/1191940391 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fde780a9980 con 0x7fde7c1045d0 2024-05-09T01:39:22.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.729+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde7c19c0c0 con 0x7fde7c1045d0 2024-05-09T01:39:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.730+0000 7fde727fc700 1 -- 172.21.15.19:0/1191940391 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fde780a9280 con 0x7fde7c1045d0 2024-05-09T01:39:22.726 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:39:22.727 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14565,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/523355371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":523355371},{"type":"v1","addr":"172.21.15.19:6835","nonce":523355371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3245904831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3245904831},{"type":"v1","addr":"172.21.15.19:6837","nonce":3245904831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T01:22:41.711277+0000","modified":"2024-05-09T01:25:56.623597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24466,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/909874272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":909874272},{"type":"v1","addr":"172.21.15.19:6839","nonce":909874272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/555235417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":555235417},{"type":"v1","addr":"172.21.15.19:6841","nonce":555235417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/2736793414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2736793414},{"type":"v1","addr":"172.21.15.196:6833","nonce":2736793414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/3715662430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3715662430},{"type":"v1","addr":"172.21.15.59:6833","nonce":3715662430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1184717757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1184717757},{"type":"v1","addr":"172.21.15.196:6835","nonce":1184717757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/3887501643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3887501643},{"type":"v1","addr":"172.21.15.196:6837","nonce":3887501643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6835/3872629659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3872629659},{"type":"v1","addr":"172.21.15.59:6835","nonce":3872629659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6839/165915317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":165915317},{"type":"v1","addr":"172.21.15.196:6839","nonce":165915317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.59:6837/2120139910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2120139910},{"type":"v1","addr":"172.21.15.59:6837","nonce":2120139910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/1954332745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1954332745},{"type":"v1","addr":"172.21.15.59:6839","nonce":1954332745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24466,24452,24473,24451,24497]},"id":1}]} 2024-05-09T01:39:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.732+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fde680b2880 msgr2=0x7fde680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.732+0000 7fde837d3700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fde680b2880 0x7fde680b4d90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fde7c103440 tx=0x7fde6c00b410 comp rx=0 tx=0).stop 2024-05-09T01:39:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.732+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c1045d0 msgr2=0x7fde7c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.732+0000 7fde837d3700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c1045d0 0x7fde7c19adb0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fde7800d950 tx=0x7fde7800dc60 comp rx=0 tx=0).stop 2024-05-09T01:39:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.732+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 shutdown_connections 2024-05-09T01:39:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.732+0000 7fde837d3700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7c109850 0x7fde7c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.733+0000 7fde837d3700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7fde680b2880 0x7fde680b4d90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.733+0000 7fde837d3700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde7c1045d0 0x7fde7c19adb0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.733+0000 7fde837d3700 1 --2- 172.21.15.19:0/1191940391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde7c108f00 0x7fde7c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.733+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 >> 172.21.15.19:0/1191940391 conn(0x7fde7c0ffea0 msgr2=0x7fde7c108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.733+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 shutdown_connections 2024-05-09T01:39:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:22.733+0000 7fde837d3700 1 -- 172.21.15.19:0/1191940391 wait complete. 2024-05-09T01:39:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T01:39:22.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:22 smithi059 ceph-mon[34351]: pgmap v793: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:39:22.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:22 smithi019 ceph-mon[30158]: pgmap v793: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:39:22.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:22 smithi196 ceph-mon[35041]: pgmap v793: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:39:23.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:39:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:39:23.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:23 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/1191940391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:39:23.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:23 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/1191940391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:39:23.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:23 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/1191940391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:39:24.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:24 smithi019 ceph-mon[30158]: pgmap v794: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T01:39:24.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:24 smithi059 ceph-mon[34351]: pgmap v794: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T01:39:24.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:24 smithi196 ceph-mon[35041]: pgmap v794: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T01:39:25.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.312+0000 7f406cd30700 1 -- 172.21.15.19:0/3243934486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40681034c0 msgr2=0x7f40681038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:25.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.312+0000 7f406cd30700 1 --2- 172.21.15.19:0/3243934486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40681034c0 0x7f40681038d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f40580099c0 tx=0x7f4058009cd0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.313+0000 7f406cd30700 1 -- 172.21.15.19:0/3243934486 shutdown_connections 2024-05-09T01:39:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.313+0000 7f406cd30700 1 --2- 172.21.15.19:0/3243934486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4068108740 0x7f40681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.313+0000 7f406cd30700 1 --2- 172.21.15.19:0/3243934486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40681034c0 0x7f40681038d0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.313+0000 7f406cd30700 1 --2- 172.21.15.19:0/3243934486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4068107df0 0x7f4068108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.313+0000 7f406cd30700 1 -- 172.21.15.19:0/3243934486 >> 172.21.15.19:0/3243934486 conn(0x7f40680fedc0 msgr2=0x7f40681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.313+0000 7f406cd30700 1 -- 172.21.15.19:0/3243934486 shutdown_connections 2024-05-09T01:39:25.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.313+0000 7f406cd30700 1 -- 172.21.15.19:0/3243934486 wait complete. 2024-05-09T01:39:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.313+0000 7f406cd30700 1 Processor -- start 2024-05-09T01:39:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f406cd30700 1 -- start start 2024-05-09T01:39:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f406cd30700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40681034c0 0x7f406819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f406cd30700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4068107df0 0x7f406819c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f406cd30700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068108740 0x7f406819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f406cd30700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4068113370 con 0x7f4068108740 2024-05-09T01:39:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f406cd30700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40681131f0 con 0x7f40681034c0 2024-05-09T01:39:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f406cd30700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40681134f0 con 0x7f4068107df0 2024-05-09T01:39:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f4066d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068108740 0x7f406819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f4066d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068108740 0x7f406819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40000/0 (socket says 172.21.15.19:40000) 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f4066d9d700 1 -- 172.21.15.19:0/2066735819 learned_addr learned my addr 172.21.15.19:0/2066735819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f406659c700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40681034c0 0x7f406819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f4065d9b700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4068107df0 0x7f406819c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f4066d9d700 1 -- 172.21.15.19:0/2066735819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4068107df0 msgr2=0x7f406819c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f4066d9d700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4068107df0 0x7f406819c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f4066d9d700 1 -- 172.21.15.19:0/2066735819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40681034c0 msgr2=0x7f406819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f4066d9d700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40681034c0 0x7f406819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.314+0000 7f4066d9d700 1 -- 172.21.15.19:0/2066735819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40681a6fc0 con 0x7f4068108740 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.315+0000 7f4066d9d700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068108740 0x7f406819c710 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f406000ada0 tx=0x7f406000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.315+0000 7f405f7fe700 1 -- 172.21.15.19:0/2066735819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060017020 con 0x7f4068108740 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.315+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40681a72b0 con 0x7f4068108740 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.315+0000 7f405f7fe700 1 -- 172.21.15.19:0/2066735819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f406000ce30 con 0x7f4068108740 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.315+0000 7f405f7fe700 1 -- 172.21.15.19:0/2066735819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060005350 con 0x7f4068108740 2024-05-09T01:39:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.315+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40681a77f0 con 0x7f4068108740 2024-05-09T01:39:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.316+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40680512c0 con 0x7f4068108740 2024-05-09T01:39:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.317+0000 7f405f7fe700 1 -- 172.21.15.19:0/2066735819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f406000c980 con 0x7f4068108740 2024-05-09T01:39:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.318+0000 7f405f7fe700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f40540b2880 0x7f40540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.319+0000 7f406659c700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f40540b2880 0x7f40540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.319+0000 7f405f7fe700 1 -- 172.21.15.19:0/2066735819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f40600e1f00 con 0x7f4068108740 2024-05-09T01:39:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.319+0000 7f406659c700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f40540b2880 0x7f40540b4d90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4058009990 tx=0x7f4058000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:25.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.322+0000 7f405f7fe700 1 -- 172.21.15.19:0/2066735819 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4060021020 con 0x7f4068108740 2024-05-09T01:39:25.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.538+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f406802e5f0 con 0x7f4068108740 2024-05-09T01:39:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.540+0000 7f405f7fe700 1 -- 172.21.15.19:0/2066735819 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22716 (secure 0 0 0) 0x7f40600a9c30 con 0x7f4068108740 2024-05-09T01:39:25.537 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:39:25.537 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"58c4b1c8-0da1-11ef-bc97-c7b262605968","created":"2024-05-09T01:14:18.495533+0000","modified":"2024-05-09T01:23:22.298486+0000","last_up_change":"2024-05-09T01:21:36.095747+0000","last_in_change":"2024-05-09T01:21:15.652470+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T01:17:49.120859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T01:22:34.932415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T01:22:38.555946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ab01f11-6b26-4cb2-a61d-fadfc3d4ea88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6803","nonce":2513726191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6805","nonce":2513726191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6809","nonce":2513726191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2513726191},{"type":"v1","addr":"172.21.15.19:6807","nonce":2513726191}]},"public_addr":"172.21.15.19:6803/2513726191","cluster_addr":"172.21.15.19:6805/2513726191","heartbeat_back_addr":"172.21.15.19:6809/2513726191","heartbeat_front_addr":"172.21.15.19:6807/2513726191","state":["exists","up"]},{"osd":1,"uuid":"d6305a55-e657-43de-83fd-8e9568e91363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6801","nonce":2880894912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6803","nonce":2880894912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6807","nonce":2880894912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2880894912},{"type":"v1","addr":"172.21.15.59:6805","nonce":2880894912}]},"public_addr":"172.21.15.59:6801/2880894912","cluster_addr":"172.21.15.59:6803/2880894912","heartbeat_back_addr":"172.21.15.59:6807/2880894912","heartbeat_front_addr":"172.21.15.59:6805/2880894912","state":["exists","up"]},{"osd":2,"uuid":"632fb397-639e-4ccf-b0a2-31e1f8c7aa65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6801","nonce":1599368048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6803","nonce":1599368048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6807","nonce":1599368048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1599368048},{"type":"v1","addr":"172.21.15.196:6805","nonce":1599368048}]},"public_addr":"172.21.15.196:6801/1599368048","cluster_addr":"172.21.15.196:6803/1599368048","heartbeat_back_addr":"172.21.15.196:6807/1599368048","heartbeat_front_addr":"172.21.15.196:6805/1599368048","state":["exists","up"]},{"osd":3,"uuid":"067c1dbe-5d98-4603-bc43-64343344993e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6811","nonce":713020110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6813","nonce":713020110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6817","nonce":713020110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":713020110},{"type":"v1","addr":"172.21.15.19:6815","nonce":713020110}]},"public_addr":"172.21.15.19:6811/713020110","cluster_addr":"172.21.15.19:6813/713020110","heartbeat_back_addr":"172.21.15.19:6817/713020110","heartbeat_front_addr":"172.21.15.19:6815/713020110","state":["exists","up"]},{"osd":4,"uuid":"322cb4e2-84e1-494f-93d6-bd5b09ce3a82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6809","nonce":2419703664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6811","nonce":2419703664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6815","nonce":2419703664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2419703664},{"type":"v1","addr":"172.21.15.59:6813","nonce":2419703664}]},"public_addr":"172.21.15.59:6809/2419703664","cluster_addr":"172.21.15.59:6811/2419703664","heartbeat_back_addr":"172.21.15.59:6815/2419703664","heartbeat_front_addr":"172.21.15.59:6813/2419703664","state":["exists","up"]},{"osd":5,"uuid":"c53b8d7f-d032-46d4-98c0-99a6b77d9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6809","nonce":1450867392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6811","nonce":1450867392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6815","nonce":1450867392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1450867392},{"type":"v1","addr":"172.21.15.196:6813","nonce":1450867392}]},"public_addr":"172.21.15.196:6809/1450867392","cluster_addr":"172.21.15.196:6811/1450867392","heartbeat_back_addr":"172.21.15.196:6815/1450867392","heartbeat_front_addr":"172.21.15.196:6813/1450867392","state":["exists","up"]},{"osd":6,"uuid":"95d655ea-93ee-4077-96be-3bbddc722520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6819","nonce":3027498508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6821","nonce":3027498508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6825","nonce":3027498508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3027498508},{"type":"v1","addr":"172.21.15.19:6823","nonce":3027498508}]},"public_addr":"172.21.15.19:6819/3027498508","cluster_addr":"172.21.15.19:6821/3027498508","heartbeat_back_addr":"172.21.15.19:6825/3027498508","heartbeat_front_addr":"172.21.15.19:6823/3027498508","state":["exists","up"]},{"osd":7,"uuid":"792a321f-5e7a-4ec4-b6bd-020b1fb8bf07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6817","nonce":2903314697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6819","nonce":2903314697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6823","nonce":2903314697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2903314697},{"type":"v1","addr":"172.21.15.59:6821","nonce":2903314697}]},"public_addr":"172.21.15.59:6817/2903314697","cluster_addr":"172.21.15.59:6819/2903314697","heartbeat_back_addr":"172.21.15.59:6823/2903314697","heartbeat_front_addr":"172.21.15.59:6821/2903314697","state":["exists","up"]},{"osd":8,"uuid":"6a9e69b9-50a5-4379-be85-ef221d1b6cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6817","nonce":2419396882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6819","nonce":2419396882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6823","nonce":2419396882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2419396882},{"type":"v1","addr":"172.21.15.196:6821","nonce":2419396882}]},"public_addr":"172.21.15.196:6817/2419396882","cluster_addr":"172.21.15.196:6819/2419396882","heartbeat_back_addr":"172.21.15.196:6823/2419396882","heartbeat_front_addr":"172.21.15.196:6821/2419396882","state":["exists","up"]},{"osd":9,"uuid":"a7437a16-b125-4a63-80f3-987d87d3365d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6827","nonce":3240481605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6829","nonce":3240481605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6833","nonce":3240481605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3240481605},{"type":"v1","addr":"172.21.15.19:6831","nonce":3240481605}]},"public_addr":"172.21.15.19:6827/3240481605","cluster_addr":"172.21.15.19:6829/3240481605","heartbeat_back_addr":"172.21.15.19:6833/3240481605","heartbeat_front_addr":"172.21.15.19:6831/3240481605","state":["exists","up"]},{"osd":10,"uuid":"56aa036a-6771-45e1-80a9-eb0fbee10f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6825","nonce":1290773390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6827","nonce":1290773390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6831","nonce":1290773390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1290773390},{"type":"v1","addr":"172.21.15.59:6829","nonce":1290773390}]},"public_addr":"172.21.15.59:6825/1290773390","cluster_addr":"172.21.15.59:6827/1290773390","heartbeat_back_addr":"172.21.15.59:6831/1290773390","heartbeat_front_addr":"172.21.15.59:6829/1290773390","state":["exists","up"]},{"osd":11,"uuid":"3b4010a5-eb86-4f92-93de-4d7733ff5ec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6825","nonce":1918691791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6827","nonce":1918691791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6831","nonce":1918691791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1918691791},{"type":"v1","addr":"172.21.15.196:6829","nonce":1918691791}]},"public_addr":"172.21.15.196:6825/1918691791","cluster_addr":"172.21.15.196:6827/1918691791","heartbeat_back_addr":"172.21.15.196:6831/1918691791","heartbeat_front_addr":"172.21.15.196:6829/1918691791","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:16:57.104403+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:21.949835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:17:45.987676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:10.666307+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:18:35.734567+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:00.307058+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:26.171519+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:19:52.377793+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:16.496754+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:20:42.705593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:08.796141+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T01:21:33.552761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3911883227":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/272437452":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/582848527":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/455701507":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6800/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:0/292723758":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6801/112627556":"2024-05-10T01:15:21.027129+0000","172.21.15.19:6800/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:6801/3332485717":"2024-05-10T01:14:53.392608+0000","172.21.15.19:0/3768545435":"2024-05-10T01:14:53.392608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f40540b2880 msgr2=0x7f40540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f40540b2880 0x7f40540b4d90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4058009990 tx=0x7f4058000bc0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068108740 msgr2=0x7f406819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068108740 0x7f406819c710 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f406000ada0 tx=0x7f406000c330 comp rx=0 tx=0).stop 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 shutdown_connections 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f40540b2880 0x7f40540b4d90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4068108740 0x7f406819c710 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40681034c0 0x7f406819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 --2- 172.21.15.19:0/2066735819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4068107df0 0x7f406819c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 >> 172.21.15.19:0/2066735819 conn(0x7f40680fedc0 msgr2=0x7f40681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 shutdown_connections 2024-05-09T01:39:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:25.542+0000 7f406cd30700 1 -- 172.21.15.19:0/2066735819 wait complete. 2024-05-09T01:39:25.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:25 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2066735819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:39:25.854 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:25 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2066735819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:39:25.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:25 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2066735819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:39:26.225 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-09T01:39:26.487 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:39:26.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:26 smithi019 ceph-mon[30158]: pgmap v795: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T01:39:26.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:26 smithi059 ceph-mon[34351]: pgmap v795: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T01:39:26.943 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:26 smithi196 ceph-mon[35041]: pgmap v795: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T01:39:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.222+0000 7f33dffb3700 1 -- 172.21.15.19:0/1011333657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8108740 msgr2=0x7f33d81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.222+0000 7f33dffb3700 1 --2- 172.21.15.19:0/1011333657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8108740 0x7f33d81100a0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f33d000b0d0 tx=0x7f33d000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.222+0000 7f33dffb3700 1 -- 172.21.15.19:0/1011333657 shutdown_connections 2024-05-09T01:39:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.222+0000 7f33dffb3700 1 --2- 172.21.15.19:0/1011333657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8108740 0x7f33d81100a0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.222+0000 7f33dffb3700 1 --2- 172.21.15.19:0/1011333657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d81034c0 0x7f33d81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.222+0000 7f33dffb3700 1 --2- 172.21.15.19:0/1011333657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d8107df0 0x7f33d8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.222+0000 7f33dffb3700 1 -- 172.21.15.19:0/1011333657 >> 172.21.15.19:0/1011333657 conn(0x7f33d80fedc0 msgr2=0x7f33d81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.222+0000 7f33dffb3700 1 -- 172.21.15.19:0/1011333657 shutdown_connections 2024-05-09T01:39:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.222+0000 7f33dffb3700 1 -- 172.21.15.19:0/1011333657 wait complete. 2024-05-09T01:39:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dffb3700 1 Processor -- start 2024-05-09T01:39:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dffb3700 1 -- start start 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dffb3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d81034c0 0x7f33d819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dffb3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8107df0 0x7f33d819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dffb3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d8108740 0x7f33d81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dffb3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33d8113370 con 0x7f33d8107df0 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dffb3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81131f0 con 0x7f33d81034c0 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dffb3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81134f0 con 0x7f33d8108740 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dd54e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8107df0 0x7f33d819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dd54e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8107df0 0x7f33d819d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40012/0 (socket says 172.21.15.19:40012) 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.223+0000 7f33dd54e700 1 -- 172.21.15.19:0/2551453238 learned_addr learned my addr 172.21.15.19:0/2551453238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33ddd4f700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d81034c0 0x7f33d819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33de550700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d8108740 0x7f33d81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33dd54e700 1 -- 172.21.15.19:0/2551453238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d8108740 msgr2=0x7f33d81a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33dd54e700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d8108740 0x7f33d81a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33dd54e700 1 -- 172.21.15.19:0/2551453238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d81034c0 msgr2=0x7f33d819d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33dd54e700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d81034c0 0x7f33d819d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33dd54e700 1 -- 172.21.15.19:0/2551453238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d81a1f00 con 0x7f33d8107df0 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33dd54e700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8107df0 0x7f33d819d5d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f33c800d980 tx=0x7f33c800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33ceffd700 1 -- 172.21.15.19:0/2551453238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8017070 con 0x7f33d8107df0 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d81a21f0 con 0x7f33d8107df0 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33ceffd700 1 -- 172.21.15.19:0/2551453238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c8010460 con 0x7f33d8107df0 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33ceffd700 1 -- 172.21.15.19:0/2551453238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c800de60 con 0x7f33d8107df0 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.224+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d81a9b10 con 0x7f33d8107df0 2024-05-09T01:39:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.225+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d80512c0 con 0x7f33d8107df0 2024-05-09T01:39:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.227+0000 7f33ceffd700 1 -- 172.21.15.19:0/2551453238 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f33c8010610 con 0x7f33d8107df0 2024-05-09T01:39:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.228+0000 7f33ceffd700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f33c40b2880 0x7f33c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.228+0000 7f33ddd4f700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f33c40b2880 0x7f33c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.228+0000 7f33ceffd700 1 -- 172.21.15.19:0/2551453238 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f33c80e1de0 con 0x7f33d8107df0 2024-05-09T01:39:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.228+0000 7f33ddd4f700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f33c40b2880 0x7f33c40b4d90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f33d4005950 tx=0x7f33d40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.231+0000 7f33ceffd700 1 -- 172.21.15.19:0/2551453238 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33c80a9a80 con 0x7f33d8107df0 2024-05-09T01:39:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.490+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f33d802e5f0 con 0x7f33d8107df0 2024-05-09T01:39:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.493+0000 7f33ceffd700 1 -- 172.21.15.19:0/2551453238 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f33c80a9380 con 0x7f33d8107df0 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f33c40b2880 msgr2=0x7f33c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f33c40b2880 0x7f33c40b4d90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f33d4005950 tx=0x7f33d40058e0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8107df0 msgr2=0x7f33d819d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8107df0 0x7f33d819d5d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f33c800d980 tx=0x7f33c800dd40 comp rx=0 tx=0).stop 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 shutdown_connections 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d8108740 0x7f33d81a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d81034c0 0x7f33d819d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f33c40b2880 0x7f33c40b4d90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 --2- 172.21.15.19:0/2551453238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d8107df0 0x7f33d819d5d0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 >> 172.21.15.19:0/2551453238 conn(0x7f33d80fedc0 msgr2=0x7f33d810ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 shutdown_connections 2024-05-09T01:39:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:28.497+0000 7f33dffb3700 1 -- 172.21.15.19:0/2551453238 wait complete. 2024-05-09T01:39:28.495 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T01:39:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mon[34351]: pgmap v796: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:39:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2551453238' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T01:39:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mon[34351]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T01:39:28.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mon[34351]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T01:39:28.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2551453238' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T01:39:28.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mon[34351]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T01:39:28.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mon[34351]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T01:39:28.847 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k[64201]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.847 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k[64201]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.847 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mds[64205]: -- 172.21.15.59:0/3345639213 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e33cbb5a40 con 0x55e33cbfd600 2024-05-09T01:39:28.847 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k[64201]: starting mds.k at 2024-05-09T01:39:28.847 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.848 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.848 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mds[65143]: -- 172.21.15.59:0/96513220 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f7c177a40 con 0x558f7c1bf180 2024-05-09T01:39:28.848 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: starting mds.b at 2024-05-09T01:39:28.848 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-e[64520]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.848 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-e[64520]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.848 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mds[64524]: -- 172.21.15.59:0/3650745616 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb3bdd7a40 con 0x55bb3be1f180 2024-05-09T01:39:28.848 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-e[64520]: starting mds.e at 2024-05-09T01:39:28.848 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-h[64830]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.848 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-h[64830]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.849 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:39:28 smithi059 ceph-mds[64834]: -- 172.21.15.59:0/1556714405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e52e17a40 con 0x561e52e5f180 2024-05-09T01:39:28.849 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:39:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-h[64830]: starting mds.h at 2024-05-09T01:39:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a[30154]: 2024-05-09T01:39:28.491+0000 7f4e87298700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T01:39:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-mon[30158]: pgmap v796: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:39:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2551453238' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T01:39:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-mon[30158]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T01:39:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-mon[30158]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T01:39:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2551453238' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T01:39:28.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-mon[30158]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T01:39:28.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-mon[30158]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T01:39:28.863 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-a[72664]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.863 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-a[72664]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.863 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-mds[72681]: -- 172.21.15.19:0/3731536010 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a785d1da40 con 0x55a785d65a80 2024-05-09T01:39:28.863 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:39:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-a[72664]: starting mds.a at 2024-05-09T01:39:28.863 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:39:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-g[72213]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.863 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:39:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-g[72213]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.863 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:39:28 smithi019 ceph-mds[72217]: -- 172.21.15.19:0/801879331 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5249b3a40 con 0x55c5249fb180 2024-05-09T01:39:28.864 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:39:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-g[72213]: starting mds.g at 2024-05-09T01:39:28.944 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f[65399]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.944 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f[65399]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.944 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f[65399]: starting mds.f at 2024-05-09T01:39:28.944 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i[66021]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.945 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i[66021]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.945 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mds[66025]: -- 172.21.15.196:0/2904833681 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605d4d19a40 con 0x5605d4d61600 2024-05-09T01:39:28.945 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i[66021]: starting mds.i at 2024-05-09T01:39:28.945 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c[65710]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.945 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c[65710]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.945 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c[65710]: starting mds.c at 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l[65075]: ignoring --setuser ceph since I am not root 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l[65075]: ignoring --setgroup ceph since I am not root 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mds[65079]: -- 172.21.15.196:0/1692434075 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e0420364e0 con 0x55e042081a80 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mds[65079]: -- 172.21.15.196:0/1692434075 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e042039a40 con 0x55e042081a80 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:39:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l[65075]: starting mds.l at 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mon[35041]: pgmap v796: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2551453238' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mon[35041]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mon[35041]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T01:39:28.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2551453238' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T01:39:28.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mon[35041]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T01:39:28.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:28 smithi196 ceph-mon[35041]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T01:39:29.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T01:39:29.439 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:39:30.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: pgmap v798: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-09T01:39:30.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6833/2503433612,v1:172.21.15.196:6834/2503433612] up:boot 2024-05-09T01:39:30.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6832/4199948711,v1:172.21.15.196:6835/4199948711] up:boot 2024-05-09T01:39:30.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.19:6838/2209188541,v1:172.21.15.19:6839/2209188541] up:boot 2024-05-09T01:39:30.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6832/1924474177,v1:172.21.15.59:6834/1924474177] up:boot 2024-05-09T01:39:30.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.19:6840/3663047636,v1:172.21.15.19:6841/3663047636] up:boot 2024-05-09T01:39:30.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6833/1042501749,v1:172.21.15.59:6835/1042501749] up:boot 2024-05-09T01:39:30.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6837/2404535858,v1:172.21.15.196:6839/2404535858] up:boot 2024-05-09T01:39:30.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.196:6836/27672261,v1:172.21.15.196:6838/27672261] up:boot 2024-05-09T01:39:30.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6836/2719653167,v1:172.21.15.59:6837/2719653167] up:boot 2024-05-09T01:39:30.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: mds.? [v2:172.21.15.59:6838/2540407621,v1:172.21.15.59:6839/2540407621] up:boot 2024-05-09T01:39:30.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T01:39:30.589 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T01:39:30.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T01:39:30.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T01:39:30.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T01:39:30.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T01:39:30.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T01:39:30.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T01:39:30.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T01:39:30.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T01:39:30.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:30 smithi019 ceph-mon[30158]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:39:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: pgmap v798: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-09T01:39:30.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6833/2503433612,v1:172.21.15.196:6834/2503433612] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6832/4199948711,v1:172.21.15.196:6835/4199948711] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.19:6838/2209188541,v1:172.21.15.19:6839/2209188541] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6832/1924474177,v1:172.21.15.59:6834/1924474177] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.19:6840/3663047636,v1:172.21.15.19:6841/3663047636] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6833/1042501749,v1:172.21.15.59:6835/1042501749] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6837/2404535858,v1:172.21.15.196:6839/2404535858] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.196:6836/27672261,v1:172.21.15.196:6838/27672261] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6836/2719653167,v1:172.21.15.59:6837/2719653167] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: mds.? [v2:172.21.15.59:6838/2540407621,v1:172.21.15.59:6839/2540407621] up:boot 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T01:39:30.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T01:39:30.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T01:39:30.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T01:39:30.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T01:39:30.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T01:39:30.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T01:39:30.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T01:39:30.848 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:30 smithi059 ceph-mon[34351]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:39:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: pgmap v798: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-09T01:39:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6833/2503433612,v1:172.21.15.196:6834/2503433612] up:boot 2024-05-09T01:39:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6832/4199948711,v1:172.21.15.196:6835/4199948711] up:boot 2024-05-09T01:39:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.19:6838/2209188541,v1:172.21.15.19:6839/2209188541] up:boot 2024-05-09T01:39:30.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6832/1924474177,v1:172.21.15.59:6834/1924474177] up:boot 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.19:6840/3663047636,v1:172.21.15.19:6841/3663047636] up:boot 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6833/1042501749,v1:172.21.15.59:6835/1042501749] up:boot 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6837/2404535858,v1:172.21.15.196:6839/2404535858] up:boot 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.196:6836/27672261,v1:172.21.15.196:6838/27672261] up:boot 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6836/2719653167,v1:172.21.15.59:6837/2719653167] up:boot 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: mds.? [v2:172.21.15.59:6838/2540407621,v1:172.21.15.59:6839/2540407621] up:boot 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T01:39:30.945 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T01:39:30.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T01:39:30.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T01:39:30.946 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:30 smithi196 ceph-mon[35041]: from='mgr.14150 172.21.15.19:0/1466947116' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:39:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.113+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/1618716644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44075d30 msgr2=0x7f3c44076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.113+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/1618716644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44075d30 0x7f3c44076140 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000b0a0 tx=0x7f3c4000b3b0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.113+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/1618716644 shutdown_connections 2024-05-09T01:39:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.113+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/1618716644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c44110ba0 0x7f3c44113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.113+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/1618716644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44075d30 0x7f3c44076140 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.113+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/1618716644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c44076680 0x7f3c44110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.113+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/1618716644 >> 172.21.15.19:0/1618716644 conn(0x7f3c44100040 msgr2=0x7f3c44102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.114+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/1618716644 shutdown_connections 2024-05-09T01:39:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.114+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/1618716644 wait complete. 2024-05-09T01:39:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.114+0000 7f3c4b7e9700 1 Processor -- start 2024-05-09T01:39:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c4b7e9700 1 -- start start 2024-05-09T01:39:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c4b7e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c44075d30 0x7f3c4419d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:31.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c4b7e9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c44076680 0x7f3c4419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c4b7e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44110ba0 0x7f3c441a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c4b7e9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44116710 con 0x7f3c44110ba0 2024-05-09T01:39:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c4b7e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44116590 con 0x7f3c44075d30 2024-05-09T01:39:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c4b7e9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44116890 con 0x7f3c44076680 2024-05-09T01:39:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c49d86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44110ba0 0x7f3c441a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c49585700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c44075d30 0x7f3c4419d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c49d86700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44110ba0 0x7f3c441a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41718/0 (socket says 172.21.15.19:41718) 2024-05-09T01:39:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c49d86700 1 -- 172.21.15.19:0/392757008 learned_addr learned my addr 172.21.15.19:0/392757008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:39:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c48d84700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c44076680 0x7f3c4419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c49d86700 1 -- 172.21.15.19:0/392757008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c44076680 msgr2=0x7f3c4419d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c49d86700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c44076680 0x7f3c4419d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c49d86700 1 -- 172.21.15.19:0/392757008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c44075d30 msgr2=0x7f3c4419d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c49d86700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c44075d30 0x7f3c4419d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.115+0000 7f3c49d86700 1 -- 172.21.15.19:0/392757008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c441a1f40 con 0x7f3c44110ba0 2024-05-09T01:39:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.116+0000 7f3c49d86700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44110ba0 0x7f3c441a1950 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00b480 tx=0x7f3c3c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.116+0000 7f3c3a7fc700 1 -- 172.21.15.19:0/392757008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c015020 con 0x7f3c44110ba0 2024-05-09T01:39:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.116+0000 7f3c3a7fc700 1 -- 172.21.15.19:0/392757008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c3c011ce0 con 0x7f3c44110ba0 2024-05-09T01:39:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.116+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c441a2230 con 0x7f3c44110ba0 2024-05-09T01:39:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.116+0000 7f3c3a7fc700 1 -- 172.21.15.19:0/392757008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c010bd0 con 0x7f3c44110ba0 2024-05-09T01:39:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.116+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c441a2770 con 0x7f3c44110ba0 2024-05-09T01:39:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.117+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c440512c0 con 0x7f3c44110ba0 2024-05-09T01:39:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.118+0000 7f3c3a7fc700 1 -- 172.21.15.19:0/392757008 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3c3c018020 con 0x7f3c44110ba0 2024-05-09T01:39:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.119+0000 7f3c3a7fc700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3c300b2900 0x7f3c300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.119+0000 7f3c49585700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3c300b2900 0x7f3c300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.119+0000 7f3c3a7fc700 1 -- 172.21.15.19:0/392757008 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8699+0+0 (secure 0 0 0) 0x7f3c3c0e1a00 con 0x7f3c44110ba0 2024-05-09T01:39:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.120+0000 7f3c49585700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3c300b2900 0x7f3c300b4e10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009e60 tx=0x7f3c4000b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:31.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.124+0000 7f3c3a7fc700 1 -- 172.21.15.19:0/392757008 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c3c01f020 con 0x7f3c44110ba0 2024-05-09T01:39:31.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.385+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3c4419e480 con 0x7f3c44110ba0 2024-05-09T01:39:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.503+0000 7f3c3a7fc700 1 -- 172.21.15.19:0/392757008 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f3c3c0a9550 con 0x7f3c44110ba0 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3c300b2900 msgr2=0x7f3c300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3c300b2900 0x7f3c300b4e10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009e60 tx=0x7f3c4000b7b0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44110ba0 msgr2=0x7f3c441a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44110ba0 0x7f3c441a1950 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00b480 tx=0x7f3c3c00b790 comp rx=0 tx=0).stop 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 shutdown_connections 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3c300b2900 0x7f3c300b4e10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c44110ba0 0x7f3c441a1950 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c44075d30 0x7f3c4419d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 --2- 172.21.15.19:0/392757008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c44076680 0x7f3c4419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 >> 172.21.15.19:0/392757008 conn(0x7f3c44100040 msgr2=0x7f3c44100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 shutdown_connections 2024-05-09T01:39:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:31.506+0000 7f3c4b7e9700 1 -- 172.21.15.19:0/392757008 wait complete. 2024-05-09T01:39:31.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:31 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/392757008' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:39:31.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:31 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/392757008' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:39:31.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:31 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/392757008' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:39:32.152 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T01:39:32.419 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:39:32.747 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:32 smithi059 ceph-mon[34351]: pgmap v799: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:39:32.747 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:32 smithi059 ceph-mon[34351]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T01:39:32.747 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:32 smithi059 ceph-mon[34351]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T01:39:32.747 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:32 smithi059 ceph-mon[34351]: Cluster is now healthy 2024-05-09T01:39:32.747 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:32 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/392757008' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T01:39:32.747 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:32 smithi059 ceph-mon[34351]: fsmap 12 up:standby 2024-05-09T01:39:32.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:32 smithi019 ceph-mon[30158]: pgmap v799: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:39:32.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:32 smithi019 ceph-mon[30158]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T01:39:32.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:32 smithi019 ceph-mon[30158]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T01:39:32.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:32 smithi019 ceph-mon[30158]: Cluster is now healthy 2024-05-09T01:39:32.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:32 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/392757008' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T01:39:32.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:32 smithi019 ceph-mon[30158]: fsmap 12 up:standby 2024-05-09T01:39:32.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:32 smithi196 ceph-mon[35041]: pgmap v799: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:39:32.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:32 smithi196 ceph-mon[35041]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T01:39:32.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:32 smithi196 ceph-mon[35041]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T01:39:32.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:32 smithi196 ceph-mon[35041]: Cluster is now healthy 2024-05-09T01:39:32.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:32 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/392757008' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T01:39:32.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:32 smithi196 ceph-mon[35041]: fsmap 12 up:standby 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.110+0000 7f3dce516700 1 -- 172.21.15.19:0/2337354298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dc810e8f0 msgr2=0x7f3dc8110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.110+0000 7f3dce516700 1 --2- 172.21.15.19:0/2337354298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dc810e8f0 0x7f3dc8110dd0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000b0a0 tx=0x7f3dc000b3b0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.110+0000 7f3dce516700 1 -- 172.21.15.19:0/2337354298 shutdown_connections 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.110+0000 7f3dce516700 1 --2- 172.21.15.19:0/2337354298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dc810e8f0 0x7f3dc8110dd0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.110+0000 7f3dce516700 1 --2- 172.21.15.19:0/2337354298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dc8105480 0x7f3dc8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.110+0000 7f3dce516700 1 --2- 172.21.15.19:0/2337354298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105dd0 0x7f3dc810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.110+0000 7f3dce516700 1 -- 172.21.15.19:0/2337354298 >> 172.21.15.19:0/2337354298 conn(0x7f3dc80fef30 msgr2=0x7f3dc8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.110+0000 7f3dce516700 1 -- 172.21.15.19:0/2337354298 shutdown_connections 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.110+0000 7f3dce516700 1 -- 172.21.15.19:0/2337354298 wait complete. 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.111+0000 7f3dce516700 1 Processor -- start 2024-05-09T01:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.111+0000 7f3dce516700 1 -- start start 2024-05-09T01:39:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dce516700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dc8105480 0x7f3dc81a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dce516700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105dd0 0x7f3dc81a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dce516700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dc810e8f0 0x7f3dc81ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dce516700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81144a0 con 0x7f3dc8105480 2024-05-09T01:39:34.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dce516700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8114320 con 0x7f3dc810e8f0 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dce516700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8114620 con 0x7f3dc8105dd0 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dccab3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dc810e8f0 0x7f3dc81ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dccab3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dc810e8f0 0x7f3dc81ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:48416/0 (socket says 172.21.15.19:48416) 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dccab3700 1 -- 172.21.15.19:0/2389241267 learned_addr learned my addr 172.21.15.19:0/2389241267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dbffff700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105dd0 0x7f3dc81a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dc7fff700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dc8105480 0x7f3dc81a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dccab3700 1 -- 172.21.15.19:0/2389241267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105dd0 msgr2=0x7f3dc81a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dccab3700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105dd0 0x7f3dc81a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dccab3700 1 -- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dc8105480 msgr2=0x7f3dc81a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dccab3700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dc8105480 0x7f3dc81a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.112+0000 7f3dccab3700 1 -- 172.21.15.19:0/2389241267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc81ad740 con 0x7f3dc810e8f0 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.113+0000 7f3dccab3700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dc810e8f0 0x7f3dc81ad200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000b070 tx=0x7f3dc0009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.113+0000 7f3dc5ffb700 1 -- 172.21.15.19:0/2389241267 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0021070 con 0x7f3dc810e8f0 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.113+0000 7f3dc5ffb700 1 -- 172.21.15.19:0/2389241267 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc0004e40 con 0x7f3dc810e8f0 2024-05-09T01:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.113+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc81ad910 con 0x7f3dc810e8f0 2024-05-09T01:39:34.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.113+0000 7f3dc5ffb700 1 -- 172.21.15.19:0/2389241267 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0004630 con 0x7f3dc810e8f0 2024-05-09T01:39:34.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.113+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81add00 con 0x7f3dc810e8f0 2024-05-09T01:39:34.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.114+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc806c920 con 0x7f3dc810e8f0 2024-05-09T01:39:34.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.116+0000 7f3dc5ffb700 1 -- 172.21.15.19:0/2389241267 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3dc0007980 con 0x7f3dc810e8f0 2024-05-09T01:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.117+0000 7f3dc5ffb700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3db40b2880 0x7f3db40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.117+0000 7f3dc5ffb700 1 -- 172.21.15.19:0/2389241267 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8699+0+0 (secure 0 0 0) 0x7f3dc000f020 con 0x7f3dc810e8f0 2024-05-09T01:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.117+0000 7f3dc7fff700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3db40b2880 0x7f3db40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.117+0000 7f3dc7fff700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3db40b2880 0x7f3db40b4d90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009c60 tx=0x7f3db8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:34.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.120+0000 7f3dc5ffb700 1 -- 172.21.15.19:0/2389241267 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3dc002a020 con 0x7f3dc810e8f0 2024-05-09T01:39:34.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.336+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3dc80512c0 con 0x7f3dc810e8f0 2024-05-09T01:39:34.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.514+0000 7f3dc5ffb700 1 -- 172.21.15.19:0/2389241267 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f3dc80512c0 con 0x7f3dc810e8f0 2024-05-09T01:39:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3db40b2880 msgr2=0x7f3db40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3db40b2880 0x7f3db40b4d90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009c60 tx=0x7f3db8009260 comp rx=0 tx=0).stop 2024-05-09T01:39:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dc810e8f0 msgr2=0x7f3dc81ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dc810e8f0 0x7f3dc81ad200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000b070 tx=0x7f3dc0009c90 comp rx=0 tx=0).stop 2024-05-09T01:39:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 shutdown_connections 2024-05-09T01:39:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dc810e8f0 0x7f3dc81ad200 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f3db40b2880 0x7f3db40b4d90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dc8105480 0x7f3dc81a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 --2- 172.21.15.19:0/2389241267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105dd0 0x7f3dc81a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:34.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.517+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 >> 172.21.15.19:0/2389241267 conn(0x7f3dc80fef30 msgr2=0x7f3dc81030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:34.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.518+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 shutdown_connections 2024-05-09T01:39:34.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:34.518+0000 7f3dce516700 1 -- 172.21.15.19:0/2389241267 wait complete. 2024-05-09T01:39:34.515 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-05-09T01:39:34.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:34 smithi059 ceph-mon[34351]: pgmap v800: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:39:34.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:34 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2389241267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:39:34.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:34 smithi059 ceph-mon[34351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:39:34.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:34 smithi019 ceph-mon[30158]: pgmap v800: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:39:34.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:34 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2389241267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:39:34.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:34 smithi019 ceph-mon[30158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:39:34.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:34 smithi196 ceph-mon[35041]: pgmap v800: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T01:39:34.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:34 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2389241267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:39:34.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:34 smithi196 ceph-mon[35041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:39:35.240 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T01:39:35.494 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:39:35.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:35 smithi019 ceph-mon[30158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:39:35.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:35 smithi019 ceph-mon[30158]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T01:39:35.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:35 smithi059 ceph-mon[34351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:39:35.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:35 smithi059 ceph-mon[34351]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T01:39:35.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:35 smithi196 ceph-mon[35041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:39:35.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:35 smithi196 ceph-mon[35041]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T01:39:36.594 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:36 smithi019 ceph-mon[30158]: pgmap v802: 65 pgs: 65 active+clean; 577 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:39:36.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:36 smithi059 ceph-mon[34351]: pgmap v802: 65 pgs: 65 active+clean; 577 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:39:36.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:36 smithi196 ceph-mon[35041]: pgmap v802: 65 pgs: 65 active+clean; 577 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:39:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.198+0000 7f1e49378700 1 -- 172.21.15.19:0/3145152008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 msgr2=0x7f1e44108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.198+0000 7f1e49378700 1 --2- 172.21.15.19:0/3145152008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 0x7f1e44108200 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f1e380099c0 tx=0x7f1e38009cd0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.198+0000 7f1e49378700 1 -- 172.21.15.19:0/3145152008 shutdown_connections 2024-05-09T01:39:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.198+0000 7f1e49378700 1 --2- 172.21.15.19:0/3145152008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e44108740 0x7f1e441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.198+0000 7f1e49378700 1 --2- 172.21.15.19:0/3145152008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e441034c0 0x7f1e441038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.198+0000 7f1e49378700 1 --2- 172.21.15.19:0/3145152008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 0x7f1e44108200 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.198+0000 7f1e49378700 1 -- 172.21.15.19:0/3145152008 >> 172.21.15.19:0/3145152008 conn(0x7f1e440fedf0 msgr2=0x7f1e44101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.198+0000 7f1e49378700 1 -- 172.21.15.19:0/3145152008 shutdown_connections 2024-05-09T01:39:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.198+0000 7f1e49378700 1 -- 172.21.15.19:0/3145152008 wait complete. 2024-05-09T01:39:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.199+0000 7f1e49378700 1 Processor -- start 2024-05-09T01:39:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.199+0000 7f1e49378700 1 -- start start 2024-05-09T01:39:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.199+0000 7f1e49378700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e441034c0 0x7f1e4419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.199+0000 7f1e49378700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 0x7f1e4419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.199+0000 7f1e49378700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e44108740 0x7f1e4419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e49378700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44113350 con 0x7f1e44107df0 2024-05-09T01:39:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e49378700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441131d0 con 0x7f1e441034c0 2024-05-09T01:39:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e49378700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441134d0 con 0x7f1e44108740 2024-05-09T01:39:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e427fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 0x7f1e4419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e427fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 0x7f1e4419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41772/0 (socket says 172.21.15.19:41772) 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e427fc700 1 -- 172.21.15.19:0/2341572705 learned_addr learned my addr 172.21.15.19:0/2341572705 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e437fe700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e44108740 0x7f1e4419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e42ffd700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e441034c0 0x7f1e4419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e427fc700 1 -- 172.21.15.19:0/2341572705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e44108740 msgr2=0x7f1e4419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e427fc700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e44108740 0x7f1e4419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e427fc700 1 -- 172.21.15.19:0/2341572705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e441034c0 msgr2=0x7f1e4419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e427fc700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e441034c0 0x7f1e4419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e427fc700 1 -- 172.21.15.19:0/2341572705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4419fca0 con 0x7f1e44107df0 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e42ffd700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e441034c0 0x7f1e4419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e427fc700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 0x7f1e4419b370 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f1e380098e0 tx=0x7f1e380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e2bfff700 1 -- 172.21.15.19:0/2341572705 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38027070 con 0x7f1e44107df0 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.200+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4419ff30 con 0x7f1e44107df0 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.201+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e441a78b0 con 0x7f1e44107df0 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.201+0000 7f1e2bfff700 1 -- 172.21.15.19:0/2341572705 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e38003f20 con 0x7f1e44107df0 2024-05-09T01:39:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.201+0000 7f1e2bfff700 1 -- 172.21.15.19:0/2341572705 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38022ce0 con 0x7f1e44107df0 2024-05-09T01:39:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.202+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e30005360 con 0x7f1e44107df0 2024-05-09T01:39:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.203+0000 7f1e2bfff700 1 -- 172.21.15.19:0/2341572705 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1e3802a070 con 0x7f1e44107df0 2024-05-09T01:39:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.204+0000 7f1e2bfff700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e2c0b2900 0x7f1e2c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:39:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.204+0000 7f1e2bfff700 1 -- 172.21.15.19:0/2341572705 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8116+0+0 (secure 0 0 0) 0x7f1e380e2fe0 con 0x7f1e44107df0 2024-05-09T01:39:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.204+0000 7f1e42ffd700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e2c0b2900 0x7f1e2c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:39:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.205+0000 7f1e42ffd700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e2c0b2900 0x7f1e2c0b4e10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1e34005950 tx=0x7f1e340058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:39:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.208+0000 7f1e2bfff700 1 -- 172.21.15.19:0/2341572705 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e38013070 con 0x7f1e44107df0 2024-05-09T01:39:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.421+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1e30005650 con 0x7f1e44107df0 2024-05-09T01:39:37.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.523+0000 7f1e2bfff700 1 -- 172.21.15.19:0/2341572705 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7f1e380aadb0 con 0x7f1e44107df0 2024-05-09T01:39:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.527+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e2c0b2900 msgr2=0x7f1e2c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.527+0000 7f1e49378700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e2c0b2900 0x7f1e2c0b4e10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1e34005950 tx=0x7f1e340058e0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.527+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 msgr2=0x7f1e4419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:39:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.527+0000 7f1e49378700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 0x7f1e4419b370 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f1e380098e0 tx=0x7f1e380057c0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.528+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 shutdown_connections 2024-05-09T01:39:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.528+0000 7f1e49378700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e44108740 0x7f1e4419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.528+0000 7f1e49378700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e441034c0 0x7f1e4419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.528+0000 7f1e49378700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:6800/2331192605,v1:172.21.15.19:6801/2331192605] conn(0x7f1e2c0b2900 0x7f1e2c0b4e10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.528+0000 7f1e49378700 1 --2- 172.21.15.19:0/2341572705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e44107df0 0x7f1e4419b370 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:39:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.528+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 >> 172.21.15.19:0/2341572705 conn(0x7f1e440fedf0 msgr2=0x7f1e44075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:39:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.528+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 shutdown_connections 2024-05-09T01:39:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T01:39:37.528+0000 7f1e49378700 1 -- 172.21.15.19:0/2341572705 wait complete. 2024-05-09T01:39:37.526 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-05-09T01:39:37.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:37 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2341572705' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:39:37.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:37 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2341572705' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:39:37.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:37 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2341572705' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:39:38.190 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:39:38.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:39:38.255 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:39:38.293 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T01:39:38.293 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T01:39:38.310 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T01:39:38.328 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T01:39:38.353 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T01:39:38.354 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T01:39:38.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.a 2024-05-09T01:39:38.522 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:38 smithi019 systemd[1]: Stopping Ceph mon.a for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:39:38.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:38 smithi196 ceph-mon[35041]: pgmap v803: 65 pgs: 65 active+clean; 577 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:39:38.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:38 smithi196 ceph-mon[35041]: from='client.? 172.21.15.19:0/2341572705' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:39:38.694 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:38 smithi196 ceph-mon[35041]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T01:39:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:38 smithi059 ceph-mon[34351]: pgmap v803: 65 pgs: 65 active+clean; 577 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:39:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:38 smithi059 ceph-mon[34351]: from='client.? 172.21.15.19:0/2341572705' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:39:38.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:38 smithi059 ceph-mon[34351]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T01:39:38.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:38 smithi019 ceph-mon[30158]: pgmap v803: 65 pgs: 65 active+clean; 577 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:39:38.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:38 smithi019 ceph-mon[30158]: from='client.? 172.21.15.19:0/2341572705' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:39:38.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:38 smithi019 ceph-mon[30158]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T01:39:38.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:38 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a[30154]: 2024-05-09T01:39:38.650+0000 7f4e8caa3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:39:38.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:38 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a[30154]: 2024-05-09T01:39:38.650+0000 7f4e8caa3700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T01:39:38.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:38 smithi019 podman[124640]: 2024-05-09 01:39:38.682069612 +0000 UTC m=+0.127821702 container died 308e13fd3f8c631b12dfa25c4a23bb7ed4cabb10f49a74beb513e4f2670ad903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:39:39.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:39 smithi019 podman[124640]: 2024-05-09 01:39:39.034537489 +0000 UTC m=+0.480289587 container cleanup 308e13fd3f8c631b12dfa25c4a23bb7ed4cabb10f49a74beb513e4f2670ad903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:39:39.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:39:39 smithi019 bash[124640]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-a 2024-05-09T01:39:39.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.a.service' 2024-05-09T01:39:39.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:39:39.464 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T01:39:39.464 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T01:39:39.464 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.b 2024-05-09T01:39:39.787 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:39 smithi059 systemd[1]: Stopping Ceph mon.b for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:39:40.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-b[34347]: 2024-05-09T01:39:39.789+0000 7f299d9da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:39:40.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-b[34347]: 2024-05-09T01:39:39.789+0000 7f299d9da700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T01:39:40.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:40 smithi059 podman[70057]: 2024-05-09 01:39:40.016254215 +0000 UTC m=+0.344572507 container died 9a88f306158aadb37780c72fc373a098022f32baccaa78e53895cb6c14ccd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:39:40.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:40 smithi059 podman[70057]: 2024-05-09 01:39:40.408014255 +0000 UTC m=+0.736332527 container cleanup 9a88f306158aadb37780c72fc373a098022f32baccaa78e53895cb6c14ccd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-09T01:39:40.596 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 01:39:40 smithi059 bash[70057]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-b 2024-05-09T01:39:40.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.b.service' 2024-05-09T01:39:40.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:39:40.874 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T01:39:40.874 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T01:39:40.874 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.c 2024-05-09T01:39:41.180 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:40 smithi196 systemd[1]: Stopping Ceph mon.c for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:39:41.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:41 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-c[35037]: 2024-05-09T01:39:41.183+0000 7f182df42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:39:41.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:41 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-c[35037]: 2024-05-09T01:39:41.183+0000 7f182df42700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T01:39:41.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:41 smithi196 podman[71588]: 2024-05-09 01:39:41.408472469 +0000 UTC m=+0.333377612 container died 98589ef238fd1a4bfe2246cd2f2bebdae22af19b07f2741dc5af63cf8a31f8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True) 2024-05-09T01:39:41.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:41 smithi196 podman[71588]: 2024-05-09 01:39:41.78356363 +0000 UTC m=+0.708468760 container cleanup 98589ef238fd1a4bfe2246cd2f2bebdae22af19b07f2741dc5af63cf8a31f8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-09T01:39:41.944 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 01:39:41 smithi196 bash[71588]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mon-c 2024-05-09T01:39:42.161 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mon.c.service' 2024-05-09T01:39:42.206 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:39:42.207 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T01:39:42.207 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T01:39:42.207 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.x 2024-05-09T01:39:42.527 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:39:42 smithi019 systemd[1]: Stopping Ceph mgr.x for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:39:42.828 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:39:42 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x[30392]: 2024-05-09T01:39:42.530+0000 7f92db1c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:39:42.828 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:39:42 smithi019 podman[124763]: 2024-05-09 01:39:42.571693558 +0000 UTC m=+0.161155961 container died d07233fe87ec017c0fd98c86e3499ae1055e1438a4af20b7cc8048469baad868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:39:43.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:39:42 smithi019 podman[124763]: 2024-05-09 01:39:42.981105298 +0000 UTC m=+0.570567743 container cleanup d07233fe87ec017c0fd98c86e3499ae1055e1438a4af20b7cc8048469baad868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T01:39:43.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:39:42 smithi019 bash[124763]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x 2024-05-09T01:39:43.659 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:39:43 smithi019 podman[124777]: 2024-05-09 01:39:43.408329971 +0000 UTC m=+0.839781892 container remove d07233fe87ec017c0fd98c86e3499ae1055e1438a4af20b7cc8048469baad868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-x, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T01:39:43.675 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-09T01:39:43.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:39:43.711 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T01:39:43.711 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T01:39:43.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.y 2024-05-09T01:39:44.038 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:39:43 smithi059 systemd[1]: Stopping Ceph mgr.y for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:39:44.346 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:39:44 smithi059 podman[70184]: 2024-05-09 01:39:44.04203667 +0000 UTC m=+0.129601481 container died ccad773eb2af100890de4b04110aa0e59affb63df636888a6835920d392b167e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True) 2024-05-09T01:39:44.744 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:39:44 smithi059 podman[70184]: 2024-05-09 01:39:44.546364273 +0000 UTC m=+0.633929084 container cleanup ccad773eb2af100890de4b04110aa0e59affb63df636888a6835920d392b167e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True) 2024-05-09T01:39:44.744 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 01:39:44 smithi059 bash[70184]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-y 2024-05-09T01:39:44.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-09T01:39:44.926 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:39:44.926 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T01:39:44.926 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T01:39:44.926 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.z 2024-05-09T01:39:45.246 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:39:44 smithi196 systemd[1]: Stopping Ceph mgr.z for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:39:45.501 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:39:45 smithi196 podman[71711]: 2024-05-09 01:39:45.250478383 +0000 UTC m=+0.132614717 container died 9b182b9c488027d72d87e9fd933b91b59c08c038947662d84879440b5b64844c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T01:39:45.777 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:39:45 smithi196 podman[71711]: 2024-05-09 01:39:45.596598516 +0000 UTC m=+0.478734837 container cleanup 9b182b9c488027d72d87e9fd933b91b59c08c038947662d84879440b5b64844c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T01:39:45.778 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 01:39:45 smithi196 bash[71711]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mgr-z 2024-05-09T01:39:46.005 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-09T01:39:46.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:39:46.039 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T01:39:46.039 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T01:39:46.039 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.0 2024-05-09T01:39:46.357 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:46 smithi019 systemd[1]: Stopping Ceph osd.0 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:39:46.644 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0[44990]: 2024-05-09T01:39:46.360+0000 7f5256d84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:39:46.644 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0[44990]: 2024-05-09T01:39:46.360+0000 7f5256d84700 -1 osd.0 86 *** Got signal Terminated *** 2024-05-09T01:39:46.644 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0[44990]: 2024-05-09T01:39:46.360+0000 7f5256d84700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:39:52.140 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:51 smithi019 podman[124886]: 2024-05-09 01:39:51.863871286 +0000 UTC m=+5.618098077 container died b28402698792403948590763db6bad9c9c0d3ab6fe3c93dc6ec0ee5e968e66f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD) 2024-05-09T01:39:52.394 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:52 smithi019 podman[124886]: 2024-05-09 01:39:52.310994784 +0000 UTC m=+6.065221567 container cleanup b28402698792403948590763db6bad9c9c0d3ab6fe3c93dc6ec0ee5e968e66f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T01:39:52.394 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:52 smithi019 bash[124886]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0 2024-05-09T01:39:52.894 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:52 smithi019 podman[124903]: 2024-05-09 01:39:52.512522832 +0000 UTC m=+0.641431860 container remove b28402698792403948590763db6bad9c9c0d3ab6fe3c93dc6ec0ee5e968e66f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T01:39:53.228 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:52 smithi019 podman[124996]: 2024-05-09 01:39:52.9469978 +0000 UTC m=+0.113306216 container create 088fe65ba0fa8301608d620710b83fd6c55d9931b8db5457819c5d4a53c239e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T01:39:53.228 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:52 smithi019 podman[124996]: 2024-05-09 01:39:52.852310396 +0000 UTC m=+0.018618814 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:39:53.228 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:53 smithi019 podman[124996]: 2024-05-09 01:39:53.158421104 +0000 UTC m=+0.324729524 container init 088fe65ba0fa8301608d620710b83fd6c55d9931b8db5457819c5d4a53c239e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T01:39:53.229 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:53 smithi019 podman[124996]: 2024-05-09 01:39:53.161783305 +0000 UTC m=+0.328091723 container start 088fe65ba0fa8301608d620710b83fd6c55d9931b8db5457819c5d4a53c239e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:39:53.229 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:53 smithi019 podman[124996]: 2024-05-09 01:39:53.232731071 +0000 UTC m=+0.399039502 container attach 088fe65ba0fa8301608d620710b83fd6c55d9931b8db5457819c5d4a53c239e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-09T01:39:53.644 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 01:39:53 smithi019 podman[124996]: 2024-05-09 01:39:53.475213045 +0000 UTC m=+0.641521462 container died 088fe65ba0fa8301608d620710b83fd6c55d9931b8db5457819c5d4a53c239e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T01:39:54.155 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.0.service' 2024-05-09T01:39:54.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:39:54.201 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T01:39:54.201 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T01:39:54.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.1 2024-05-09T01:39:54.596 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:39:54 smithi059 systemd[1]: Stopping Ceph osd.1 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:39:55.096 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:39:54 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1[38944]: 2024-05-09T01:39:54.642+0000 7fe61eb9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:39:55.096 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:39:54 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1[38944]: 2024-05-09T01:39:54.642+0000 7fe61eb9a700 -1 osd.1 86 *** Got signal Terminated *** 2024-05-09T01:39:55.096 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:39:54 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1[38944]: 2024-05-09T01:39:54.642+0000 7fe61eb9a700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:40:00.449 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:00 smithi059 podman[70310]: 2024-05-09 01:40:00.151777248 +0000 UTC m=+5.744753383 container died c96899f4b54123695ae5730ee51f6de7d926e827554d16ee0306700120b2000e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T01:40:00.757 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:00 smithi059 podman[70310]: 2024-05-09 01:40:00.551227928 +0000 UTC m=+6.144204062 container cleanup c96899f4b54123695ae5730ee51f6de7d926e827554d16ee0306700120b2000e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T01:40:00.757 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:00 smithi059 bash[70310]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1 2024-05-09T01:40:01.096 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:00 smithi059 podman[70328]: 2024-05-09 01:40:00.761069625 +0000 UTC m=+0.611289761 container remove c96899f4b54123695ae5730ee51f6de7d926e827554d16ee0306700120b2000e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:40:01.410 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:01 smithi059 podman[70419]: 2024-05-09 01:40:01.103574889 +0000 UTC m=+0.114258117 container create 3cd4b32f2352a008b929d8d9b37130abc697f58727c0640dd3c8c111ba0c50db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-09T01:40:01.410 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:01 smithi059 podman[70419]: 2024-05-09 01:40:01.008127498 +0000 UTC m=+0.018810702 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:40:01.410 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:01 smithi059 podman[70419]: 2024-05-09 01:40:01.331044681 +0000 UTC m=+0.341727867 container init 3cd4b32f2352a008b929d8d9b37130abc697f58727c0640dd3c8c111ba0c50db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T01:40:01.411 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:01 smithi059 podman[70419]: 2024-05-09 01:40:01.334246326 +0000 UTC m=+0.344929521 container start 3cd4b32f2352a008b929d8d9b37130abc697f58727c0640dd3c8c111ba0c50db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:40:01.411 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:01 smithi059 podman[70419]: 2024-05-09 01:40:01.414257547 +0000 UTC m=+0.424940771 container attach 3cd4b32f2352a008b929d8d9b37130abc697f58727c0640dd3c8c111ba0c50db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:40:01.846 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 01:40:01 smithi059 podman[70419]: 2024-05-09 01:40:01.640596432 +0000 UTC m=+0.651279622 container died 3cd4b32f2352a008b929d8d9b37130abc697f58727c0640dd3c8c111ba0c50db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:40:02.354 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.1.service' 2024-05-09T01:40:02.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:40:02.399 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T01:40:02.400 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T01:40:02.400 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.2 2024-05-09T01:40:02.694 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:02 smithi196 systemd[1]: Stopping Ceph osd.2 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:40:02.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:02 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2[40189]: 2024-05-09T01:40:02.731+0000 7f6d43459700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:40:02.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:02 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2[40189]: 2024-05-09T01:40:02.731+0000 7f6d43459700 -1 osd.2 86 *** Got signal Terminated *** 2024-05-09T01:40:02.958 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:02 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2[40189]: 2024-05-09T01:40:02.731+0000 7f6d43459700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:40:08.405 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:08 smithi196 podman[71835]: 2024-05-09 01:40:08.134315254 +0000 UTC m=+5.545209721 container died 4841ccf6b9241763603569dab4052cbba0ac66730abe951043484c716225e953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T01:40:08.694 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:08 smithi196 podman[71835]: 2024-05-09 01:40:08.499640253 +0000 UTC m=+5.910534727 container cleanup 4841ccf6b9241763603569dab4052cbba0ac66730abe951043484c716225e953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T01:40:08.694 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:08 smithi196 bash[71835]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2 2024-05-09T01:40:09.114 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:08 smithi196 podman[71864]: 2024-05-09 01:40:08.801314297 +0000 UTC m=+0.661777582 container remove 4841ccf6b9241763603569dab4052cbba0ac66730abe951043484c716225e953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T01:40:09.434 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:09 smithi196 podman[71956]: 2024-05-09 01:40:09.020724477 +0000 UTC m=+0.018002102 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:40:09.435 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:09 smithi196 podman[71956]: 2024-05-09 01:40:09.118776049 +0000 UTC m=+0.116053658 container create 9c1729ffee125de16d66b8f118e49ec26779d248771c5b9fd48bd74671521480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T01:40:09.435 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:09 smithi196 podman[71956]: 2024-05-09 01:40:09.363759912 +0000 UTC m=+0.361037522 container init 9c1729ffee125de16d66b8f118e49ec26779d248771c5b9fd48bd74671521480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:40:09.435 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:09 smithi196 podman[71956]: 2024-05-09 01:40:09.367049877 +0000 UTC m=+0.364327495 container start 9c1729ffee125de16d66b8f118e49ec26779d248771c5b9fd48bd74671521480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True) 2024-05-09T01:40:09.435 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:09 smithi196 podman[71956]: 2024-05-09 01:40:09.438139597 +0000 UTC m=+0.435417221 container attach 9c1729ffee125de16d66b8f118e49ec26779d248771c5b9fd48bd74671521480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True) 2024-05-09T01:40:09.694 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 01:40:09 smithi196 podman[71956]: 2024-05-09 01:40:09.668994551 +0000 UTC m=+0.666272162 container died 9c1729ffee125de16d66b8f118e49ec26779d248771c5b9fd48bd74671521480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T01:40:10.231 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.2.service' 2024-05-09T01:40:10.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:40:10.272 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T01:40:10.272 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T01:40:10.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.3 2024-05-09T01:40:10.577 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:10 smithi019 systemd[1]: Stopping Ceph osd.3 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:40:10.893 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:10 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3[51497]: 2024-05-09T01:40:10.580+0000 7f8c5c2bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:40:10.894 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:10 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3[51497]: 2024-05-09T01:40:10.580+0000 7f8c5c2bd700 -1 osd.3 86 *** Got signal Terminated *** 2024-05-09T01:40:10.894 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:10 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3[51497]: 2024-05-09T01:40:10.580+0000 7f8c5c2bd700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:40:12.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:12 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:12.306+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:12.893 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:12 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3[51497]: 2024-05-09T01:40:12.481+0000 7f8c5486b700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.930513+0000 front 2024-05-09T01:39:49.930570+0000 (oldest deadline 2024-05-09T01:40:12.230693+0000) 2024-05-09T01:40:13.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:12 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:12.983+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:13.596 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:13 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:13.132+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:13.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:13 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:13.265+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:13.894 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:13 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3[51497]: 2024-05-09T01:40:13.457+0000 7f8c5486b700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.930513+0000 front 2024-05-09T01:39:49.930570+0000 (oldest deadline 2024-05-09T01:40:12.230693+0000) 2024-05-09T01:40:13.894 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:13 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:13.701+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:14.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:14.013+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:14.596 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:14 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:14.118+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:14.597 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:14 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:14.255+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:14.646 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:14 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3[51497]: 2024-05-09T01:40:14.408+0000 7f8c5486b700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.930513+0000 front 2024-05-09T01:39:49.930570+0000 (oldest deadline 2024-05-09T01:40:12.230693+0000) 2024-05-09T01:40:14.647 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:14 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:14.304+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:15.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:14 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:14.714+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:15.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:15.007+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:15.596 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:15 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:15.157+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:15.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:15 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:15.291+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:15.644 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:15 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3[51497]: 2024-05-09T01:40:15.394+0000 7f8c5486b700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.930513+0000 front 2024-05-09T01:39:49.930570+0000 (oldest deadline 2024-05-09T01:40:12.230693+0000) 2024-05-09T01:40:15.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:15 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:15.296+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:16.144 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:15 smithi019 podman[125119]: 2024-05-09 01:40:15.942571666 +0000 UTC m=+5.462474582 container died a701beef54eb9efac1841089e84ea6981cc7468efd7bc89bcd95ae3b8fd25f27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:40:16.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:15 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:15.695+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:16.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:15.969+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:16.596 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:16 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:16.147+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:16.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:16 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:16.301+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:16.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:16 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:16.248+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:16.644 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:16 smithi019 podman[125119]: 2024-05-09 01:40:16.439475371 +0000 UTC m=+5.959378318 container cleanup a701beef54eb9efac1841089e84ea6981cc7468efd7bc89bcd95ae3b8fd25f27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:40:16.644 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:16 smithi019 bash[125119]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3 2024-05-09T01:40:16.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:16 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:16.612+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:17.143 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:16 smithi019 podman[125136]: 2024-05-09 01:40:16.832391581 +0000 UTC m=+0.880011946 container remove a701beef54eb9efac1841089e84ea6981cc7468efd7bc89bcd95ae3b8fd25f27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T01:40:17.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:16 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:16.726+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:17.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:17.018+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:17.481 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:17 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:17.290+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:17.481 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:17 smithi019 podman[125227]: 2024-05-09 01:40:17.12791599 +0000 UTC m=+0.018970496 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:40:17.481 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:17 smithi019 podman[125227]: 2024-05-09 01:40:17.258395661 +0000 UTC m=+0.149450169 container create e6cbff418df8883d46731f417dfaf309572070d975b5e5ae213bcfe988afd133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-05-09T01:40:17.596 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:17 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:17.125+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:17.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:17 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:17.297+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:17.752 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:17 smithi019 podman[125227]: 2024-05-09 01:40:17.485722521 +0000 UTC m=+0.376777032 container init e6cbff418df8883d46731f417dfaf309572070d975b5e5ae213bcfe988afd133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:40:17.752 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:17 smithi019 podman[125227]: 2024-05-09 01:40:17.489349346 +0000 UTC m=+0.380403852 container start e6cbff418df8883d46731f417dfaf309572070d975b5e5ae213bcfe988afd133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T01:40:17.753 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:17 smithi019 podman[125227]: 2024-05-09 01:40:17.577256468 +0000 UTC m=+0.468310991 container attach e6cbff418df8883d46731f417dfaf309572070d975b5e5ae213bcfe988afd133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=) 2024-05-09T01:40:17.943 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:17.581+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:18.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:17 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:17.755+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:18.144 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:17 smithi019 podman[125227]: 2024-05-09 01:40:17.793934762 +0000 UTC m=+0.684989258 container died e6cbff418df8883d46731f417dfaf309572070d975b5e5ae213bcfe988afd133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T01:40:18.443 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:18.067+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:18.506 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:18 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:18.292+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:18.507 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:18 smithi019 podman[125253]: 2024-05-09 01:40:18.497704345 +0000 UTC m=+0.698223998 container remove e6cbff418df8883d46731f417dfaf309572070d975b5e5ae213bcfe988afd133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T01:40:18.507 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:18 smithi019 systemd[1]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.3.service: Deactivated successfully. 2024-05-09T01:40:18.507 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 01:40:18 smithi019 systemd[1]: Stopped Ceph osd.3 for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:40:18.509 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.3.service' 2024-05-09T01:40:18.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:40:18.551 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T01:40:18.551 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T01:40:18.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.4 2024-05-09T01:40:18.579 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:18 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:18.099+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:18.579 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:18 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:18.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:18.846 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:18 smithi059 systemd[1]: Stopping Ceph osd.4 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:40:18.894 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:18 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:18.718+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:18.943 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:18.573+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:19.298 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:18 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:18.870+0000 7f1b3b64a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:40:19.299 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:18 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:18.870+0000 7f1b3b64a700 -1 osd.4 86 *** Got signal Terminated *** 2024-05-09T01:40:19.299 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:18 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:18.870+0000 7f1b3b64a700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:40:19.299 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:19 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:19.083+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:19.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:19.090+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:19.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:19 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:19.302+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:19.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:19 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:19.270+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:19.943 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:19.538+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:20.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:19 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:19.726+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:20.323 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:20 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:20.066+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:20.346 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:20 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:20.095+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:20.346 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:20 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:20.342+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:20.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:20 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:20.307+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:20.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:20 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:20.558+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:21.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:20 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:20.723+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:21.346 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:21 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:21.062+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:21.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:21 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:21.100+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:21.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:21 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:21.100+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:21.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:21 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:21.287+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:21.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:21 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:21.287+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:21.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:21 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:21.350+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:21.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:21 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:21.572+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:21.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:21 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:21.572+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:22.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:21 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:21.682+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:22.344 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:22 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:22.037+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:22.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:22 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:22.062+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:22.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:22 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:22.062+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:22.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:22 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:22.347+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:22.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:22 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:22.256+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:22.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:22 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:22.256+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:22.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:22 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:22.578+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:22.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:22 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:22.578+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:23.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:22 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:22.718+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:23.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:22 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:22.718+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:23.346 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:23 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4[45889]: 2024-05-09T01:40:23.060+0000 7f1b33bf8700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:46.476304+0000 front 2024-05-09T01:39:46.476365+0000 (oldest deadline 2024-05-09T01:40:12.376428+0000) 2024-05-09T01:40:23.346 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:23 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:23.305+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:23.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:23 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:23.080+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:23.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:23 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:23.080+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:23.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:23 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:23.223+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:23.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:23 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:23.223+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:23.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:23 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:23.606+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:23.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:23 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:23.606+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:24.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:23 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:23.742+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:24.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:23 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:23.761+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:24.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:23 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:23.761+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:24.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:24 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:24.042+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:24.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:24 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:24.042+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:24.485 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:24 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:24.309+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:24.485 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:24 smithi059 podman[70539]: 2024-05-09 01:40:24.212515656 +0000 UTC m=+5.460397898 container died f4aa789beefa814f1441d2bdec780c0d31a5d0e3ef1194dc95674829a2353893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T01:40:24.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:24 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:24.217+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:24.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:24 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:24.217+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:24.846 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:24 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:24.705+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:24.846 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:24 smithi059 podman[70539]: 2024-05-09 01:40:24.713087391 +0000 UTC m=+5.960969632 container cleanup f4aa789beefa814f1441d2bdec780c0d31a5d0e3ef1194dc95674829a2353893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True) 2024-05-09T01:40:24.846 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:24 smithi059 bash[70539]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4 2024-05-09T01:40:24.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:24 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:24.603+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:24.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:24 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:24.603+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:25.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:24 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:24.809+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:25.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:24 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:24.809+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:25.265 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:24 smithi059 podman[70556]: 2024-05-09 01:40:24.939341795 +0000 UTC m=+0.721831034 container remove f4aa789beefa814f1441d2bdec780c0d31a5d0e3ef1194dc95674829a2353893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True) 2024-05-09T01:40:25.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:25 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:25.037+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:25.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:25 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:25.037+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:25.517 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:25 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:25.345+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:25.517 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:25 smithi059 podman[70643]: 2024-05-09 01:40:25.172378271 +0000 UTC m=+0.018781690 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:40:25.517 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:25 smithi059 podman[70643]: 2024-05-09 01:40:25.315138 +0000 UTC m=+0.161541407 container create 6c0e64d2ab8ea0167035a80eae4e5724afe1ad1c07f978a76ac0c163bc9b0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T01:40:25.517 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:25 smithi059 podman[70643]: 2024-05-09 01:40:25.518129546 +0000 UTC m=+0.364532967 container init 6c0e64d2ab8ea0167035a80eae4e5724afe1ad1c07f978a76ac0c163bc9b0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T01:40:25.518 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:25 smithi059 podman[70643]: 2024-05-09 01:40:25.521255032 +0000 UTC m=+0.367658440 container start 6c0e64d2ab8ea0167035a80eae4e5724afe1ad1c07f978a76ac0c163bc9b0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:40:25.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:25 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:25.194+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:25.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:25 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:25.194+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:25.819 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:25 smithi059 podman[70643]: 2024-05-09 01:40:25.609686243 +0000 UTC m=+0.456089666 container attach 6c0e64d2ab8ea0167035a80eae4e5724afe1ad1c07f978a76ac0c163bc9b0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5) 2024-05-09T01:40:25.820 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:25 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:25.725+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:25.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:25 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:25.610+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:25.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:25 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:25.610+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:26.092 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:25 smithi059 podman[70643]: 2024-05-09 01:40:25.822977103 +0000 UTC m=+0.669380510 container died 6c0e64d2ab8ea0167035a80eae4e5724afe1ad1c07f978a76ac0c163bc9b0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T01:40:26.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:25 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:25.813+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:26.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:25 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:25.813+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:26.384 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:26 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:26.035+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:26.384 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:26 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:26.035+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:26.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.4.service' 2024-05-09T01:40:26.640 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:26 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:26.365+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:26.640 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:26 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:26.365+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:26.640 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:26 smithi059 podman[70669]: 2024-05-09 01:40:26.546764412 +0000 UTC m=+0.717899556 container remove 6c0e64d2ab8ea0167035a80eae4e5724afe1ad1c07f978a76ac0c163bc9b0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:40:26.640 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:26 smithi059 systemd[1]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-09T01:40:26.640 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:26 smithi059 systemd[1]: Stopped Ceph osd.4 for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:40:26.640 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 01:40:26 smithi059 systemd[1]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.4.service: Consumed 13.947s CPU time. 2024-05-09T01:40:26.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:26 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:26.163+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:26.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:26 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:26.163+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:26.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:40:26.659 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T01:40:26.659 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T01:40:26.659 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.5 2024-05-09T01:40:26.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:26 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:26.388+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:26.685 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:26 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:26.578+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:26.686 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:26 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:26.578+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:26.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:26 smithi196 systemd[1]: Stopping Ceph osd.5 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:40:27.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:26 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:26.742+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:27.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:26 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:26.769+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:27.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:26 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:26.769+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:27.380 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:27 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:27.028+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:27.380 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:27 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:27.028+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:27.380 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:26 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:26.968+0000 7f7e953d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:40:27.380 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:26 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:26.968+0000 7f7e953d7700 -1 osd.5 86 *** Got signal Terminated *** 2024-05-09T01:40:27.381 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:26 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:26.968+0000 7f7e953d7700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:40:27.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:27 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:27.158+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:27.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:27 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:27.158+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:27.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:27 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:27.384+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:27.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:27 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:27.614+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:27.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:27 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:27.614+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:27.727 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:27 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:27.388+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:27.728 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:27 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:27.388+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:28.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:27 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:27.731+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:28.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:27 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:27.744+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:28.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:27 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:27.744+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:28.331 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:28.059+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:28.331 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:28.059+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:28.620 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:28.334+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:28.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:28.206+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:28.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:28.206+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:28.708 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:28.397+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:28.708 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:28.397+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:28.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:28.624+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:28.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:28 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:28.624+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:29.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:28.712+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:29.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:28 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:28.712+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:29.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:28.713+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:29.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:28 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:28.713+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:29.359 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:29 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:29.074+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:29.359 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:29 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:29.074+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:29.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:29 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:29.241+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:29.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:29 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:29.241+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:29.693 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:29 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:29.362+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:29.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:29 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:29.593+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:29.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:29 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:29.593+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:29.747 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:29 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:29.392+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:29.747 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:29 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:29.392+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:29.747 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:29 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:29.392+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:30.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:29 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:29.750+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:30.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:29 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:29.750+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:30.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:29 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:29.691+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:30.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:29 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:29.691+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:30.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:29 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:29.691+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:30.390 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:30 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:30.116+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:30.391 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:30 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:30.116+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:30.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:30 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:30.214+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:30.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:30 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:30.214+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:30.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:30 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:30.394+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:30.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:30 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:30.596+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:30.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:30 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:30.596+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:30.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:30 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:30.596+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:04.672847+0000 front 2024-05-09T01:40:04.672866+0000 (oldest deadline 2024-05-09T01:40:29.973074+0000) 2024-05-09T01:40:30.747 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:30 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:30.426+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:30.747 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:30 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:30.426+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:30.747 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:30 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:30.426+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:31.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:30 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:30.751+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:31.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:30 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:30.751+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:31.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:30 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:30.648+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:31.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:30 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:30.648+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:31.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:30 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:30.648+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:31.383 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:31.112+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:31.384 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:31.112+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:31.384 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:31.112+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:31.624 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:31.226+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:31.624 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:31.226+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:31.624 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:31.226+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:31.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:31.387+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:31.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:31.387+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:31.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:31.581+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.569107+0000 front 2024-05-09T01:39:48.569174+0000 (oldest deadline 2024-05-09T01:40:15.569722+0000) 2024-05-09T01:40:31.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:31.581+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.171696+0000 front 2024-05-09T01:39:57.171726+0000 (oldest deadline 2024-05-09T01:40:20.671460+0000) 2024-05-09T01:40:31.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:31 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5[47135]: 2024-05-09T01:40:31.581+0000 7f7e8d985700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:04.672847+0000 front 2024-05-09T01:40:04.672866+0000 (oldest deadline 2024-05-09T01:40:29.973074+0000) 2024-05-09T01:40:31.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:31 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:31.379+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:31.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:31 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:31.379+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:31.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:31 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:31.379+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:31.893 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:31.623+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:31.894 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:31.623+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:31.894 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:31.623+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:32.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:31 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:31.719+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:32.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:31 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:31.719+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:32.341 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:32 smithi196 podman[72061]: 2024-05-09 01:40:32.278569092 +0000 UTC m=+5.408906112 container died 262bb24ed4f1f4198fbba214c986d2ed59c63bed2f87305094bc9c2e61fd68b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-09T01:40:32.342 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:32 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:32.076+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:32.342 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:32 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:32.076+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:32.342 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:32 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:32.076+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:32.614 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:32 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:32.344+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:32.614 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:32 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:32.345+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:32.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:32.241+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:32.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:32.241+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:32.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:32.241+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:32.689 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:32 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:32.428+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:32.689 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:32 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:32.428+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:32.689 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:32 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:32.428+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:32.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:32 smithi196 podman[72061]: 2024-05-09 01:40:32.617960261 +0000 UTC m=+5.748297280 container cleanup 262bb24ed4f1f4198fbba214c986d2ed59c63bed2f87305094bc9c2e61fd68b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True) 2024-05-09T01:40:32.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:32 smithi196 bash[72061]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5 2024-05-09T01:40:32.944 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:32 smithi196 podman[72080]: 2024-05-09 01:40:32.810794845 +0000 UTC m=+0.528332734 container remove 262bb24ed4f1f4198fbba214c986d2ed59c63bed2f87305094bc9c2e61fd68b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T01:40:32.992 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:32 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:32.691+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:32.993 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:32 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:32.691+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:33.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:32.658+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:33.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:32.658+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:33.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:32 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:32.658+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:33.337 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:33 smithi196 podman[72174]: 2024-05-09 01:40:33.145459586 +0000 UTC m=+0.018224666 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:40:33.337 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:33 smithi196 podman[72174]: 2024-05-09 01:40:33.244872953 +0000 UTC m=+0.117638028 container create 6e61e38aa3145221bf3b9e006a849f6d032250eec7d63e61d415a689e1fe898e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T01:40:33.337 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:33.313+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:33.337 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:33.313+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:33.337 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:33.077+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:33.338 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:33.077+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:33.338 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:33 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:33.077+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:33.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:33 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:33.259+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:33.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:33 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:33.259+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:33.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:33 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:33.259+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:33.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:33 smithi196 podman[72174]: 2024-05-09 01:40:33.430780627 +0000 UTC m=+0.303545703 container init 6e61e38aa3145221bf3b9e006a849f6d032250eec7d63e61d415a689e1fe898e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T01:40:33.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:33 smithi196 podman[72174]: 2024-05-09 01:40:33.433800517 +0000 UTC m=+0.306565602 container start 6e61e38aa3145221bf3b9e006a849f6d032250eec7d63e61d415a689e1fe898e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T01:40:33.694 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:33 smithi196 podman[72174]: 2024-05-09 01:40:33.506505398 +0000 UTC m=+0.379270487 container attach 6e61e38aa3145221bf3b9e006a849f6d032250eec7d63e61d415a689e1fe898e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:40:33.846 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:33 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:33.655+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:33.846 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:33 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:33.655+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:33.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:33 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:33.417+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:33.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:33 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:33.417+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:33.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:33 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:33.417+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:34.013 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 01:40:33 smithi196 podman[72174]: 2024-05-09 01:40:33.729824248 +0000 UTC m=+0.602589332 container died 6e61e38aa3145221bf3b9e006a849f6d032250eec7d63e61d415a689e1fe898e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:40:34.143 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:33 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:33.700+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:34.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:33 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:33.700+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:34.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:33 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:33.700+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:34.277 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:34 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:34.117+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:34.278 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:34 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:34.117+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:34.278 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:34 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:34.117+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:34.330 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.5.service' 2024-05-09T01:40:34.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:40:34.409 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T01:40:34.409 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T01:40:34.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.6 2024-05-09T01:40:34.425 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:34 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:34.224+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:34.425 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:34 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:34.224+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:34.426 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:34 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:34.224+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:34.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:34 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:34.281+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:34.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:34 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:34.281+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:34.700 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:34 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:34.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:34.701 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:34 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:34.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:34.701 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:34 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:34.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:34.731 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:34 smithi019 systemd[1]: Stopping Ceph osd.6 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:40:35.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:34 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:34.703+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:35.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:34 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:34.703+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:35.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:34 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:34.736+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:35.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:34 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:34.736+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:35.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:34 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:34.736+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:35.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:34 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:34.741+0000 7f63c569f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:40:35.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:34 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:34.741+0000 7f63c569f700 -1 osd.6 86 *** Got signal Terminated *** 2024-05-09T01:40:35.144 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:34 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:34.741+0000 7f63c569f700 -1 osd.6 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:40:35.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:35.267+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:35.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:35.267+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:35.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:35.073+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:35.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:35.073+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:35.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:35 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:35.073+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:35.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:35 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:35.187+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:35.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:35 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:35.187+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:35.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:35 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:35.187+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:35.747 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:35 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:35.483+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:35.747 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:35 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:35.483+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:35.747 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:35 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:35.483+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:36.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:35 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:35.749+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:36.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:35 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:35.749+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:36.141 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:35 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:35.756+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:36.142 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:35 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:35.756+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:36.142 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:35 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:35.756+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:36.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:36 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:36.145+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:36.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:36 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:36.145+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:36.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:36 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:36.145+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:36.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:36.226+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:36.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:36.226+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:36.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:36.106+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:36.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:36.106+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:36.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:36 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:36.106+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:36.796 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:36.475+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:36.796 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:36.475+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:36.796 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:36.475+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:36.797 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:36.475+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:37.094 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:36 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:36.709+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:37.094 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:36 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:36.709+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:37.095 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:36 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:36.709+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:37.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:36.799+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:37.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:36 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:36.799+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:37.393 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:37 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:37.098+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:37.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:37 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:37.098+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:37.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:37 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:37.098+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:37.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:37.264+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:37.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:37.264+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:37.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:37.130+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:37.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:37.130+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:37.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:37.130+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:37.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:37 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:37.130+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:37.815 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:37.450+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:37.815 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:37.450+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:37.816 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:37.450+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:37.816 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:37.450+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:38.052 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:37 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:37.701+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:38.052 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:37 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:37.701+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:38.052 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:37 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:37.701+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:38.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:37.818+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:38.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:37 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:37.818+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:38.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:38 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:38.056+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:38.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:38 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:38.056+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:38.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:38 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:38.056+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:38.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:38 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:38.056+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:38.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:38 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:38.285+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:38.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:38 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:38.285+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:38.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:38 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:38.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:38.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:38 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:38.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:38.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:38 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:38.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:38.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:38 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:38.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:38.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:38.404+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:38.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:38.404+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:38.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:38.404+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:38.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:38.404+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:39.028 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:38 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:38.733+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:48.624270+0000 front 2024-05-09T01:39:48.624310+0000 (oldest deadline 2024-05-09T01:40:12.724394+0000) 2024-05-09T01:40:39.029 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:38 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:38.733+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.225660+0000 front 2024-05-09T01:39:56.225478+0000 (oldest deadline 2024-05-09T01:40:22.125473+0000) 2024-05-09T01:40:39.029 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:38 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6[58501]: 2024-05-09T01:40:38.733+0000 7f63bdc4d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.526897+0000 front 2024-05-09T01:40:05.527033+0000 (oldest deadline 2024-05-09T01:40:29.626906+0000) 2024-05-09T01:40:39.346 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:38.859+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:39.346 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:38.859+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:39.346 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:38 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:38.859+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:39.393 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:39 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:39.033+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:39.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:39 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:39.033+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:39.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:39 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:39.033+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:39.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:39 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:39.033+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:39.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:39 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:39.258+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:39.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:39 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:39.258+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:39.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:39 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:39.185+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:39.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:39 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:39.185+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:39.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:39 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:39.185+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:39.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:39 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:39.185+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:39.818 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:39.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:39.819 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:39.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:39.819 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:39.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:39.819 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:39.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:40.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:39.821+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:40.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:39.821+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:40.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:39.821+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:40.338 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:40 smithi019 podman[125332]: 2024-05-09 01:40:40.072759968 +0000 UTC m=+5.434751381 container died 2039531a3d37001b0982017032a11e03bd41d840d2f3578f7b24fa7c018354ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T01:40:40.338 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:40 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:40.077+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:40.338 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:40 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:40.077+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:40.338 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:40 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:40.077+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:40.338 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:40 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:40.077+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:40.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:40 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:40.220+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:40.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:40 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:40.220+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:40.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:40 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:40.220+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:40.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:40 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:40.167+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:40.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:40 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:40.167+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:40.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:40 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:40.167+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:40.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:40 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:40.167+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:40.643 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:40 smithi019 podman[125332]: 2024-05-09 01:40:40.483220473 +0000 UTC m=+5.845211889 container cleanup 2039531a3d37001b0982017032a11e03bd41d840d2f3578f7b24fa7c018354ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T01:40:40.644 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:40 smithi019 bash[125332]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6 2024-05-09T01:40:40.829 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:40 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:40.368+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:40.829 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:40 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:40.368+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:40.829 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:40 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:40.368+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:40.829 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:40 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:40.368+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:41.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:40 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:40.832+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:41.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:40 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:40.832+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:41.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:40 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:40.832+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:41.111 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:40 smithi019 podman[125360]: 2024-05-09 01:40:40.710143189 +0000 UTC m=+0.638121424 container remove 2039531a3d37001b0982017032a11e03bd41d840d2f3578f7b24fa7c018354ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-09T01:40:41.394 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:41 smithi019 podman[125450]: 2024-05-09 01:40:41.077329006 +0000 UTC m=+0.018660130 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:40:41.394 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:41 smithi019 podman[125450]: 2024-05-09 01:40:41.202566149 +0000 UTC m=+0.143897300 container create e13dec54e85a25683465f6955c17945536b6e9ffe5f72f8f87dda1ead033af3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:40:41.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:41 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:41.114+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:41.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:41 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:41.114+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:41.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:41 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:41.114+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:41.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:41 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:41.114+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:41.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:41 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:41.196+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:41.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:41 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:41.196+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:41.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:41 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:41.196+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:41.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:41 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:41.204+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:41.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:41 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:41.204+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:41.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:41 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:41.204+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:41.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:41 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:41.204+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:41.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:41 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:41.339+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:41.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:41 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:41.339+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:41.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:41 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:41.339+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:41.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:41 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:41.339+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:41.759 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:41 smithi019 podman[125450]: 2024-05-09 01:40:41.455294087 +0000 UTC m=+0.396625208 container init e13dec54e85a25683465f6955c17945536b6e9ffe5f72f8f87dda1ead033af3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:40:41.759 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:41 smithi019 podman[125450]: 2024-05-09 01:40:41.459350148 +0000 UTC m=+0.400681261 container start e13dec54e85a25683465f6955c17945536b6e9ffe5f72f8f87dda1ead033af3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T01:40:41.759 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:41 smithi019 podman[125450]: 2024-05-09 01:40:41.55489767 +0000 UTC m=+0.496228796 container attach e13dec54e85a25683465f6955c17945536b6e9ffe5f72f8f87dda1ead033af3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-09T01:40:42.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:41 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:41.812+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:42.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:41 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:41.812+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:42.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:41 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:41.812+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:42.119 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 01:40:41 smithi019 podman[125450]: 2024-05-09 01:40:41.763890477 +0000 UTC m=+0.705221589 container died e13dec54e85a25683465f6955c17945536b6e9ffe5f72f8f87dda1ead033af3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T01:40:42.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:42 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:42.126+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:42.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:42 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:42.126+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:42.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:42 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:42.126+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:42.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:42 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:42.126+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:42.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.6.service' 2024-05-09T01:40:42.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:40:42.527 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T01:40:42.527 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T01:40:42.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.7 2024-05-09T01:40:42.574 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:42.311+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:42.574 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:42.311+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:42.574 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:42.311+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:42.574 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:42.311+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:42.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:42 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:42.226+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:42.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:42 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:42.226+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:42.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:42 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:42.226+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:42.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:42 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:42.236+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:42.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:42 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:42.236+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:42.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:42 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:42.236+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:42.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:42 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:42.236+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:42.830 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:42 smithi059 systemd[1]: Stopping Ceph osd.7 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:40:42.830 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:42.785+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:42.830 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:42.785+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:42.830 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:42.785+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:42.830 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:42.832+0000 7f153617a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:40:42.830 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:42.832+0000 7f153617a700 -1 osd.7 86 *** Got signal Terminated *** 2024-05-09T01:40:42.830 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:42 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:42.832+0000 7f153617a700 -1 osd.7 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:40:43.393 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:43 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:43.103+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:43.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:43 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:43.103+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:43.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:43 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:43.103+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:43.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:43 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:43.103+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:43.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:43 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:43.188+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:43.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:43 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:43.188+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:43.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:43 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:43.188+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:43.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:43 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:43.278+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:43.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:43 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:43.278+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:43.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:43 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:43.278+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:43.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:43 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:43.278+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:43.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:43 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:43.322+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:43.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:43 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:43.322+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:43.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:43 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:43.322+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:43.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:43 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:43.322+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:44.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:43 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:43.817+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:44.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:43 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:43.817+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:44.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:43 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:43.817+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:44.393 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:44 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:44.093+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:44.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:44 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:44.093+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:44.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:44 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:44.093+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:44.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:44 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:44.093+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:44.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:44 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:44.274+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:44.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:44 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:44.274+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:44.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:44 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:44.274+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:44.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:44 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:44.274+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:44.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:44 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:44.209+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:44.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:44 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:44.209+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:44.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:44 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:44.209+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:44.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:44 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:44.296+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:44.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:44 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:44.296+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:44.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:44 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:44.296+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:44.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:44 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:44.296+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:45.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:44 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:44.802+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:45.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:44 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:44.802+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:45.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:44 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:44.802+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:45.400 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:45 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:45.112+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:45.400 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:45 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:45.112+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:45.400 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:45 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:45.112+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:45.401 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:45 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:45.112+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:45.597 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:45.303+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:45.597 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:45.303+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:45.597 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:45.303+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:45.597 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:45.303+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:45.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:45 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:45.206+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:45.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:45 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:45.206+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:45.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:45 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:45.206+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:45.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:45 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:45.206+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:45.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:45 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:45.318+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:45.695 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:45 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:45.318+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:45.695 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:45 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:45.318+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:45.695 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:45 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:45.318+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:45.695 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:45 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:45.318+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:46.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:45.768+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:46.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:45.768+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:46.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:45.768+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:46.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:45.768+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:22.232461+0000 front 2024-05-09T01:40:22.232379+0000 (oldest deadline 2024-05-09T01:40:45.132564+0000) 2024-05-09T01:40:46.393 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:46.128+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:46.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:46.128+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:46.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:46.128+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:46.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:46 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:46.128+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:46.446 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:46.161+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:46.446 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:46.161+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:46.446 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:46.161+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:46.446 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:46.161+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:46.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:46.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:46.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:46.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:46.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:46.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:46.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:46.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:46.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:46 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:46.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:46.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:46 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:46.278+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:46.597 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:46 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:46.278+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:46.597 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:46 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:46.278+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:46.597 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:46 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:46.278+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:47.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:46 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:46.740+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:47.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:46 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:46.740+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:47.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:46 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:46.740+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:47.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:46 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:46.740+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:22.232461+0000 front 2024-05-09T01:40:22.232379+0000 (oldest deadline 2024-05-09T01:40:45.132564+0000) 2024-05-09T01:40:47.393 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:47 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:47.105+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:47.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:47 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:47.105+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:47.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:47 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:47.105+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:47.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:47 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:47.105+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:47.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:47 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:47.195+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:47.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:47 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:47.195+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:47.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:47 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:47.195+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:47.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:47 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:47.195+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:47.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:47 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:47.307+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:47.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:47 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:47.307+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:47.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:47 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:47.307+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:47.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:47 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:47.307+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:47.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:47 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:47.307+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:47.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:47.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:47.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:47.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:47.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:47.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:47.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:47.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:47.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:47.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:48.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:47.761+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:57.628797+0000 front 2024-05-09T01:39:57.628734+0000 (oldest deadline 2024-05-09T01:40:23.528857+0000) 2024-05-09T01:40:48.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:47.761+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:03.529264+0000 front 2024-05-09T01:40:03.529286+0000 (oldest deadline 2024-05-09T01:40:28.229409+0000) 2024-05-09T01:40:48.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:47.761+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:12.330892+0000 front 2024-05-09T01:40:12.330662+0000 (oldest deadline 2024-05-09T01:40:38.230835+0000) 2024-05-09T01:40:48.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:47 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7[52902]: 2024-05-09T01:40:47.761+0000 7f1532730700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:22.232461+0000 front 2024-05-09T01:40:22.232379+0000 (oldest deadline 2024-05-09T01:40:45.132564+0000) 2024-05-09T01:40:48.393 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:48 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:48.112+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:48.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:48 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:48.112+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:48.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:48 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:48.112+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:48.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:48 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:48.112+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:48.397 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:48 smithi059 podman[70759]: 2024-05-09 01:40:48.148205327 +0000 UTC m=+5.419150668 container died 7f926f41f4df881441d605746de38b91cf081a06c5be83f56e55e3c8cc116bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T01:40:48.397 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:48 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:48.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:48.397 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:48 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:48.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:48.397 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:48 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:48.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:48.398 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:48 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:48.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:48.398 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:48 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:48.316+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:48.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:48 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:48.208+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:48.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:48 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:48.208+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:48.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:48 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:48.208+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:48.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:48 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:48.208+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:48.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:48 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:48.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:48.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:48 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:48.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:48.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:48 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:48.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:48.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:48 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:48.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:48.695 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:48 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:48.274+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:48.788 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:48 smithi059 podman[70759]: 2024-05-09 01:40:48.507505158 +0000 UTC m=+5.778450492 container cleanup 7f926f41f4df881441d605746de38b91cf081a06c5be83f56e55e3c8cc116bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-09T01:40:48.788 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:48 smithi059 bash[70759]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7 2024-05-09T01:40:49.096 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:48 smithi059 podman[70777]: 2024-05-09 01:40:48.79206987 +0000 UTC m=+0.637134594 container remove 7f926f41f4df881441d605746de38b91cf081a06c5be83f56e55e3c8cc116bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-09T01:40:49.393 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:49 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:49.110+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:49.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:49 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:49.110+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:49.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:49 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:49.110+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:49.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:49 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:49.110+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:49.434 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:49 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:49.348+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:49.434 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:49 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:49.348+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:49.434 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:49 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:49.348+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:49.435 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:49 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:49.348+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:49.435 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:49 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:49.348+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:49.435 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:49 smithi059 podman[70869]: 2024-05-09 01:40:49.034632529 +0000 UTC m=+0.018238486 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:40:49.435 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:49 smithi059 podman[70869]: 2024-05-09 01:40:49.142949027 +0000 UTC m=+0.126554964 container create d39eaed505c1d089c0c78bf833a9a358e76898fd91f099e1bc1997b5609cc374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T01:40:49.435 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:49 smithi059 podman[70869]: 2024-05-09 01:40:49.370106281 +0000 UTC m=+0.353712236 container init d39eaed505c1d089c0c78bf833a9a358e76898fd91f099e1bc1997b5609cc374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:40:49.435 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:49 smithi059 podman[70869]: 2024-05-09 01:40:49.37331049 +0000 UTC m=+0.356916429 container start d39eaed505c1d089c0c78bf833a9a358e76898fd91f099e1bc1997b5609cc374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T01:40:49.435 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:49 smithi059 podman[70869]: 2024-05-09 01:40:49.436684399 +0000 UTC m=+0.420290348 container attach d39eaed505c1d089c0c78bf833a9a358e76898fd91f099e1bc1997b5609cc374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:40:49.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:49 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:49.158+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:49.446 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:49 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:49.158+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:49.446 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:49 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:49.158+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:49.446 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:49 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:49.158+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:49.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:49 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:49.252+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:49.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:49 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:49.252+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:49.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:49 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:49.252+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:49.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:49 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:49.252+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:49.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:49 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:49.252+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:49.846 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 01:40:49 smithi059 podman[70869]: 2024-05-09 01:40:49.680823008 +0000 UTC m=+0.664428969 container died d39eaed505c1d089c0c78bf833a9a358e76898fd91f099e1bc1997b5609cc374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:40:50.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.7.service' 2024-05-09T01:40:50.302 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:40:50.303 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T01:40:50.303 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T01:40:50.303 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.8 2024-05-09T01:40:50.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:50 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:50.102+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:50.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:50 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:50.102+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:50.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:50 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:50.102+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:50.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:50 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:50.102+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:50.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:50.155+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:50.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:50.155+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:50.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:50.155+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:50.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:50.155+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:50.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:50 smithi196 systemd[1]: Stopping Ceph osd.8 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:40:50.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:50.222+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:50.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:50.222+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:50.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:50.222+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:50.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:50.222+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:50.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:50.222+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:50.530 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:50 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:50.343+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:50.530 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:50 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:50.343+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:50.530 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:50 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:50.343+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:50.530 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:50 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:50.343+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:50.530 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:50 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:50.343+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:50.944 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:50.602+0000 7f012f034700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:40:50.944 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:50.602+0000 7f012f034700 -1 osd.8 86 *** Got signal Terminated *** 2024-05-09T01:40:50.944 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:50 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:50.602+0000 7f012f034700 -1 osd.8 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:40:51.393 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:51 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:51.126+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:51.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:51 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:51.126+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:51.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:51 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:51.126+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:51.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:51 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:51.126+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:51.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:51 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:51.189+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:51.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:51 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:51.189+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:51.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:51 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:51.189+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:51.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:51 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:51.189+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:51.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:51 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:51.210+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:51.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:51 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:51.210+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:51.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:51 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:51.210+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:51.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:51 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:51.210+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:51.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:51 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:51.210+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:51.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:51 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:51.371+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:51.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:51 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:51.371+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:51.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:51 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:51.371+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:51.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:51 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:51.371+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:51.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:51 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:51.372+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:52.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:52 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:52.143+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:52.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:52 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:52.143+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:52.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:52 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:52.143+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:52.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:52 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:52.143+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:52.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:52 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:52.140+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:52.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:52 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:52.140+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:52.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:52 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:52.140+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:52.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:52 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:52.140+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:52.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:52 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:52.170+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:52.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:52 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:52.170+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:52.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:52 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:52.170+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:52.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:52 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:52.170+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:52.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:52 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:52.170+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:52.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:52 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:52.421+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:52.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:52 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:52.421+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:52.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:52 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:52.421+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:52.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:52 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:52.421+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:52.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:52 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:52.421+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:53.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:53.195+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:53.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:53.195+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:53.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:53.195+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:53.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:53.195+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:53.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:53.195+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:53.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:53.135+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:53.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:53.135+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:53.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:53.135+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:53.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:53.135+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:53.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:53.135+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.190324+0000 front 2024-05-09T01:40:30.190289+0000 (oldest deadline 2024-05-09T01:40:52.490432+0000) 2024-05-09T01:40:53.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:53 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:53.186+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:53.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:53 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:53.186+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:53.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:53 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:53.186+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:53.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:53 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:53.186+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:53.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:53 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:53.465+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:53.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:53 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:53.465+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:53.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:53 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:53.465+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:53.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:53 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:53.465+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:53.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:53 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:53.465+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:53.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:53 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:53.465+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:40:54.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:54.178+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:54.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:54.178+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:54.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:54.178+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:54.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:54.178+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:54.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:54.178+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:54.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:54.144+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:54.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:54.144+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:54.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:54.144+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:54.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:54.144+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:54.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:54 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:54.144+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.190324+0000 front 2024-05-09T01:40:30.190289+0000 (oldest deadline 2024-05-09T01:40:52.490432+0000) 2024-05-09T01:40:54.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:54 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:54.169+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:54.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:54 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:54.169+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:54.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:54 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:54.169+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:54.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:54 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:54.169+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:54.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:54 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:54.427+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:54.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:54 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:54.427+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:54.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:54 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:54.427+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:54.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:54 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:54.427+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:54.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:54 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:54.427+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:54.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:54 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:54.427+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:40:55.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:55.139+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:55.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:55.139+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:55.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:55.139+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:55.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:55.140+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:55.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:55.140+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:55.444 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:55.129+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.384939+0000 front 2024-05-09T01:39:59.385036+0000 (oldest deadline 2024-05-09T01:40:25.785567+0000) 2024-05-09T01:40:55.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:55.129+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.785922+0000 front 2024-05-09T01:40:05.785955+0000 (oldest deadline 2024-05-09T01:40:31.086136+0000) 2024-05-09T01:40:55.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:55.129+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.587170+0000 front 2024-05-09T01:40:11.587223+0000 (oldest deadline 2024-05-09T01:40:39.787655+0000) 2024-05-09T01:40:55.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:55.129+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:19.788340+0000 front 2024-05-09T01:40:19.788485+0000 (oldest deadline 2024-05-09T01:40:44.488281+0000) 2024-05-09T01:40:55.445 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:55 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8[53685]: 2024-05-09T01:40:55.129+0000 7f01275e2700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.190324+0000 front 2024-05-09T01:40:30.190289+0000 (oldest deadline 2024-05-09T01:40:52.490432+0000) 2024-05-09T01:40:55.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:55 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:55.160+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:55.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:55 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:55.160+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:55.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:55 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:55.160+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:55.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:55 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:55.160+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:55.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:55 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:55.160+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:31.023152+0000 front 2024-05-09T01:40:31.022959+0000 (oldest deadline 2024-05-09T01:40:55.122933+0000) 2024-05-09T01:40:55.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:55 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:55.436+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:55.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:55 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:55.436+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:55.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:55 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:55.436+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:55.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:55 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:55.436+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:55.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:55 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:55.436+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:55.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:55 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:55.436+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:40:56.310 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:56 smithi196 podman[72279]: 2024-05-09 01:40:56.042339054 +0000 UTC m=+5.539876095 container died 35c16b4ebe3562844e3e1991982b2ae054b646bf12a0ed1f921e441d554a6e8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T01:40:56.310 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:56 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:56.146+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:56.310 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:56 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:56.147+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:56.310 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:56 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:56.147+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:56.311 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:56 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:56.147+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:56.311 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:56 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:56.147+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:56.311 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:56 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:56.147+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:40:56.398 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:56 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:56.133+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:56.398 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:56 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:56.133+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:56.398 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:56 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:56.133+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:56.398 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:56 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:56.133+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:56.398 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:56 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:56.133+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:31.023152+0000 front 2024-05-09T01:40:31.022959+0000 (oldest deadline 2024-05-09T01:40:55.122933+0000) 2024-05-09T01:40:56.695 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:56 smithi196 podman[72279]: 2024-05-09 01:40:56.404189636 +0000 UTC m=+5.901726672 container cleanup 35c16b4ebe3562844e3e1991982b2ae054b646bf12a0ed1f921e441d554a6e8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T01:40:56.696 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:56 smithi196 bash[72279]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8 2024-05-09T01:40:56.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:56 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:56.439+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:56.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:56 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:56.439+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:56.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:56 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:56.439+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:56.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:56 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:56.439+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:56.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:56 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:56.439+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:56.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:56 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:56.439+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:40:57.019 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:56 smithi196 podman[72296]: 2024-05-09 01:40:56.706103261 +0000 UTC m=+0.661650595 container remove 35c16b4ebe3562844e3e1991982b2ae054b646bf12a0ed1f921e441d554a6e8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T01:40:57.314 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:57 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:57.188+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:57.314 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:57 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:57.188+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:57.315 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:57 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:57.188+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:57.315 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:57 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:57.188+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:57.315 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:57 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:57.188+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:57.315 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:57 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:57.188+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:40:57.315 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:57 smithi196 podman[72389]: 2024-05-09 01:40:56.926403677 +0000 UTC m=+0.018002633 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:40:57.315 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:57 smithi196 podman[72389]: 2024-05-09 01:40:57.023446839 +0000 UTC m=+0.115045768 container create cd3a78508fa05c4099863318af1165ea9745b73b39d5dde8400417c5cb98c05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-05-09T01:40:57.315 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:57 smithi196 podman[72389]: 2024-05-09 01:40:57.217818146 +0000 UTC m=+0.309417083 container init cd3a78508fa05c4099863318af1165ea9745b73b39d5dde8400417c5cb98c05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-09T01:40:57.315 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:57 smithi196 podman[72389]: 2024-05-09 01:40:57.220760031 +0000 UTC m=+0.312358961 container start cd3a78508fa05c4099863318af1165ea9745b73b39d5dde8400417c5cb98c05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T01:40:57.315 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:57 smithi196 podman[72389]: 2024-05-09 01:40:57.318401295 +0000 UTC m=+0.410000225 container attach cd3a78508fa05c4099863318af1165ea9745b73b39d5dde8400417c5cb98c05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:40:57.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:57.120+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:57.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:57.120+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:57.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:57.120+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:57.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:57.120+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:57.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:57 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:57.120+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:31.023152+0000 front 2024-05-09T01:40:31.022959+0000 (oldest deadline 2024-05-09T01:40:55.122933+0000) 2024-05-09T01:40:57.694 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 01:40:57 smithi196 podman[72389]: 2024-05-09 01:40:57.517946146 +0000 UTC m=+0.609545086 container died cd3a78508fa05c4099863318af1165ea9745b73b39d5dde8400417c5cb98c05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:40:57.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:57 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:57.473+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:57.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:57 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:57.473+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:57.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:57 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:57.473+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:57.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:57 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:57.473+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:57.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:57 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:57.473+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:57.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:57 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:57.473+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:40:58.087 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.8.service' 2024-05-09T01:40:58.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:40:58.129 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T01:40:58.129 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T01:40:58.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.9 2024-05-09T01:40:58.340 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:58 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:58.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:58.340 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:58 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:58.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:58.340 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:58 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:58.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:58.340 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:58 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:58.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:58.340 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:58 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:58.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:58.340 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:58 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:58.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:40:58.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:58.143+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:58.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:58.143+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:58.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:58.143+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:58.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:58.143+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:58.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:58.143+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:31.023152+0000 front 2024-05-09T01:40:31.022959+0000 (oldest deadline 2024-05-09T01:40:55.122933+0000) 2024-05-09T01:40:58.394 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:58 smithi019 systemd[1]: Stopping Ceph osd.9 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:40:58.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:58 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:58.474+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:58.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:58 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:58.474+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:58.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:58 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:58.474+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:58.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:58 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:58.474+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:58.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:58 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:58.474+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:58.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:58 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:58.474+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:40:58.893 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:58.436+0000 7fef2bae3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:40:58.893 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:58.436+0000 7fef2bae3700 -1 osd.9 86 *** Got signal Terminated *** 2024-05-09T01:40:58.894 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:58 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:58.436+0000 7fef2bae3700 -1 osd.9 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:40:59.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:59 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:59.165+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:40:59.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:59 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:59.165+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:40:59.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:59 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:59.165+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:40:59.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:59 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:59.165+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:40:59.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:59 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:59.165+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:40:59.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:40:59 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:40:59.165+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:40:59.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:59 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:59.149+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:40:59.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:59 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:59.149+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:40:59.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:59 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:59.149+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:40:59.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:59 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:59.149+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:40:59.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:40:59 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:40:59.149+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:31.023152+0000 front 2024-05-09T01:40:31.022959+0000 (oldest deadline 2024-05-09T01:40:55.122933+0000) 2024-05-09T01:40:59.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:59 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:59.447+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:40:59.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:59 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:59.447+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:40:59.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:59 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:59.447+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:40:59.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:59 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:59.447+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:40:59.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:59 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:59.447+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:40:59.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:40:59 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:40:59.447+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:00.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:00 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:00.160+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:00.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:00 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:00.160+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:00.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:00 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:00.160+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:00.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:00 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:00.160+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:00.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:00 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:00.160+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:00.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:00 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:00.160+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:00.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:00 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:00.179+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:41:00.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:00 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:00.179+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:41:00.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:00 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:00.179+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:41:00.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:00 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:00.179+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:41:00.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:00 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:00.179+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:31.023152+0000 front 2024-05-09T01:40:31.022959+0000 (oldest deadline 2024-05-09T01:40:55.122933+0000) 2024-05-09T01:41:00.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:00 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:00.471+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:00.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:00 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:00.471+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:00.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:00 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:00.471+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:00.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:00 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:00.471+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:00.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:00 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:00.471+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:00.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:00 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:00.471+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:01.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:01 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:01.162+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:41:01.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:01 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:01.162+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:41:01.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:01 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:01.162+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:41:01.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:01 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:01.162+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:41:01.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:01 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:01.162+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:31.023152+0000 front 2024-05-09T01:40:31.022959+0000 (oldest deadline 2024-05-09T01:40:55.122933+0000) 2024-05-09T01:41:01.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:01.198+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:01.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:01.198+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:01.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:01.198+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:01.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:01.198+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:01.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:01.198+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:01.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:01.198+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:01.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:01 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:01.453+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:01.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:01 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:01.453+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:01.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:01 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:01.453+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:01.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:01 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:01.453+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:01.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:01 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:01.453+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:01.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:01 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:01.453+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:02.595 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:02 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:02.226+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:02.595 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:02 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:02.226+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:02.595 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:02 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:02.226+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:02.595 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:02 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:02.226+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:02.595 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:02 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:02.226+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:02.595 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:02 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:02.226+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:02.643 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:02 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:02.163+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:41:02.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:02 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:02.163+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:41:02.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:02 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:02.163+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:41:02.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:02 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:02.163+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:41:02.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:02 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:02.163+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:31.023152+0000 front 2024-05-09T01:40:31.022959+0000 (oldest deadline 2024-05-09T01:40:55.122933+0000) 2024-05-09T01:41:02.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:02 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:02.163+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:39.724274+0000 front 2024-05-09T01:40:39.724296+0000 (oldest deadline 2024-05-09T01:41:02.024343+0000) 2024-05-09T01:41:02.748 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:02 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:02.498+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:02.748 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:02 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:02.498+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:02.748 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:02 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:02.498+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:02.748 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:02 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:02.498+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:02.748 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:02 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:02.498+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:02.748 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:02 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:02.498+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:03.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:03 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:03.202+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:49.414155+0000 front 2024-05-09T01:39:49.414009+0000 (oldest deadline 2024-05-09T01:40:13.514232+0000) 2024-05-09T01:41:03.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:03 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:03.202+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:56.216247+0000 front 2024-05-09T01:39:56.216279+0000 (oldest deadline 2024-05-09T01:40:20.316216+0000) 2024-05-09T01:41:03.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:03 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:03.202+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:06.717857+0000 front 2024-05-09T01:40:06.717824+0000 (oldest deadline 2024-05-09T01:40:30.217874+0000) 2024-05-09T01:41:03.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:03 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:03.202+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:14.819466+0000 front 2024-05-09T01:40:14.819652+0000 (oldest deadline 2024-05-09T01:40:37.119619+0000) 2024-05-09T01:41:03.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:03 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:03.202+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:31.023152+0000 front 2024-05-09T01:40:31.022959+0000 (oldest deadline 2024-05-09T01:40:55.122933+0000) 2024-05-09T01:41:03.644 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:03 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9[65488]: 2024-05-09T01:41:03.202+0000 7fef28099700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:39.724274+0000 front 2024-05-09T01:40:39.724296+0000 (oldest deadline 2024-05-09T01:41:02.024343+0000) 2024-05-09T01:41:03.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:03 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:03.227+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:03.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:03 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:03.227+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:03.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:03 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:03.227+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:03.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:03 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:03.227+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:03.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:03 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:03.227+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:03.695 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:03 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:03.227+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:03.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:03 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:03.502+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:03.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:03 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:03.502+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:03.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:03 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:03.502+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:03.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:03 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:03.502+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:03.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:03 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:03.502+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:03.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:03 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:03.502+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:03.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:03 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:03.502+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:37.835222+0000 front 2024-05-09T01:40:37.835199+0000 (oldest deadline 2024-05-09T01:41:03.135011+0000) 2024-05-09T01:41:04.143 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:03 smithi019 podman[125555]: 2024-05-09 01:41:03.838627462 +0000 UTC m=+5.499286775 container died 004b242765f9722e938dfed76b168bb0afb7a09ff81def5acdd4addda1293d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T01:41:04.440 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:04 smithi019 podman[125555]: 2024-05-09 01:41:04.260574146 +0000 UTC m=+5.921233465 container cleanup 004b242765f9722e938dfed76b168bb0afb7a09ff81def5acdd4addda1293d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:41:04.440 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:04 smithi019 bash[125555]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9 2024-05-09T01:41:04.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:04 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:04.211+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:04.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:04 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:04.211+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:04.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:04 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:04.211+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:04.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:04 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:04.211+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:04.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:04 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:04.211+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:04.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:04 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:04.211+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:04.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:04 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:04.549+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:04.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:04 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:04.549+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:04.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:04 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:04.549+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:04.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:04 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:04.549+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:04.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:04 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:04.549+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:04.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:04 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:04.549+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:04.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:04 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:04.549+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:37.835222+0000 front 2024-05-09T01:40:37.835199+0000 (oldest deadline 2024-05-09T01:41:03.135011+0000) 2024-05-09T01:41:04.893 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:04 smithi019 podman[125572]: 2024-05-09 01:41:04.444978593 +0000 UTC m=+0.603154436 container remove 004b242765f9722e938dfed76b168bb0afb7a09ff81def5acdd4addda1293d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T01:41:05.202 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:04 smithi019 podman[125665]: 2024-05-09 01:41:04.805068749 +0000 UTC m=+0.018528108 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:41:05.202 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:04 smithi019 podman[125665]: 2024-05-09 01:41:04.904229896 +0000 UTC m=+0.117689296 container create 1c3ea78752b201b4870fb9d3549549900172f3ff339dfeff2affe2dd0fc695ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:41:05.202 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:05 smithi019 podman[125665]: 2024-05-09 01:41:05.131427389 +0000 UTC m=+0.344886758 container init 1c3ea78752b201b4870fb9d3549549900172f3ff339dfeff2affe2dd0fc695ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T01:41:05.202 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:05 smithi019 podman[125665]: 2024-05-09 01:41:05.135115896 +0000 UTC m=+0.348575256 container start 1c3ea78752b201b4870fb9d3549549900172f3ff339dfeff2affe2dd0fc695ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T01:41:05.202 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:05 smithi019 podman[125665]: 2024-05-09 01:41:05.206255162 +0000 UTC m=+0.419714531 container attach 1c3ea78752b201b4870fb9d3549549900172f3ff339dfeff2affe2dd0fc695ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:41:05.469 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 01:41:05 smithi019 podman[125665]: 2024-05-09 01:41:05.454801998 +0000 UTC m=+0.668261357 container died 1c3ea78752b201b4870fb9d3549549900172f3ff339dfeff2affe2dd0fc695ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T01:41:05.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:05 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:05.257+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:05.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:05 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:05.257+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:05.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:05 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:05.257+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:05.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:05 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:05.257+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:05.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:05 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:05.257+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:05.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:05 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:05.257+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:05.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:05 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:05.524+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:05.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:05 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:05.524+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:05.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:05 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:05.524+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:05.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:05 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:05.524+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:05.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:05 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:05.524+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:05.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:05 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:05.524+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:05.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:05 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:05.524+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:37.835222+0000 front 2024-05-09T01:40:37.835199+0000 (oldest deadline 2024-05-09T01:41:03.135011+0000) 2024-05-09T01:41:06.148 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.9.service' 2024-05-09T01:41:06.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:06.188 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T01:41:06.188 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T01:41:06.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.10 2024-05-09T01:41:06.487 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 systemd[1]: Stopping Ceph osd.10 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:06.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:06 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:06.225+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:06.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:06 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:06.225+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:06.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:06 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:06.225+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:06.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:06 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:06.225+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:06.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:06 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:06.225+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:06.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:06 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:06.225+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:06.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.490+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:06.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.490+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:06.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.490+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:06.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.490+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:06.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.490+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:06.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.490+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:06.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.490+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:37.835222+0000 front 2024-05-09T01:40:37.835199+0000 (oldest deadline 2024-05-09T01:41:03.135011+0000) 2024-05-09T01:41:06.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.642+0000 7f94c72ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:06.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.642+0000 7f94c72ea700 -1 osd.10 86 *** Got signal Terminated *** 2024-05-09T01:41:06.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:06 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:06.642+0000 7f94c72ea700 -1 osd.10 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:41:07.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:07 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:07.216+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:07.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:07 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:07.216+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:07.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:07 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:07.216+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:07.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:07 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:07.216+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:07.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:07 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:07.216+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:07.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:07 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:07.216+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:07.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:07 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:07.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:07.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:07 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:07.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:07.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:07 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:07.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:07.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:07 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:07.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:07.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:07 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:07.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:07.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:07 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:07.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:07.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:07 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:07.442+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:37.835222+0000 front 2024-05-09T01:40:37.835199+0000 (oldest deadline 2024-05-09T01:41:03.135011+0000) 2024-05-09T01:41:08.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:08 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:08.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:08.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:08 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:08.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:08.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:08 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:08.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:08.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:08 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:08.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:08.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:08 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:08.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:08.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:08 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:08.186+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:08.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:08 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:08.448+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:08.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:08 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:08.448+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:08.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:08 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:08.448+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:08.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:08 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:08.448+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:08.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:08 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:08.448+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:08.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:08 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:08.448+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:08.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:08 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:08.448+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:37.835222+0000 front 2024-05-09T01:40:37.835199+0000 (oldest deadline 2024-05-09T01:41:03.135011+0000) 2024-05-09T01:41:09.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:09 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:09.161+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:09.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:09 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:09.161+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:09.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:09 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:09.161+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:09.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:09 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:09.161+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:09.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:09 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:09.161+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:09.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:09 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:09.161+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:09.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:09 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:09.472+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:09.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:09 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:09.472+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:09.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:09 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:09.472+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:09.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:09 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:09.472+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:09.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:09 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:09.472+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:09.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:09 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:09.472+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:09.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:09 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:09.472+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:37.835222+0000 front 2024-05-09T01:40:37.835199+0000 (oldest deadline 2024-05-09T01:41:03.135011+0000) 2024-05-09T01:41:10.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:10 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:10.182+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:10.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:10 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:10.182+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:10.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:10 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:10.182+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:10.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:10 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:10.182+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:10.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:10 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:10.182+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:10.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:10 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:10.182+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:10.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:10 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:10.432+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:10.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:10 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:10.432+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:10.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:10 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:10.432+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:10.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:10 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:10.432+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:10.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:10 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:10.432+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:10.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:10 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:10.432+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:10.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:10 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:10.432+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:37.835222+0000 front 2024-05-09T01:40:37.835199+0000 (oldest deadline 2024-05-09T01:41:03.135011+0000) 2024-05-09T01:41:11.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:11 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:11.228+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:11.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:11 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:11.228+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:11.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:11 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:11.228+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:11.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:11 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:11.228+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:11.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:11 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:11.228+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:11.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:11 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:11.228+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:11.695 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:11 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:11.228+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T01:40:46.698890+0000 front 2024-05-09T01:40:46.698908+0000 (oldest deadline 2024-05-09T01:41:10.798624+0000) 2024-05-09T01:41:11.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:11 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:11.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:47.226271+0000 front 2024-05-09T01:39:47.226187+0000 (oldest deadline 2024-05-09T01:40:11.926226+0000) 2024-05-09T01:41:11.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:11 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:11.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.527936+0000 front 2024-05-09T01:39:59.527902+0000 (oldest deadline 2024-05-09T01:40:25.428119+0000) 2024-05-09T01:41:11.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:11 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:11.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:05.429000+0000 front 2024-05-09T01:40:05.428785+0000 (oldest deadline 2024-05-09T01:40:28.928716+0000) 2024-05-09T01:41:11.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:11 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:11.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:13.430971+0000 front 2024-05-09T01:40:13.431176+0000 (oldest deadline 2024-05-09T01:40:36.330933+0000) 2024-05-09T01:41:11.846 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:11 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:11.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:20.832951+0000 front 2024-05-09T01:40:20.832975+0000 (oldest deadline 2024-05-09T01:40:46.733083+0000) 2024-05-09T01:41:11.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:11 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:11.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:29.634498+0000 front 2024-05-09T01:40:29.634328+0000 (oldest deadline 2024-05-09T01:40:52.534214+0000) 2024-05-09T01:41:11.847 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:11 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10[59920]: 2024-05-09T01:41:11.409+0000 7f94c38a0700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T01:40:37.835222+0000 front 2024-05-09T01:40:37.835199+0000 (oldest deadline 2024-05-09T01:41:03.135011+0000) 2024-05-09T01:41:12.299 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:12 smithi059 podman[70998]: 2024-05-09 01:41:12.037779756 +0000 UTC m=+5.644588010 container died fc8a1ca79e1366cfa49212aac852c7bbe7f5ab5cbffba161279b437a31c95988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:41:12.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:12 smithi059 podman[70998]: 2024-05-09 01:41:12.409427224 +0000 UTC m=+6.016235477 container cleanup fc8a1ca79e1366cfa49212aac852c7bbe7f5ab5cbffba161279b437a31c95988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T01:41:12.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:12 smithi059 bash[70998]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10 2024-05-09T01:41:12.650 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:12 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:12.218+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:12.650 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:12 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:12.218+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:12.650 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:12 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:12.218+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:12.650 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:12 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:12.218+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:12.650 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:12 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:12.218+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:12.650 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:12 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:12.218+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:12.650 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:12 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:12.218+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T01:40:46.698890+0000 front 2024-05-09T01:40:46.698908+0000 (oldest deadline 2024-05-09T01:41:10.798624+0000) 2024-05-09T01:41:13.086 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:12 smithi059 podman[71015]: 2024-05-09 01:41:12.768914273 +0000 UTC m=+0.732576712 container remove fc8a1ca79e1366cfa49212aac852c7bbe7f5ab5cbffba161279b437a31c95988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T01:41:13.338 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:13 smithi059 podman[71103]: 2024-05-09 01:41:12.993848401 +0000 UTC m=+0.018440097 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:41:13.338 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:13 smithi059 podman[71103]: 2024-05-09 01:41:13.103114154 +0000 UTC m=+0.127705847 container create 37f0c0846b341845177319beff8c3e2b747e2a443f8677c8c3a3bbf4ed8d075e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T01:41:13.338 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:13 smithi059 podman[71103]: 2024-05-09 01:41:13.338630864 +0000 UTC m=+0.363222560 container init 37f0c0846b341845177319beff8c3e2b747e2a443f8677c8c3a3bbf4ed8d075e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T01:41:13.338 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:13 smithi059 podman[71103]: 2024-05-09 01:41:13.34185268 +0000 UTC m=+0.366444373 container start 37f0c0846b341845177319beff8c3e2b747e2a443f8677c8c3a3bbf4ed8d075e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T01:41:13.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:13 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:13.169+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:13.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:13 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:13.169+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:13.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:13 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:13.169+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:13.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:13 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:13.169+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:13.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:13 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:13.169+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:13.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:13 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:13.169+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:13.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:13 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:13.169+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T01:40:46.698890+0000 front 2024-05-09T01:40:46.698908+0000 (oldest deadline 2024-05-09T01:41:10.798624+0000) 2024-05-09T01:41:13.596 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:13 smithi059 podman[71103]: 2024-05-09 01:41:13.438743744 +0000 UTC m=+0.463335449 container attach 37f0c0846b341845177319beff8c3e2b747e2a443f8677c8c3a3bbf4ed8d075e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T01:41:13.962 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 01:41:13 smithi059 podman[71103]: 2024-05-09 01:41:13.667824408 +0000 UTC m=+0.692416104 container died 37f0c0846b341845177319beff8c3e2b747e2a443f8677c8c3a3bbf4ed8d075e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:41:14.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.10.service' 2024-05-09T01:41:14.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:14.302 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T01:41:14.302 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T01:41:14.302 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.11 2024-05-09T01:41:14.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.128+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:14.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.128+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:14.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.128+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:14.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.128+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:14.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.128+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:14.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.128+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:14.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.128+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T01:40:46.698890+0000 front 2024-05-09T01:40:46.698908+0000 (oldest deadline 2024-05-09T01:41:10.798624+0000) 2024-05-09T01:41:14.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 systemd[1]: Stopping Ceph osd.11 for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:14.944 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.606+0000 7f045fdd3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:14.944 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.606+0000 7f045fdd3700 -1 osd.11 86 *** Got signal Terminated *** 2024-05-09T01:41:14.944 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:14 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:14.606+0000 7f045fdd3700 -1 osd.11 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:41:15.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:15.095+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:15.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:15.095+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:15.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:15.095+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:15.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:15.095+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:15.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:15.095+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:15.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:15.095+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:15.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:15 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:15.095+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T01:40:46.698890+0000 front 2024-05-09T01:40:46.698908+0000 (oldest deadline 2024-05-09T01:41:10.798624+0000) 2024-05-09T01:41:16.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:16 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:16.119+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:16.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:16 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:16.119+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:16.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:16 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:16.119+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:16.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:16 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:16.119+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:16.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:16 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:16.119+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:16.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:16 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:16.119+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:16.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:16 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:16.119+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T01:40:46.698890+0000 front 2024-05-09T01:40:46.698908+0000 (oldest deadline 2024-05-09T01:41:10.798624+0000) 2024-05-09T01:41:17.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:17.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:17.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:17.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:17.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:17.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:17.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:17.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:17.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:17.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:17.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:17.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:17.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:17.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T01:40:46.698890+0000 front 2024-05-09T01:40:46.698908+0000 (oldest deadline 2024-05-09T01:41:10.798624+0000) 2024-05-09T01:41:17.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:17 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:17.141+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-05-09T01:40:51.899395+0000 front 2024-05-09T01:40:51.899414+0000 (oldest deadline 2024-05-09T01:41:16.599326+0000) 2024-05-09T01:41:18.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:18.129+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:18.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:18.129+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:18.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:18.129+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:18.446 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:18.129+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:18.450 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:18.129+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:18.450 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:18.129+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:18.450 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:18.129+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T01:40:46.698890+0000 front 2024-05-09T01:40:46.698908+0000 (oldest deadline 2024-05-09T01:41:10.798624+0000) 2024-05-09T01:41:18.451 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:18 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:18.129+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-05-09T01:40:51.899395+0000 front 2024-05-09T01:40:51.899414+0000 (oldest deadline 2024-05-09T01:41:16.599326+0000) 2024-05-09T01:41:19.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:19.113+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T01:39:51.087942+0000 front 2024-05-09T01:39:51.087927+0000 (oldest deadline 2024-05-09T01:40:12.787823+0000) 2024-05-09T01:41:19.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:19.113+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T01:39:59.491443+0000 front 2024-05-09T01:39:59.491385+0000 (oldest deadline 2024-05-09T01:40:20.591240+0000) 2024-05-09T01:41:19.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:19.113+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T01:40:07.593068+0000 front 2024-05-09T01:40:07.593043+0000 (oldest deadline 2024-05-09T01:40:31.093089+0000) 2024-05-09T01:41:19.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:19.113+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T01:40:11.093661+0000 front 2024-05-09T01:40:11.093574+0000 (oldest deadline 2024-05-09T01:40:36.993649+0000) 2024-05-09T01:41:19.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:19.113+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T01:40:21.694952+0000 front 2024-05-09T01:40:21.694889+0000 (oldest deadline 2024-05-09T01:40:45.194751+0000) 2024-05-09T01:41:19.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:19.113+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T01:40:30.495827+0000 front 2024-05-09T01:40:30.495956+0000 (oldest deadline 2024-05-09T01:40:55.796008+0000) 2024-05-09T01:41:19.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:19.113+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T01:40:46.698890+0000 front 2024-05-09T01:40:46.698908+0000 (oldest deadline 2024-05-09T01:41:10.798624+0000) 2024-05-09T01:41:19.444 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:19 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11[60685]: 2024-05-09T01:41:19.113+0000 7f0458381700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-05-09T01:40:51.899395+0000 front 2024-05-09T01:40:51.899414+0000 (oldest deadline 2024-05-09T01:41:16.599326+0000) 2024-05-09T01:41:20.370 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:20 smithi196 podman[72506]: 2024-05-09 01:41:20.085520487 +0000 UTC m=+5.581299034 container died 0efac5a8c5f4372dbca5a43a89cf1f28240b6254872af25694f314c8f531e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:41:20.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:20 smithi196 podman[72506]: 2024-05-09 01:41:20.464119893 +0000 UTC m=+5.959898467 container cleanup 0efac5a8c5f4372dbca5a43a89cf1f28240b6254872af25694f314c8f531e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T01:41:20.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:20 smithi196 bash[72506]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11 2024-05-09T01:41:21.128 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:20 smithi196 podman[72523]: 2024-05-09 01:41:20.773655591 +0000 UTC m=+0.676920299 container remove 0efac5a8c5f4372dbca5a43a89cf1f28240b6254872af25694f314c8f531e254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T01:41:21.407 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:21 smithi196 podman[72616]: 2024-05-09 01:41:21.034697748 +0000 UTC m=+0.018097944 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:41:21.407 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:21 smithi196 podman[72616]: 2024-05-09 01:41:21.149563007 +0000 UTC m=+0.132963188 container create d1352e410a753f2897425d3cf49b4bd5d1f41938072d11aa5cd085fe607d2da8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T01:41:21.407 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:21 smithi196 podman[72616]: 2024-05-09 01:41:21.335088617 +0000 UTC m=+0.318488798 container init d1352e410a753f2897425d3cf49b4bd5d1f41938072d11aa5cd085fe607d2da8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True) 2024-05-09T01:41:21.407 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:21 smithi196 podman[72616]: 2024-05-09 01:41:21.338168701 +0000 UTC m=+0.321568885 container start d1352e410a753f2897425d3cf49b4bd5d1f41938072d11aa5cd085fe607d2da8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-09T01:41:21.407 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:21 smithi196 podman[72616]: 2024-05-09 01:41:21.409953317 +0000 UTC m=+0.393353509 container attach d1352e410a753f2897425d3cf49b4bd5d1f41938072d11aa5cd085fe607d2da8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:41:21.694 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 01:41:21 smithi196 podman[72616]: 2024-05-09 01:41:21.623043396 +0000 UTC m=+0.606443578 container died d1352e410a753f2897425d3cf49b4bd5d1f41938072d11aa5cd085fe607d2da8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T01:41:22.167 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@osd.11.service' 2024-05-09T01:41:22.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:22.203 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T01:41:22.203 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T01:41:22.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.a 2024-05-09T01:41:22.516 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:41:22 smithi019 systemd[1]: Stopping Ceph mds.a for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:22.893 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:41:22 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-a[72664]: 2024-05-09T01:41:22.519+0000 7fdd0effb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:22.893 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:41:22 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-a[72664]: 2024-05-09T01:41:22.519+0000 7fdd0effb700 -1 mds.a *** got signal Terminated *** 2024-05-09T01:41:24.883 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:41:24 smithi019 podman[125782]: 2024-05-09 01:41:24.594304924 +0000 UTC m=+2.185515091 container died 60e1d1c7825b57ef378296da2e1350fc7b5c74271bc67cdafa7315d3855bffce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507) 2024-05-09T01:41:25.143 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:41:25 smithi019 podman[125782]: 2024-05-09 01:41:25.053110488 +0000 UTC m=+2.644320703 container cleanup 60e1d1c7825b57ef378296da2e1350fc7b5c74271bc67cdafa7315d3855bffce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-09T01:41:25.144 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 01:41:25 smithi019 bash[125782]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-a 2024-05-09T01:41:25.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.a.service' 2024-05-09T01:41:25.557 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:25.557 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T01:41:25.557 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T01:41:25.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.d 2024-05-09T01:41:25.894 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 01:41:25 smithi019 systemd[1]: Stopping Ceph mds.d for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:26.394 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 01:41:25 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-d[71378]: 2024-05-09T01:41:25.898+0000 7fd9065f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:26.394 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 01:41:25 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-d[71378]: 2024-05-09T01:41:25.898+0000 7fd9065f8700 -1 mds.d *** got signal Terminated *** 2024-05-09T01:41:30.187 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 01:41:29 smithi019 podman[125903]: 2024-05-09 01:41:29.913532365 +0000 UTC m=+4.123675891 container died 5805afed24c745b2c619fba814f0b39d7224aab1c17487d2cb0f1cfe5269fe41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-d, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:41:30.625 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 01:41:30 smithi019 podman[125903]: 2024-05-09 01:41:30.360769264 +0000 UTC m=+4.570912795 container cleanup 5805afed24c745b2c619fba814f0b39d7224aab1c17487d2cb0f1cfe5269fe41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T01:41:30.625 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 01:41:30 smithi019 bash[125903]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-d 2024-05-09T01:41:30.793 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.d.service' 2024-05-09T01:41:30.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:30.827 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T01:41:30.827 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T01:41:30.827 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.g 2024-05-09T01:41:31.144 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:41:30 smithi019 systemd[1]: Stopping Ceph mds.g for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:31.643 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:41:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-g[72213]: 2024-05-09T01:41:31.167+0000 7fb114de9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:31.644 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:41:31 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-g[72213]: 2024-05-09T01:41:31.167+0000 7fb114de9700 -1 mds.g *** got signal Terminated *** 2024-05-09T01:41:32.894 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:41:32 smithi019 podman[126026]: 2024-05-09 01:41:32.601871762 +0000 UTC m=+1.544596235 container died 6750fe50565813b35c84a96fbce9bf16c1a457daa649195512403363ab20d8ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:41:33.325 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:41:33 smithi019 podman[126026]: 2024-05-09 01:41:33.007159389 +0000 UTC m=+1.949883862 container cleanup 6750fe50565813b35c84a96fbce9bf16c1a457daa649195512403363ab20d8ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-09T01:41:33.325 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:41:33 smithi019 bash[126026]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-g 2024-05-09T01:41:33.325 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 01:41:33 smithi019 podman[126040]: 2024-05-09 01:41:33.233440303 +0000 UTC m=+0.627822215 container remove 6750fe50565813b35c84a96fbce9bf16c1a457daa649195512403363ab20d8ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-g, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:41:33.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.g.service' 2024-05-09T01:41:33.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:33.382 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T01:41:33.382 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T01:41:33.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.j 2024-05-09T01:41:33.643 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:41:33 smithi019 systemd[1]: Stopping Ceph mds.j for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:34.143 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:41:33 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-j[70949]: 2024-05-09T01:41:33.744+0000 7fc006db7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:34.144 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:41:33 smithi019 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-j[70949]: 2024-05-09T01:41:33.744+0000 7fc006db7700 -1 mds.j *** got signal Terminated *** 2024-05-09T01:41:38.144 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:41:37 smithi019 podman[126146]: 2024-05-09 01:41:37.761307253 +0000 UTC m=+4.147359133 container died ed0d63ac177663e9b002a49825edeef8d663704a04bb84785014186cdf629a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T01:41:38.576 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:41:38 smithi019 podman[126146]: 2024-05-09 01:41:38.329563993 +0000 UTC m=+4.715615873 container cleanup ed0d63ac177663e9b002a49825edeef8d663704a04bb84785014186cdf629a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-j, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:41:38.576 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 01:41:38 smithi019 bash[126146]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-j 2024-05-09T01:41:38.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.j.service' 2024-05-09T01:41:38.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:38.759 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T01:41:38.759 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T01:41:38.759 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.b 2024-05-09T01:41:39.096 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:41:38 smithi059 systemd[1]: Stopping Ceph mds.b for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:39.374 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:41:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: 2024-05-09T01:41:39.109+0000 7f4edbdb8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:39.374 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:41:39 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b[65139]: 2024-05-09T01:41:39.109+0000 7f4edbdb8700 -1 mds.b *** got signal Terminated *** 2024-05-09T01:41:40.865 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:41:40 smithi059 podman[71208]: 2024-05-09 01:41:40.605974655 +0000 UTC m=+1.645806478 container died b3d7d32a0ba77efdb56d691f69dc014d325a1ca2b4e55a3ccceae7009d586ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-09T01:41:41.223 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:41:40 smithi059 podman[71208]: 2024-05-09 01:41:40.992771113 +0000 UTC m=+2.032602933 container cleanup b3d7d32a0ba77efdb56d691f69dc014d325a1ca2b4e55a3ccceae7009d586ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T01:41:41.223 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 01:41:40 smithi059 bash[71208]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-b 2024-05-09T01:41:41.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.b.service' 2024-05-09T01:41:41.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:41.401 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T01:41:41.401 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T01:41:41.402 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.e 2024-05-09T01:41:41.846 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:41:41 smithi059 systemd[1]: Stopping Ceph mds.e for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:41.846 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:41:41 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-e[64520]: 2024-05-09T01:41:41.721+0000 7f17e1e9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:41.846 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:41:41 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-e[64520]: 2024-05-09T01:41:41.721+0000 7f17e1e9d700 -1 mds.e *** got signal Terminated *** 2024-05-09T01:41:44.868 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:41:44 smithi059 podman[71339]: 2024-05-09 01:41:44.60508803 +0000 UTC m=+2.979528094 container died 0d40aa63be46422d35d627fdd51074ac7d47708d4e521c51fd025a37a72f1cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-09T01:41:45.260 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:41:44 smithi059 podman[71339]: 2024-05-09 01:41:44.962965102 +0000 UTC m=+3.337405157 container cleanup 0d40aa63be46422d35d627fdd51074ac7d47708d4e521c51fd025a37a72f1cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T01:41:45.260 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 01:41:44 smithi059 bash[71339]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-e 2024-05-09T01:41:45.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.e.service' 2024-05-09T01:41:45.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:45.433 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T01:41:45.433 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T01:41:45.433 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.h 2024-05-09T01:41:45.596 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:45 smithi059 systemd[1]: Stopping Ceph mds.h for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:46.096 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-h[64830]: 2024-05-09T01:41:45.765+0000 7fba861d6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:46.096 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:45 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-h[64830]: 2024-05-09T01:41:45.765+0000 7fba861d6700 -1 mds.h *** got signal Terminated *** 2024-05-09T01:41:48.872 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:48 smithi059 podman[71460]: 2024-05-09 01:41:48.610999981 +0000 UTC m=+2.955989763 container died 1e7f4e161b91396ce708df38488e79dd9acfdc701fb0409e67c20d6ef1a6cb01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-h, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:41:49.139 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:48 smithi059 podman[71460]: 2024-05-09 01:41:48.94982436 +0000 UTC m=+3.294814138 container cleanup 1e7f4e161b91396ce708df38488e79dd9acfdc701fb0409e67c20d6ef1a6cb01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:41:49.139 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:48 smithi059 bash[71460]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-h 2024-05-09T01:41:49.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.h.service' 2024-05-09T01:41:49.406 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:49 smithi059 podman[71475]: 2024-05-09 01:41:49.142706976 +0000 UTC m=+0.530377662 container remove 1e7f4e161b91396ce708df38488e79dd9acfdc701fb0409e67c20d6ef1a6cb01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-09T01:41:49.406 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:49 smithi059 systemd[1]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-09T01:41:49.406 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:49 smithi059 systemd[1]: Stopped Ceph mds.h for 58c4b1c8-0da1-11ef-bc97-c7b262605968. 2024-05-09T01:41:49.406 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 01:41:49 smithi059 systemd[1]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.h.service: Consumed 2.349s CPU time. 2024-05-09T01:41:49.416 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:49.417 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T01:41:49.417 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T01:41:49.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.k 2024-05-09T01:41:49.741 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:41:49 smithi059 systemd[1]: Stopping Ceph mds.k for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:50.096 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:41:49 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k[64201]: 2024-05-09T01:41:49.744+0000 7fad01ce4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:50.096 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:41:49 smithi059 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k[64201]: 2024-05-09T01:41:49.744+0000 7fad01ce4700 -1 mds.k *** got signal Terminated *** 2024-05-09T01:41:52.899 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:41:52 smithi059 podman[71583]: 2024-05-09 01:41:52.610492269 +0000 UTC m=+2.957992681 container died 5cd8671a4aa277a3e30026dec4de02f1b04f716baaf0247dd3aa9f6447cf0206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:41:52.899 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:41:52 smithi059 podman[71583]: 2024-05-09 01:41:52.903080206 +0000 UTC m=+3.250580612 container cleanup 5cd8671a4aa277a3e30026dec4de02f1b04f716baaf0247dd3aa9f6447cf0206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:41:53.243 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:41:52 smithi059 bash[71583]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k 2024-05-09T01:41:53.243 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 01:41:53 smithi059 podman[71598]: 2024-05-09 01:41:53.128982137 +0000 UTC m=+0.517287001 container remove 5cd8671a4aa277a3e30026dec4de02f1b04f716baaf0247dd3aa9f6447cf0206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T01:41:53.267 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.k.service' 2024-05-09T01:41:53.299 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:53.300 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T01:41:53.300 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T01:41:53.300 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.c 2024-05-09T01:41:53.599 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:41:53 smithi196 systemd[1]: Stopping Ceph mds.c for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:53.860 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:41:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c[65710]: 2024-05-09T01:41:53.603+0000 7f59b0026700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:53.860 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:41:53 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c[65710]: 2024-05-09T01:41:53.603+0000 7f59b0026700 -1 mds.c *** got signal Terminated *** 2024-05-09T01:41:56.941 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:41:56 smithi196 podman[72722]: 2024-05-09 01:41:56.600147042 +0000 UTC m=+3.098895890 container died e5a570786a3e44fbc9d79bf6f227bd25d2d39fbfdb2518023e477e5ad42bdd6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T01:41:57.194 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:41:57 smithi196 podman[72722]: 2024-05-09 01:41:57.036161705 +0000 UTC m=+3.534910563 container cleanup e5a570786a3e44fbc9d79bf6f227bd25d2d39fbfdb2518023e477e5ad42bdd6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:41:57.194 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 01:41:57 smithi196 bash[72722]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-c 2024-05-09T01:41:57.416 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.c.service' 2024-05-09T01:41:57.451 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:41:57.451 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T01:41:57.451 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T01:41:57.451 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.f 2024-05-09T01:41:57.944 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:41:57 smithi196 systemd[1]: Stopping Ceph mds.f for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:41:57.944 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:41:57 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f[65399]: 2024-05-09T01:41:57.765+0000 7fe6ec21d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:41:57.944 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:41:57 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f[65399]: 2024-05-09T01:41:57.765+0000 7fe6ec21d700 -1 mds.f *** got signal Terminated *** 2024-05-09T01:42:00.853 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:42:00 smithi196 podman[72854]: 2024-05-09 01:42:00.603249406 +0000 UTC m=+2.935293423 container died 7a5bab63e85e8ac0bb25bde20f769efe95cca5f1927fcd6d49670bd6ae9ea2e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T01:42:01.194 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:42:00 smithi196 podman[72854]: 2024-05-09 01:42:00.956502186 +0000 UTC m=+3.288546193 container cleanup 7a5bab63e85e8ac0bb25bde20f769efe95cca5f1927fcd6d49670bd6ae9ea2e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T01:42:01.194 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 01:42:00 smithi196 bash[72854]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-f 2024-05-09T01:42:01.379 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.f.service' 2024-05-09T01:42:01.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:42:01.411 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T01:42:01.411 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T01:42:01.411 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.i 2024-05-09T01:42:01.694 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:42:01 smithi196 systemd[1]: Stopping Ceph mds.i for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:42:02.194 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:42:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i[66021]: 2024-05-09T01:42:01.735+0000 7fa54ca1c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:42:02.194 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:42:01 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i[66021]: 2024-05-09T01:42:01.735+0000 7fa54ca1c700 -1 mds.i *** got signal Terminated *** 2024-05-09T01:42:04.875 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:42:04 smithi196 podman[72975]: 2024-05-09 01:42:04.587131456 +0000 UTC m=+2.960549021 container died ae57a6613523a7075718f55cbfe72b9a1cfaf55fa3fecf750f48beb97f515a1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:42:05.194 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:42:04 smithi196 podman[72975]: 2024-05-09 01:42:04.984643515 +0000 UTC m=+3.358061079 container cleanup ae57a6613523a7075718f55cbfe72b9a1cfaf55fa3fecf750f48beb97f515a1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:42:05.194 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 01:42:04 smithi196 bash[72975]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-i 2024-05-09T01:42:05.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.i.service' 2024-05-09T01:42:05.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:42:05.380 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T01:42:05.380 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T01:42:05.380 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.l 2024-05-09T01:42:05.693 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:42:05 smithi196 systemd[1]: Stopping Ceph mds.l for 58c4b1c8-0da1-11ef-bc97-c7b262605968... 2024-05-09T01:42:05.944 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:42:05 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l[65075]: 2024-05-09T01:42:05.696+0000 7f9032509700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:42:05.944 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:42:05 smithi196 ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l[65075]: 2024-05-09T01:42:05.696+0000 7f9032509700 -1 mds.l *** got signal Terminated *** 2024-05-09T01:42:08.927 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:42:08 smithi196 podman[73095]: 2024-05-09 01:42:08.589958439 +0000 UTC m=+2.993465795 container died e2168d199905b8017f860093dfec4ddfb5333e134354136383166d78c8a022cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:42:09.194 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:42:08 smithi196 podman[73095]: 2024-05-09 01:42:08.996658271 +0000 UTC m=+3.400165622 container cleanup e2168d199905b8017f860093dfec4ddfb5333e134354136383166d78c8a022cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T01:42:09.194 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:42:08 smithi196 bash[73095]: ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l 2024-05-09T01:42:09.194 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 01:42:09 smithi196 podman[73109]: 2024-05-09 01:42:09.164018996 +0000 UTC m=+0.569057204 container remove e2168d199905b8017f860093dfec4ddfb5333e134354136383166d78c8a022cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T01:42:09.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-58c4b1c8-0da1-11ef-bc97-c7b262605968@mds.l.service' 2024-05-09T01:42:09.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:42:09.407 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T01:42:09.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T01:42:09.610 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:42:12.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T01:42:12.360 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:42:14.864 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T01:42:15.055 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:42:17.372 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:42:17.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:42:17.439 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:42:17.463 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T01:42:17.465 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/remote/smithi019/crash 2024-05-09T01:42:17.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/crash -- . 2024-05-09T01:42:17.510 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T01:42:17.511 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-05-09T01:42:17.513 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/remote/smithi059/crash 2024-05-09T01:42:17.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/crash -- . 2024-05-09T01:42:17.552 INFO:teuthology.orchestra.run.smithi059.stderr:tar: /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T01:42:17.552 INFO:teuthology.orchestra.run.smithi059.stderr:tar: Error is not recoverable: exiting now 2024-05-09T01:42:17.554 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/remote/smithi196/crash 2024-05-09T01:42:17.555 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/crash -- . 2024-05-09T01:42:17.614 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T01:42:17.615 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-05-09T01:42:17.616 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T01:42:17.616 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T01:42:17.660 INFO:tasks.cephadm:Compressing logs... 2024-05-09T01:42:17.660 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T01:42:17.712 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T01:42:17.733 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T01:42:17.742 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T01:42:17.742 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log 2024-05-09T01:42:17.742 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T01:42:17.742 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-09T01:42:17.742 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-09T01:42:17.743 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-09T01:42:17.743 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-09T01:42:17.743 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-09T01:42:17.743 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-09T01:42:17.743 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-09T01:42:17.743 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mon.a.log: 87.8% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T01:42:17.744 INFO:teuthology.orchestra.run.smithi019.stderr: 90.3% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T01:42:17.746 INFO:teuthology.orchestra.run.smithi019.stderr: 94.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-09T01:42:17.746 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T01:42:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-09T01:42:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T01:42:17.747 INFO:teuthology.orchestra.run.smithi019.stderr: 82.9% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T01:42:17.747 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-09T01:42:17.757 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-09T01:42:17.758 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log 2024-05-09T01:42:17.758 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-05-09T01:42:17.758 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-09T01:42:17.758 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-09T01:42:17.759 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T01:42:17.759 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-09T01:42:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T01:42:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log 2024-05-09T01:42:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T01:42:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-09T01:42:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-09T01:42:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-09T01:42:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-09T01:42:17.761 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T01:42:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-09T01:42:17.769 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-09T01:42:17.769 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-09T01:42:17.769 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-09T01:42:17.769 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T01:42:17.769 INFO:teuthology.orchestra.run.smithi059.stderr: 90.5%/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.h.log: -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T01:42:17.770 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T01:42:17.770 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-09T01:42:17.770 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T01:42:17.770 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T01:42:17.772 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T01:42:17.773 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.e.log: 93.0% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T01:42:17.781 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-09T01:42:17.781 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-09T01:42:17.781 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-09T01:42:17.781 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-09T01:42:17.781 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mgr.z.log: 87.8% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T01:42:17.781 INFO:teuthology.orchestra.run.smithi196.stderr: 90.5% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T01:42:17.781 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T01:42:17.781 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-09T01:42:17.781 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T01:42:17.782 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-09T01:42:17.782 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.11.log: 93.1% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T01:42:17.782 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T01:42:17.782 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-09T01:42:17.784 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-09T01:42:17.788 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T01:42:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-09T01:42:17.793 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T01:42:17.795 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T01:42:17.805 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% 91.0% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T01:42:17.806 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T01:42:17.808 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T01:42:17.811 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T01:42:17.818 INFO:teuthology.orchestra.run.smithi019.stderr: 94.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-09T01:42:17.826 INFO:teuthology.orchestra.run.smithi019.stderr: 94.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-09T01:42:17.833 INFO:teuthology.orchestra.run.smithi059.stderr: 93.5% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-09T01:42:17.835 INFO:teuthology.orchestra.run.smithi196.stderr: 93.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-09T01:42:17.852 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-09T01:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr: 93.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-09T01:42:17.877 INFO:teuthology.orchestra.run.smithi059.stderr: 94.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-09T01:42:17.889 INFO:teuthology.orchestra.run.smithi196.stderr: 94.6% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-09T01:42:17.985 INFO:teuthology.orchestra.run.smithi019.stderr: 90.1% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T01:42:18.047 INFO:teuthology.orchestra.run.smithi196.stderr: 94.3% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-09T01:42:18.161 INFO:teuthology.orchestra.run.smithi196.stderr: 91.9% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-09T01:42:18.281 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-09T01:42:18.290 INFO:teuthology.orchestra.run.smithi059.stderr: 91.8% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-09T01:42:18.330 INFO:teuthology.orchestra.run.smithi196.stderr: 92.4% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-09T01:42:18.395 INFO:teuthology.orchestra.run.smithi196.stderr: 92.4% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-09T01:42:18.406 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-09T01:42:18.477 INFO:teuthology.orchestra.run.smithi196.stderr: 92.7% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-09T01:42:18.516 INFO:teuthology.orchestra.run.smithi059.stderr: 92.1% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-09T01:42:18.533 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-09T01:42:18.538 INFO:teuthology.orchestra.run.smithi019.stderr: 92.2% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-09T01:42:18.714 INFO:teuthology.orchestra.run.smithi059.stderr: 92.7% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-09T01:42:18.739 INFO:teuthology.orchestra.run.smithi059.stderr: 92.5% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-09T01:42:18.838 INFO:teuthology.orchestra.run.smithi019.stderr: 90.3% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-09T01:42:18.840 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-05-09T01:42:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m1.115s 2024-05-09T01:42:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m4.184s 2024-05-09T01:42:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m0.146s 2024-05-09T01:42:19.104 INFO:teuthology.orchestra.run.smithi196.stderr: 93.1% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-09T01:42:19.107 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-05-09T01:42:19.107 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m1.359s 2024-05-09T01:42:19.107 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m4.056s 2024-05-09T01:42:19.107 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.152s 2024-05-09T01:42:19.119 INFO:teuthology.orchestra.run.smithi059.stderr: 93.4% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-09T01:42:19.514 INFO:teuthology.orchestra.run.smithi059.stderr: 92.8% -- replaced with /var/log/ceph/58c4b1c8-0da1-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-09T01:42:19.516 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-09T01:42:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:real 0m1.769s 2024-05-09T01:42:19.517 INFO:teuthology.orchestra.run.smithi059.stderr:user 0m6.329s 2024-05-09T01:42:19.517 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m0.226s 2024-05-09T01:42:19.517 INFO:tasks.cephadm:Archiving logs... 2024-05-09T01:42:19.517 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/remote/smithi019/log 2024-05-09T01:42:19.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T01:42:20.190 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/remote/smithi059/log 2024-05-09T01:42:20.191 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T01:42:21.099 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/remote/smithi196/log 2024-05-09T01:42:21.100 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T01:42:21.710 INFO:tasks.cephadm:Removing cluster... 2024-05-09T01:42:21.710 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 --force 2024-05-09T01:42:21.909 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:42:22.186 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 --force 2024-05-09T01:42:22.386 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:42:22.658 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 58c4b1c8-0da1-11ef-bc97-c7b262605968 --force 2024-05-09T01:42:22.848 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 58c4b1c8-0da1-11ef-bc97-c7b262605968 2024-05-09T01:42:23.102 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T01:42:23.102 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T01:42:23.123 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T01:42:23.146 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T01:42:23.161 INFO:tasks.cephadm:Teardown complete 2024-05-09T01:42:23.161 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T01:42:23.183 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T01:42:23.183 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T01:42:23.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T01:42:23.191 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T01:42:23.327 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T01:42:23.345 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T01:42:23.346 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:42:23.349 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:42:23.352 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:42:23.364 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:42:23.365 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:42:23.367 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:42:23.369 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:42:23.369 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-05-09T01:42:23.369 INFO:teuthology.orchestra.run.smithi019.stdout:^- hv01.front.sepia.ceph.com 3 6 377 33 +3498us[+3529us] +/- 71ms 2024-05-09T01:42:23.370 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -559us[ -527us] +/- 74ms 2024-05-09T01:42:23.370 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +584us[ +615us] +/- 75ms 2024-05-09T01:42:23.370 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:42:23.372 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:42:23.373 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-05-09T01:42:23.373 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 34 +2313us[+2313us] +/- 71ms 2024-05-09T01:42:23.373 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -1738us[-1660us] +/- 74ms 2024-05-09T01:42:23.373 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -585us[ -585us] +/- 75ms 2024-05-09T01:42:23.373 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:42:23.375 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:42:23.375 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-05-09T01:42:23.376 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 34 +2366us[+2349us] +/- 71ms 2024-05-09T01:42:23.376 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -1689us[-1706us] +/- 74ms 2024-05-09T01:42:23.376 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -539us[ -556us] +/- 75ms 2024-05-09T01:42:23.376 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:42:23.377 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T01:42:23.407 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T01:42:23.407 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T01:42:23.431 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T01:42:23.524 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T01:42:23.590 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T01:42:23.656 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:42:23.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:42:23.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:42:23.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:42:23.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:42:23.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:42:23.747 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T01:42:23.761 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T01:42:23.772 INFO:teuthology.task.internal:Duration was 2123.028587 seconds 2024-05-09T01:42:23.773 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T01:42:23.785 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T01:42:23.785 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T01:42:23.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T01:42:23.790 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T01:42:23.825 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:42:23.826 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:42:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:42:24.189 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T01:42:24.189 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T01:42:24.189 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T01:42:24.252 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-05-09T01:42:24.252 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T01:42:24.279 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-05-09T01:42:24.279 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T01:42:24.315 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T01:42:24.315 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T01:42:24.318 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T01:42:24.321 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T01:42:24.381 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T01:42:24.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T01:42:24.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T01:42:24.388 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T01:42:25.735 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T01:42:25.749 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T01:42:25.749 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T01:42:25.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T01:42:25.805 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T01:42:25.830 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T01:42:25.843 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:42:25.845 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:42:25.848 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:42:25.868 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-05-09T01:42:25.873 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-05-09T01:42:25.892 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-05-09T01:42:25.910 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:42:25.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:42:25.937 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:42:25.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:42:25.962 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:42:25.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:42:25.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T01:42:25.988 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T01:42:25.989 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/remote/smithi019 2024-05-09T01:42:25.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T01:42:26.061 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/remote/smithi059 2024-05-09T01:42:26.061 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T01:42:26.102 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698932/remote/smithi196 2024-05-09T01:42:26.103 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T01:42:26.148 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T01:42:26.148 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T01:42:26.150 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T01:42:26.153 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T01:42:26.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T01:42:26.210 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T01:42:26.210 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T01:42:26.221 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T01:42:26.221 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T01:42:26.224 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T01:42:26.226 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T01:42:26.237 INFO:teuthology.orchestra.run.smithi019.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 01:42 /home/ubuntu/cephtest 2024-05-09T01:42:26.242 INFO:teuthology.orchestra.run.smithi059.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 01:42 /home/ubuntu/cephtest 2024-05-09T01:42:26.252 INFO:teuthology.orchestra.run.smithi196.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 01:42 /home/ubuntu/cephtest 2024-05-09T01:42:26.253 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T01:42:26.265 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T01:42:26.293 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/test_o_trunc}} duration: 2123.028586626053 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-09T01:42:26.294 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T01:42:26.381 INFO:teuthology.run:pass